Skip to content

drm/asahi: per-context GPU job completions stall ~50 ms every ~100 ms after ~30 min of a D3D12 game (other contexts unaffected; restarting the process fixes it) #666

Open

Description

@alexpomsft

Summary

In a long-running Diablo II: Resurrected session (D3D12 → vkd3d-proton → Honeykrisp), the game becomes jittery after ~20–45 minutes. Host gpu_scheduler tracepoints show the game's GPU context starts having frequent ~50 ms windows with no job completions, roughly every ~100 ms, while the job submission rate is unchanged. The compositor's GPU contexts keep completing normally during those windows, so the GPU is not globally stalled. Restarting only the game process (new GPU context/VM) restores normal behaviour; the host and VM stay up. Nothing is logged in dmesg.

Environment

  • Hardware: MacBook Pro 14" M2 Max (2023), GPU G14C, 30 active cores
  • Distro: Omarchy (Arch Linux ARM based, Asahi stack), kernel 7.1.12-2-12.2-sep-ARCH (linux-aurora 7.1.12.aurora2-12.2, a downstream fork from https://github.com/iconidentify/aurora-linux, not the official linux-asahi package; I can retest on an official kernel once Omarchy 4.5 is out and there is a reasonable upgrade path), 16K pages
  • Host Mesa / vulkan-asahi 26.2.4
  • Guest: muvm 0.7.0 / libkrun 1.19.6 (virtio-gpu DRM native context), guest kernel 6.12.109
  • x86-64 Mesa (FEX overlay) mesa-fex-emu-overlay-x86_64 26.2.3; FEX-Emu 2610 (locally patched); Proton Experimental 11.0-20261027b; vkd3d-proton cae2e003b
  • Display: 4K60 external monitor (DP), no VRR; game windowed, VSync on

Steps to reproduce

  1. Launch Steam under muvm/FEX and start Diablo II: Resurrected (offline character).
  2. Play normally (move between areas) for 20–45 minutes.
  3. Jitter appears and persists; Save & Exit + relaunching only the game fixes it until it reoccurs ~20–45 minutes later. Reproduced in 5 sessions.

Measurements

Host ftrace of gpu_scheduler events (drm_sched_job_queue/run/done), 20 s windows, same game process at ~5 min (smooth) vs ~36 min (jittery). Game context = fence context 46607290.

Metric (game context) ~5 min (smooth) ~36 min (jittery)
Jobs/s 2,947 2,909
run→done p50 / p95 / p99 0.16 / 0.88 / 12.4 ms 0.35 / 13.5 / 14.6 ms
Jobs with run→done 5–15 ms ~1,000 (2%) ~22,000 (38%)
Mean hw job count 1.6 14.5
Windows >20 ms with no completions (20 s) 9 (mostly loading) 166
Length of those windows p50 / p95 — 50 / 70 ms
Interval between windows p50 — ~105 ms
Time with ≥1 job in flight 81% 70%
  • Compositor contexts (14, 128) keep completing normally during the game's stall windows, which points to something per-context or per-queue.
  • The workload is ~50 small, mostly serialised jobs per frame. Lowering resolution or disabling heavy effects (AO, shadows, AA) does not help.
  • Over the session, the VMM holds ~9k → ~11.5k dmabufs (one per guest BO), then plateaus. The game's RSS grows from ~3.0 to ~4.5 GB. The engine is known to churn committed resources.
  • VKD3D_CONFIG=single_queue and VKD3D_SWAPCHAIN_PRESENT_MODE=MAILBOX do not fix it. VKD3D_SHADER_CACHE_PATH=0 causes heavy tile-pattern texture corruption on this stack (separate issue).
  • Ruled out: thermals, CPU saturation, host memory or IO pressure, and the libkrun fence path (the kernel queue→run wait did not grow).

Raw traces (~50 MB each), the analysis scripts, and asahi-diagnose output are available on request.

Possibly related

Questions

  • Is there firmware or kernel state per GPU VM/context that grows with BO bind/unbind churn, such as page-table or TLB-invalidation work, or firmware buffer or heap growth?
  • Are there asahi debug flags or tracepoints that would show what the firmware is waiting on for this queue?

Activity

  1. alexpomsft commented on Nov 5, 2026

    @alexpomsft
    Author

    Retest after upgrading to Omarchy 4.5

    The problem still reproduces on the updated stack, though it's milder. Jitter started about 36 minutes into the session, the same as before.

    What changed:

    • Kernel: linux-aurora 7.2.11.aurora14 (https://github.com/omacom/linux-aurora; still not the official linux-asahi)
    • Host Mesa: mesa-aurora/vulkan-aurora 26.3.0-devel (20261031, a1969111475)
    • linux-firmware 20260916; omarchy-aurora-firmware 20261014; m1n1-aurora 1.6.1

    Unchanged: guest x86-64 Mesa overlay 26.2.3, muvm 0.7.0, libkrun 1.19.6, FEX 2610, Proton Experimental 11.0-20261027b.

    Same gpu_scheduler trace method as before. The game's fence context is now 969.

    Metric (game context) Old kernel ~5 min Old kernel ~36 min 4.5 ~5 min 4.5 ~36 min (jittery)
    Jobs/s 2,947 2,909 3,791 3,403
    run→done p95 / p99 0.88 / 12.4 ms 13.5 / 14.6 ms 0.53 / 2.6 ms 4.9 / 8.3 ms
    Windows >20 ms with no completions (20 s) 9 166 1 67
    Length of those windows p50 / p95 — 50 / 70 ms — 39 / 253 ms
    Interval between windows p50 — ~105 ms — ~122 ms
    Mean hw job count 1.6 14.5 1.1 3.7
    Time with ≥1 job in flight 81% 70% 94% 69%

    As before, the compositor context (14) kept completing normally during the game's stall windows (p95 1.2 ms). The pattern is unchanged: about 30 minutes in, the game's context stops completing jobs in bursts while other contexts are unaffected, and restarting the game process clears it.

  2. alexpomsft commented on Nov 5, 2026

    @alexpomsft
    Author

    Follow-up: it keeps getting worse within the same session

    I took a second 20 s trace about 6 minutes after the previous one, with the same game process and context 969. Over those few minutes the numbers moved back toward the old kernel's jittery state:

    Metric (game context) 4.5, ~36 min 4.5, ~43 min Old kernel, jittery
    run→done p95 / p99 4.9 / 8.3 ms 6.3 / 9.6 ms 13.5 / 14.6 ms
    Windows >20 ms with no completions (20 s) 67 139 166
    Length of those windows p50 / p95 39 / 253 ms 37 / 57 ms 50 / 70 ms
    Interval between windows p50 ~122 ms ~107 ms ~105 ms
    Mean hw job count 3.7 7.3 14.5
    Time with ≥1 job in flight 69% 68% 70%

    The compositor context (14) was still unaffected: run→done p95 1.1 ms, queue→run ~0. So on the new kernel the degradation appears to build up more slowly but still progresses with the context's lifetime, and restarting the game process remains the only thing that clears it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions