nx2: read GPU-written command lists from copies instead of draining

The trees fix (#52) drains the split and flushes for every command list
on a GPU-written page: in a Mario Kart World race ~13,500 lists and ~400
GPU round trips a second, during which the back has nothing to do. The
replay ran at ~40 fps instead of ~45.

The front now marks the pages those lists are on. The buffer cache notes
writes to them; after a compute dispatch or DMA copy that writes one it
copies the page into host memory and submits. The front waits for the
back only up to the last compute or copy work it sent (not everything
queued), then reads the copy once the GPU is done with it.

Pages the GPU wrote once, long before, and the CPU now writes lists to
have no copy: their first list drains and flushes the whole page, which
is then read from guest memory until GPU work writes it again. CPU
writes and inline uploads reach guest memory, so they keep such a page
synced but make a copy stale. A page a draw writes always drains.

Mario Kart World time-trial replay, Radeon 780M: ~40 -> ~45 fps, trees
drawn, no list drains left.

1 files changed, 28 insertions, 2 deletions

diff --git a/src/video_core/gpu_split.cpp b/src/video_core/gpu_split.cpp
index ac0d3f25f1..ef0cdae123 100644
--- a/src/video_core/gpu_split.cpp
+++ b/src/video_core/gpu_split.cpp
@@ -138,6 +138,7 @@ void GpuSplit::Stop() {
     // never ran, it waited on its syncpoint forever), and the engines it held were freed.
     ring.reset();
     write_position = 0;
+    last_writer_position = 0;
     published_local = 0;
     consumed_cache = 0;
     publish_every = PublishEvery;
@@ -216,6 +217,28 @@ void GpuSplit::Drain() {
     }
 }
 
+void GpuSplit::WaitForWriters() {
+    const u64 target = std::min(last_writer_position, write_position);
+    consumed_cache = consumed.load(std::memory_order_acquire);
+    if (consumed_cache >= target) {
+        return;
+    }
+    wait_target.store(target, std::memory_order_seq_cst);
+    Publish();
+    for (u32 spins = 0;; ++spins) {
+        consumed_cache = consumed.load(std::memory_order_acquire);
+        if (consumed_cache >= target) {
+            wait_target.store(~u64{0}, std::memory_order_relaxed);
+            return;
+        }
+        if (spins < 16384) {
+            Pause();
+        } else {
+            std::this_thread::yield();
+        }
+    }
+}
+
 void GpuSplit::SwitchEngine(Tegra::Engines::EngineInterface* engine) {
     front_engine = engine;
     u64* const entry = Reserve(2);
@@ -346,8 +369,11 @@ void GpuSplit::BackLoop(std::stop_token stop_token) {
             const size_t index = read_position & RingMask;
             const size_t size = Execute(&ring[index]);
             read_position += size != 0 ? size : RingWords - index;
-            // Hand ring space back now and then while working through a long batch.
-            if (read_position - last_store >= 4096 && read_position != available) {
+            // Hand ring space back now and then while working through a long batch, and at once
+            // when the front waits for this position (WaitForWriters).
+            if ((read_position - last_store >= 4096 ||
+                 read_position >= wait_target.load(std::memory_order_relaxed)) &&
+                read_position != available) {
                 consumed.store(read_position, std::memory_order_release);
                 last_store = read_position;
             }