nx2: read command lists the GPU wrote from the GPU's copy

Mario Kart World runs small command lists (20-byte entries) that its
culling dispatches fill in each frame. Accurate GPU mode reads every
command list through the caches; Fast mode reads guest memory directly
(on the split's front thread too), where those entries were still the
words of an older frame, so the draws behind them were left out: the
trees around the courses vanished in Fast mode only.

The buffer cache now also marks the device pages the GPU writes (shader
stores, copies of GPU-written data) in a bitmap the split's front can
read. A command list on such a page waits for the back to run what was
queued before it and is flushed when its words are still only in host
buffers. In a race that is ~5% of command lists (13,500 a second of
262,000) and ~400 flushes a second.

1 files changed, 7 insertions, 0 deletions

diff --git a/src/video_core/buffer_cache/buffer_cache.h b/src/video_core/buffer_cache/buffer_cache.h
index 42d7ba066d..a8f1edc68d 100644
--- a/src/video_core/buffer_cache/buffer_cache.h
+++ b/src/video_core/buffer_cache/buffer_cache.h
@@ -17,6 +17,7 @@
 #include "video_core/buffer_cache/buffer_cache_base.h"
 #include "video_core/guest_memory.h"
 #include "video_core/host1x/gpu_device_memory_manager.h"
+#include "video_core/nx2.h"
 #include "video_core/nx2_profile.h"
 #include "video_core/texture_cache/util.h"
 
@@ -289,6 +290,9 @@ bool BufferCache<P>::DMACopy(GPUVAddr src_address, GPUVAddr dest_address, u64 am
     const bool has_new_downloads = tmp_intervals.size() != 0;
     for (const auto& pair : tmp_intervals) {
         gpu_modified_ranges.Add(pair.first, pair.second);
+        if (Tegra::g_nx2_ampere.load(std::memory_order_relaxed)) {
+            Tegra::Nx2MarkGpuWritten(pair.first, pair.second);
+        }
     }
     const auto& copy = copies[0];
     src_buffer.MarkUsage(copy.src_offset, copy.size);
@@ -1606,6 +1610,9 @@ void BufferCache<P>::MarkWrittenBuffer(BufferId buffer_id, DAddr device_addr, u3
     if (AsyncBufferDownloads()) {
         uncommitted_gpu_modified_ranges.Add(device_addr, size);
     }
+    if (Tegra::g_nx2_ampere.load(std::memory_order_relaxed)) {
+        Tegra::Nx2MarkGpuWritten(device_addr, size);
+    }
 }
 
 template <class P>