texture_cache: give images over unwritten sparse blocks no device memory

Mario Kart World creates nine terrain layer pools in sparse regions (four
R8 16384x16384, four RG16 8192x8192, one R16 8192x8192: 2.9 GB of device
memory). In 180 s of title and attract demo not one of their blocks is
written and nothing renders into them, so they hold zeros; on the console
only the few tiles the game maps have memory.

nx2:gpu now publishes which 4 MiB blocks of each sparse region have been
written (nx2.h, a leaf lock the texture cache may take). An image of 32
MiB or more lying in one sparse region with no written block is created
as a sparse resident image with nothing bound: reads return zero, as an
unmapped tile does. The first write into its range, or the first time the
GPU renders into it (that range then counts as written), marks it
remapped, so a backed image replaces it at its next use.

Mario Kart World, 180 s: the collector deletes 0-80 images per 600 frames
instead of thousands, frames over 100 ms per 600 presents drop from 1-9 to
0-1, peak working set 14.7 GB. One render target in a small sparse region
gets replaced once (logged).

2 files changed, 17 insertions, 0 deletions

diff --git a/src/video_core/texture_cache/image_base.h b/src/video_core/texture_cache/image_base.h
index 20d3883130..cfb839d17e 100644
--- a/src/video_core/texture_cache/image_base.h
+++ b/src/video_core/texture_cache/image_base.h
@@ -50,6 +50,9 @@ enum class ImageFlagBits : u32 {
     /// in VRAM); uploads skip the rest, downloads are never done.
     TrimmedLevels = 1 << 19,
     ZeroFilled = 1 << 20, ///< Cleared to zero blocks before its first asynchronous decode.
+    /// Over Switch 2 sparse blocks nobody wrote: all zeros, so the host image has no memory
+    /// (Unbacked) until the first write, which makes a backed image replace it.
+    UnwrittenSparse = 1 << 21,
 };
 DECLARE_ENUM_FLAG_OPERATORS(ImageFlagBits)
 
diff --git a/src/video_core/texture_cache/texture_cache.h b/src/video_core/texture_cache/texture_cache.h
index d582f520f7..168df50c8b 100644
--- a/src/video_core/texture_cache/texture_cache.h
+++ b/src/video_core/texture_cache/texture_cache.h
@@ -28,6 +28,7 @@
 #include "video_core/texture_cache/samples_helper.h"
 #include "video_core/texture_cache/texture_cache_base.h"
 #include "video_core/texture_cache/util.h"
+#include "video_core/nx2.h"
 #include "video_core/nx2_profile.h"
 #include "video_core/textures/decoders.h"
 
@@ -717,6 +718,11 @@ FramebufferId TextureCache<P>::GetFramebufferId(const RenderTargets& key) {
 template <class P>
 void TextureCache<P>::WriteMemory(DAddr cpu_addr, size_t size) {
     ForEachImageInRegion(cpu_addr, size, [this, cpu_addr, size](ImageId image_id, Image& image) {
+        if (True(image.flags & ImageFlagBits::UnwrittenSparse)) {
+            // Written at last: a backed image replaces it where it is used next.
+            image.flags |= ImageFlagBits::CpuModified | ImageFlagBits::Remapped;
+            return;
+        }
         if (UploadsByLayer(image)) {
             MarkLayersWritten(image, cpu_addr, size);
             return;
@@ -2918,6 +2924,14 @@ void TextureCache<P>::PrepareImage(ImageId image_id, bool is_modification, bool
         }
     }
     if (is_modification) {
+        if (True(image.flags & ImageFlagBits::UnwrittenSparse) &&
+            False(image.flags & ImageFlagBits::Remapped)) {
+            // The GPU renders into it: these writes are lost, later uses get a backed image.
+            LOG_WARNING(HW_GPU, "GPU write into unwritten sparse image {:#x}; replacing it",
+                        image.gpu_addr);
+            Tegra::Nx2SparseMarkWritten(image.gpu_addr, image.guest_size_bytes);
+            image.flags |= ImageFlagBits::CpuModified | ImageFlagBits::Remapped;
+        }
         MarkModification(image);
     }
     lru_cache.Touch(image.lru_index, frame_tick);