[compat] Fix other DragonFlyBSD/NixOS issues (#2860)

Uh - the other pr that had "Fix dragonfly" got commits lost and I didn't notice... oops; cherry picked them back through :)
Signed-off-by: lizzie <lizzie@eden-emu.dev>

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/2860
Reviewed-by: crueter <crueter@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
Co-authored-by: lizzie <lizzie@eden-emu.dev>
Co-committed-by: lizzie <lizzie@eden-emu.dev>

11 files changed, 100 insertions, 47 deletions

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 4181c8d7b7..b9012d70b4 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -13,6 +13,8 @@ elseif (${CMAKE_SYSTEM_NAME} STREQUAL "OpenBSD")
     set(PLATFORM_OPENBSD ON)
 elseif (${CMAKE_SYSTEM_NAME} STREQUAL "NetBSD")
     set(PLATFORM_NETBSD ON)
+elseif (${CMAKE_SYSTEM_NAME} STREQUAL "DragonFly")
+    set(PLATFORM_DRAGONFLYBSD ON)
 elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Haiku")
     set(PLATFORM_HAIKU ON)
 elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
@@ -209,9 +211,8 @@ if (MSVC AND NOT CXX_CLANG)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /W3 /WX-")
 endif()
 
-if (PLATFORM_FREEBSD)
+if (PLATFORM_FREEBSD OR PLATFORM_DRAGONFLYBSD)
     set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -L${CMAKE_SYSROOT}/usr/local/lib")
-
 endif()
 
 # Set bundled sdl2/qt as dependent options.
diff --git a/docs/Caveats.md b/docs/Caveats.md
index 1ddf08b24c..ac847d052d 100644
--- a/docs/Caveats.md
+++ b/docs/Caveats.md
@@ -30,6 +30,13 @@ macOS is largely untested. Expect crashes, significant Vulkan issues, and other
 
 ## Solaris
 
+Always consult [the OpenIndiana package list](https://pkg.openindiana.org/hipster/en/index.shtml) to cross-verify availability.
+
+Run the usual update + install of essential toolings: `sudo pkg update && sudo pkg install git cmake`.
+
+- **gcc**: `sudo pkg install developer/gcc-14`.
+- **clang**: Version 20 is broken, use `sudo pkg install developer/clang-19`.
+
 Qt Widgets appears to be broken. For now, add `-DENABLE_QT=OFF` to your configure command. In the meantime, a Qt Quick frontend is in the works--check back later!
 
 This is needed for some dependencies that call cc directly (tz):
@@ -74,7 +81,7 @@ Still will not run flawlessly until `mesa-24` is available. Modify CMakeCache.tx
 
 After configuration, you may need to modify `externals/ffmpeg/CMakeFiles/ffmpeg-build/build.make` to use `-j$(nproc)` instead of just `-j`.
 
-`-lc++-experimental` doesn't exist in OpenBSD but the LLVM driver still tries to link against it, to solve just symlink `ln -s /usr/lib/libc++.a /usr/lib/libc++experimental.a`.
+`-lc++-experimental` doesn't exist in OpenBSD but the LLVM driver still tries to link against it, to solve just symlink `ln -s /usr/lib/libc++.a /usr/lib/libc++experimental.a`. Builds are currently not working due to lack of `std::jthread` and such, either compile libc++ manually or wait for ports to catch up.
 
 If clang has errors, try using `g++-11`.
 
@@ -107,6 +114,21 @@ cmake --build build -- -j`nproc`
 cmake --install build
 ```
 
+# DragonFlyBSD
+
+If `libstdc++.so.6` is not found (`GLIBCXX_3.4.30`) then attempt:
+```sh
+rm /usr/local/lib/gcc11/libstdc++.so.6
+ln -s /usr/local/lib/gcc14/libstdc++.so /usr/local/lib/gcc11/libstdc++.so.6
+```
+This may have unforeseen consequences of which we don't need to worry about for now.
+
+Default `g++` (and the libstdc++) is too outdated - so install `gcc14` and redirect CMake to the new compiler toolchain  `-DCMAKE_CXX_COMPILER=gcc14 -DCMAKE_C_COMPILER=g++14`.
+
+There is also `llvm18` and use `-DCMAKE_CXX_COMPILER=clang++18 -DCMAKE_C_COMPILER=clang18` (note the `18` suffix at the end). NOTE: It doesn't have an updated libcxx so `<span>` will be missing, either build it manually or use gcc.
+
+If build hangs, use `hammer2 bulkfree`.
+
 ## MSYS2
 
 `qt6-static` isn't supported yet.
@@ -169,3 +191,9 @@ find ./*/ -name "*.dll" | while read -r dll; do deps "$dll"; done
 DirectX 12 is not available - simply copy and paste a random DLL and name it `d3d12.dll`.
 
 Install [Qt6 compatibility libraries](github.com/ANightly/qt6windows7) specifically Qt 6.9.5.
+
+## RedoxOS
+
+The package install may randomly hang at times, in which case it has to be restarted. ALWAYS do a `sudo pkg update` or the chances of it hanging will be close to 90%. If "multiple" installs fail at once, try installing 1 by 1 the packages.
+
+When CMake invokes certain file syscalls - it may sometimes cause crashes or corruptions on the (kernel?) address space - so reboot the system if there is a "hang" in CMake.
diff --git a/docs/Coding.md b/docs/Coding.md
index ec524cbbf8..8d02df4088 100644
--- a/docs/Coding.md
+++ b/docs/Coding.md
@@ -8,6 +8,8 @@ But for new developers you may find that following these guidelines will make ev
 
 Simply put, types/classes are named as `PascalCase`, same for methods and functions like `AddElement`. Variables are named `like_this_snake_case` and constants are `IN_SCREAMING_CASE`.
 
+Except for Qt MOC where `functionName` is preferred.
+
 Template typenames prefer short names like `T`, `I`, `U`, if a longer name is required either `Iterator` or `perform_action` are fine as well.
 
 Macros must always be in `SCREAMING_CASE`. Do not use short letter macros as systems like Solaris will conflict with them; a good rule of thumb is >5 characters per macro - i.e `THIS_MACRO_IS_GOOD`, `AND_ALSO_THIS_ONE`.
diff --git a/docs/Deps.md b/docs/Deps.md
index f66bf05e90..15ffff28a8 100644
--- a/docs/Deps.md
+++ b/docs/Deps.md
@@ -196,7 +196,6 @@ apk add g++ git cmake make mbedtls-dev mbedtls-static mesa-dev qt6-qtbase-dev qt
 `mbedtls-static` has to be specified otherwise `libeverest.a` and `libp256m.a` will fail to be found.
 
 </details>
-
 <details>
 <summary>Void Linux</summary>
 
@@ -236,7 +235,7 @@ brew install molten-vk vulkan-loader
 <details>
 <summary>FreeBSD</summary>
 
-As root run: `pkg install devel/cmake devel/sdl20 devel/boost-libs devel/catch2 devel/libfmt devel/nlohmann-json devel/ninja devel/nasm devel/autoconf devel/pkgconf devel/qt6-base devel/simpleini net/enet multimedia/ffnvcodec-headers multimedia/ffmpeg audio/opus archivers/liblz4 lang/gcc12 graphics/glslang graphics/vulkan-utility-libraries graphics/spirv-tools www/cpp-httplib devel/jwt-cpp devel/unordered-dense`
+As root run: `pkg install devel/cmake devel/sdl20 devel/boost-libs devel/catch2 devel/libfmt devel/nlohmann-json devel/ninja devel/nasm devel/autoconf devel/pkgconf devel/qt6-base devel/simpleini net/enet multimedia/ffnvcodec-headers multimedia/ffmpeg audio/opus archivers/liblz4 lang/gcc12 graphics/glslang graphics/vulkan-utility-libraries graphics/spirv-tools www/cpp-httplib devel/jwt-cpp devel/unordered-dense mbedtls3 vulkan-headers quazip-qt6`
 
 If using FreeBSD 12 or prior, use `devel/pkg-config` instead.
 
@@ -263,16 +262,21 @@ pkg_add cmake nasm git boost unzip--iconv autoconf-2.72p0 bash ffmpeg glslang gm
 
 </details>
 <details>
-<summary>Solaris / OpenIndiana</summary>
+<summary>DragonFlyBSD</summary>
 
-Always consult [the OpenIndiana package list](https://pkg.openindiana.org/hipster/en/index.shtml) to cross-verify availability.
+```sh
+pkg install gcc14 git cmake unzip nasm autoconf bash pkgconf ffmpeg glslang gmake jq nlohmann-json enet spirv-tools sdl2 vulkan-utility-libraries vulkan-headers catch2 libfmt openssl liblz4 boost-libs cpp-httplib qt6-base quazip-qt6 unordered-dense libva-vdpau-driver libva-utils libva-intel-driver
+```
 
-Run the usual update + install of essential toolings: `sudo pkg update && sudo pkg install git cmake`.
+[Caveats](./Caveats.md#dragonflybsd).
 
-- **gcc**: `sudo pkg install developer/gcc-14`.
-- **clang**: Version 20 is broken, use `sudo pkg install developer/clang-19`.
+</details>
+<details>
+<summary>Solaris / OpenIndiana</summary>
 
-Then install the libraries: `sudo pkg install qt6 boost glslang libzip library/lz4 libusb-1 nlohmann-json openssl opus sdl2 zlib compress/zstd unzip pkg-config nasm autoconf mesa library/libdrm header-drm developer/fmt`.
+```sh
+sudo pkg install qt6 boost glslang libzip library/lz4 libusb-1 nlohmann-json openssl opus sdl2 zlib compress/zstd unzip pkg-config nasm autoconf mesa library/libdrm header-drm developer/fmt
+```
 
 [Caveats](./Caveats.md#solaris).
 
@@ -316,16 +320,16 @@ pkgman install git cmake patch libfmt_devel nlohmann_json lz4_devel opus_devel b
 [Caveats](./Caveats.md#haikuos).
 
 </details>
-
 <details>
 <summary>RedoxOS</summary>
 
-TODO: Fix syscall crashes (heavy IO stalls and hangup due to net mutexes?)
 ```sh
-sudo pkg update && sudo pkg install git cmake
-sudo pkg install ffmpeg6 sdl2 zlib llvm18
+sudo pkg update
+sudo pkg install git cmake ffmpeg6 sdl2 zlib llvm18
 ```
 
+[Caveats](./Caveats.md#redoxos).
+
 </details>
 
 ## All Done
diff --git a/shell.nix b/shell.nix
index 549c91959d..3c91286738 100644
--- a/shell.nix
+++ b/shell.nix
@@ -9,11 +9,15 @@ pkgs.mkShellNoCC {
     # libraries
     openssl boost fmt nlohmann_json lz4 zlib zstd
     enet libopus vulkan-headers vulkan-utility-libraries
-    spirv-tools spirv-headers simpleini vulkan-memory-allocator
-    vulkan-loader unzip mbedtls glslang python3 httplib
-    cpp-jwt ffmpeg-headless libusb1 cubeb
-    qt6.full # eden
-    SDL2 # eden-cli
-    discord-rpc gamemode # optional components
+    spirv-tools spirv-headers vulkan-loader unzip mbedtls
+    glslang python3 httplib cpp-jwt ffmpeg-headless
+    libusb1 cubeb
+    # eden
+    qt6.qtbase qt6.qtmultimedia qt6.qtwayland qt6.qttools
+    qt6.qtwebengine qt6.qt5compat
+    # eden-cli
+    SDL2
+    # optional components
+    discord-rpc gamemode
   ];
 }
\ No newline at end of file
diff --git a/src/common/host_memory.cpp b/src/common/host_memory.cpp
index 6e47fa286d..5400b97018 100644
--- a/src/common/host_memory.cpp
+++ b/src/common/host_memory.cpp
@@ -521,14 +521,29 @@ public:
 #else
         fd = memfd_create("HostMemory", 0);
 #endif
-        ASSERT_MSG(fd >= 0, "memfd_create failed: {}", strerror(errno));
-
-        // Defined to extend the file with zeros
-        int ret = ftruncate(fd, backing_size);
-        ASSERT_MSG(ret == 0, "ftruncate failed with {}, are you out-of-memory?", strerror(errno));
-
-        backing_base = static_cast<u8*>(
-            mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0));
+        bool use_anon = false;
+        if (fd <= 0) {
+            LOG_WARNING(Common_Memory, "memfd_create: {}", strerror(errno));
+            use_anon = true;
+        }
+        if (!use_anon) {
+            // Defined to extend the file with zeros
+            int ret = ftruncate(fd, backing_size);
+            if (ret != 0) {
+                LOG_WARNING(Common_Memory, "ftruncate: {} (likely out-of-emory)", strerror(errno));
+                use_anon = true;
+            }
+        }
+        if (use_anon) {
+            LOG_WARNING(Common_Memory, "Using private mappings instead of shared ones");
+            backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0));
+            if (fd > 0) {
+                fd = -1;
+                close(fd);
+            }
+        } else {
+            backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0));
+        }
         ASSERT_MSG(backing_base != MAP_FAILED, "mmap failed: {}", strerror(errno));
 
         // Virtual memory initialization
@@ -552,22 +567,18 @@ public:
         free_manager.AllocateBlock(virtual_base + virtual_offset, length);
 
         // Deduce mapping protection flags.
-        int flags = PROT_NONE;
-        if (True(perms & MemoryPermission::Read)) {
-            flags |= PROT_READ;
-        }
-        if (True(perms & MemoryPermission::Write)) {
-            flags |= PROT_WRITE;
-        }
+        int prot_flags = PROT_NONE;
+        if (True(perms & MemoryPermission::Read))
+            prot_flags |= PROT_READ;
+        if (True(perms & MemoryPermission::Write))
+            prot_flags |= PROT_WRITE;
 #ifdef ARCHITECTURE_arm64
-        if (True(perms & MemoryPermission::Execute)) {
-            flags |= PROT_EXEC;
-        }
+        if (True(perms & MemoryPermission::Execute))
+            prot_flags |= PROT_EXEC;
 #endif
-
-        void* ret = mmap(virtual_base + virtual_offset, length, flags, MAP_SHARED | MAP_FIXED, fd,
-                         host_offset);
-        ASSERT_MSG(ret != MAP_FAILED, "mmap failed: {}", strerror(errno));
+        int flags = (fd > 0 ? MAP_SHARED : MAP_PRIVATE) | MAP_FIXED;
+        void* ret = mmap(virtual_base + virtual_offset, length, prot_flags, flags, fd, host_offset);
+        ASSERT_MSG(ret != MAP_FAILED, "mmap: {}", strerror(errno));
     }
 
     void Unmap(size_t virtual_offset, size_t length) {
diff --git a/src/common/settings.cpp b/src/common/settings.cpp
index 0022d242c3..6800c20f69 100644
--- a/src/common/settings.cpp
+++ b/src/common/settings.cpp
@@ -169,7 +169,7 @@ bool IsFastmemEnabled() {
     if (values.cpu_accuracy.GetValue() == CpuAccuracy::Unsafe) {
         return bool(values.cpuopt_unsafe_host_mmu);
     }
-#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) || defined(__HAIKU__)
+#if !defined(__APPLE__) && !defined(__linux__) && !defined(__ANDROID__) && !defined(_WIN32)
     return false;
 #else
     return true;
diff --git a/src/common/settings.h b/src/common/settings.h
index 2aef8ba85a..360a49c6c6 100644
--- a/src/common/settings.h
+++ b/src/common/settings.h
@@ -296,7 +296,7 @@ struct Values {
                                               Category::CpuDebug};
 
     SwitchableSetting<bool> cpuopt_unsafe_host_mmu{linkage,
-#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) || defined(__HAIKU__)
+#if !defined(__APPLE__) && !defined(__linux__) && !defined(__ANDROID__) && !defined(_WIN32)
                                         false,
 #else
                                         true,
diff --git a/src/core/arm/dynarmic/arm_dynarmic_32.cpp b/src/core/arm/dynarmic/arm_dynarmic_32.cpp
index 5fcc928b86..eca2bb33bb 100644
--- a/src/core/arm/dynarmic/arm_dynarmic_32.cpp
+++ b/src/core/arm/dynarmic/arm_dynarmic_32.cpp
@@ -292,7 +292,7 @@ std::shared_ptr<Dynarmic::A32::Jit> ArmDynarmic32::MakeJit(Common::PageTable* pa
     // Curated optimizations
     case Settings::CpuAccuracy::Auto:
         config.unsafe_optimizations = true;
-#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) || defined(__HAIKU__)
+#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) || defined(__HAIKU__) || defined(__DragonFly__)
         config.fastmem_pointer = std::nullopt;
         config.fastmem_exclusive_access = false;
 #endif
diff --git a/src/core/arm/dynarmic/arm_dynarmic_64.cpp b/src/core/arm/dynarmic/arm_dynarmic_64.cpp
index 55975f4cdc..a87130f6dc 100644
--- a/src/core/arm/dynarmic/arm_dynarmic_64.cpp
+++ b/src/core/arm/dynarmic/arm_dynarmic_64.cpp
@@ -351,7 +351,7 @@ std::shared_ptr<Dynarmic::A64::Jit> ArmDynarmic64::MakeJit(Common::PageTable* pa
     // Safe optimisations
     case Settings::CpuAccuracy::Auto:
         config.unsafe_optimizations = true;
-#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) || defined(__HAIKU__)
+#if !defined(__APPLE__) && !defined(__linux__) && !defined(__ANDROID__) && !defined(_WIN32)
         config.fastmem_pointer = std::nullopt;
         config.fastmem_exclusive_access = false;
 #endif
diff --git a/src/dynarmic/src/dynarmic/common/context.h b/src/dynarmic/src/dynarmic/common/context.h
index ea2c1ef251..941289cb94 100644
--- a/src/dynarmic/src/dynarmic/common/context.h
+++ b/src/dynarmic/src/dynarmic/common/context.h
@@ -72,6 +72,9 @@
 #    elif defined(__sun__)
 #        define CTX_RIP (mctx.gregs[REG_RIP])
 #        define CTX_RSP (mctx.gregs[REG_RSP])
+#    elif defined(__DragonFly__)
+#        define CTX_RIP (mctx.mc_rip)
+#        define CTX_RSP (mctx.mc_rsp)
 #    else
 #        error "Unknown platform"
 #    endif