Kernel+UE: Add MAP_FIXED_NOREPLACE mmap() flag

This feature was introduced in version 4.17 of the Linux kernel, and
while it's not specified by POSIX, I think it will be a nice addition to
our system.

MAP_FIXED_NOREPLACE provides a less error-prone alternative to
MAP_FIXED: while regular fixed mappings would cause any intersecting
ranges to be unmapped, MAP_FIXED_NOREPLACE returns EEXIST instead. This
ensures that we don't corrupt our process's address space if something
is already at the requested address.

Note that the more portable way to do this is to use regular
MAP_ANONYMOUS, and check afterwards whether the returned address matches
what we wanted. This, however, has a large performance impact on
programs like Wine which try to reserve large portions of the address
space at once, as the non-matching addresses have to be unmapped
separately.

2 files changed, 5 insertions, 4 deletions

diff --git a/Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp b/Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp
index 7a46c74e78..07ca7dade8 100644
--- a/Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp
+++ b/Userland/DevTools/UserspaceEmulator/Emulator_syscalls.cpp
@@ -873,10 +873,11 @@ u32 Emulator::virt$mmap(u32 params_addr)
     Optional<Range> result;
     if (params.flags & MAP_RANDOMIZED) {
         result = m_range_allocator.allocate_randomized(requested_size, params.alignment);
-    } else if (params.flags & MAP_FIXED) {
+    } else if (params.flags & MAP_FIXED || params.flags & MAP_FIXED_NOREPLACE) {
         if (params.addr) {
             // If MAP_FIXED is specified, existing mappings that intersect the requested range are removed.
-            virt$munmap(params.addr, requested_size);
+            if (params.flags & MAP_FIXED)
+                virt$munmap(params.addr, requested_size);
             result = m_range_allocator.allocate_specific(VirtualAddress { params.addr }, requested_size);
         } else {
             // mmap(nullptr, …, MAP_FIXED) is technically okay, but tends to be a bug.
diff --git a/Userland/DevTools/UserspaceEmulator/MmapRegion.cpp b/Userland/DevTools/UserspaceEmulator/MmapRegion.cpp
index 9ddd3cfda3..4351e0763f 100644
--- a/Userland/DevTools/UserspaceEmulator/MmapRegion.cpp
+++ b/Userland/DevTools/UserspaceEmulator/MmapRegion.cpp
@@ -36,8 +36,8 @@ NonnullOwnPtr<MmapRegion> MmapRegion::create_anonymous(u32 base, u32 size, u32 p
 
 NonnullOwnPtr<MmapRegion> MmapRegion::create_file_backed(u32 base, u32 size, u32 prot, int flags, int fd, off_t offset, String name)
 {
-    // Since we put the memory to an arbitrary location, do not pass MAP_FIXED to the Kernel.
-    auto real_flags = flags & ~MAP_FIXED;
+    // Since we put the memory to an arbitrary location, do not pass MAP_FIXED and MAP_FIXED_NOREPLACE to the Kernel.
+    auto real_flags = flags & ~(MAP_FIXED | MAP_FIXED_NOREPLACE);
     auto* data = (u8*)mmap_with_name(nullptr, size, prot, real_flags, fd, offset, name.is_empty() ? nullptr : String::formatted("(UE) {}", name).characters());
     VERIFY(data != MAP_FAILED);
     auto* shadow_data = (u8*)mmap_initialized(size, 1, "MmapRegion ShadowData");