UserspaceEmulator: Handle MAP_FIXED with invalid addresses better

Old behavior: Crash due to VERIFY, unless we're completely and entirely out of
memory (m_available_ranges being empty), in which case it would return -ENOMEM.

New behavior: Return ENOMEM (and don't crash). In the case of nullptr,
also emit a helpful diagnostic.

Note that MAP_FIXED with nullptr is technically okay, but tends to be a sign
that something went wrong.

Also, this should improve mmap performance marginally, as it pulls the check out
of a loop that does not modify any parts of the check.

UE is now self-hosting! Fixes #5709.

However, this still needs some love: "ue UserspaceEmulator true" spits out tons
of error messages, probably false-positives, and takes about 229 seconds to run.
Then again, true-in-ue-in-ue-in-Qemu is three levels of emulation, so no wonder
it takes a long time! :D

1 files changed, 4 insertions, 1 deletions

diff --git a/Userland/DevTools/UserspaceEmulator/RangeAllocator.cpp b/Userland/DevTools/UserspaceEmulator/RangeAllocator.cpp
index 7dfc8c6e49..60ba47e233 100644
--- a/Userland/DevTools/UserspaceEmulator/RangeAllocator.cpp
+++ b/Userland/DevTools/UserspaceEmulator/RangeAllocator.cpp
@@ -149,9 +149,12 @@ Optional<Range> RangeAllocator::allocate_specific(VirtualAddress base, size_t si
     VERIFY((size % PAGE_SIZE) == 0);
 
     Range allocated_range(base, size);
+    if (!m_total_range.contains(allocated_range)) {
+        dbgln("Unallocatable mmap request?! {:p}+{:p}", base.get(), size);
+        return {};
+    }
     for (size_t i = 0; i < m_available_ranges.size(); ++i) {
         auto& available_range = m_available_ranges[i];
-        VERIFY(m_total_range.contains(allocated_range));
         if (!available_range.contains(base, size))
             continue;
         if (available_range == allocated_range) {