LibJS: Fix wraparound UB in `Value::to_u{8,16}`

If we call these two functions on a negative value, undefined behavior
occurs due to casting a negative double to an unsigned integer. These
functions are defined to perform modular arithmetic, so negative values
can be fixed up by adding 2^8/2^16.

The reason why this step is not mentioned in ECMA-262 is that it defines
modular arithmetic so that `x mod m` had the same sign as `m`, while
LibM's `fmod(x, m)` copies `x`'s sign.

This issue was found by UBSAN with the Clang toolchain.

1 files changed, 4 insertions, 0 deletions

diff --git a/Userland/Libraries/LibJS/Runtime/Value.cpp b/Userland/Libraries/LibJS/Runtime/Value.cpp
index 76c6d4cc41..f2336f51cb 100644
--- a/Userland/Libraries/LibJS/Runtime/Value.cpp
+++ b/Userland/Libraries/LibJS/Runtime/Value.cpp
@@ -684,6 +684,8 @@ u16 Value::to_u16(GlobalObject& global_object) const
     if (signbit(value))
         int_val = -int_val;
     auto int16bit = fmod(int_val, NumericLimits<u16>::max() + 1.0);
+    if (int16bit < 0)
+        int16bit += NumericLimits<u16>::max() + 1.0;
     return static_cast<u16>(int16bit);
 }
 
@@ -720,6 +722,8 @@ u8 Value::to_u8(GlobalObject& global_object) const
     if (signbit(value))
         int_val = -int_val;
     auto int8bit = fmod(int_val, NumericLimits<u8>::max() + 1.0);
+    if (int8bit < 0)
+        int8bit += NumericLimits<u8>::max() + 1.0;
     return static_cast<u8>(int8bit);
 }