diff options
author | Linus Groh <mail@linusgroh.de> | 2021-11-16 09:17:55 +0000 |
---|---|---|
committer | Linus Groh <mail@linusgroh.de> | 2021-11-16 10:06:53 +0000 |
commit | 58c6a156bf3e6dcf0e37a3a73ba6dd4365ce1203 (patch) | |
tree | 25a7ee6d30da8c3a5a5f37860d1cb054ab7deafd /Userland/Libraries/LibCrypto | |
parent | 014840eecae67a89b220ec1e33efbb01308d49d3 (diff) | |
download | serenity-58c6a156bf3e6dcf0e37a3a73ba6dd4365ce1203.zip |
LibCrypto: Fix subtracting two negative `SignedBigInteger`s
Currently, we get the following results
-1 - -2 = -1
-2 - -1 = 1
Correct would be:
-1 - -2 = 1
-2 - -1 = -1
This was already attempted to be fixed in 7ed8970, but that change was
incorrect. This directly translates to LibJS BigInts having the same
incorrect behavior - it even was tested.
Diffstat (limited to 'Userland/Libraries/LibCrypto')
-rw-r--r-- | Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp | 10 |
1 files changed, 7 insertions, 3 deletions
diff --git a/Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp b/Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp index 1ff34050a6..5299e7dd5e 100644 --- a/Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp +++ b/Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp @@ -109,11 +109,15 @@ FLATTEN SignedBigInteger SignedBigInteger::minus(const SignedBigInteger& other) // -x - -y = y - x if (m_unsigned_data < other.m_unsigned_data) { // The result will be positive. - return SignedBigInteger { other.m_unsigned_data.minus(m_unsigned_data), true }; + return SignedBigInteger { other.m_unsigned_data.minus(m_unsigned_data) }; } - // The result will be either zero, or negative. // y - x = - (x - y) - return SignedBigInteger { m_unsigned_data.minus(other.m_unsigned_data) }; + if (m_unsigned_data > other.m_unsigned_data) { + // The result will be negative. + return SignedBigInteger { m_unsigned_data.minus(other.m_unsigned_data), true }; + } + // Both operands have the same magnitude, the result is positive zero. + return SignedBigInteger { 0 }; } FLATTEN SignedBigInteger SignedBigInteger::plus(const UnsignedBigInteger& other) const |