summaryrefslogtreecommitdiff
path: root/AK/GenericLexer.h
AgeCommit message (Collapse)Author
2021-08-19AK: Add GenericLexer API to consume an escaped Unicode code pointTimothy Flynn
This parsing is already duplicated between LibJS and LibRegex, and will shortly be needed in more places in those libraries. Move it to AK to prevent further duplication. This API will consume escaped Unicode code points of the form: \\u{code point} \\unnnn (where each n is a hexadecimal digit) \\unnnn\\unnnn (where the two escaped values are a surrogate pair)
2021-07-12AK: Add a retreat(count) method to GenericLexerIdan Horowitz
This method can be used to rewind a constant amount backwards in the source instead of one by one with retract()
2021-04-22AK/GenericLexer: constexpr where possibleLenny Maiorani
Problem: - Much of the `GenericLexer` can be `constexpr`, but is not. Solution: - Make it `constexpr` and de-duplicate code. - Extend some of `StringView` with `constexpr` to support. - Add tests to ensure `constexpr` behavior. Note: - Construction of `StringView` from pointer and length is not `constexpr`-compatible at the moment because the VERIFY cannot be, yet.
2021-04-22Everything: Move to SPDX license identifiers in all files.Brian Gianforcaro
SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2020-10-29AK: Add GenericLexer::retreat()Linus Groh
This allows going back one character at a time, and then re-consume previously consumed chars. The code I need this for looks something like this: ASSERT(lexer.consume_specific('\\')); if (lexer.next_is("foo")) ... lexer.retreat(); lexer.consume_escaped_character(); // This expects lexer.peek() == '\\'
2020-10-22AK: Add `GenericLexer::consume_escaped_character()'AnotherTest
...and use it in `consume_and_unescape_string()'.
2020-10-02AK+Format: Do some housekeeping in the format implementation.asynts
2020-09-27AK: Remove the ctype adapters and use the actual ctype functions insteadBenoît Lormeau
This finally takes care of the kind-of excessive boilerplate code that were the ctype adapters. On the other hand, I had to link `LibC/ctype.cpp` to the Kernel (for `AK/JsonParser.cpp` and `AK/Format.cpp`). The previous commit actually makes sense now: the `string.h` includes in `ctype.{h,cpp}` would require to link more LibC stuff to the Kernel when it only needs the `_ctype_` array of `ctype.cpp`, and there wasn't any string stuff used in ctype. Instead of all this I could have put static derivatives of `is_any_of()` in the concerned AK files, however that would have meant more boilerplate and workarounds; so I went for the Kernel approach.
2020-09-27AK: Use templates instead of Function for Conditions in the GenericLexerBenoit Lormeau
Since commit 1ec59f28cea56f1afced0994127e2df62300e618 turns the ctype macros into functions we can now feed them directly to a GenericLexer! This will lead to removing the ctype adapters that were kind-of excessive boilerplate, but needed as the Kernel doesn't compile with the LibC.
2020-09-26AK: Enhance GenericLexer's string consumptionBenoit Lormeau
The `consume_quoted_string()` can now take an escape character. This allows it (for example) to capture a string's enclosing quotes. The escape character is optional by default. You can also consume and unescape a quoted string with the eponymous method `consume_and_unescape_string()`. It takes an escape character as parameter (backslash by default). It builds a String in which common escape sequences get... unescaped :^) (e.g. \n, \r, \t...).
2020-09-26AK: Added explanatory comments in GenericLexer.hBenoit Lormeau
2020-09-26AK: Add GenericLexer::remaining.asynts
This is useful for debugging with printf :^).
2020-08-31AK: Add is_any_of(StringView) to GenericLexerAnotherTest
2020-08-21LibWeb: Use GenericLexer in WrapperGeneratorNico Weber
2020-08-09AK: Add a GenericLexer and extend the JsonParser with it (#2696)Benoît Lormeau