| Age | Commit message (Collapse) | Author |
|---|
|
|
|
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
|
|
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
|
|
Previously the s_decomposition_mappings variable would refer to other
data in s_decomposition_mappings_data. This would cause thousands of
avoidable relocations at load time.
This saves about 128kB RAM for each process which uses LibUnicode.
|
|
Otherwise, we end up propagating those dependencies into targets that
link against that library, which creates unnecessary link-time
dependencies.
Also included are changes to readd now missing dependencies to tools
that actually need them.
|
|
We have logic for serenity_generated_sources which works well for source
files that are specified in GENERATED_SOURCES prior to calling
serenity_lib or serenity_bin. However, code generated with
invoke_generator, and the LibWeb generators do not always follow the
pattern of the IDL and GML files.
For the LibWeb generators, we can just add_dependencies to LibWeb at the
time we declare the generate_Foo custom target. However for LibLocale,
LibTimeZone, and LibUnicode, we don't have the name of the target
available, so export the name in a variable to set into
GENERATED_SOURCES.
To make this work for Lagom, we need to make sure that lagom_lib and
serenity_bin in Lagom/CMakeLists.txt call serenity_generated_sources on
the target.
This enables the Xcode generator on macOS hosts, at least for Lagom.
|
|
This fixes `combine_hangul_code_points` which would try to combine
a LVT syllable with a trailing consonant, resulting in a wrong
character.
Also added a test for this specific case.
|
|
|
|
The mappings are exposed via `Unicode::code_point_decomposition(u32)`
and `Unicode::code_point_decompositions()`, the latter being useful for
reverse searching a code point from its decomposition.
The normalization code does not make use of `Quick_Check` props (https://www.unicode.org/reports/tr44/#Decompositions_and_Normalization),
meaning no quick check optimizations.
|
|
Parse emoji from emoji-serenity.txt to allow displaying their names and
grouping them together in the EmojiInputDialog.
This also adds an "Unknown" value to the EmojiGroup enum. This will be
useful for emoji that aren't found in the UCD, or for when UCD downloads
are disabled.
|
|
According to TR #51, the "best definition of the full set [of emojis] is
in the emoji-test.txt file". This defines not only the emoji themselves,
but the order in which they should be displayed, and what "group" of
emojis they belong to.
|
|
Everything is now setup to create the LibLocale library and link it
where needed.
|
|
The UCD only cares about a few locales for special casing rules (az, lt,
and tr). Unfortunately, LibUnicode cannot use LibLocale once the
libraries are separate because LibLocale will need to use LibUnicode for
many more things; thus there would be a circular dependency. Instead,
just generate the small enum needed for this one use case.
|
|
These are still included in LibUnicode, but this updates their location
and the include paths of other files which include them.
|
|
|
|
This is generated by GenerateLocaleData, which will soon be in the
Locale namespace. Move it out of CurrencyCode.h, as that will continue
to live in the Unicode namespace.
|
|
They are still included into LibUnicode, but this moves their generated
location to be under LibLocale.
|
|
Currently, LibUnicodeData contains the generated UCD and CLDR data. Move
the UCD data to the main LibUnicode library, and rename LibUnicodeData
to LibLocaleData. This is another prepatory change to migrate to
LibLocale.
|
|
To prepare for placing all CLDR generated data in a new library,
LibLocale, this moves the code generators for the CLDR data to the
LibLocale subfolder.
|
|
The generated locale data contains an enum also named Variant, as
variants are part of locale strings. This hasn't been an issue, but as
includes are reordered, the order in which the enum and AK::Variant are
included may cause an ambiguity error.
|
|
|
|
Note that not all locales have this day period.
|
|
This algorithm is to inject spacing around the range separator under
certain conditions. For example, in en-US, the range [3, 5] should be
formatted as "3–5" if unitless, but as "$3 – $5" for currency.
|
|
|
|
This has been superseded by get_preferred_keyword_value_for_locale,
which doesn't require allocating a Vector just to return its first
element.
|
|
|
|
|
|
This also ensures we only include values we actually support in the
generated list of available values.
|
|
|
|
|
|
To prepare for using plural rules within number & duration format, this
removes the NumberFormat::Plurality enumeration.
This also adds PluralCategory::ExactlyZero & PluralCategory::ExactlyOne.
These are used in locales like French, where PluralCategory::One really
means any value from 0.00 to 1.99. PluralCategory::ExactlyOne means only
the value 1, as the name implies. These exact rules are not known by the
general plural rules, they are explicitly for number / currency format.
|
|
The PluralCategory enum is currently generated for plural rules. Instead
of generating it, this moves the enum to the public LibUnicode header.
While it was nice to auto-discover these values, they are well defined
by TR-35, and we will need their values from within the number format
code generator (which can't rely on the plural rules generator having
run yet). Further, number format will require additional values in the
enum that plural rules doesn't know about.
|
|
The Polish test cases added here cover previous failures from test262,
due to the way that 0 is specified to be "many" in Polish.
|
|
Separate lists are generated for cardinal and ordinal form.
|
|
Plural rules in the CLDR are of the form:
"cs": {
"pluralRule-count-one": "i = 1 and v = 0 @integer 1",
"pluralRule-count-few": "i = 2..4 and v = 0 @integer 2~4",
"pluralRule-count-many": "v != 0 @decimal 0.0~1.5, 10.0, 100.0 ...",
"pluralRule-count-other": "@integer 0, 5~19, 100, 1000, 10000 ..."
}
The syntax is described here:
https://unicode.org/reports/tr35/tr35-numbers.html#Plural_rules_syntax
There are up to 2 sets of rules for each locale, a cardinal set and an
ordinal set. The approach here is to generate a C++ function for each
set of rules. Each condition in the rules (e.g. "i = 1 and v = 0") is
transpiled to a C++ if-statement within its function. Then lookup tables
are generated to match locales to their generated functions.
NOTE: -Wno-parentheses-equality is added to the LibUnicodeData compile
flags because the generated plural rules have lots of extra parentheses
(because e.g. we need to selectively negate and combine rules). The code
to generate only exactly the right number of parentheses is quite hairy,
so this just tells the compiler to ignore the extras.
|
|
This includes:
* The minimum number of days in a week for that week to count as the
first week of a new year.
* The day to be shown as the first day of the week in a calendar.
* The start/end days of the weekend.
Like the existing hour cycle data, week data is presented per-region in
the CLDR, rather than per-locale. The method to add likely subtags to a
locale to perform region lookups is the same.
The list of regions in the CLDR for hour cycle, minimum days, first day,
and weekend days are quite different. So rather than changing the
existing HourCycleRegion enum to a generic Region enum, we generate
separate enums for each of the week data fields. This allows each lookup
into these fields to remain simple array-based index access, without any
"jumps" for regions that don't have CLDR data for a field.
|
|
Currently contains just each locale's character order, but is set up to
easily add other text layout fields from the CLDR if ECMA-402 eventually
requires them.
|
|
This commit has no behavior changes.
In particular, this does not fix any of the wrong uses of the previous
default parameter (which used to be 'false', meaning "only replace the
first occurence in the string"). It simply replaces the default uses by
String::replace(..., ReplaceMode::FirstOnly), leaving them incorrect.
|
|
This will be used by Intl.DurationFormat
|
|
Release notes: https://cldr.unicode.org/index/downloads/cldr-41
Note that the HourCycleRegion enum now contains 272 entires, thus needs
to be bumped from u8 to u16.
|
|
|
|
There isn't too much of an effective difference here other than that the
BCP 47 data contains some aliases we would otherwise not handle.
|
|
BCP 47 will be the single source of truth for known calendar and number
system keywords, and their aliases (e.g. "gregory" is an alias for
"gregorian"). Move the generation of available keywords to where we
parse the BCP 47 data, so that hard-coded aliases may be removed from
other generators.
|
|
We have a fair amount of hard-coded keywords / aliases that can now be
replaced with real data from BCP 47. As a result, the also changes the
awkward way we were previously generating keys. Before, we were more or
less generating keywords as a CSV list of keys, e.g. for the "nu" key,
we'd generate "latn,arab,grek" (ordered by locale preference). Then at
runtime, we'd split on the comma. We now just generate spans of keywords
directly.
|
|
This parses Blocks.txt for CharacterType properties and creates
a global display array for use in apps.
|
|
|
|
|
|
|
|
|
|
|
