LibCpp: Add SemanticSyntaxHighlighter

The SemanticSyntaxHighlighter uses TokenInfo results from the
language server to provide 'semantic' syntax highlighting, which
provides more fin-grained text spans results.

For example, the SemanticSyntaxHighlighter can color function calls,
member fields references and user-defined types in different colors.

With the simple lexer-only syntax highlighter, all of these tokens were
given the same text highlighting span type.

Since we have to provide immediate highlighting feedback to the user
after each edit and before we get the result for the language server,
we use a heuristic which computes the diff between the current tokens
and the last known tokens with compete semantic information
(We use LibDiff for this).

This heuristic is not very performant, and starts feeling sluggish with
bigger (~200 LOC) files.
A possible future improvement would be only computing the diff for
tokens in text ranges that have changes since the last commit.

1 files changed, 49 insertions, 0 deletions

diff --git a/Userland/Libraries/LibCpp/SemanticSyntaxHighlighter.h b/Userland/Libraries/LibCpp/SemanticSyntaxHighlighter.h
new file mode 100644
index 0000000000..7983c35d4f
--- /dev/null
+++ b/Userland/Libraries/LibCpp/SemanticSyntaxHighlighter.h
@@ -0,0 +1,49 @@
+/*
+ * Copyright (c) 2022, Itamar S. <itamar8910@gmail.com>
+ * Copyright (c) 2022, the SerenityOS developers.
+ *
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#pragma once
+
+#include "SyntaxHighlighter.h"
+#include "Token.h"
+#include <LibGUI/AutocompleteProvider.h>
+#include <LibSyntax/Highlighter.h>
+#include <LibThreading/Mutex.h>
+
+namespace Cpp {
+
+class SemanticSyntaxHighlighter final : public Syntax::Highlighter {
+
+public:
+    SemanticSyntaxHighlighter() { }
+    virtual ~SemanticSyntaxHighlighter() override = default;
+
+    virtual bool is_identifier(u64 token) const override;
+    virtual bool is_navigatable(u64 token) const override;
+
+    virtual Syntax::Language language() const override { return Syntax::Language::Cpp; }
+    virtual void rehighlight(Palette const&) override;
+
+    void update_tokens_info(Vector<GUI::AutocompleteProvider::TokenInfo>);
+
+    virtual bool is_cpp_semantic_highlighter() const override { return true; }
+
+protected:
+    virtual Vector<MatchingTokenPair> matching_token_pairs_impl() const override { return m_simple_syntax_highlighter.matching_token_pairs_impl(); };
+    virtual bool token_types_equal(u64 token1, u64 token2) const override { return m_simple_syntax_highlighter.token_types_equal(token1, token2); };
+
+private:
+    void update_spans(Vector<GUI::AutocompleteProvider::TokenInfo> const&, Gfx::Palette const&);
+
+    Cpp::SyntaxHighlighter m_simple_syntax_highlighter;
+    Vector<GUI::AutocompleteProvider::TokenInfo> m_tokens_info;
+    String m_saved_tokens_text;
+    Vector<Token> m_saved_tokens;
+    Threading::Mutex m_lock;
+};
+}
+template<>
+inline bool Syntax::Highlighter::fast_is<Cpp::SemanticSyntaxHighlighter>() const { return is_cpp_semantic_highlighter(); }