Syntax Highlighting

Syntax highlighting engine for Kate syntax definitions

Table of contents

1. Introduction
2. Out of scope
3. Syntax definition files
4. Color theme files
5. Build it
6. How to contribute
   • Licensing
   • Tips for contributing to syntax definition files
   • Adding unit tests for a syntax definition
7. Report bug or help to fix them
8. Updating the syntax & themes pages of the kate-editor.org website

Introduction

This is a stand-alone implementation of the Kate syntax highlighting engine. It's meant as a building block for text editors as well as for simple highlighted text rendering (e.g. as HTML), supporting both integration with a custom editor as well as a ready-to-use QSyntaxHighlighter sub-class.

Besides a C++ API, a QML API is also provided.

Out of scope

To not turn this into yet another text editor, the following things are considered out of scope:

• code folding, beyond providing folding range information
• auto completion
• spell checking
• user interface for configuration
• management of text buffers or documents

If you need any of this, check out KTextEditor.

Syntax definition files

This library uses Kate syntax definition files for the actual highlighting, the file format is documented here.

More than 300 syntax definition files are included, that are located in data/syntax/ and have the .xml extension. Some files are generated at compile time, all xml files are available here. Additional ones are picked up from the file system if present, so you can easily extend this by application-specific syntax definitions for example.

To install or test a syntax definition file locally, place it in org.kde.syntax-highlighting/syntax/, which is located in your user directory. Usually it is:

For local user	$HOME/.local/share/org.kde.syntax-highlighting/syntax/
For Flatpak packages	$HOME/.var/app/package-name/data/org.kde.syntax-highlighting/syntax/
For Snap packages	$HOME/snap/package-name/current/.local/share/org.kde.syntax-highlighting/syntax/
On Windows®	%USERPROFILE%\AppData\Local\org.kde.syntax-highlighting\syntax\
On macOS®	$HOME/Library/Application Support/org.kde.syntax-highlighting/syntax/

For more details, see "The Highlight Definition XML Format" (Working with Syntax Highlighting, KDE Documentation).

Also, in data/schema/ there is a script to validate the syntax definition XML files. Use the command validatehl.sh mySyntax.xml.

Color theme files

This library includes the color themes, which are documented here.

The color theme files use the JSON format and are located in data/themes/ with the .theme extension. Additional ones are also picked up from the file system if present, in the org.kde.syntax-highlighting/themes/ folder of your user directory, allowing you to easily add custom color theme files. This location is the same as shown in the table of the previous section, replacing the syntax folder with themes. For more details, see "The Color Themes JSON Format" (Working with Color Themes, KDE Documentation).

The KTextEditor library (used by Kate, Kile and KDevelop, for example) provides a user interface for editing and creating KSyntaxHighlighting color themes, including a tool for exporting and importing the JSON theme files.

Note that in KDE text editors, the KSyntaxHighlighting color themes are used since KDE Frameworks 5.75, released on October 10, 2020. Previously, Kate's color schemes (KConfig based schema config) were used and are now deprecated. The tool utils/schema-converter/ and the script utils/kateschema_to_theme_converter.py convert the old Kate schemas to KSyntaxHighlighting themes.

Also see "Submit a KSyntaxHighlighting Color Theme" (Kate Editor Website).

Build it

1. Create and change into a build directory. Usually, a folder called build is created inside the syntax-highlighting source directory.

   mkdir <build-directory>
   cd <build-directory>

2. Run the configure process with cmake and compile:

   cmake <source-directory>
   make

   For example:

   git clone git@invent.kde.org:frameworks/syntax-highlighting.git
   mkdir ./syntax-highlighting/build
   cd ./syntax-highlighting/build
   cmake ../
   make

   For more details see "Building Kate from Sources on Linux" (Kate Editor Website).

   NOTE: If running cmake shows an error related to your version of KDE Frameworks, you edit the CMakeLists.txt file in the line find_package(ECM 5.XX.X ...).

3. To run tests:

   make test

   The tests are located in the autotests directory. This command can be used to check changes to units test after modifying some syntax definition file. To add a unit test or update the references, see the section "Adding unit tests for a syntax definition".

How to contribute

KDE uses a GitLab instance at invent.kde.org for code review. The official repository of the KSyntaxHighlighting framework is here.

All the necessary information to send contributions is here.

Licensing

Contributions to KSyntaxHighlighting shall be licensed under MIT.

All files shall contain a proper "SPDX-License-Identifier: MIT" identifier inside a header like:

/*
    SPDX-FileCopyrightText: 2020 Christoph Cullmann <cullmann@kde.org>
 
    SPDX-License-Identifier: MIT
*/

Tips for contributing to syntax definition files

• If you are modifying an existing syntax definition XML file, you must increase the version number of the language.

• Do not use hard-coded colors, as they may not look good or be illegible in some color themes. Prefer to use the default color styles.

  For more information, see:

  • Available Default Styles (Working with Syntax Highlighting, KDE Documentation)
  • Kate Part (KF5): New Default Styles for better Color Schemes (Kate Editor Website)

• While writing syntax, you can use ksyntaxhighlighter6 (previously named kate-syntax-highlighter) to display the style or others informations associated with each part of text. For example ksyntaxhighlighter6 --output-format=ansi --syntax-trace=format test.cpp.

  On Linux and Unix systems, you can isolate the syntax files to be modified by using the XDG_DATA_DIRS environment variable to point to a folder containing org.kde.syntax-highlighting/syntax/ in which you will put all the files required for a syntax:

  XDG_DATA_DIRS=mydir ksyntaxhighlighter6 ....

• Add test files, these are found in autotests/input/. If you are going to add a new syntax XML file, create a new test file; if you are going to modify a XML file, adds examples to existing test files.

  Then, it is necessary to generate and update the files in autotests/folding/, autotests/html/ and autotests/reference/, which must be included in the patches. Instructions are below.

Adding unit tests for a syntax definition

1. Add an input file into the autotests/input/ folder, lets call it test.<language-extension>.
2. If the file extension is not sufficient to trigger the right syntax definition, you can add an second file testname.<language-extension>.syntax that contains the syntax definition name to enforce the use of the right extension.

3. Do make && make test.

   Note that after adding or modifying something in <source-directory>/autotests/input/, an error will be showed when running make test, because the references in the source directory do not match the ones now generated.

4. Inspect the outputs found in your binary directory autotests/folding.out/, autotests/html.output/ and autotests/output/.
5. If OK, run in the binary folder ./autotests/update-reference-data.sh to copy the results to the right location. That script updates the references in the source directory in autotest/folding/, autotest/html/ and autotest/reference/.
6. Add the result references after the copying to the git.

Report bug or help to fix them

KDE uses Bugzilla to management of bugs at bugs.kde.org. You can see the bugs reported of frameworks-syntax-highlighting here.

Also, you can report a bug here.

However, some users often report bugs related to syntax highlighting in kate/syntax and kile/editor.

Updating the syntax & themes pages of the kate-editor.org website

To update the kate-editor.org/syntax and kate-editor.org/themes websites including the update site & all linked examples/files, please run after successful build & test the following make target:

make update_kate_editor_org

This will clone the kate-editor.org git from invent.kde.org into kate-editor-org inside the build directory and update the needed things.

You can afterwards step into kate-editor-org and commit & push the change after review.

The kate-editor.org webserver will update itself periodically from the repository on invent.kde.org.