Sunday, September 29, 2024

How to build and run the Rust implementation of tree-sitter

 How to build the Rust implementation of tree-sitter


$ sudo ./script/build-wasm


  (Note just running the script might go wrong due to the permission error when it attempts to access the docker.)


$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh


  (This installs rustc, the latest rust compiler at ~/.cargo as guided in https://rustup.rs/.)


$ source ~/.cargo/env


 (This activates the path of the installed rust toolchains.)


$ cargo build


 (This builds tree-sitter with the installed rust toolchains. With no option to the build mode, tree-sitter is built in a debug mode that provides various debug options. Alternatively, one can give --release to the build mode for a release.)


How to run tree-sitter, say, over tree-sitter-python?


$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash


  (Note one needs to setup nvm to install npm and other javascript relevant tools.)

$ nvm  install --lts

  (Install the latest npm and others.)

$ git clone https://github.com/tree-sitter/tree-sitter-python

  (For example, let us consider tree-sitter-python, which is a python parser using tree-sitter and uses an external its own lexer implementation not using tree-sitter.)

$ cd tree-sitter-python

$ $TREE-SITTER/target/tree-sitter/release/tree-sitter generate --debug-build

 (Assuming $TREE-SITTER is a path to the tree-sitter directory, it generates src/parser.c and others. At this moment, it is unclear to me what is the effect of giving --debug-build to the generate mode.)

$TREE-SITTER/target/tree-sitter/release/tree-sitter build --debug

 (It changes ./src/node-types.json.)

$ $TREE-SITTER/target/tree-sitter/release/tree-sitter parse --debug YOUR-PYTHON-PROGRAM.py

 (It prints various logs including parsing actions and states, and then it prints an abstract syntax tree for the python program.)