Here we perform a shallow fetch using builtins.fetchTree. In order
to make this work, we need to specify the `ref' for any repository
that doesn't have `master' as its default branch (I'm not sure why
this limitation exists since we don't need this when performing
the shallow fetch in `--grammar build')
This `ref' field is ignored by helix, so I have left it undocumented
for now, but I could be open to documenting it.
This commit replaces the out-of-date builder in the flake which relied
on submodules for fetching and the compiler for building. Now we
disable fetching and building explicitly with the environment variable
and then use builtins.fetchGit and a derivation mostly derived from
upstream to compile the grammars.
Anecdotally, this is still quite slow as builtins.fetchGit does not
seem to do shallow clones. I'm not sure I see a way around it though
without recording sha256s, which seems cumbersome.
This restores much of the behavior that existed before this PR:
helix will build the grammars when compiling. The difference is that
now fetching is also done during the build phase and is done much
more quickly - both shallow and in parallel.
This is a bit of a micro-optimization: in the current setup we waste
a thread in the pool for a local grammar only to println! a message
saying we're skipping fetching because it's a local grammar.
This is a rather large refactor that moves most of the code for
loading, fetching, and building grammars into a new helix-loader
module. This works well with the [[grammars]] syntax for
languages.toml defined earlier: we only have to depend on the types
for GrammarConfiguration in helix-loader and can leave all the
[[language]] entries for helix-core.
The vision with 'use-grammars' is to allow the long-requested feature
of being able to declare your own set of grammars that you would like.
A simple schema with only/except grammar names controls the list
of grammars that is fetched and built. It does not (yet) control which
grammars may be loaded at runtime if they already exist.
build_grammars adapts the functionality that previously came from
helix-syntax to be used at runtime from the command line flags.
fetch_grammars wraps command-line git to perform the same actions
previously done in the scripts in #1560.
This is not strictly speaking necessary. tree_sitter_library was used by
just one grammar: llvm-mir-yaml, which uses the yaml grammar. This will
make the language more consistent, though. Each language can explicitly
say that they use Some(grammar), defaulting when None to the grammar that
has a grammar_id matching the language's language_id.
helix-syntax mostly existed for the sake of the build task which
checks and compiles the submodules. Since we won't be relying on
that process anymore, it doesn't end up making much sense to have
a very thin crate just for some functions that we could port to
helix-core.
The remaining build-related code is moved to helix-term which will
be able to provide grammar builds through the --build-grammars CLI
flag.
Here we add syntax to the languages.toml languge
[[grammar]]
name = "<name>"
source = { .. }
Which can be used to specify a tree-sitter grammar separately of
the language that defines it, and we make this distinction for
two reasons:
* In later commits, we will separate this code from helix-core
and bring it to a new helix-loader crate. Using separate schemas
for language and grammar configurations allows for a nice divide
between the types needed to be declared in helix-loader and in
helix-core/syntax
* Two different languages may use the same grammar. This is currently
the case with llvm-mir-yaml and yaml. We could accomplish a config
that works for this with just `[[languages]]`, but it gets a bit
dicey with languages depending on one another. If you enable
llvm-mir-yaml and disable yaml, does helix still need to fetch and
build tree-sitter-yaml? It could be a matter of interpretation.
The submodules system is being replaced with a command-line flag
hx --fetch-grammars
Which shallow-clones grammar repositories at the given revision and
hx --build-grammars
For building grammars separate of the initial compilation of helix.
Why remove submodules?
* Cloning helix in general takes a long time because of the submodules,
especially when the submodules are not fetched as shallow
* Packaging is consistently painful no matter the package-manager
* It is quite difficult to devise a scheme where users can declare
a desired set of grammars and implement it with submodules
This commit fully removes the existing tree-sitter submodules from
the tree (as well as the .gitmodules file which is no longer used).
* Move runtime file location definitions to core
* Add basic --health command
* Add language specific --health
* Show summary for all langs with bare --health
* Use TsFeature from xtask for --health
* cargo fmt
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
https://github.com/nix-community/dream2nix is a fairly new and
cool-looking project for adapting upstream package manager outputs
(lockfiles mostly it would seem) for nix.
This should improve the ability to cross-compile. As a more concrete
measure of improvement, `nix flake check' now succeeds 🎉
* Add arrow key mappings for tree-sitter parent/child/sibling nav
This helps my use case, where I use a non-qwerty layout with a
programmable mechanical keyboard, and use a layer switching key (think
fn) to send left down up right from the traditional hjkl positions.
* Add new bindings to docs
closes#1746
The queries for Go were updated in ddbf03613d.
The old ref was before this commit, so running helix from the flake
nix flake run github:helix-editor/helix/d62ad8b595a4f901b9c5dba1bb6e8f70ece395bf -- path/to/file.go
will crash because the old grammar's query analysis will fail (because `iota`
was not yet a named node).
This commit updates the version of the grammars that we pull down when building
the flake so that the queries match the grammars. We'll have to do an update like
this whenever a grammar is bumped in a breaking way (which happens fairly often
in tree-sitter) until #1659 comes along and the version of the grammar becomes
tied to the version declared in source.
