LSP outside of the OCaml Platform

[gr1mpatr0n]

Hi OCamlers -

I raise a query here that has been asked before on the forums. It received a negative response, so I am satisfied if the response now is the same.

At my organisation, we do not use Dune as our build system for a plethora of reasons. I do not intend for this to be a criticism of Dune, as it clearly serves the ecosystem well. It also relates to our requirements in a large, multi-language environment. Alas, some reasons include:

• Generally slow maintenance and over-decade-old bugs.
• Inconsistent caching behaviour - spurious whole-compiler rebuilds among other things.
• Need to run dune build for LSP to become active (see the Rust ecosystem for how this should be managed)
• Inferior file watching.
• No support (?) for distributed, concurrent, multi-platform builds?
• Awkward requirements around separating directories when using dynamic_include, as well as confusing limitations.
• Unpleasant (in our view) syntax.

We instead use BUCK2, which has good support for OCaml, in combination with some Python scripts which manage switches and creating Starlark rules based on OPAM packages. This works wonders for us - super fast builds with excellent incremental builds and caching, easy integration of PPX, JS stuff and so on.

One trade off however, is that we now have no LSP. It seems that the reason raised in the past was that the LSP communicates with Dune to understand artifact locations etc.

My question, therefore, is - is there any way we can plumb up our Buck2 build system to offer that same information for the LSP? If not, can abstractions be made so that we can do such a thing?

I look forward to your comments.

Best,
Ben

[rgrinberg]

If not, can abstractions be made so that we can do such a thing?

Yes, But. The vast majority of ocamllsp users use dune, so it wouldn't be sensible for us to invest in other build systems. You will need to lead the way on this if you'd like to see this happen.

[gr1mpatr0n]

No problem - would this project be open to merging support for Buck2, or would they have to be kept as downstream patches? Might also be interesting to hear someone from Meta's take on this.

[rgrinberg]

That entirely depends on how the project is going to look like. A solid implementation that is not a maintenance burden will certainly be welcome.

[rgrinberg]

Some preliminary caution though: some of the work you have in mind might need changes in merlin itself. I have no idea if are open to this idea. As a solid first step, perhaps you should teach your build system to generate .merlin files. This is an alternative way in which ocamllsp and merlin can already communicate with other build systems.

[gr1mpatr0n]

@rgrinberg Thanks for the pointers. It looks as though the necessary .merlin stanzas are quite simple to generate. I will give it a go. What exactly am I missing out on if I use the fallback-merlin mode?

Looking into it, it seems this mode is pretty well suited for me already :)

[voodoos]

I can add a few more pointers here:

• Using .merlin comes with one drawback: these files provide per-directory configuration. The Dune connector allows for per-module configuration. If you go that direction anyway, we have a plan to improve the behavior of the --fallback-read-dot-merlin flag. Help on this would be very welcomed.
• To get it's configuration, Merlin (and thus OCaml-LSP) uses a "configuration reader" which is a small server that speaks a very simple protocol. We currently have two readers: dot-merlin-reader to read .merlin files and dune ocaml-merlin that loads configuration from Dune artifacts. So you could write a new one, for use with Buck2 (or more generic). We could then make it configurable when starting OCaml-LSP.