Integrated Development Environment support #1226

New issue

2024-05-05T13:02:03Z

vyzo commented

2024-05-05 13:02:03 +00:00

(Migrated from github.com)

There is already work in progress to support swank/slime and LSP, and it is becoming apparent that there is a lot of common structure between IDE support programs.

We propose an :std/ide package, where we provide coomon structures for all IDEs and packages for swank, lsp etc underneath.

Common Functionality

In general there are three main usage patterns we want to support for IDEs:

reading/navigating code
writing new code or editing existing modules
interactive evaluation in an appropriate context.

This is the most fundamental interaction mode. We need to support:

navigation to definition of a symbol in source
documentation lookup and display in the user's IDE.
type signatures for all procedures from compiled modules; we already have the information in the compiler, so it is a matter of surfacing it.

Interactive Code Evaluation

This can happen in two contexts:

a top interacton context, where a user evaluates some code for exploratory or testing purposes.
a module context, where the user evaluates code they are working with, usually for testing purposes.

In order to facilitate interaction, the IDE server can be instructed to evaluate code in a specific module context. Furthermore all the symbols in the current code should support introspection, as in navigation above.

Code Editing

This is perhaps most important, but also the most complex to implement.
The server should keep track of the code as it is being edited, even though it cannot expand it yet.
It should create a shadow module context, where it can provide information and completions as the user is editing.
It should also provide interactive evaluation support in the partial module, provided it is syntactically correct and can expand.

Generally desirable features for interactive editing:

basic completions: imports and symbols should be completable on demand.
context dependent completions: type annotations and instances ofr using should be parsed so that the user can complete the dot, and even the types in annotations/using
when suggesting a completion or when processing arguments the IDE server should provide type signatures and links to documentation.

There is already work in progress to support swank/slime and LSP, and it is becoming apparent that there is a lot of common structure between IDE support programs. We propose an `:std/ide` package, where we provide coomon structures for all IDEs and packages for swank, lsp etc underneath. # Common Functionality In general there are three main usage patterns we want to support for IDEs: - reading/navigating code - writing new code or editing existing modules - interactive evaluation in an appropriate context. ## Supporting Code navigation and reading This is the most fundamental interaction mode. We need to support: - navigation to definition of a symbol in source - documentation lookup and display in the user's IDE. - type signatures for all procedures from compiled modules; we already have the information in the compiler, so it is a matter of surfacing it. ## Interactive Code Evaluation This can happen in two contexts: - a top interacton context, where a user evaluates some code for exploratory or testing purposes. - a module context, where the user evaluates code they are working with, usually for testing purposes. In order to facilitate interaction, the IDE server can be instructed to evaluate code in a specific module context. Furthermore all the symbols in the current code should support introspection, as in navigation above. ## Code Editing This is perhaps most important, but also the most complex to implement. The server should keep track of the code as it is being edited, even though it cannot expand it yet. It should create a shadow module context, where it can provide information and completions as the user is editing. It should also provide interactive evaluation support in the partial module, provided it is syntactically correct and can expand. Generally desirable features for interactive editing: - basic completions: imports and symbols should be completable on demand. - context dependent completions: type annotations and instances ofr `using` should be parsed so that the user can complete the dot, and even the types in annotations/`using` - when suggesting a completion or when processing arguments the IDE server should provide type signatures and links to documentation. # Related Issues and PRs - #781 - #782 - #803 - #1186

metaleap commented

2024-06-11 17:44:05 +00:00

(Migrated from github.com)

interactive evaluation in an appropriate context

In my thinking of the VSCode extension and dev-ex I have in mind (but really not just vsc but practically any & all other LSP clients), for real rapid-iteration live-coding ideally the server side (lsp or ide package) should maintain a kind of longrunning / ongoing "interpreter session" that begins by loading up all source files in the "project" or editor session, and from then on tracks all on-the-fly client-side edits live (if the LSP client transmits them — the vsc one would — otherwise.. file-watching I guess?), whether they're already stored-on-disk or not.

This is of course already kinda-sorta-somewhat prerequisite for on-the-fly / as-you-type diagnostics — but really if done in a way that preserves "program state" (of non-function-typed globals and closure-captured-vars even as they're affected/mutated by one's ongoing interactive eval requests), you have "image-based development (without the image)". Enables lots of live-coding with less frequent need for always-another "run program from main, and debug" cycle.

> interactive evaluation in an appropriate context In my thinking of the VSCode extension and dev-ex I have in mind (but really not just vsc but practically any & all other LSP clients), for real rapid-iteration live-coding ideally the server side (`lsp` or `ide` package) should maintain a kind of longrunning / ongoing "interpreter session" that begins by loading up all source files in the "project" or editor session, and from then on tracks all on-the-fly client-side edits live (_if_ the LSP client transmits them — the vsc one _would_ — otherwise.. file-watching I guess?), whether they're already stored-on-disk or not. This is of course already kinda-sorta-somewhat prerequisite for on-the-fly / as-you-type diagnostics — but really if done in a way that preserves "program state" (of non-function-typed globals and closure-captured-vars even as they're affected/mutated by one's ongoing interactive eval requests), you have "image-based development (without the image)". Enables lots of live-coding with less frequent need for always-another "run program from main, and debug" cycle.

metaleap commented

2024-06-17 16:31:07 +00:00

(Migrated from github.com)

My list of to-be-exposed ide API functionality suggestions, with common caveats and subtleties covered — looks like a wall-of-text, but less so if one just attacks one little thing at a time (they're guess-ordered for presumed maximum reuse/build-upon-ability =)

[My list of to-be-exposed `ide` API functionality suggestions](https://github.com/metaleap/gerbil-lsp/blob/main/lsp/gerbil/README.md), with common caveats and subtleties covered — looks like a wall-of-text, but less so if one just attacks one little thing at a time (they're guess-ordered for presumed maximum reuse/build-upon-ability =)

Sign in to join this conversation.