package architecture

import "src.elv.sh/docs/architecture"

Overview

This file documents how Elvish's codebase is structured on a high level. You can read it either in a code editor, or in a godoc viewer such as https://pkg.go.dev/src.elv.sh@master/docs/architecture.

Elvish is a Go project. If you are not familiar with how Go code is organized, start with how to write Go code.

Go code in the Elvish repo lives under two directories:

• The cmd directory contains Elvish's entrypoints, but it contains very little code.
• The pkg directory has most of Elvish's Go code. It has a lot of subdirectories, so it can be a bit hard to find your bearing just by exploring the file tree.

We will cover the cmd directory first, and then focus on the most important subdirectories under pkg.

The Elvish repo also contains other directories. They are not technically part of the Go program, so we won't cover them here. Read their respective README files to learn more.

Module, package and symbol names

Elvish's module name is src.elv.sh. You can think of it as an alias to where the code is actually hosted (currently github.com/elves/elvish).

The import paths of all the packages start with the module name src.elv.sh. For example, the import path of the package in pkg/parse is src.elv.sh/pkg/parse.

When referring to a symbol from a package, we'll use just the last component of the package's import path. For example, the Evaler type from the src.elv.sh/pkg/eval package is simply eval.Evaler. (This is consistent with Go's syntax.)

Entrypoints (cmd/elvish and pkg/prog)

The default entrypoint of Elvish is src.elv.sh/cmd/elvish. It has a main function that does the following:

• Assemble a "composite program" from multiple subprograms, most notably shell.Program.
• Call prog.Run.

You can read about the advantage of this approach in the godoc of src.elv.sh/pkg/prog.

There are other main packages, like src.elv.sh/cmd/withpprof/elvish. They follow the same structure and only differ in which subprograms they include.

The shell subprogram (pkg/shell)

The shell subprogram has two slightly different "modes", interactive and non-interactive, depending on the command-line arguments. The doc for shell.Program contains more details.

In both modes, the shell subprogram uses the interpreter implemented in src.elv.sh/pkg/eval to evaluate code.

In interactive mode, the shell also uses the line editor implemented in src.elv.sh/pkg/edit to read commands interactively. Some features of the editor depend on persistent storage; the shell subprogram also takes care of initializing that, using src.elv.sh/pkg/daemon.

The interpreter (pkg/eval)

The src.elv.sh/pkg/eval package is perhaps the most important package in Elvish, as it implements the Elvish language and the builtin module.

The interpreter is represented by eval.Evaler, which is created with eval.NewEvaler. The method eval.Evaler.Eval (yes, that's 3 "evals"s) evaluates Elvish code, and does so in several steps:

1. Invoke the parser to get an AST.
2. Compile the AST into an "operation tree".
3. Run the operation tree.

This approach is chosen mainly for its simplicity. It's probably not very performant.

The compilation of each AST node into its corresponding operation node, as well as how each operation node runs, is defined in the several compile_*.go files. These files are where most of the language semantics is implemented.

Another sizable chunk of this package is the various builtin_fn_*.go files, which implement functions of the builtin module. These may be moved to a different package in future.

Some other packages important for the interpreter are:

• src.elv.sh/pkg/eval/vals implements a standard set of operations for Elvish values.
• src.elv.sh/pkg/persistent implements Elvish's lists and maps, modeled after Clojure's vectors and maps.
• Subdirectories of src.elv.sh/pkg/mods implement the various builtin modules.

The parser (pkg/parse)

The src.elv.sh/pkg/parse package implements parsing of Elvish code, with the parse.Parse as the entrypoint.

The parsing algorithm is a handwritten recursive descent one, with the slightly unusual property that there's no separate tokenization phase. Read the package's godoc for more details.

The editor (pkg/edit)

The src.elv.sh/pkg/edit package contains Elvish's interactive line editor, represented by edit.Editor. The traditional term "line editor" is a bit of a misnomer; modern line editors (including Elvish's) are similar to full-blown TUI applications like Vim, except that they usually restrict themselves to the last N lines of the terminal rather than the entire screen.

The editor is built on top of the more low-level src.elv.sh/pkg/cli package (which is also a bit of a misnomer), in particular the cli.App type.

The entire TUI stack is due for a rewrite soon.

The editor relies on persistent storage for features like the directory history and the command history. As mentioned above, the initialization of the storage is done in pkg/shell, using pkg/daemon.

The storage daemon (pkg/daemon)

Support for persistent storage is is currently provided by a storage daemon. The src.elv.sh/pkg/daemon packages implements two things:

• A subprogram implementing the storage daemon (daemon.Program).
• A client to talk to the daemon (returned by daemon.Activate).

The daemon is launched and terminated on demand:

• The first interactive Elvish shell launches the daemon.
• Subsequent interactive shells talk to the same daemon.
• When the last interactive Elvish shell quits, the daemon also quits.

Internally, the daemon uses bbolt as the database engine.

In future (subject to evaluation) Elvish might get a custom database, and the daemon might go away.

Closing remarks

This should have given you a rough idea of the most important bits of Elvish's implementation. The implementation prioritizes readability, and most exported symbols are documented, so feel free to dive into the source code!

If you have questions, feel free to ask in the user group or DM xiaq.

Index ¶

Source Files ¶

doc.go