Nushell: Introduction to a new kind of shell

Comment on Mastodon

What is nushell

Let me introduce you to a nice project I found while lurking on the Internet. It's called nushell and is a non-POSIX shell, so most of your regular shells knowledge (zsh, bash, ksh, etc…) can't be applied on it, and using it feels like doing functional programming.

It's a good tool for creating robust data manipulation pipelines, you can think of it like a mix of a shell which would include awk's power, behave like a SQL database, and which knows how to import/export XML/JSON/YAML/TOML natively.

You may want to try nushell only as a tool, and not as your main shell, it's perfectly fine.

With a regular shell, iterating over a command output can be complex when it involves spaces or newlines, for instance, that's why `find` and `xargs` have a `-print0` parameter to have a special delimited between "items", but it doesn't compose well with other tools. Nushell handles correctly this situation as its manipulates the data using indexed entries, given you correctly parsed the input at the beginning.

Nushell official project page

Nushell documentation website

How to get it

Nushell is a rust program, so it should work on every platform where Rust/Cargo are supported. I packaged it for OpenBSD, so it's available on -current (and will be in releases after 7.3 is out), the port could be used on 7.2 with no effort.

With Nix, it's packaged under the name `nushell`, the binary name is `nu`.

For other platforms, it's certainly already packaged, otherwise you can find installation instructions to build it from sources.

Nushell documentation: Building nushell from sources

Configuration

At first run, you are prompted to use default configuration files, I'd recommend accepting, you will have files created in `~/.config/nushell/`.

The only change I made from now is to make Tab completion case-sensitive, so `D[TAB]` completes to `Downloads` instead of asking between `dev` and `Downloads`. Look for `case_sensitive_completions` in `.config/nushell/config.nu` and set it to `true`.

Examples

If you are like me, and you prefer learning by doing instead of reading a lot of documentation, I prepared a bunch of real world use case you can experiment with. The documentation is still required to learn the many commands and syntax, but examples are a nice introduction.

Getting help

Help from nushell can be parsed directly with nu commands, it's important to understand where to find information about commands.

Use `help a-command` to learn from a single command:

Use `help commands` to list all available commands (I'm limiting to 5 between there are a lot of commands)

Add `sort-by category` to list them... sorted by category.

Use `where category == filters` to only list commands from the `filters` category.

Use `find foobar` to return lines containing `foobar`.

General examples

Converting a data structure into another

This is just an example from YAML to JSON, but you can convert much more formats into other formats.

Parsing sysctl output

Because the output would be too long, here is how you get 10 random keys from sysctl.

Recursively convert FLAC files to OPUS

A complicated task using a regular shell, recursively find files matching a pattern and then run a given command on each of them, in parallel. Which is exactly what you need if you want to convert your music library into another format, let's convert everything from FLAC to OPUS in this example.

In the following command line, we will look for every `.flac` file in the subdirectories, then run in parallel using `par-each` the command `ffmpeg` on it, from its current name to the old name with `.flac` changed to `.opus`.

The `let convert` and `| complete` commands are used to store the output of each command into a result table, and store it in the variable `convert` so we can query it after the job is done.

Now, we have a structure in `convert` that contains the columns `stdout`, `stderr` and `exit_code`, so we can look if all the commands did run correctly using the following query.

Synchronize a music library to a compressed one

I had a special need for my phone and my huge music library, I wanted to have a lower quality version of it synced with syncthing, but I needed this to be easy to update when adding new files.

It takes all the music files in `/home/user/Music/` and creates a 64K opus file in `/home/user/Stream/` by keeping the same file tree hierarchy, and if the opus destination file exists it's skipped.

Convert PDF/CBR/CBZ pages into webp and CBZ archives

I have a lot of digitalized books/mangas/comics, this conversion is a handy operation reducing the size of the files by 40% (up to 70%).

Parse gnu tar output

Opening spreadsheets

We can format new strings from columns values.

Filter and sort a JSON

There is a website listing packages that can be updated on OpenBSD at https://portroach.openbsd.org, it provides json of data for rendering.

We can use this data to sort which maintainer has the most up to date percentage, but only if they manage more than 30 packages.

NixOS examples

Query profiles packages

Query flakes

Parse a flake.lock file

OpenBSD examples

Parse /etc/fstab

Parse /var/log/messages

Parse pkg_info output

Conclusion

Nushell is very fun, it's terribly different from regular shells, but it comes with a powerful language and tooling. I always liked shells because of pipes commands, allowing to construct a complex transformation/analysis step by step, and easily inspect any step, or be able to replace a step by another.

With nushell, it feels like I finally have a better tool to create more reliable, robust, portable and faster command pipelines. The learning curve didn't feel too hard, but maybe it's because I'm already used to functional programming.

Proxied content from gemini://perso.pw/blog/articles/nushell.gmi (external content)

Gemini request details:

Original URL
gemini://perso.pw/blog/articles/nushell.gmi
Status code
Success
Meta
text/gemini
Proxied by
kineto

Be advised that no attempt was made to verify the remote SSL certificate.