I've been on a longish search for a bash replacement (or at least "extra substitute") in the realm of "more complex commandline scripting" (functions, utility scripts, etc.). Naturally, every option is a set of different compromises.
I think I've finally settled on Lua- It's small, coherent, surprisingly fast (especially with LuaJIT) and has the "virtually no startup cost" I was looking for (which ruled out, for example, Elixir for a lot of things).
And, I've finally figured out how to bang my nix-darwin flake.nix config in a way that gets it AND a bunch of popular libraries installed together and all seeing each other.
Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.
Is Guile really that good at the shell scripting "thing"? Like, the bash superpower is that it makes it extremely easy to run subprocesses in incredibly flexible ways (in the background, foreground, in pipelines, with stdout/stderr redirected to files, etc. etc.). Can Guile do that smoothly? I generally find that doing things like this is way more awkward in most "real" programming languages compared to shell scripting languages.
I know I’m in my own little corner, but stringing process together is the shell/unix superpower.
The “Unix way” stands on the shoulders of stdin, stdout, pipe, and fork.
And the shells make that a first class concept. Breaking up the command line and linking processes together is the #1 job of a shell, everything else is gravy.
The scheme and lisp shell takes fail here, just because you have to go through the gymnastics of syntax.
csort is caching sort: it caches the result of the key function through which the elements are projected to form the argument of the comparison.
In other words, so we don't wastefully call get-nums multiple times for the same path.
I guess we should distinguish "the shell environment" (which might stay Bash) with "the shell script language" (which could be any one of a number of options, just use the right hashbang header)
One that strove to do both WOULD have to do all the things you suggested as well (job management, stdout/stderr redirection etc.) So options like oil shell, nushell, fish etc.
I was just talking about "a commandline tooling language" in general, as far as use-cases. And for that, Lua (and Moonscript, which compiles to Lua but is way nicer and has been featured on HN a few times over the years) seems to fit the bill (1-based indexing aside, ugh)
I love that in Guix, Guile is used as the configuration language all the way "from the metal" (the bootloader) to the declarative environment config in userspace.
I'm one of those folks that still can't do LISPs though. If there was a way to replace some or all of the parens with significant indentation instead (which I believe Racket makes possible... but that only works in Racket), I might reconsider it.
You probably heard this one before. After a while the parens do disappear. When conventionally indented, lisp users see only the indentation and not the parens.
With structural editing, no one ever have to match parens or count parens at the end...
Can you expand on using Guile for shell scripting?
I was aware of it, but not familiar until I looked at the link you shared. Looks like a Scheme, which for the uninitiated is a different approach for shell scripts.
Yes, there are some basic examples here [0] – otherwise it's just shebang and you're good to go; it can be a rabbit hole where you start liking it too much than what you really should (you wanted simple scripts, now you are finishing scheme book).
It's consistent, "infinitely" terse (you can create dsl-like apis for your problem domains), proper programming language.
First time I used it for scripting was when I had to process some xmls and do bunch of processing and gluing together – I was impressed how quick and easy it was without knowing anything about it and how readable the whole thing ended up being.
I’m the author of Murex, so I’ve definitely stuck with that. It’s been my primary shell for 6 or 7 years now. The only time I touch Bash is the odd occasion I SSH into a box.
However the other shells are also highly polished. I’d definitely recommend each and all of them.
I write a lot of JS/TS for my day job, so zx (https://github.com/google/zx) has been a nice tool for bash scripts that start getting a little too complex.
I keep wanting to get into Raku. It has such a rich pedigree of everything learned from decades of Perl, in a clean slate. It seems to suffer from lack of adoption but it seems better than *sh in every way except for ubiquity.
Indeed Raku is a delight to use for 'shell scripting'.
Perl was originally written as an amalgamation of grep,sed,tr, awk and I am sure a few more unix utilities with their own mini languages. The idea was to use one language instead of tying together half a dozen mini-languages in a shell script. And it worked really well. Perl being a demon with text munging didn't hurt.
Raku keeps this heritage but adds so much more (for better or worse :-) ). It inherits ideas from Lisp and functional programming languages. The thing that impressed me was, how easy it was to use concurrency.
It is much more complicated than Perl. Every feature you could ever want, in multiple ways, it seems like. Perl isn’t too complicated or large of a language.
I try Perl 6/Raku every few years, so it seems like I've tried it half a dozen times by now since it started. I like some things about it, but in a way it seems like an academic project, more suited for experimentation than doing work. I always come back to Perl, which has been my favorite hammer for 30 years.
I've actually already written a lot of the functionality I see here as bash (and other) functions, lol (things like log, filter, debug, retry, repeat, map, fetch (a URL, with retry)... etc.). (things like: enumeration functions, table pretty-printing, etc.)
> Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.
If you're familiar with Python, give xonsh (https://xon.sh/) a go. It's a Bash-like shell but the syntax is Python. It has the same ease of writing shell scripts as Bash does, sans the insane language.
How does the startup time and execution speed of Python compare to other options?
(I'm not particularly inclined to Python, unfortunately, due to the dependency issues and some personally negative opinions about its design... but I like the idea of Xonsh if you're a Python person! I wish I had something like that for Elixir!)
We've transitioned all of our Bash scripts over to Ruby and are pretty happy with it. We almost exclusively use only Ruby's standard library. 'fileutils' and 'rake' have a bunch of nice functions to replace Bash scripts.
See, FileUtils for things like rm_rf, rake provides sh which can execute commands, then there's FileList and Pathname. It even comes with its own template engine (erb) if you need that. Regexes as first class citizens are awesome for shell scripting.
I can also recommend using optprase + ostruct for commandline argument parsing.
I usually go with python when my .sh/.bat script files gets too complex. So I guess we wento down the same road.
Like ruby (I guess), python is very pleasant to work with for small scripts and it's portable between linux and windows, which is really cool (at least in my use cases)
In python, glob, Path and subprocess are mostly all I need...
coming back to raku after basically any other language is a breath of fresh air. i haven't even been worrying much about when the ast stuff lands cause i can't think of anything i would even want to mess with that much, even though i seem fundamentally incapable of engaging with lisp/prolog/forth without giving in to the siren song of metaprogramming. to me that's a very strong sign of how much it gets right as a base language.
i do all my automation with one tiny 20 line raku script that manages a thousand things. hell, the entire monitoring site for our frog is basically just a pile of mixins with a neat little pipeline interface unifying everything from request profiling and html templating to reading from and filling that cache in parallel whenever it remotely pulls the sensor data off various things and gets the current webcam frame.
I see Penlight (which is Python-flavored, but also has some functional programming constructs) https://github.com/lunarmodules/Penlight , Moses if you like functional programming https://yonaba.github.io/Moses/, likely others, but you're right in that there's no standard, "opinionated" stdlib. No bigint support on LuaJIT etc.
I'm an incredibly happy user of nushell, which brings all the best features of other shells terse pipelining syntax and all the best features of more well designed scripting languages (functions, modules, lexical scope, data structures, completely optional types) in one awesome package that also comes with editor (LSP) support and excellent documentation
(The intro page may be a bit misleading. You can freely mix-and-match existing, unstructured as well as nushell-built-in structured commands in the pipeline, as long as you convert to/from string streams - its not mandatory to use the structured built-ins. For example if an existing cli tool has json output, you can use `tool | from json` to turn it into structured data. There are also commands like `detect columns` that parses classic column output, and so on - the tools to mix-and-match structured and unstructured data are convenient and expressive)
Addendum: Its not my login shell. I run it ad-hoc as soon as the command pipeline i'm writing starts getting too complicated, or to write scripts (which of course can be run from outside nushell too, so long as they have the correct shebang)
That's because you don't use the native lib pathlib from python ! It was a delivery when I knew about it. For example, to join paths, you can do so much easily, see the section "operators" of this link : https://docs.python.org/3/library/pathlib.html
The mark of an advanced shell scripter is knowing not to do advanced shell scripting.
(This statement primarily applies to existing stringly-typed scripting languages, which are nightmarish to maintain and debug. PowerShell, nushell, or similar solutions have a much higher complexity ceiling.)
Not everything, and this already changed a while back.
> The core of the Tcl interpreter has been replaced with an on-the-fly compiler that translates Tcl scripts to byte codes; a new interpreter then executes the byte codes. In earlier versions of Tcl, strings were used as a universal representation; in Tcl 8.0 strings are replaced with Tcl_Obj structures ("objects") that can hold both a string value and an internal form such as a binary integer or compiled bytecodes.
I have followed the advice in this story for all my scripts. They are just better interfaces to collection of tools that already exists, not new tools. I write them when the subproblems are already solved by specific programs and the coordination does not require complex data transformation.
Great presentation. I've been writing bash for waaaay too long, and, while I nodded and said yup, yup, a few times, I also learned some new tricks, so thank you!
The social graces having been observed, if you were to update this presentation to reflect modern bash, and especially reference variables, what would be your top 3 additions?
(I am a big fan of reference variables. They make it possible to, among other things, do some funky things that would otherwise require liberal use of eval, which I eschew. Figuring out how to ensure that the target of a reference variable was in fact an associative array while running under `set -u` was a trick of which I am proud. Note: what I did works and isn't totally ugly, there may be a better way, YMMV, IANAL, etc.)
Second Q, if you will permit: Given the memory leaks in associative array management that have been corrected over the last several years, what are your thoughts on a) whether or not bash should have ever gone that route, and, potentially more controversially, b) the current state of bash maintenance?
(I'm not going to ask about embedding multiple levels of ${...#...} or ${...#...}, because even if it worked, readability would tend to 0 very, very quickly. I'd rather be repetitive and readable and maintainable. Well. Most of the time.)
Website seems to be down or dead, consider using the archived link [1]. Also, since these are slides, it'd sure be nice to have the presentation on video form since a lot of context is missing from the slides. It does not seem to be available (well, it is from October 2006).
From the archived site's abstract [2]:
>"bash (and scripting languages in general) act as the glue that hold other system components together. This presentation will focus on the under utilized features of bash that are critical to building production quality scripts. Demos will show you how and why to turn these features on."
EDIT: seems the website being down is a me issue (?), but still, having the archives doesn't hurt.
---
You will like the final version of the traperr ERR function. It looks to create a "stack trace" type dump of where you are in the script when it fails.
This trope comes up every time bash is mentioned, without fail. Unless your team enjoys reading this, then no, a general purpose programming language is not a good replacement for what is effectively a "subprocess orchestration language":
from subprocess import run, PIPE
home = os.getenv("HOME")
try:
os.chdir(f"{home}/dev")
except FileNotFoundError:
sys.exit(1)
out = run(("git ls-remote https://github.com/%(repo)s refs/tags/%(tag)s"
% {"repo": shlex.quote("myorg/myrepo"), "tag": shlex.quote("v1.2.3+deadbeef")}).split(" "),
stdout=PIPE, stderr=PIPE)
The line noise is outrageous as compared to a language built for changing directories and running commands
That's not even getting into the pty portion, where the progress of the output isn't visible until the end of it, which is terrible DX
Nice rube goldberg machine. Apart from imports, that is equivalent to
repo = "myorg/myrepo"
tag = "v1.2.3+deadbeef"
out = check_output(["git", "ls-remote", f"https://github.com/{repo}", f"refs/tags/{tag}"], cwd=os.path.expanduser("~/dev"), stderr=PIPE)
Converting exceptions to exits is an anti-pattern that hides the source of the error, f-strings has been a thing for 10 years and argument quoting happens automatically. Progress of output isn't visible in bash either when you capture output with $(), unless you "tee", which opens up another can of "pipefail"-worms.
IMO, using something like Python or JavaScript is a good idea if any of the following criteria is met:
- Working with data more than executing other binaries
- Dominant data processing path is structured (e.g. JSON, XML, binary)
- Script requires complex flow control to be manageable (e.g. functions, if, case)
- Shell UI requirements that involve handling command-line options
- You find yourself going _deep_ into the docs for jq to get the job done
.
Conversely, BASH is better under the following circumstances:
- Orchestrating other binaries the majority of the time
- Simple jobs that work with newline-separated text lines, or all data can be comfortably "stringly typed"
- Launching and job control for other processes
- You just need to glue some other shell commands and binaries together
- Are just manipulating shell environment vars, or providing a custom user shell (e.g. Python venv `activate`)
- Need to wrap a CLI tool in a simple way
.
The minute you have to reach for BASH arrays, case statements, math... stop everything and seriously consider using a language that has stronger support for all that.
Since you’re the only other person in this thread to mention it, I’m surprised the industry has been sleeping so hard on QuickJS as a scripting language runtime.
I think the lack of batteries-included stdlib is probably the biggest holdup, but JS can be much nicer than python for anything that can be decomposed into map/reduce/filter string munging.
I agree. For me, the major stumbling block is what interpreters ship with your typical Linux distro. Python and BASH are usually just... there. Everything else just takes more steps.
yes. I think if you find yourself dropping to sed/grep/awk/find too much, switch to python.
The language itself imho seems more readable. Strings, lists and hashes seem better especially when you choke on quoting.
And the python standard library makes a huge difference when you grow above "simple". I love argparse for readable arguments with help. os.path lets you manipulate paths and filenames in a readable and robust way. you can read and write json, csv and more. datetime lets you manipulate dates and times.
Greg's BASH Wiki is the best resource that I've found for tips, tricks and discussion of the many, many mistakes that BASH almost encourages people to make.
Also, ShellCheck is an invaluable resource to use on all your scripts. When it complains about something, read up on why it doesn't like it and then decide to either fix your code or put in an exception to tell ShellCheck to ignore that particular issue for the next line.
historically, I did not write a lot of code in python - but since an LLM can, now i especially have no problem doing more scripts in it. it's so much more scalable than bash and pretty much ubiquitous.
What an interesting, eye-catching subject matter that certainly can bring an old programmer's attention to more productive endeavours than messy ideological discussion.
Linux distribution maintainers believe Bash is an inferior shell for scripting. It may be a superior shell for interactive use, depending on personal preference.
No one is prevented from using whatever shell they prefer for whatever purpose. For example I prefer to use a smaller, slightly modified scripting shell for interactive use. Some folks prefer to use a large, interactive shell for scripting.
Choice of scripting shell is personal preference. Preferences may differ amongst computer owners. For example, I prefer NetBSD sh. On Linux I use a slightly modified Dash.
As for the preferences of others, I note the default scripting shell on the largest Linux distributions is not Bash. It is of course Dash, which is derived from NetBSD sh, which is derived from the Almquist shell (ash). Indeed, ash is a popular choice for scripting. It is smaller and faster than Bash.
The most valuable skill in bash scripting is to know when to stop writing bash and switch to a proper language before the script turns to unmaintainable chunk of unmanageable complexity.
I was a co-op at a company in the 90s that specialzed in mechanical simulation of virtual prototypes. The software was primarily used on Unix (most of us used SGI boxes) but there was also a Windows port for smaller jobs and laptop use and (at least the dev environment) was all built on Cygwin.
I've been on a longish search for a bash replacement (or at least "extra substitute") in the realm of "more complex commandline scripting" (functions, utility scripts, etc.). Naturally, every option is a set of different compromises.
I think I've finally settled on Lua- It's small, coherent, surprisingly fast (especially with LuaJIT) and has the "virtually no startup cost" I was looking for (which ruled out, for example, Elixir for a lot of things).
And, I've finally figured out how to bang my nix-darwin flake.nix config in a way that gets it AND a bunch of popular libraries installed together and all seeing each other.
Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.
GNU Guile [0] is pretty cool.
[0] https://www.gnu.org/software/guile
Is Guile really that good at the shell scripting "thing"? Like, the bash superpower is that it makes it extremely easy to run subprocesses in incredibly flexible ways (in the background, foreground, in pipelines, with stdout/stderr redirected to files, etc. etc.). Can Guile do that smoothly? I generally find that doing things like this is way more awkward in most "real" programming languages compared to shell scripting languages.
I know I’m in my own little corner, but stringing process together is the shell/unix superpower.
The “Unix way” stands on the shoulders of stdin, stdout, pipe, and fork.
And the shells make that a first class concept. Breaking up the command line and linking processes together is the #1 job of a shell, everything else is gravy.
The scheme and lisp shell takes fail here, just because you have to go through the gymnastics of syntax.
My fantasy would be to have something like this:
Piping system commands into my own functions within the environment.Sure, I could do:
But then I can do that with anything, not just scheme (there’s that Unix superpower again).So basically using the shell of something like Xerox's PARC Interlisp-D and threading macros.
I would be something like this,
Followed by "Select => Execute" with the mouse, or "Execute last form" with keyboard shorcut.You also have to go through gymnastics of syntax to write a shell program that anyone could reliably use on any data, with any file names.
How about we parse digit sequences out of the path names and combine them with the originals, then sort on those digit sequences turned into integers: csort is caching sort: it caches the result of the key function through which the elements are projected to form the argument of the comparison. In other words, so we don't wastefully call get-nums multiple times for the same path.I guess we should distinguish "the shell environment" (which might stay Bash) with "the shell script language" (which could be any one of a number of options, just use the right hashbang header)
One that strove to do both WOULD have to do all the things you suggested as well (job management, stdout/stderr redirection etc.) So options like oil shell, nushell, fish etc.
I was just talking about "a commandline tooling language" in general, as far as use-cases. And for that, Lua (and Moonscript, which compiles to Lua but is way nicer and has been featured on HN a few times over the years) seems to fit the bill (1-based indexing aside, ugh)
I love that in Guix, Guile is used as the configuration language all the way "from the metal" (the bootloader) to the declarative environment config in userspace.
I'm one of those folks that still can't do LISPs though. If there was a way to replace some or all of the parens with significant indentation instead (which I believe Racket makes possible... but that only works in Racket), I might reconsider it.
You probably heard this one before. After a while the parens do disappear. When conventionally indented, lisp users see only the indentation and not the parens.
With structural editing, no one ever have to match parens or count parens at the end...
YMMV ofcourse.
Can you expand on using Guile for shell scripting?
I was aware of it, but not familiar until I looked at the link you shared. Looks like a Scheme, which for the uninitiated is a different approach for shell scripts.
Yes, there are some basic examples here [0] – otherwise it's just shebang and you're good to go; it can be a rabbit hole where you start liking it too much than what you really should (you wanted simple scripts, now you are finishing scheme book).
It's consistent, "infinitely" terse (you can create dsl-like apis for your problem domains), proper programming language.
First time I used it for scripting was when I had to process some xmls and do bunch of processing and gluing together – I was impressed how quick and easy it was without knowing anything about it and how readable the whole thing ended up being.
[0] https://www.gnu.org/software/guile/manual/html_node/Scriptin...
The most interesting ones I’ve come across are:
- Elvish https://elv.sh
- Murex https://murex.rocks
- Oil https://oils.pub
Same. Do you use any of them or have you stuck with any of them? Opinions?
I’m the author of Murex, so I’ve definitely stuck with that. It’s been my primary shell for 6 or 7 years now. The only time I touch Bash is the odd occasion I SSH into a box.
However the other shells are also highly polished. I’d definitely recommend each and all of them.
I write a lot of JS/TS for my day job, so zx (https://github.com/google/zx) has been a nice tool for bash scripts that start getting a little too complex.
I think it kinda was one of Perl’s goals at the time. But the syntax is rather unusual.
I keep wanting to get into Raku. It has such a rich pedigree of everything learned from decades of Perl, in a clean slate. It seems to suffer from lack of adoption but it seems better than *sh in every way except for ubiquity.
Indeed Raku is a delight to use for 'shell scripting'.
Perl was originally written as an amalgamation of grep,sed,tr, awk and I am sure a few more unix utilities with their own mini languages. The idea was to use one language instead of tying together half a dozen mini-languages in a shell script. And it worked really well. Perl being a demon with text munging didn't hurt.
Raku keeps this heritage but adds so much more (for better or worse :-) ). It inherits ideas from Lisp and functional programming languages. The thing that impressed me was, how easy it was to use concurrency.
It is much more complicated than Perl. Every feature you could ever want, in multiple ways, it seems like. Perl isn’t too complicated or large of a language.
I try Perl 6/Raku every few years, so it seems like I've tried it half a dozen times by now since it started. I like some things about it, but in a way it seems like an academic project, more suited for experimentation than doing work. I always come back to Perl, which has been my favorite hammer for 30 years.
As someone old enough to have been a Perl user during its glory days, I wonder what is unusual about it, it feels right at home in UNIX.
Have you tried NGS (https://ngs-lang.org)? I found its approach of giving you that power while remaining shell-focused to be very refreshing.
Thanks for the kind words. I phrase it as "fully fledged programming language with domain specific facilities". Hope it makes sense.
Somebody ought to package it for Linux OSs. I filed a ticket.
It has similar aims to https://www.nushell.sh/
interesting.
I've actually already written a lot of the functionality I see here as bash (and other) functions, lol (things like log, filter, debug, retry, repeat, map, fetch (a URL, with retry)... etc.). (things like: enumeration functions, table pretty-printing, etc.)
Hi. Author here.
> I've actually already written a lot of the functionality I see here as bash (and other) functions
That's exactly the reason these are part of stdlib. No reason for everybody to reimplemnt these with slight variations and copy it around.
> Wondering what others have found to address this. Or if they just stuck with Bash, since it's just not going away anytime soon.
If you're familiar with Python, give xonsh (https://xon.sh/) a go. It's a Bash-like shell but the syntax is Python. It has the same ease of writing shell scripts as Bash does, sans the insane language.
Been using it since 2018.
The nicest thing, I think, is that you can make a new shell command simply by writing a python function.
How does the startup time and execution speed of Python compare to other options?
(I'm not particularly inclined to Python, unfortunately, due to the dependency issues and some personally negative opinions about its design... but I like the idea of Xonsh if you're a Python person! I wish I had something like that for Elixir!)
Obviously slower, but since when is the speed of a Bash script a critical factor?
Lua is my favourite, too, but i found writing in awk to be more pragmatic.
So Awk for everything (or in my case gawk or frawk)?
Or just text-processing things, where it's most suitable?
(I think Awk is underrated, for sure!)
We've transitioned all of our Bash scripts over to Ruby and are pretty happy with it. We almost exclusively use only Ruby's standard library. 'fileutils' and 'rake' have a bunch of nice functions to replace Bash scripts.
See, FileUtils for things like rm_rf, rake provides sh which can execute commands, then there's FileList and Pathname. It even comes with its own template engine (erb) if you need that. Regexes as first class citizens are awesome for shell scripting.
I can also recommend using optprase + ostruct for commandline argument parsing.
I usually go with python when my .sh/.bat script files gets too complex. So I guess we wento down the same road.
Like ruby (I guess), python is very pleasant to work with for small scripts and it's portable between linux and windows, which is really cool (at least in my use cases)
In python, glob, Path and subprocess are mostly all I need...
I found https://github.com/cosmos72/schemesh lately and - once you accept LISPism - is pretty nice to use.
I moved to using perl first, but these days I'm mostly using python except where regular expressions are needed in which case I'm still using perl
I find python is not a natural fit for shell script replacement (may be I am doing it wrong).
Executing a process and bringing back status, stdout and stderr seem tedious. While in Perl, it is as natural as it is in a bash script (well almost).
Raku on the other hand is _Fun_.
coming back to raku after basically any other language is a breath of fresh air. i haven't even been worrying much about when the ast stuff lands cause i can't think of anything i would even want to mess with that much, even though i seem fundamentally incapable of engaging with lisp/prolog/forth without giving in to the siren song of metaprogramming. to me that's a very strong sign of how much it gets right as a base language.
i do all my automation with one tiny 20 line raku script that manages a thousand things. hell, the entire monitoring site for our frog is basically just a pile of mixins with a neat little pipeline interface unifying everything from request profiling and html templating to reading from and filling that cache in parallel whenever it remotely pulls the sensor data off various things and gets the current webcam frame.
Same but I found it a really bad choice for a Bash replacement
You have to reinvent the wheel every other day since the standard library doesn't come with much included
I see Penlight (which is Python-flavored, but also has some functional programming constructs) https://github.com/lunarmodules/Penlight , Moses if you like functional programming https://yonaba.github.io/Moses/, likely others, but you're right in that there's no standard, "opinionated" stdlib. No bigint support on LuaJIT etc.
I'm an incredibly happy user of nushell, which brings all the best features of other shells terse pipelining syntax and all the best features of more well designed scripting languages (functions, modules, lexical scope, data structures, completely optional types) in one awesome package that also comes with editor (LSP) support and excellent documentation
https://www.nushell.sh/
(The intro page may be a bit misleading. You can freely mix-and-match existing, unstructured as well as nushell-built-in structured commands in the pipeline, as long as you convert to/from string streams - its not mandatory to use the structured built-ins. For example if an existing cli tool has json output, you can use `tool | from json` to turn it into structured data. There are also commands like `detect columns` that parses classic column output, and so on - the tools to mix-and-match structured and unstructured data are convenient and expressive)
Some highlights:
- automatic command line arguments and help by defining a main function and adding comments to each argument - e.g. https://github.com/nushell/nushell/discussions/11969
- run commands with controlled parallelism: https://www.nushell.sh/commands/docs/par-each.html
- easy parsing of raw input https://www.nushell.sh/commands/docs/parse.html
- support for a wide variety of data formats https://www.nushell.sh/commands/categories/formats.html
- built-in support for talking to SQLite databases: https://www.nushell.sh/book/loading_data.html#sqlite
edit: it looks like Mitchell Hashimoto was recently impressed too https://x.com/mitchellh/status/1907849319052386577 - rich functional programming library that blends with pipeline syntax https://www.nushell.sh/book/nushell_map_functional.html
Addendum: Its not my login shell. I run it ad-hoc as soon as the command pipeline i'm writing starts getting too complicated, or to write scripts (which of course can be run from outside nushell too, so long as they have the correct shebang)
NetBSD's base system includes both the best scripting shell, ash, and lua. Thankfully, no python, no startup cost.
if bash/sh scripts get too long (or long-lived), I switch to python when possible.
As a minor example, I hate having to remember (or explain) ${var##pattern}
It is useful for manipulating paths, but it is cryptic.
in comparison, something like:
IMHO is more self-explanatory and robust.Also, argparse is the best.
That's because you don't use the native lib pathlib from python ! It was a delivery when I knew about it. For example, to join paths, you can do so much easily, see the section "operators" of this link : https://docs.python.org/3/library/pathlib.html
The mark of an advanced shell scripter is knowing not to do advanced shell scripting.
(This statement primarily applies to existing stringly-typed scripting languages, which are nightmarish to maintain and debug. PowerShell, nushell, or similar solutions have a much higher complexity ceiling.)
2025 update of this 2006 presentation has a better title, https://news.ycombinator.com/item?id=43714928
Thank you for linking this.
I really want to reach for that pun and suggest: "Seat Belts and Airbags for bash safety" as an even better title.
Are you bashing bash?
All day, every day. And that's coming from someone that _likes_ working with it.
Aren't TCL and Perl stringly-typed scripting languages?
No for Perl, in Tcl kind of, it changed in Tcl 8, already long time ago.
http://www.ira.inaf.it/Computing/manuals/tcl/man-8.0/Changes...
You won't catch me defending them ;)
Yes - at least in Tcl, _everything_ is a string
Not everything, and this already changed a while back.
> The core of the Tcl interpreter has been replaced with an on-the-fly compiler that translates Tcl scripts to byte codes; a new interpreter then executes the byte codes. In earlier versions of Tcl, strings were used as a universal representation; in Tcl 8.0 strings are replaced with Tcl_Obj structures ("objects") that can hold both a string value and an internal form such as a binary integer or compiled bytecodes.
http://www.ira.inaf.it/Computing/manuals/tcl/man-8.0/Changes...
> This statement primarily applies to existing stringly-typed scripting languages
IMO it applies to all languages. Fancy language features usually make for poor maintainability, e.g. metaprogramming.
New readers might enjoy the tale of Emperor Sh and the Traveller: https://sanctum.geek.nz/etc/emperor-sh-and-the-traveller.txt
I have followed the advice in this story for all my scripts. They are just better interfaces to collection of tools that already exists, not new tools. I write them when the subproblems are already solved by specific programs and the coordination does not require complex data transformation.
I am the author of this presentation. Feel free to ask me questions and send me corrections to the pdf and youtube video.
Here is are the slides: http://uniforumchicago.org/slides/bash_2025-03-25.pdf
Here is the recording of the video: https://www.youtube.com/watch?v=DvDu8_A2uhs
Here is the stringent.sh library that is shown in the presentation: https://github.com/pottmi/stringent.sh
Great presentation. I've been writing bash for waaaay too long, and, while I nodded and said yup, yup, a few times, I also learned some new tricks, so thank you!
The social graces having been observed, if you were to update this presentation to reflect modern bash, and especially reference variables, what would be your top 3 additions?
(I am a big fan of reference variables. They make it possible to, among other things, do some funky things that would otherwise require liberal use of eval, which I eschew. Figuring out how to ensure that the target of a reference variable was in fact an associative array while running under `set -u` was a trick of which I am proud. Note: what I did works and isn't totally ugly, there may be a better way, YMMV, IANAL, etc.)
Second Q, if you will permit: Given the memory leaks in associative array management that have been corrected over the last several years, what are your thoughts on a) whether or not bash should have ever gone that route, and, potentially more controversially, b) the current state of bash maintenance?
(I'm not going to ask about embedding multiple levels of ${...#...} or ${...#...}, because even if it worked, readability would tend to 0 very, very quickly. I'd rather be repetitive and readable and maintainable. Well. Most of the time.)
Is there an https link to this pdf? So rare to see http nowadays.
I contacted the guys that run the site and asked them to add https.
I will throw the pdf on my gitlab account that has the stringent.sh script.
https://github.com/pottmi/stringent.sh
Thanks for carrying a 20 year torch on bash maintainability!
Website seems to be down or dead, consider using the archived link [1]. Also, since these are slides, it'd sure be nice to have the presentation on video form since a lot of context is missing from the slides. It does not seem to be available (well, it is from October 2006).
From the archived site's abstract [2]:
>"bash (and scripting languages in general) act as the glue that hold other system components together. This presentation will focus on the under utilized features of bash that are critical to building production quality scripts. Demos will show you how and why to turn these features on."
EDIT: seems the website being down is a me issue (?), but still, having the archives doesn't hurt. ---
[1]: https://web.archive.org/web/20241212102046/https://uniforumc...
[2]: https://web.archive.org/web/20240719092417/http://uniforumch...
> it'd sure be nice to have the presentation on video form
March 2025 presentation by the same author, "Seat belts and Airbags for bash"
slides: http://uniforumchicago.org/slides/bash_2025-03-25.pdf
video (87m): https://www.youtube.com/watch?v=DvDu8_A2uhs
Here is a https link to the presentation and the stringent.sh library:
https://github.com/pottmi/stringent.sh
Not down, just not on https. http link works just fine, but browser redirects to https by default.
One of the first things I do is turn on errexit and pipefail options, but trapping ERR and combining with LINENO to catch silent exits is brilliant!
You will like the final version of the traperr ERR function. It looks to create a "stack trace" type dump of where you are in the script when it fails.
See it here: https://github.com/pottmi/stringent.sh
Isn’t it better to replace bash scripting with Python or similar, unless the script is simple?
This trope comes up every time bash is mentioned, without fail. Unless your team enjoys reading this, then no, a general purpose programming language is not a good replacement for what is effectively a "subprocess orchestration language":
The line noise is outrageous as compared to a language built for changing directories and running commandsThat's not even getting into the pty portion, where the progress of the output isn't visible until the end of it, which is terrible DX
Nice rube goldberg machine. Apart from imports, that is equivalent to
Converting exceptions to exits is an anti-pattern that hides the source of the error, f-strings has been a thing for 10 years and argument quoting happens automatically. Progress of output isn't visible in bash either when you capture output with $(), unless you "tee", which opens up another can of "pipefail"-worms.This falls under the case “unless the bash script is simple “.
If you consider more complex tasks, bash quickly gets unwieldy, and the situation is reversed.
Pythons has a lot of advantages, including full flexibility, error handling etc.
In xonsh[0], that would be:
(Not sure about the equivalent of shlex.quote, but in the worst case, you can just use "from shlex import quote as q" or something).So yes, there are good alternatives to bash - even Python based.
[0] https://xon.sh/
IMO, using something like Python or JavaScript is a good idea if any of the following criteria is met:
- Working with data more than executing other binaries
- Dominant data processing path is structured (e.g. JSON, XML, binary)
- Script requires complex flow control to be manageable (e.g. functions, if, case)
- Shell UI requirements that involve handling command-line options
- You find yourself going _deep_ into the docs for jq to get the job done
.
Conversely, BASH is better under the following circumstances:
- Orchestrating other binaries the majority of the time
- Simple jobs that work with newline-separated text lines, or all data can be comfortably "stringly typed"
- Launching and job control for other processes
- You just need to glue some other shell commands and binaries together
- Are just manipulating shell environment vars, or providing a custom user shell (e.g. Python venv `activate`)
- Need to wrap a CLI tool in a simple way
.
The minute you have to reach for BASH arrays, case statements, math... stop everything and seriously consider using a language that has stronger support for all that.
Since you’re the only other person in this thread to mention it, I’m surprised the industry has been sleeping so hard on QuickJS as a scripting language runtime.
I think the lack of batteries-included stdlib is probably the biggest holdup, but JS can be much nicer than python for anything that can be decomposed into map/reduce/filter string munging.
I agree. For me, the major stumbling block is what interpreters ship with your typical Linux distro. Python and BASH are usually just... there. Everything else just takes more steps.
don't listen to the others.
yes. I think if you find yourself dropping to sed/grep/awk/find too much, switch to python.
The language itself imho seems more readable. Strings, lists and hashes seem better especially when you choke on quoting.
And the python standard library makes a huge difference when you grow above "simple". I love argparse for readable arguments with help. os.path lets you manipulate paths and filenames in a readable and robust way. you can read and write json, csv and more. datetime lets you manipulate dates and times.
yes, unless you need it to run somewhere that you know only has bash.
Greg's BASH Wiki is the best resource that I've found for tips, tricks and discussion of the many, many mistakes that BASH almost encourages people to make.
https://mywiki.wooledge.org/BashGuide
Also, ShellCheck is an invaluable resource to use on all your scripts. When it complains about something, read up on why it doesn't like it and then decide to either fix your code or put in an exception to tell ShellCheck to ignore that particular issue for the next line.
https://www.shellcheck.net/
+1 for shellcheck recommendation, it's fairly essential if you want to be good at Bash
Is shellcheck written in Haskell?
Yes, it's mentioned on their homepage - "written in Haskell, if you're into that sort of thing."
https://www.shellcheck.net/
Please, don't do any advanced shell scripting, period. I've had enough :(
historically, I did not write a lot of code in python - but since an LLM can, now i especially have no problem doing more scripts in it. it's so much more scalable than bash and pretty much ubiquitous.
15 years ago my 'mentor' taught me
buggy_command || :
and 15 years later i am still begging people not to litter our deployment scripts with this pattern. i call this "the penny in the fusebox"
That. Is. Evil.
Nuke it from orbit, or something like that.
What an interesting, eye-catching subject matter that certainly can bring an old programmer's attention to more productive endeavours than messy ideological discussion.
Anyone knows a good book for bash scripting 5.x ???
Linux distribution maintainers believe Bash is an inferior shell for scripting. It may be a superior shell for interactive use, depending on personal preference.
No one is prevented from using whatever shell they prefer for whatever purpose. For example I prefer to use a smaller, slightly modified scripting shell for interactive use. Some folks prefer to use a large, interactive shell for scripting.
Choice of scripting shell is personal preference. Preferences may differ amongst computer owners. For example, I prefer NetBSD sh. On Linux I use a slightly modified Dash.
As for the preferences of others, I note the default scripting shell on the largest Linux distributions is not Bash. It is of course Dash, which is derived from NetBSD sh, which is derived from the Almquist shell (ash). Indeed, ash is a popular choice for scripting. It is smaller and faster than Bash.
Any recommendation on small shells? ash?
60 slides from a bash workshop?
It is not how to code in bash, it is how to avoid the bash pitfalls.
The most valuable skill in bash scripting is to know when to stop writing bash and switch to a proper language before the script turns to unmaintainable chunk of unmanageable complexity.
[flagged]
Strange that PowerPoint was used for the slides. That's pretty much at the complete opposite end of the software spectrum to GNU Bash.
The slides look to me a lot closer to laTex beamer than PowerPoint.
Edit: I take it back, the PDF metadata says:
Creator: PowerPoint
Producer: Mac OS X 10.4.8 Quartz PDFContext
It was done in keynote on a mac in 2006. I still maintain the 2025 version in keynote on a mac.
Maybe it was written on OpenOffice or something and saved to ppt(x)? The linked file is PDF though.
Thanks to WSL, bash is ̶a̶v̶a̶i̶l̶a̶b̶l̶e̶ ̶o̶n̶ shipped with Windows.
Git also ships bash for windows and doesn't require WSL. Can be even set as a default shell for OpenSSH so you can just ssh into windows bash...
bash has been available on Windows since at least before the mid '90s.
Yes I remember using cygwin back in 2004ish to provide bash when I had to use a windows pc at work (very briefly)
I was a co-op at a company in the 90s that specialzed in mechanical simulation of virtual prototypes. The software was primarily used on Unix (most of us used SGI boxes) but there was also a Windows port for smaller jobs and laptop use and (at least the dev environment) was all built on Cygwin.