Hacker News new | past | comments | ask | show | jobs | submit

A shell colon does nothing. Use it anyway

https://refp.se/articles/your-shell-and-the-magic-colon
Articles like this are fun but they all come from posix shell syntax being fundamentally bad for scripting/programming. All the piping stuff is great, of course. And the overall ecosystem is great. But the interpretation of the script itself working by a series of string substitutions is a mechanism we wouldn't accept in a regular programming language. And there's no excuse for it really, except that shell syntax is really, really old.

For example, what does `$foo` mean in shell syntax? In any reasonable language (perl or powershell, for example, or python if you drop the `$`), it's an expression that evaluates to whatever value's inside that variable. In shell, `$foo` isn't an expression in that sense, and what it does depends on what's inside it via a variety of string substitution rules.

This is the main reason we have arcane articles like this.

That said, nice article.

I feel the need to push back on this perspective.

Old doesn't imply arcane. It instead can (and does, in this case) mean that it won out over the course of decades against other less worthy alternatives.

Although you can write "programs" in sh, the shell syntax was never meant to be anything like a systems or application programming language. It was meant to efficiently automate systems tasks. It is kind of like the fork lift of computing. It does it's job very well, is bad at doing anything else, yet no other kind of vehicle can do it's job nearly as efficiently.

Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!

loading story #49058951
Your `$foo` example confused me at first because I thought the backticks are part of the example.
I will immediately remove all the shell scripts on my system based on this comment alone.

Posted from my phone since my system came crashing down.

Yes a large percentage of bash scripts fails if you feed them filenames with spaces or other special characters. Or a large amount of filenames near the Unix maximum commandline length.

Bash et al are great for command line use, but produce a situation of really bad engineering hygiene when used in scripts.

Do not use. Stay away. Bash et al considered harmful.

Well that's exciting. I learned a lot of uses for ":" today.

However, the only one I already knew...

    if some-command; then
        :                            # command required
    else
        echo "command failed"
    fi
I used to do that until I learned of

    if ! some-command; then
        echo "command failed"
    fi
It's in the POSIX standard so it's not just a bashism: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...

> If the pipeline does not begin with the "!" reserved word, the exit status shall be the exit status of the last command specified in the pipeline. Otherwise, the exit status shall be the logical NOT of the exit status of the last command

loading story #49056528
loading story #49057788
I am not a huge fan of most of these, but a few do seem useful.

    : "${1:?missing argument, aborting!}"
I wouldn't use this because I would want to give $1 a name for the rest of the script, so I would assign. But it can be a nice way to give a clear error for missing required environment variables.

Many of the others (like truncating files) are probably more clearly written with dedicated commands, but may come in useful if you are going to extreme lengths to avoid dependencies outside of the shell.

loading story #49055140
Agreed; author does note

> Why use the null-command when I could do VAR=${VAR:-default-value}?

and points out it's one less thing to typo, but that assumes the name is the same; I like e.g.

  TARGETFILE="${1:?need input file}"
  OPTIONALVAL="${2:-defaultvalue}"
loading story #49055153
loading story #49059619
loading story #49055651
loading story #49055160
I use the colon as EDITOR with Git when I want to do an interactive rebase combined with auto squash without having to edit the todo list.

I have an alias[1] for that which I call a quick interactive rebase:

    riq = -c sequence.editor=: rebase --interactive
[1]: https://github.com/fphilipe/dotfiles/blob/94f2ff70bade070694...
loading story #49056473
loading story #49056563
I actually did absolutely need it recently when I golfed together a shell script that is simultaneously a valid YAML file [0]. Sometimes having no-op tokens is nice!

[0]: https://domi.work/blog/posts/compose_polyglot/

Why take a perfectly readable if-statement and turn it into something, 99.9% of people would need to lookup. Concise != better. You can make it one line with:

    [ -z "$1" ] && { echo "missing argument, aborting." 1>&2; exit 1 }
loading story #49056608
loading story #49056476
loading story #49060439
My personal favourite use of the colon command is what I've written about some years ago: https://johannes.truschnigg.info/writing/2021-12_colodebug/
This "hidden knowledge" is fun to read but pain to remember and use, especially when working with multi-platform environments //

Switched from bash to plain python scripts for shell stuff everywhere several years ago, and never looked back into bash zoo anymore. Stable syntax across Win/Mac/Linux, no bash/zsh/msys2 obscure differences, normal errors and Clause writes scaffolds quick and flawless anyway

loading story #49056970
Off-topic, but I am reminded of Larry's First and Second Laws of Language Redesign (which Larry Wall discovered/stated when he designed Perl 6, which is now Raku):

    1. Everyone wants the colon.
    2. Larry gets the colon.
Nice one! I love those weird bash tricks.

Some of the examples here are interesting, but they show parameter substitution more than colon itself: https://tldp.org/LDP/abs/html/parameter-substitution.html

In small scopes, I tend to inline the `:?` validation inside the arg of the command. `echo "${1:? first param required}"`

Another usecase is to use colon in the body of a while loop, while doing work in the condition of the loop.

    while rlwrap -o -S'>> ' tr a-z A-Z ; do :; done
Gives you the "do X while it succeeds. stop when it returns non-0" semantics.

I've also written about this and other bash tricks over the years in https://github.com/kidd/scripting-field-guide/blob/master/bo.... You might like them :)

loading story #49056754
I have probably the worst use case, but I like it. I have a very specifically structured ZDOTDIR, and I write everything in a way that is self-documenting.

After a while, you probably know more commands and utilities than you know what to do with, and you'll forget they exist when you need them. In order to not waste time looking for a program for a particular and infrequent purpose, I create "do nothing" aliases like `: alias f3probe` so I can realize I just forgot I already have something for it. Nice predictable pattern to grep with `^: alias` to look through all of these.

life is way too short to deal with this nightmare of a language and its 50000 footguns for anything longer than a 2 line script, especially in the age of LLMs. Just write a python/TS/any real language script instead. Bash is great for the command line, it should be limited to use there.
loading story #49055049
loading story #49055425
loading story #49055460
loading story #49056809
loading story #49055296
What if there was a less cryptic way to have mandatory arguments, something harder to get wrong, like

  param(
    [parameter(mandatory)] $name
  )

  "Hello, $name!"
And then:

  $ ./script.ps1 Dave
  Hello, Dave!

  $ ./script.ps1 -name Dave
  Hello, Dave!

  $ ./script.ps1
  cmdlet script.ps1 at command pipeline position 1
  Supply values for the following parameters:
  name: <cursor here>
Or non interactively:

  $ pwsh -nonint ./script.ps1
  script.ps1: Cannot process command because of one or more missing mandatory parameters: name.
loading story #49055957
loading story #49055929
loading story #49056159
I always wanted to learn more about scripting. But today I am not as passionate as before because LLMs write working scripts most of the time. I am wondering if it is true for most programming techniques and quirks. Are we going to write code to solve low level problems?

The other day there was a blog post about learning SIMD. I think in future "programmers" will just nag about the speed of the program and the coding assistant will eventually introduce SIMD to the source code.

It is a little sad but we have to go with the current if we want to survive.

loading story #49057372
loading story #49059127
Good to know, but looks less readable than `if` example.
I often use : to set default values of configuration envs. e.g, in my dotfiles bootstrap script I have:

  : "${DOTFILES_PATH:=$HOME/.dotfiles}"
Which will use $DOTFILES_PATH value if it's set, otherwise it's going to be $HOME/.dotfiles
> ( : < dataset.json ) && echo YES # is dataset.json readable?

The subshell execution parentheses and the colon are superfluous here, just:

   < dataset.json && echo YES
Redirections do not require a colon command to hang off of, and there is no need to fork a subshell to execute such a command.

> ( : >> result.json ) && echo YES # is result.json writable?

As a go-to idiom for a writability test, it gives me pause. If the file didn't exist, we created a zero-length one. That might be okay if we are going to write to it anyway as the next action.

If we are testing because we intend to overwrite it, why not just "> result.json" (which is by itself an idiom for truncating a file to zero length).

When would we every do this? Maybe before some command which takes the file name as a destination file argument rather than using output redirection, and which performs a lengthy computation before trying to open the file for writing. We can catch the permission error early.

I don't think I've ever coded such a test; normally you just do the operation that writes to the file and let that fail.

In POSIX C, there is a function access() for doing these kinds of tests. But it has a special purpose: it is meant to be used by a setuid root process to perform a permission test as if it were the real user/group (the one which elevated privilege to root). I.e. it's not can we do this operation, but should we do this operation (would we still be allowed, if we dropped privileges back to the original user).

They are NOT superflous, and all you need to prove it is `zsh` (but there are others that follow suit in similar fashions):

  zsh% echo "hello world" > data
  
  zsh% < data && echo "READABLE"   # <- will print the contents of data
  hello world
  READABLE
  
  zsh% : < data && echo "READABLE" # <- this will not
  READABLE
So, if you want something that "everyone can use" without going into details about the difference between commonly used shells.. you'd use the null-command.

---

and given that we use the null-command, it _WILL_ behave different with or without subshell.. and all you need is `bash --posix` to prove it:

    % cat subshell.sh
    #!/bin/bash --posix
    ( : < missing.json ); echo AFTER    # <- will echo AFTER
    
    % ./subshell.sh
    ./subshell.sh: line 2: missing.json: No such file or directory
    AFTER
    
    % cat no-subshell.sh
    #!/bin/bash --posix
    : < missing.json; echo AFTER        # <- this will not
    
    % ./no-subshell.sh
    ./no-subshell.sh: line 2: missing.json: No such file or directory

    % : ^- apparently.. there is a difference

The output above is not truncated, `no-subshell.sh` will stop executing due to the broken read.

---

One should never trust things just because they are written, but that also applies to comments on HN. Originally when I read your message I actually thought I made a mistake, I was very close to writing an apology comment and adding a note to the blog post, but not close enough - I had to test it again.

I'm thankful for the watchful eyes and scrutiny when reading things online, that's good - keep it up, but your message is factually wrong - on so many levels.

{"deleted":true,"id":49055670,"parent":49055636,"time":1785051616,"type":"comment"}
It’s very common that the most upvoted comment on a post is a mistaken correction. People seem to love comments seemingly proving that the post is wrong without actually checking anything. And once it has enough upvotes a big discussion may follow up with personal attacks on the author or other questionable comments, while the people trying to point out that the post actually is accurate get either buried under the critic storm or downvoted to oblivion because people just don’t like the truth. This happens in every controversial topic.

One of the reasons I stopped writing.

I have never been in this situation before but I do agree with you, and honestly it feels very bad. I saw the comment when it was posted and, perhaps naively, thought "oh well, no one will care - its just another fluff comment".

Fast-forward to seeing that comment climb up the ranks, posted by a person who has 270x my karma, who seemingly gets upvotes by just.. writing things? no proof? no rationale? nothing?

Yeah, feels bad. I'm super happy so many are enjoying the article, and this situation can't take too much away from that, but man.. discouraging for sure.

Don’t worry, everyone who reads the comment will read yours as well, which is itself interesting.

> Yeah, feels bad. I'm super happy so many are enjoying the article, and this situation can't take too much away from that, but man.. discouraging for sure.

Again, don’t worry too much about that. The story got upvoted to the front page and lots of people will read it, that’s what matters. Not that a misguided post got a bunch of upvotes.

{"deleted":true,"id":49055674,"parent":49055440,"time":1785051644,"type":"comment"}
the colon does nothing, which makes it the only bash command an LLM can't over-engineer
loading story #49057486
there are some early unix tapes floating around, and in those early shells, i'm fairly certain the colon was one of only two special-cased code paths after the command line was parsed. does anyone recall more specifically?
loading story #49056571
I really hate bash because of its unreadable syntax, and this does not help it in any way.

We have some large bash scripts in my company, ~10,000 LOC spread across multiple files, all sourcing each other and what not. It is truly hard to read bash, which means it is truly hard to maintain bash, which means that when the one person knowing the bash scripts in your company goes away, you're in for some "fun".

My point is, these quirks are not useful, except for some bash enthusiasts.

I've read through all of the examples in the article & they all seem to serve to sole purpose of turning readable multiple line code into one-liners.

One-liners are a cool little artifact of early shell culture & are sometimes still useful today if they're short to avoid the readability problems of `/` when copy pasting a quick shell command to run, but they have no place in scripts.

None of this seems useful to me.

> if you are like me and prefer less typing (gotta go fast)

Yeah, no.

loading story #49055863
loading story #49057868
I usually skip the get option builtin, and use : as...

while : ; do case "$1" in "") break;; -f|-foo) shift; whatever;; *) usage; exit 1;; esac done

For this... instead

  if something; then 
     true
  else
     echo ERROR
     exit 1
  fi 
Using : would be too much here.

For anything else including json etc. I usually go to duckdb. Awesome support, single file install, readable, easy to maintain.

Powershell on Linux or Unix? Just another huge dependency if you manage 1000s of machines, and good luck finding a Linux gal/guy wanting or able to touch pwsh without chemical grade gloves.

loading story #49060114
Thank you for this.

Also, I'll never use it.

Because a language feature that needs marketing is against readability, among those in my target audience who have not yet read the marketing.

I need my shell scripts to be long enough to explain to my audience exactly what they are doing.

> Though.. what if I told you the above four lines could be replaced by just... one?

I'd reject the pull request. Bash is already bad as programming language (the goodness of language for long code is inversely proportional to how nice it is for shell one-liners), this is just turning "bad" into "line noise"

If your bash script takes more than one screen, rewrite it in Python, hell, rewrite it in Perl, even that's better

loading story #49056122
the truncation is a poor example.

- it creates the file if it does not exist, not merely truncate. as a tutorial kind of blog post this incomplete description matters IMO.

- it would work the same without the colon (similar for default variable assignment examples). we generally strive not to have "extra" things, like useless use of cat.

- educationally it's useful to demonstrate that redirection, like parameter expansion, works before the command executes (the null command in this case), but the article doesn't explain that at all!

otherwise i <3 this article. some uses of colon i had never thought of or seen before. like file truncation, not sure i'd use them but it was cool to see them.

loading story #49055190
loading story #49055379
Prefer

  if x then :; else something; fi
over

  if ! x; then something; fi
Really? Colon is the appendix of the shell.
loading story #49055889
Aside: hacker news doesn't do markdown code blocks, but instead uses 2 spaces before text;

   if x then :; else something; fi
loading story #49055479
loading story #49055246
loading story #49055275
For decades I used bash and never knew and saw this syntax. And recently discovered that by a code generated by Claude.

And now I see this article. So I guess that it is a construct suddenly popularized by llm.

Cool!

Btw, can the sequence :? be called "reverse Elvis" ?

loading story #49057606
Maybe it's just the fact that I woke up 10 minutes ago, but the readability of this looks awful. Even more than just usual shell scripts.
loading story #49059328
I've never liked cleverness in scripting when clarity costs effectively nothing.

This is one of those clever things, similar to people using perl5 use trinary expressions. Like, are you TRYING to make this obtuse and hard to read?

This is an excellent article that helps people decide against writing shell scripts. I abandoned doing so shortly after I switched to linux. Since then I was also using ruby. I still do not understand why people would prefer shell scripts over ruby (or python). On systems without ruby or python, one may see a benefit in using shell scripts; other than that I fail to see why shell scripts are necessary.

Shell scripts simply suck for many reason. They are ugly, verbose, convoluted, outright stupid too such as argument passing into functions. Then there is straight up retarded stuff such as case/esac. Whoever came up with that was clearly an incompetent language designer.

loading story #49055908
loading story #49057835
loading story #49057648
loading story #49058530