Let us start 2022 with a cool little tool that I recently found out about¹.

Man page¶

And it really is little. So little, I will quote its man page in full:

Name

sponge - soak up standard input and write to a file

Synopsis

sed '...' file | grep '...' | sponge [-a] file

Description

sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This allows constructing pipelines that read from and write to the same file.

sponge preserves the permissions of the output file if it already exists.

When possible, sponge creates or updates the output file atomically by renaming a temp file into place. (This cannot be done if TMPDIR is not in the same filesystem.)

If the output file is a special file or symlink, the data will be written to it, non-atomically.

If no file is specified, sponge outputs to stdout.

Options

• -a – Replace the file with a new file that contains the file's original content, with the standard input appended to it. This is done atomically when possible.

Examples¶

Where it most used in practice though is when you are piping information that will come in a stream (even a short one).

A typical example would be appending one file to another:

At a first glance you would think that this would produce the results:

cat foo bar > foo

… but, in fact, what would happen is that foo would now contain only the same as bar. From what I understand, what happens is that first foo gets overwritten by foo itself, and then again overwritten by bar.

But when you introduce sponge in between to soak it all up, it works fine:

cat foo bar | sponge foo

… now first the contents of foo gets soaked up by sponge and after that also the content of bar. And once the data stream has ended, sponge releases it all into foo.

Similarly when you want to filter the contents of a file (e.g. to only include lines that have “yes” at the end), you could use:

grep 'yes$' answers.txt | sponge answers.txt

Final words¶

sponge is part of the MoreUtils package, which neatly expands the standard GNU CoreUtils package, while preserving the UNIX philosophy.

This year I intend to write more shorter blog posts about little tools, like this one. During the years, there have been quite a few that gathered up.

hook out → waiting to first soak it all up seems quite fitting after the challenging last two years