Let us start 2022 with a cool little tool that I recently found out about1.
And it really is little. So little, I will quote its
man page in full:
sponge- soak up standard input and write to a file
sed '...' file | grep '...' | sponge [-a] file
spongereads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This allows constructing pipelines that read from and write to the same file.
spongepreserves the permissions of the output file if it already exists.
When possible, sponge creates or updates the output file atomically by renaming a temp file into place. (This cannot be done if TMPDIR is not in the same filesystem.)
If the output file is a special file or symlink, the data will be written to it, non-atomically.
If no file is specified, sponge outputs to stdout.
-a– Replace the file with a new file that contains the file's original content, with the standard input appended to it. This is done atomically when possible.
Where it most used in practice though is when you are piping information that will come in a stream (even a short one).
A typical example would be appending one file to another:
At a first glance you would think that this would produce the results:
cat foo bar > foo
… but, in fact, what would happen is that
foo would now contain only the same as
bar. From what I understand, what happens is that first
foo gets overwritten by
foo itself, and then again overwritten by
But when you introduce
sponge in between to soak it all up, it works fine:
cat foo bar | sponge foo
… now first the contents of
foo gets soaked up by
sponge and after that also the content of
bar. And once the data stream has ended,
sponge releases it all into
Similarly when you want to filter the contents of a file (e.g. to only include lines that have “yes” at the end), you could use:
grep 'yes$' answers.txt | sponge answers.txt
sponge is part of the MoreUtils package, which neatly expands the standard GNU CoreUtils package, while preserving the UNIX philosophy.
This year I intend to write more shorter blog posts about little tools, like this one. During the years, there have been quite a few that gathered up.
hook out → waiting to first soak it all up seems quite fitting after the challenging last two years
Hat-tip to Sebastian “seabass-labrax” Crane. ↩