Sunday, October 7

filter-exec-stdin

I recently added a new script to my collection of personal utilities, filter-exec-stdin.

It’s a little hard to explain this one without reference to its close relatives, filter-exec and filter-exec-raw. These are, as the name strongly implies, filters: They take standard input, transform it, and print the result to standard output.

This makes them easy to use in a text editor like vim, where you can pass the contents of a buffer through an external program like so:

:%!some-script

I wrote filter-exec, the first of these, as a quick and dirty way to include shell commands and their output when writing documentation (like userland or DigitalOcean tutorials) in Markdown. Markdown parsers tend to ignore HTML comments, so I used comments to mark the start and end of a block, and $ to mimic a command prompt, followed by the command string:

<!-- exec -->

    $ ack --thppppt

<!-- end -->

And the filter adds the output:

<!-- exec -->

    $ ack --thppppt
    _   /|
    \'o.O'
    =(___)=
       U    ack --thppppt!

<!-- end -->

The output is indented 4 spaces, which causes Markdown parsers to treat it as preformatted text. It can be run repeatedly, and will retain the command but update the output.

Later on I was cobbling together a system for generating simple invoices, and wanted something similar without the command included in the rendered output or the indentation, so I added filter-exec-raw:

<!-- exec-raw ack --thppppt -->
_   /|
\'o.O'
=(___)=
   U    ack --thppppt!
<!-- end -->

Here the command is inside the HTML comment, and the raw output is printed inside the block, instead of an indented version.

exec-stdin is more meta. It goes something like this:

<!-- exec-stdin sort | uniq -->
1
2
3
4
<!-- end -->

…that is, it runs the command after exec-stdin, passes the contents of the block off to it, and replaces the block with the command’s output. I’m not sure how useful this is. It assumes that a command will have stable output for given input - that is, something like sort, where once the input is sorted, it won’t change again. The idea is that I might have a block of text that should be sorted, checked for duplicates, formatted, etc., and I can just invoke utilities like sort, uniq, fmt, and so forth to handle those problems.

It’s possible that a more-generally-useful filter would retain the original block while also outputting the results of the command.