p2f-exec, p2f-wait

communicate easily with interactive commands from shell scripts

Date: 2019-09-20
Version: 0.4.0
Manual section:1

Synopsis

p2f-exec [option]... command [arg]...

p2f-wait string [string]... < fifo

Description

The programs p2f-exec and p2f-wait were written with the following statement in mind: it should be as easy to communicate with an interactive command from a shell script as it is from a shell prompt.

One naive approach to communicate with an interactive command from a shell script is to redirect standard input and output streams into named pipes, also known as FIFOs. Here's an example with GDB:

#!/bin/sh

mkfifo in out

gdb < in > out &

grep '(gdb) '           < out  # issue 1
echo 'file a.out'       > in   # issue 2
grep 'Reading symbols'  < out

grep '(gdb) '           < out  # issue 3
echo 'run'              > in
grep 'Starting program' < out
grep 'exited normally'  < out

grep '(gdb) '           < out
echo 'quit'             > in

rm in out

Sadly, this naive approach suffers a couple of major issues:

  1. most text-processing tools work on a per-line line basis, thus they cannot handle lines that are not terminated by an end-of-line or an end-of-file, as it is typically the case for most command prompts.
  2. most interactive commands quit once the input stream is closed.
  3. most tools read files eagerly, thus a given content might have been already slurp unexpectedly from the FIFO by a previous read.

This is where p2f-exec and p2f-wait come in handy: p2f-exec ensures the input and output streams are kept open as long as the program is running, and p2f-wait reads only up to the specified string, nothing more. That way, the previous example can be simply rewritten as:

#!/bin/sh

p2f-exec gdb

p2f-wait '(gdb) '           < out
echo     'file a.out'       > in
p2f-wait 'Reading symbols'  < out

p2f-wait '(gdb) '           < out
echo     'run'              > in
p2f-wait 'Starting program' < out
p2f-wait 'exited normally'  < out

p2f-wait '(gdb) '           < out
echo     'quit'             > in

These p2f- commands can be seen as a minimal alternative to the famous expect interpreter, and they do not have a lot of features to respect the Unix philosophy:

In other words, time limits should be handled by timeout, regular expressions by grep, terminal settings by stty, special characters by bash for instance, et cetera. See section Tutorial for details.

Options

p2f-wait has no options, and p2f-exec has the following ones:

-f stay in foreground.
-x do not wait for output stream to be purged when command has exited. See section Purging output stream for details.
-t do not disable "\n" to "\r\n" translation in output stream. See section Default terminal line discipline for details.
-e do not disable input echo into output stream. See section Default terminal line discipline for details.
-p <prefix>

prepend <prefix> to FIFO and PID file names. For examples:

-p foo/  ->  foo/in, foo/out, foo/pid
-p foo.  ->  foo.in, foo.out, foo.pid
-p foo   ->  fooin, fooout, foopid
-p ''    ->  in, out, pid  (this is the default).

Exit status

p2f-wait returns 255 if an error occurred, 254 if no strings matched and there's nothing else to read (end of the program, typically), otherwise it returns the index of the matched string (starting from 0).

p2f-exec returns 1 only if something goes wrong during its initialization, otherwise it returns 0 even if it cannot execute the specified command.

Files

p2f-wait reads from standard input and write to standard output.

p2f-exec creates these three files:

in
a FIFO used as the input stream (stdin) of the executed command.
out
a FIFO used as the output stream (stdout, stderr) of the executed command.
pid
a regular file that contains the process identifier of the executed command.

Tutorial

Note: all the examples from this section are executed using Bash, thus the syntax must be adapted when using another shell interpreter. Also, it is assumed the terminal is configured sanely (stty sane).

By default p2f-exec creates two FIFOs named in and out that are respectively the input and output streams of the specified command. Here's a tiny example:

p2f-exec echo 'ok'
cat out  # print 'ok\n'

Here cat reads the out FIFO until it is closed on the other end, without much possibilities to interact with the executed command. As a consequence, when interaction is required, p2f-wait should be used instead of cat; this former reads only up to the specified string, then exits. For instance, with a simple interactive command:

p2f-exec sh -c 'echo -n "Your name? "; read a; echo "Hello $a"'
p2f-wait 'name? ' < out  # print 'Your name? '
echo 'John'       > in
cat out                  # print 'Hello John\n'

It is possible to wait for any special character except the null byte. One easy way to handle special characters in a bash script is to use the syntax $'…', for example with an end-of-line:

p2f-exec echo 'ok'
p2f-wait $'\n' < out  # print 'ok\n'

The bash syntax $'…' can also be used to send special character. For instance, most interactive commands quits when receiving an end-of-transmission (control-D on a keyboard):

p2f-exec cat
echo 'ok'   > in
echo $'\cD' > in
cat out  # purge output stream, print 'ok\n'

Of course special characters other than control characters can be used too:

p2f-exec echo $'\xe2\x98\xba'
p2f-wait $'\xe2\x98\xba' < out

The example above is strictly equivalent to the following one, assuming the terminal supports UTF-8:

p2f-exec echo '☺'
p2f-wait '☺' < out

Actually p2f-wait works at the byte level, so it can even be used against a byte stream. For instance:

p2f-wait $'\xe2\x98\xba' < /dev/urandom > happy-blob

However, to wait for a given number of bytes, external tools should be used instead. For example:

p2f-exec echo 'this is a test'
read -N10 foo < out
echo $foo  # print 'this is a '
read -N10 foo < out
echo $foo  # print 'test\n'

Or similarly:

p2f-exec echo 'this is a test'
dd if=out of=foo bs=1 count=10
cat foo  # print 'this is a '
dd if=out of=foo bs=1 count=10
cat foo  # print 'test\n'

Reading only a given number of lines is a little bit trickier because programs like head or grep can't be used since they read files eagerly, that is, they consume from the output stream more bytes than necessary. In that case p2f-wait must be used instead, as in the following example where it is used to carefully read the first five lines only, leaving the remaining ones untouched from the output stream:

p2f-exec cat /etc/fstab
for i in $(seq 5); do
    p2f-wait $'\n' < out
done
cat out  # print remaining lines

Since both p2f-exec and p2f-wait respect the Unix philosophy, many features are delegated to dedicated external tools, as it is the case for time limits for instance:

p2f-exec echo 'foo'
timeout 5s p2f-wait 'bar'  # exit unsuccessfully

Likewise for regular expressions:

p2f-exec bash -c '[ $RANDOM -lt 16384 ]; echo "Result: $?"'
p2f-wait 'Result: ' < out
p2f-wait $'\n' < out | grep -E '(0|1)'

Although for simple alternations, it is possible to pass several strings to p2f-wait. In that case, it will return the index of the matched string:

p2f-exec bash -c '[ $RANDOM -lt 16384 ]; echo "Result: $?"'
p2f-wait 'Result: 0' 'Result: 1' < out
case "$?" in
"0") # First string matched.
    ;;
"1") # Second string matched.
    ;;
*)   # A problem occured.
    ;;
esac

Last but not least, here's how to get the exit code of the command executed by p2f-exec:

p2f-exec sh -c 'command; echo "exit code = $?"'
p2f-wait 'exit code = ' < out
exit_code=$(cat out)

Purging output stream

In most cases, p2f-exec does not exit immediately once the command has quit, and it keeps 'out' FIFO alive. This is because there are data in the output stream that were not read yet. Here's a typical example:

p2f-exec cat -
echo ok     > in
echo $'\cD' > in
ls in out pid  # only 'in' and 'pid' have disappeared
cat out        # purge output stream, print 'ok\n'
ls out         # 'out' has disappeared

This default behavior can be changed with the -x option. In that case, p2f-exec exits as soon as the command has quit, thus data that were not read yet from the output stream are lost. For instance:

p2f-exec -x cat -
echo ok     > in
echo $'\cD' > in
ls in out pid  # they have all disappeared
cat out        # failure, 'ok\n' is lost

Default terminal line discipline

Since version 0.3.0, and for convenience purposes, p2f-exec disables by default both the "\n" to "\r\n" translation in output stream, and the input echo into output stream. Of course it is possible to tell p2f-exec not to touch these settings using the options -t and -e respectively.

Here is an example regarding the "\n" to "\r\n" translation:

p2f-exec -t printf 'foo\nbar'
p2f-wait $'foo\nbar'   < out  # failure
p2f-wait $'foo\r\nbar' < out  # success

And another example regarding the input echo:

p2f-exec -e cat
echo ok > in
cat out  # print 'ok\n' twice

whereas:

p2f-exec cat
echo ok > in
cat out  # print 'ok\n' once

This latter option is useful for debug purposes, in order to see when commands are sent to the executed command.

Notes

The prefix "p2f" is the short for "PTY to FIFOs".

Author

Software and documentation written by Cédric Vincent <cedric.vincent@st.com>.

See also

expect(1), empty(1), sexpect(1), timeout(1), grep(1), stty(1), bash(1)