Convert file I/O into pipe I/O with /dev/fd

Some Unix commands read data from files or write data to files, without offering an obvious way to use them as part of a pipeline. How can you write a program to interact with such a command in a streaming fashion? This would allow your program and the command run concurrently, without the storage and I/O overhead of a temporary file. You could create and use a named pipe, but this is a clunky solution, requiring you to create and destroy a unique underlying file name. Here’s a better approach.

Modern Unix systems offer a virtual device directory named /dev/fd/. The files in it correspond to the file descriptors of each process. To refresh your memory, a file descriptor is a small integer representing an open file endpoint; processes can access the file using its file descriptor. Each time a process opens a new file, creates a network socket, or (crucially for us) sets up a pipe, new file descriptors will pop up in the /dev/fd directory.

Moreover, open files are inherited through their descriptors every time one process executes another. This means that one process can execute another and use the /dev/fd files to pass some of its descriptors to the other process in the form of file names. For example, it can pass access to file descriptor 7 as /dev/fd/7. The Bash shell takes advantage of this facility to offer the <(...) and >(...) process redirection syntax.

Returning to our original problem, in order to run a process from within our program, and communicate with one of the files it accesses through a pipe, all we need to do is to construct a pipe, and pass to the process as a /dev/fd/N filename the pipe endpoint to which we want the process to read or write. Again, as a refresher, each pipe is constructed as an array of two file descriptors: the first represents its read endpoint and the second its write endpoint.

As a concrete example, consider the case where we want our program to read the output of the strace system call tracing command. By default, the output of strace appears in its standard error, which can make it co-mingle with the error output of the commands we want to trace. Thankfully, strace can write to a file specified with the -o option. We can therefore use this option to pass it the file descriptor to which we want it to write as a /dev/fd/N file name.

The following small C program demonstrates this by running a specified command under strace and then (as a trivial use case) converting the strace output into uppercase characters.

/*
 * Trace the specified command, outputing the results in upper case.
 * Demonstrates the use of /dev/fd
 *
 * Diomidis Spinellis, December 2019.
 */

#include <ctype.h>
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int
main(int argc, char *argv[])
{
    int pipefd[2];
    char buff[100];
    FILE *fin;
    int c;

    if (argc != 2) {
        fprintf(stderr, "Usage: %s command\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    /* Create the two file descriptors representing a pipe */
    if (pipe(pipefd) != 0)
        err(EXIT_FAILURE, "pipe");

    /* Pass write-end of pipe to async execution of strace */
    snprintf(buff, sizeof(buff), "strace -o /dev/fd/%d %s &", pipefd[1],
        argv[1]);
    if (system(buff) != 0)
        err(EXIT_FAILURE, "system");

    /* Close pipe's write-end on our own side */
    close(pipefd[1]);

    /* Read strace output from pipe's read and converting it to uppercase */
    if ((fin = fdopen(pipefd[0], "r")) == NULL)
        err(EXIT_FAILURE, "fdopen");
    while ((c = getc(fin)) != EOF)
        putchar(toupper(c));

    return (EXIT_SUCCESS);
}

Comments   Toot! Tweet Share