SCSH-Process (historical revision 27567)

You are looking at historical revision 27567 of this page. It may differ significantly from its current revision.

SCSH-Process

Description

A reimplementation for Chicken of SCSH's process notation.

Documentation

This egg implements all the special forms required to implement SCSH's convenient process notation. This notation is called "EPF", which stands for extended process form.

The procedural equivalents for these special forms are also available. Where it's required or makes sense, a handful of other procedures are also provided. This egg does not strive for full SCSH compatibility; it exists only to bring the convenience of EPF notation to Chicken.

For the same reason, currently there is no extra effort done to prevent problems with threading. This means that if you are using this egg, you should always remember that other threads will continue running whenever you fork. Please note that all of the EPF forms implicitly fork for each process in a pipeline.

This caveat may be fixed in the future as Chicken grows ways to deal with this.

Loading scsh-process will transparently cause the posix process-wait procedure to be replaced so it can update the bookkeeping information about reaped child processes. See the description in the SCSH manual about how SCSH performs process reaping. Currently only the "early" reaping strategy is supported.

Beware that if you set a signal/chld handler, you will need to remember the original handler and call it from your handler. If you don't, you must manually wait for all children. If you've installed a handler before scsh-process is loaded, it will be automatically chained by the handler installed by scsh-process.

Macros

The special forms provided by this egg are the most convenient to use, but also slightly less powerful than the underlying procedures they use.

Basic process macros

[syntax] (run pf [redirection ...])
[syntax] (& pf [redirection ...])
[syntax] (exec-epf pf [redirection ...])

These forms run a new process pipeline, described by process form pf, with optional redirections of input and output descriptors indicated by any number of redirection patterns. These processes don't interact with the calling process; if you want to capture their output there are other forms, like run/string, run/port and friends. Please also beware that these forms don't do anything to indicate nonzero exit statuses in pipelines. The && and || macros might help if you need to do status checks.

The run form simply runs the process and waits for the final process in the pipeline to exit. It returns three values relating to this final process:

Either its exit status as an integer if the process terminated normally, or the signal number that terminated/stopped the process.
#t if the process exited normally, #f if it terminated abnormally.
Its process id (an integer).

You'll note that these are just the same values returned by process-wait, but in a slightly different order. This is for compatibility reasons: In SCSH this form returns only the exit status, but because Chicken accepts multiple values in single value contexts (discarding all but the first), we can provide the other ones as extra values, thereby avoiding gratuitous incompatibility.

The & form simply runs the process in the background and returns one value: a process object representing the last process in the pipeline.

The exec-epf form never returns; it replaces the current process by the last process in the pipeline. The others are implemented in terms of this form:

(& . epf) => (process-fork (lambda () (exec-epf . epf)))
(run . epf) => (process-wait (& . epf))

A process form followed by a set of redirections is called "extended process form" in SCSH terminology. A process form can be one of the following:

(<program> <arg> ...)
(pipe <pf> ...)
(pipe+ <connect-list> <pf> ...)
(epf <pf> <redirection> ...)
(begin <s-expr> ...)

The arguments to a <program> rule are implicitly quasiquoted. The basic building blocks are <program> and <begin>, which always correspond to one process in a pipeline. The other rules are ways to combine these two.

The pipe rule will hook up standard output and standard error of each pf to the next pf's standard input, just like in a regular Unix shell pipeline.

The pipe+ rule is like pipe, but it allows you to hook up arbitrary file descriptors between two neighbouring processes. This is done through the connect-list, which is a list of fd-mappings describing how ports are connected from one process to the next. It has the form ((from-fd1 from-fd2 ... to-fd) ...). The from-fds correspond to outbound file descriptors in one process, the to-fds correspond to inbound file descriptors in the other process.

The epf rule is to get extended process forms in contexts where only process forms are accepted, like the pipe and pipe+ subforms, and in the && and || macros (so you can do file redirects here).

The begin rule allows you to write scheme code which will be run in a forked process, having its current-input-port, current-output-port and current-error-port hooked up to its neighbouring processes in the pipeline.

A redirection can be one of the following:

(> [<fd>] <file-name>)      ; Write fd (default: 1) to the given filename
(>> [<fd>] <file-name>)     ; Like >, but append instead of overwriting
(< [<fd>] <file-name>)      ; Read fd (default: 0) from the filename
(<< [<fd>] <scheme-object>) ; Like <, but use object's printed representation
(= <fd> <fd-or-port>)       ; Redirect fd to fd-or-port
(- <fd-or-port>)            ; Close fd
stdports                    ; Duplicate fd 0, 1, 2 from standard Scheme ports

The arguments to redirection rules are also implicitly quasiquoted.

To tie it all together, here are a few examples:

(use scsh-process)

;; Writes "1235" to a file called "out" in the current directory.
;; Shell equivalent:  echo 1234 + 1 | bc > out
(run (pipe (echo "1234" + 1) ("bc")) (> out))

(define message "hello, world")

;; Writes 13 to stdout, with a forked Scheme process writing the data.
;; Shell equivalent (sort of): echo 'hello, world' | wc -c
(run (pipe (begin (display message) (newline)) (wc -c)))

;; A verbose way of doing the same, using pipe+.  It connects the {{begin}}
;; form's standard output and standard error to standard input of {{wc}}:
(run (pipe+ ((1 2 0)) (begin (display message) (newline)) (wc -c)))

;; Same as above, using redirection instead of port writing:
(run (wc -c) (<< ,(string-append message "\n")))

;; Writes nothing because stdout is closed:
(run (wc -c) (<< ,message) (- 1))

;; A complex toy example using nested pipes, with input/output redirection.
;; Closest shell equivalent:
;; ((sh -c "echo foo >&2") 2>&1 | cat) | cat
(run (pipe+ ((1 0))
            (pipe+ ((2 0)) (sh -c "echo foo >&2") (cat))
            (cat)))

Process macros for interfacing with Scheme

[syntax] (run/port pf [redirection ...])
[syntax] (run/file pf [redirection ...])
[syntax] (run/string pf [redirection ...])
[syntax] (run/strings pf [redirection ...])
[syntax] (run/sexp pf [redirection ...])
[syntax] (run/sexps pf [redirection ...])

These forms are equivalent to run, except they wire up the current process to the endpoint of the pipeline, allowing you to read the standard output from the pipeline as a whole. If you also need standard error or have even more specialized needs, take a look at the run/collecting form.

The difference between these forms is in how this output is returned, and when the call returns:

run/port immediately returns after forking, and returns a port from which you can read.
run/file returns after the final process exits, resulting in a string which indicates a temporary file containing the process' output.
run/string returns when the process closes its standard output (ie, when EOF is read), collecting the standard output into a string.
run/strings is like run/string, but returns a list of strings, split at newline characters.
run/sexp reads an s-expression, and returns as soon as a complete s-expression was read.
run/sexp reads all s-expressions until eof, returning a list of s-expressions. It returns as soon as EOF is read.

Macros for conditional process control

[syntax] (&& pf ...)
[syntax] (|| pf ...)

These macros act like their counterpart shell operators; they run the given process forms in sequence and stop either on the first "false" value (nonzero exit) or on the first "true" value (zero exit), respectively.

The result value of these is #f or #t, so they act a lot like regular Scheme and and or.

Note: The name of the || macro is really the empty symbol whereas in SCSH's reader, it reads as a symbol consisting of two pipe characters. The name of these macros may change in the future if it turns out to cause too much trouble.

Collecting multiple outputs

[syntax] (run/collecting fds pf ...)

This form runs the pf form, redirecting each the file descriptors in the fds list to a separate tempfile, and waits for the process to complete.

The result of this expression is (status file ...). status is the exit status of the process. Each file entry is an opened input port for the temporary file that belongs to the file descriptor at the same offset in the fds list. If you close the port, the tempfile is removed.

See the SCSH documentation for an extended rationale of why this works the way it does.

Procedural interface

These procedures form the basis for the special forms documented above, and can be used to implement your own, more specialized macros.

Basic forking and pipeline primitives

[procedure] (fork [thunk])
[procedure] (%fork [thunk])

If thunk is provided and not #f, the child process will invoke the thunk and exit when it returns.

fork differs from the regular process-fork in its return value. Instead of a pid value, this returns a process object representing the child is returned in the parent process. When thunk is not provided, #f (not zero!) is returned in the child process.

Currently %fork/pipe is just an alias for fork/pipe.

[procedure] (wait [pid-or-process [nohang]])

Like process-wait: Suspends the current process until the child process described by pid-or-process (either a numerical process ID or a scsh-process object) has terminated using the UNIX system call waitpid(). If pid-or-process is not given, then this procedure waits for any child process. If nohang is given and not #f then the current process is not suspended.

This process returns three values, in a different order from process-wait:

Either the exit status, if the process terminated normally or the signal number that terminated/stopped the process.
#t if the process exited normally or #f otherwise.
pid or 0

All values are #f, if nohang is true and the child process has not terminated yet.

It is allowed to wait multiple times for the same process after it has completed, if you pass a process object. Process IDs can only be waited for once after they have completed and will cause an error otherwise.

[procedure] (process? object)

Is object an object representing a process?

[procedure] (fork/pipe [thunk])
[procedure] (%fork/pipe [thunk])

These fork the process as per fork or %fork, but additionally they set up a pipe between parent and child. The child's standard output and stdandard error are set up to write to the pipe, while the parent's standard input is set up to read to the pipe.

The return value is a process object or #f.

Currently fork%/pipe is just an alias for fork/pipe.

Important: These procedures only set up the file descriptors, not the Scheme ports. current-input-port, current-output-port and current-error-port still refer to their old file descriptors after a fork. This means that you'll need to reopen the descriptors to get a Scheme port that reads from the child or writes to the parent:

(use scsh-process posix)

(process-wait
  (fork/pipe (lambda ()
               (with-output-to-port (open-output-file* 1)
                 (lambda () (display "Hello, world.\n"))))))

(read-line (open-input-file* 0)) => "Hello, world"

[procedure] (fork/pipe+ conns [thunk])
[procedure] (fork%/pipe+ conns [thunk])

These are like fork/pipe and fork%/pipe, except they allow you to control how the file descriptors are wired. Conns is a list of lists, of the form ((from-fd1 from-fd2 ... to-fd) ...). See the description of pipe+ under the run special form for more information.

Currently fork%/pipe+ is just an alias for fork/pipe+.

0.3.1 - Change wait result values to all be #f if nohang is #t.
0.3 - Fix segfault properly, revert back to using standard-extension. Add wait and do not accept scsh-process objects in POSIX process-wait.
0.2.1 - Workaround for segmentation fault caused by compiled version of scsh-process in 4.8.0 due to -O3 being the default compilation of a standard-extension.
0.2 - Fix <<-redirection and increase robustness of test for it.
0.1.2 - Fix setup-file to have non-bogus version identifier. Don't rely on "bc" being present in the tests.
0.1.1 - Fix order of values returned by process-wait to be consistent with POSIX unit.
0.1 - Initial release, created at the Chicken UK 2012 hacking event.

 Copyright (c) 2012, Peter Bex
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
 
 THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR
 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
 IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY DIRECT, INDIRECT,
 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

SCSH-Process

Description

Requirements

Documentation

Macros

Basic process macros

Process macros for interfacing with Scheme

Macros for conditional process control

Collecting multiple outputs

Procedural interface

Basic forking and pipeline primitives

Pipeline procedures

Collecting multiple outputs

Changelog

Author

Repository

License