A wrapper is a shell command that creates a service dedicated to a process tree: this service is available to any descendant process by virtue of socket bound to a unique name in the filesystem. Such services are not limited by ephemeral TCP/IP ports, or to the resource limits of a single process -- they are limited only by the number of processes that may be started on the host and by the name-space available for UNIX domain sockets. In effect such services scale out to the total capacity of the supporting host.
Such a service is called a "wrapper" because it wraps the descendant
process tree, existing only to
serve until that process tree exit
s.
Wrappers have scope, in that they are intended to serve a
fixed process tree. Each can be viewed as part of the mechanics of
that environment, not just the global environment.
Contrast that with a daemon, which exists for the life of the host,
started at system boot running until shutdown, or a network service
started as-needed by inetd
or
a CGI
started by a web server, or
a service accessed via
the RPC port mapper.
Each of those has its place in the structure of an information technologies environment. The wrapper has some features that make it attractive, and that's what we'll start with below.
By scope we mean that each of the many users on a machine may have multiple instances of any wrapper active at the same time, and those instances may be enclosed in other instances of the same, or another wrapper. More on that below.
By scale we mean that we are not limited to the number of TCP ports, RPC program numbers, or a system configuration that only the super user can update. Starting many wrapper instances to get the structure required is like starting many filters to get the output format required. It might take nesting wrappers compositionally or running multiple instances in parallel to produce the desired effect.
In the sections below we examine other features of wrappers.
An example at this point helps: say a wrapper offers
a spell checking service.
At start-up the spelling wrapper would first check for an enclosing
copy of itself, then read stored preferences from any cache in the
client's home directory, then check for a cache in the current
directory (with appropriate security checks). It might even check
parent directories, if that preference is set.
The outer-most service, finding no enclosing diversion,
begins with a set tableau from the preferences located,
or from a default file like /usr/share/dict/words
.
As words are offered and corrected they might be added to permanent folder in the clients home directory, or kept only in memory. This limits any locale, language, or similar preference kept by the spelling wrapper to the domain of that program, not any client program that needs the spelling service.
Each login on the system may start as many "spelling wrappers" as needed. Nested wrapper should consult any outer wrapper (if that is the client's preference) to expand the tableau of available words and assume the active preferences. Newly corrected submissions should be propagated as the client has directed, either back to the outer-most instance or trapped below.
As more applications reuse the common spelling wrapper the work needed to correct documents becomes less for the clients; more "clues" are banked in both the running wrapper's state and any configuration files. Those clues are all scoped to the right level, which makes the whole structure better for the client.
Spelling wrappers for more than one language might be active at the same time for the same login. There is no throttle-point in the structure to prevent it -- so it should be supported.
Clients should be able to select any outer diversion, when that makes sense in the context of the application. There are two suggested ways to select an outer diversion: a `depth' count, and a `direct path'.
The command line specification for depth
is an integer prefixed by the option dash.
The unspecified depth for the outer,
-1
for the next one
out, then -2
and -3
and so-on.
As a degenerate case an explicit depth of
zero (-0
) is also allowed, but
it is better form to just leave that option out.
Some other command-line option should provide a way to specify a path to the controlling unix domain socket for any active diversion's socket. This allows a client to pick a diversion that might not be part of the current process tree. How the client gets the path to the socket is dependent on the implementation, or might be drawn from the common implementation described below.
A direct path of "-
" is another name for
the tightest enclosing diversion. That same symbol is often
taken, in a filter context, as stdin or stdout:
it may be used as either of those as well.
To keep track of the diversion we use the
name
=value
pair environment
as a fundamental abstraction. Each service selects a unique prefix
(for example a string based on the program named, followed by an
under-bar) to manage its diversions.
Two key variables are constructed in the environment
based on that prefix:
spck_link=2
spck_1=/tmp/spckXXXXXX/s1 spck_2=/tmp/spckXXXXXX/s2
spck_d=/tmp/ptbwXXXXXX/s5
The wrapper programs I've coded all use the same 300 line C module to handle all the manipulation of these variables (viz. ptbw, xclate). The same abstraction in perl is much shorter.
inetd
, fails it
impacts a large groups of services. When a wrapper fails if
should only impact a small portion of the system.
For example, misspelled words may be inserted into a spelling wrapper's tableau. This happens because the spelling wrapper, by its nature, believes the Customer. So an instance believes the Customer when she tells it that a Polish word belongs in the Spanish database. In the wrapper model this should never impact other Spanish speakers.
Usually the command-line option -m
forces the program to start a new diversion in master mode.
In some cases the client provided by the program is less feature-rich
than more specialized clients.
exit
, the enclosing master doesn't have to
know about the transient entry in this directory. Leaving junk files in
a parent name-space is really poor form.
exit
's.
The time between the process termination and the socket
shutdown is application specific.
-
).
fork
'ing a process.
xapply
stack" when we are talking
about the close relationship between
xapply
, xclate
, and
ptbw
.
xclate
is
active a new ptbw
instance uses the open name-space
for the new control socket.
esc_link=1 esc_1=/tmp/escdX1234Q/e1
When a client of that wrapper looks for $esc_link
in
the environment it notes 2 things:
esc_1
the client connects directly to
the service.
When another instance of the service (a diversion "master") opens a new diversion it finds the same information and build a new unix domain socket (possibly in the same directory)
esc_link=2 esc_2=/tmp/escdX1234Q/e2
Now clients will connect to the new (more local) diversion in preference to the older outer one. The more local diversion may pass some requests on to "e1", or answer them based on the function of the program. Results may be passed verbatim, or modified based on the parameters given to each master in the chain.
This is the general idea. Below we go into some specific details which I'm suggesting become standard features.
depth
option means.
In other words it would be unexpected for an application that
used a wrapper internally to add to the diversion stack.
Any applications that want to hide their use of a master from
the diversion stack may do so.
The unix domain socket is built in
the "d" alternate diversion environment
variable (usually specified by the command-line option
-d
).
Note that a client never falls back to the alternate diversion. It is supposed to be hidden from the normal use, so there are three ways to force it into scope:
-N
is often
the direct path option (in my wrappers). So, for example,
to force xclate
to use the alternate
diversion one might code:
xclate -N $xcl_d ....
client
process.
It is, however, supported in the clients I've coded. The
above example would be phrased as:
xcl_link=d xclate ....
-N
in combination with -md
.
When the application has a safe name-space to allocate a specific
socket name it should force the name, then send the name to any
client processes by its own means.
A good example of the use of disconnected diversions comes from
xapply
under its -m
option. Under this option xapply
uses the
top-of-stack xclate
diversion, if any exists. When none
exists it builds a new instance and uses it.
The new instances may be disconnected (under -d
)
or may start a new stack for descendent xapply
instances and xclate
clients. This
"auto-wrap" feature of xapply
makes it part of
the "stack" -- even though it is not itself a wrapper.
utility
we could use to hold
the master instance open for the entire production run of the
system? We can't really wrap init
(8).
We could wrap a sleep
command, but they do
timeout eventually, and we'd be wasting a process. It would be
even worse to code a program to sleep forever (just to waste a process).
All the wrappers I've coded accept the pseudo-utility colon (":")
as an internal command that never exits. In this case a client
may send a shutdown command to the wrapper (under -Q
).
This shutdown takes effect after that client finishes their interaction
with the diversion.
This doesn't really limit the use of the ":" command, as it is
a shell built-in command and can't be execve
'd
anyway.
As an alternative one could export the service via another transport. For example some process might provision an RPC service using a wrapper as a primary data source. In that case the master mode should wrap the server process, not meet it at a private socket address.
The global mode is not intended to "swallow" the process tree.
That is to say that we don't expect an environment variable representing
the global diversion to be inheritted throughout the whole process tree.
Any global instance is usually referenced by a site policy file, which
contains the diversion's control socket name, or a convention that the
global diversion's control socket is always in the same place under
/var/run
(or the like).
sh
or
bash
) knows it is a "login shell" if the
name of the program in argv[0]
starts
with a dash.
The behavior of a wrapper is based on knowing its own "common" name. The specification of any other name might be treated as an "option" by a wrapper, in other words it might change the behavior of the program.
There are the suggested rules for dealing with the name of the program for all wrappers. Not every wrapper implements all of these, but most should follow the intent here:
argv[0]
to an explicit value, but is not required (nor
do I have a suggested name for this option),
argv[0]
starts with a dash
then remove the dash and make this process a session leader
(see setsid
(3)).
argv[0]
is the empty string
it should be replaced with the default program name.
argv[0]
is not the
common name of the wrapper it should be used as an explicit
argv[0]
for the inferior process
argv[0]
is just a colon (":"),
and there is no explicit utility
specified
fork
's a process
with the recovered data provided through one of two channels.
The ptbw
wrapper makes an easy example.
We'll create a tableau of the first 7 ordinal numbers, then ask
a client instance to allocate the first 2, in that environment we'll
ask the shell to echo
those numbers:
ptbw -m -J7 ptbw -R2 sh -c 'echo $ptbw_list'
The example above outputs "0 1", which are the first two ordinal numbers.
The data channel that ptbw
uses to send the
shell the recovered data follows the same paradigm as
any "link" suffix variables: the common prefix for the wrapper application
specifies which data channel, the value of the variable specifies
the payload.
Sensitive data is always placed in a file, the name of the file should give little information to anyone reading the process table for arguments or environment variable values. This is just good application programming, not anything special to wrappers.
In this case we run the same master instance, but request the data
under the -A
, the client program is
/bin/echo
, rather than a shell:
ptbw -m -J7 ptbw -A -R2 echo
ptbw
uses
the common dash (-
) pseudo-client:
Which outputs a very descriptive table of the 6 tokens, their names, the fact that none of them are locked, and the request size.ptbw -m -J3 -R2 ptbw -
ptbw: master has 6 tokens, max length 1 (total space 12) Master tokens from the internal function: iota Index Lock Tokens (request in groups of 2) 0 0 0 1 0 1 2 0 2 3 0 3 4 0 4 5 0 5
screen
builds a UNIX domain
socket and enters a "master mode" unless it sees a running socket
in which case it runs as a client.
The ssh-agent
program, like screen
, builds
a UNIX domain socket, publishes an environment and acts as both the
master and the client. It also doesn't allow nested diversions.
In this case it would make a lot of sense for ssh-agent
to allow nested agents with keys that could be deleted by just
exit
'ing the utility
.
The window
program is a lot like screen
but less like a wrapper. The program doesn't ever act as a client for
itself (it just starts new masters) when enclosed in itself.
The nice
program is a very primitive wrapper.
Any enclosed processes can see the effect, and enclosed instances
can modify the nice value. The same is true for chroot
,
env
,
nohup
, su
, and
newgrp
because they all work on
attributes of a process.
Privilege escalation with sudo
or
op
might also be part of the same application class.
None of these use a socket to nest instances, and the tableau is the
process itself.
Another class would be debuggers, which start a process tree that is
served by an enclosing master: viz.
adb
,
gdb
, and
mdb
.
One might include truss
and
ptrace
in that group in-spite of their
non-interactive nature.
I've coded several wrappers:
ptbw
xclate
xapply
's collation wrapper buffers output
such that parallel processes output a single collated stream without
blocking each other, as much as they otherwise would.
xapply
ptbw
and
xclate
to provide services.
In a strange way find ... -exec
,
apply
, and xargs
provide a wrapped environment, but not nearly at the level of
power needed to call them "wrappers", even though they may
enclose instances of themselves.
The X
server provides graphics services to
processes that can see the environment variable DISPLAY
.
It is not a true wrapper as it only runs in the global mode (that is,
it doesn't wait for any wrapped process to exit).
It may allow a "remote open" service (under a command-line switch) or even a "start a process" service.
Then removes the link environment variables from the descendent environment and starts itswrapw_link
=/tmp/wrapXXXXXX/socket/token
utility
.
The wrapw diversion may be used to proxy mulitple wrappers
though a single socket, gain a list of the existing diversion
(without access to the rest of the environment.
It takes requests for
"name" return name:d,name0:name,1...name:n (with=value) "name:level" return value (direct socket name) "*" return all the names we've (ever) seen (:current-level) "-name:level" retract the name at that level
utility
enclosed by
a "wrapw" instance on the remote host which proxies
the wrapped environment over an (additional) ssh connection.
$Id: wrapper.html,v 1.16 2010/08/13 17:21:36 ksb Exp $