A wrapper is a shell command that creates a service dedicated to a process tree: this service is available to any descendant process by virtue of socket bound to a unique name in the filesystem. Such services are not limited by ephemeral TCP/IP ports, or to the resource limits of a single process -- they are limited only by the number of processes that may be started on the host and by the name-space available for UNIX domain sockets. In effect such services scale out to the total capacity of the supporting host.
Such a service is called a "wrapper" because it wraps the descendant
process tree, existing only to
serve until that process tree exits.
Wrappers have scope, in that they are intended to serve a
fixed process tree. Each can be viewed as part of the mechanics of
that environment, not just the global environment.
Contrast that with a daemon, which exists for the life of the host,
started at system boot running until shutdown, or a network service
started as-needed by inetd or
a CGI
started by a web server, or
a service accessed via
the RPC port mapper.
Each of those has its place in the structure of an information technologies environment. The wrapper has some features that make it attractive, and that's what we'll start with below.
By scope we mean that each of the many users on a machine may have multiple instances of any wrapper active at the same time, and those instances may be enclosed in other instances of the same, or another wrapper. More on that below.
By scale we mean that we are not limited to the number of TCP ports, RPC program numbers, or a system configuration that only the super user can update. Starting many wrapper instances to get the structure required is like starting many filters to get the output format required. It might take nesting wrappers compositionally or running multiple instances in parallel to produce the desired effect.
In the sections below we examine other features of wrappers.
An example at this point helps: say a wrapper offers
a spell checking service.
At start-up the spelling wrapper would first check for an enclosing
copy of itself, then read stored preferences from any cache in the
client's home directory, then check for a cache in the current
directory (with appropriate security checks). It might even check
parent directories, if that preference is set.
The outer-most service, finding no enclosing diversion,
begins with a set tableau from the preferences located,
or from a default file like /usr/share/dict/words.
As words are offered and corrected they might be added to permanent folder in the clients home directory, or kept only in memory. This limits any locale, language, or similar preference kept by the spelling wrapper to the domain of that program, not any client program that needs the spelling service.
Each login on the system may start as many "spelling wrappers" as needed. Nested wrapper should consult any outer wrapper (if that is the client's preference) to expand the tableau of available words and assume the active preferences. Newly corrected submissions should be propagated as the client has directed, either back to the outer-most instance or trapped below.
As more applications reuse the common spelling wrapper the work needed to correct documents becomes less for the clients; more "clues" are banked in both the running wrapper's state and any configuration files. Those clues are all scoped to the right level, which makes the whole structure better for the client.
Spelling wrappers for more than one language might be active at the same time for the same login. There is no throttle-point in the structure to prevent it -- so it should be supported.
Clients should be able to select any outer diversion, when that makes sense in the context of the application. There are two suggested ways to select an outer diversion: a `depth' count, and a `direct path'.
The command line specification for depth
is an integer prefixed by the option dash.
The unspecified depth for the outer,
-1 for the next one
out, then -2
and -3 and so-on.
As a degenerate case an explicit depth of
zero (-0) is also allowed, but
it is better form to just leave that option out.
Some other command-line option should provide a way to specify a path to the controlling unix domain socket for any active diversion's socket. This allows a client to pick a diversion that might not be part of the current process tree. How the client gets the path to the socket is dependent on the implementation, or might be drawn from the common implementation described below.
A direct path of "-" is another name for
the tightest enclosing diversion. That same symbol is often
taken, in a filter context, as stdin or stdout:
it may be used as either of those as well.
To keep track of the diversion we use the
name=value pair environment
as a fundamental abstraction. Each service selects a unique prefix
(for example a string based on the program named, followed by an
under-bar) to manage its diversions.
Two key variables are constructed in the environment
based on that prefix:
spck_link=2
spck_1=/tmp/spckXXXXXX/s1 spck_2=/tmp/spckXXXXXX/s2
spck_d=/tmp/ptbwXXXXXX/s5
The wrapper programs I've coded all use the same 300 line C module to handle all the manipulation of these variables (viz. ptbw, xclate). The same abstraction in perl is much shorter.
inetd, fails it
impacts a large groups of services. When a wrapper fails if
should only impact a small portion of the system.
For example, misspelled words may be inserted into a spelling wrapper's tableau. This happens because the spelling wrapper, by its nature, believes the Customer. So an instance believes the Customer when she tells it that a Polish word belongs in the Spanish database. In the wrapper model this should never impact other Spanish speakers.
Usually the command-line option -m
forces the program to start a new diversion in master mode.
In some cases the client provided by the program is less feature-rich
than more specialized clients.
exit, the enclosing master doesn't have to
know about the transient entry in this directory. Leaving junk files in
a parent name-space is really poor form.
exit's.
The time between the process termination and the socket
shutdown is application specific.
-).
fork'ing a process.
xapply stack" when we are talking
about the close relationship between
xapply, xclate, and
ptbw.
xclate is
active a new ptbw instance uses the open name-space
for the new control socket.
esc_link=1 esc_1=/tmp/escdX1234Q/e1
When a client of that wrapper looks for $esc_link in
the environment it notes 2 things:
esc_1 the client connects directly to
the service.
When another instance of the service (a diversion "master") opens a new diversion it finds the same information and build a new unix domain socket (possibly in the same directory)
esc_link=2 esc_2=/tmp/escdX1234Q/e2
Now clients will connect to the new (more local) diversion in preference to the older outer one. The more local diversion may pass some requests on to "e1", or answer them based on the function of the program. Results may be passed verbatim, or modified based on the parameters given to each master in the chain.
This is the general idea. Below we go into some specific details which I'm suggesting become standard features.
depth option means.
In other words it would be unexpected for an application that
used a wrapper internally to add to the diversion stack.
Any applications that want to hide their use of a master from
the diversion stack may do so.
The unix domain socket is built in
the "d" alternate diversion environment
variable (usually specified by the command-line option
-d).
Note that a client never falls back to the alternate diversion. It is supposed to be hidden from the normal use, so there are three ways to force it into scope:
-N is often
the direct path option (in my wrappers). So, for example,
to force xclate to use the alternate
diversion one might code:
xclate -N $xcl_d ....
client process.
It is, however, supported in the clients I've coded. The
above example would be phrased as:
xcl_link=d xclate ....
-N in combination with -md.
When the application has a safe name-space to allocate a specific
socket name it should force the name, then send the name to any
client processes by its own means.
A good example of the use of disconnected diversions comes from
xapply under its -m
option. Under this option xapply uses the
top-of-stack xclate diversion, if any exists.
When none exists it builds a new instance and uses it.
The new instances may be disconnected (under -d) or
may start a new stack for descendent xapply
instances and xclate clients. This
"auto-wrap" feature of xapply makes it part of
the "stack" -- even though it is not itself a wrapper.
utility we could use to hold
the master instance open for the entire production run of the
system? We can't really wrap init(8).
We could wrap a sleep command, but they do
timeout eventually, and we'd be wasting a process. It would be
even worse to code a program to sleep forever (just to waste a process).
Well, here is that process (in case you need it):
perl -e 'my($in,$out);pipe($in,$out);print <$in>;'
All the wrappers I've coded accept the pseudo-utility colon (":")
as an internal command that never exits. In this case a client
may send a shutdown command to the wrapper (under -Q).
This shutdown takes effect after that client finishes their interaction
with the diversion.
This doesn't really limit the use of the ":" command, as it is
a shell built-in command and can't be execve'd
anyway.
As an alternative one could export the service via another transport. For example some process might provision an RPC service using a wrapper as a primary data source. In that case the master mode should wrap the server process, not meet it at a private socket address.
The global mode is not intended to "swallow" the process tree.
That is to say that we don't expect an environment variable representing
the global diversion to be inherited throughout the whole process tree.
Any global instance is usually referenced by a site policy file, which
contains the diversion's control socket name, or a convention that the
global diversion's control socket is always in the same place under
/var/run (or the like).
sh or
bash) knows it is a "login shell" if the
name of the program in argv[0] starts
with a dash.
The behavior of a wrapper is based on knowing its own "common" name. The specification of any other name might be treated as an "option" by a wrapper, in other words it might change the behavior of the program.
There are the suggested rules for dealing with the name of the program for all wrappers. Not every wrapper implements all of these, but most should follow the intent here:
argv[0]
to an explicit value, but is not required (nor
do I have a suggested name for this option),
argv[0] starts with a dash
then remove the dash and make this process a session leader
(see setsid(2)).
argv[0] is the empty string
it should be replaced with the default program name.
argv[0] is not the
common name of the wrapper it should be used as an explicit
argv[0] for the inferior process
argv[0] is just a colon (":"),
and there is no explicit utility specified
fork's a process
with the recovered data provided through one of two channels.
The ptbw wrapper makes an easy example.
We'll create a tableau of the first 7 ordinal numbers, then ask
a client instance to allocate the first 2, in that environment we'll
ask the shell to echo those numbers:
ptbw -m -J7 ptbw -R2 sh -c 'echo $ptbw_list'
The example above outputs "0 1", which are the first two ordinal numbers.
The data channel that ptbw uses to send the
shell the recovered data follows the same paradigm as
any "link" suffix variables: the common prefix for the wrapper application
specifies which data channel, the value of the variable specifies
the payload.
Sensitive data is always placed in a file, the name of the file should give little information to anyone reading the process table for arguments or environment variable values. This is just good application programming, not anything special to wrappers.
In this case we run the same master instance, but request the data
under the -A, the client program is
/bin/echo, rather than a shell:
ptbw -m -J7 ptbw -A -R2 echo
ptbw uses
the common dash (-) pseudo-client:
Which outputs a very descriptive table of the 6 tokens, their names, the fact that none of them are locked, and the request size.ptbw -m -J3 -R2 ptbw -
ptbw: master has 6 tokens, max length 1 (total space 12) Master tokens from the internal function: iota Index Lock Tokens (request in groups of 2) 0 0 0 1 0 1 2 0 2 3 0 3 4 0 4 5 0 5
For example screen builds a UNIX domain
socket and enters a "master mode" unless it sees a running socket
in which case it runs as a client.
But doesn't allow nested diversions -- that is to say you can't get
a nested screen to exchange data over the local socket with the
master above. Which really models a diversion stack of 1 instance.
The ssh-agent program builds a UNIX
domain socket, publishes an environment and acts as a master, with
other applications (ssh-add,
ssh) to act as the clients.
In this case it would make a lot of sense for ssh-agent
to allow nested agents to chain key requests, with keys that could be deleted by
just exit'ing the utility.
But it is presently limited to 1 level, and it checks the unix credentials of
every client access, which breaks access from escalated applications
(like op, sudo,
newgrp and su). I would
believe the file permissions on the socket, because that's what they
are their for. If you want to deny access by default, change the mode after
the bind, don't check on every connect for peer credentials. If you really
can't trust the superuser on the host you shouldn't be trusting the
binary for any applications.
The window program is a lot like
screen but less like a wrapper. The program doesn't
ever act as a client for
itself (it just starts new masters) when enclosed in itself. On the other
hand, it does confine itself to the window it is running in, so it
does have some recursive functions.
The X server provides graphics services to
processes that can see the environment variable
DISPLAY. It is not a true wrapper as
it only runs in the global mode (that is, it doesn't wait for
any wrapped process to exit). If you think of Xnest
as a client that creates a new diversion then that makes a whole wrapper
stack out of X. (But both programs should take
a utility specification.)
The nice program is a very primitive wrapper.
Any enclosed processes can see the effect, and enclosed instances
can modify the nice value. The same is true for chroot,
env,
nohup, su, and
newgrp because they all work on
attributes of a process.
Privilege escalation with sudo or
op might also be part of the same application class.
None of these use a socket to nest instances, and the tableau is the
process itself.
Another class would be debuggers, which start a process tree that is
served by an enclosing master: viz.
adb,
gdb, and
mdb.
One might include truss and
ptrace in that group in-spite of their
non-interactive nature. They don't use a diversion socket, but if you
are really desperate you can use one to debug itself.
ptbw
xclate
xapply's collation wrapper buffers output
such that parallel processes output a single collated stream without
blocking each other, as much as they otherwise would.
xapply
ptbw and
xclate to provide services.
wrapw
utility. This maps all the
traffic for the whole diversion-stack through a single socket.
hxmd
xapply this is not itself a wrapper,
but it knows host to setup xclate and
gtfw diversion.
sshw
utility enclosed by
a wrapw client instance on a remote host, proxy
the wrapped environment connections over the ssh
connection.
The wrapw proxy forward of the stack allows
the proxy back from the remote machine.
gtfw
sshw evem more useful
by adding a global file proxy (requires FUSE filesystem access).
$Id: wrapper.html,v 1.21 2012/10/07 01:02:53 ksb Exp $