Much of this comes down to local policy:
The ability to complete each of those tasks with certainty is clearly the goal. The administrator (or software engineer) should know which state each the master source structure is in as she commits changes and updates client structures. Driving a powerful software change agent without clear rules-of-the-road always leads to tragedy.
Stability is knowing that you are applying the correct change to the correct hosts at the proper time and revision.
That need alone should drive any local installation to adopt something
like msync, even if the local policy is not
exactly the same as my local policy.
What we are onto here is a policy that drives all software builds in a way that makes them all look the same to the installer. A few well documented commands and rules give enough structure to aid us without limiting our powers. Counter to our more childish reasoning some structure actually removes limits.
make
recipe file. They let msrc select the values
for SUBDIR, IGNORE,
HXINCLUDE, MAP,
SEND, and MODE while
only explicitly setting INTO.
This is because copies of
the directory unpacked under /tmp would not be
able to find the correct value of INTO.
The macros listed in the above are not the only macros we'll need to
assure that we have all the files we need to run msrc.
We may need files that m4
include's, or files that
provide ssh keys... or who-knows-what else.
For just this reason msrc depends on the
make target __msrc.
We can bind the gathering of our resource to that target.
Here is an example from the source to msync itself:
# ...Id: Makefile,v 4.3 2009/02/15 16:19:00 ksb... # meta makefile for msync, msrc version 2008 INTO= /usr/src/local/sbin/msync GEN= SOURCE= Makefile Makefile.host README msync.html \ msync.ksh msync.man clean: FRC rm -f ${GEN} source: ${SOURCE} ${GEN} ${SOURCE}: co -q $@ FRC: # msrc patch to get up-to-date and checked-out sources __msrc: source
The two macros added by this convention are
the ones msync expects GEN
and SOURCE.
The pseudo target source
is a prerequisite for __msrc, this
requirement run co to gather files from
RCS (in the example). It could gather other files from the
GEN macro, if we had recipes for any
of them.
The clean target removes any
generated files to restore the directory to a "pure" state.
rcs) as an
example, and stubbed in some code for CVS. There is no reason why
git could not work as well.
No program which shares data between logins will work without cooperation from the file-system: the modes and groups on the master sourced files must allow access to all the controlling parties. Usually a group (like "source", or "staff") selects the logins that are in control.
That's why msrc checks the group access and
group membership for each file and directory.
That kind of disruptive change can ruin even the best laid support plan. To make that even worse the odds are pretty fair that support for the older version will continue for some time after the introduction of the new version.
Always include some mechanism which allows the human controlling the
structure to get solid feed-back that the update staged is the update
intended. That is one of the key benefits of msync.
msrc application reads a few
msrc macros from the recipe file to
select the disposition and role of each file in the directory
(see the HTML document for details).
The msync application reads another macro:
SOURCE.
As it says in the msrc documentation, it is
pretty standard the use the SOURCE macro
as a list of all the files required to update a client host.
This is so common that msync requires it,
but it does not require the either of the explicit dependency lines:
or__msrc: ${SOURCE} ${GEN}
__msrc: source
If the SOURCE macro doesn't contain a list
of all the plain files in the target directory msync
carps about the missing or extra files. This should help assure
that there is a way for the make recipe file
to summon the sources from your local revision control system when
they are missing.
The files listed in the GEN macro are
exempt from the paragraph above. They shouldn't overlap with
the SOURCE list, but make
will complain about that for us, so we don't check that.
msync to your site's
local policy. The expected source control group is the first.
I always assume that not every UNIX™ login on the host is authorized to make changes to source files. The ones that are should have a common group membership, so that file permissions (viz. group write) are enough to let engineers do their work.
I always use the group named "source" to carry this credential, but
you may pick another name. If you need to change it, set the
environment variable MSYNC_GROUP to the name of
the expected group.
If you need to change that for a specific directory, you can use a marked line in the control recipe:
# ...Id: Msrc.mk,v 2.13 2012/02/15 16:19:00 agt... # meta makefile for superusers... # $Msync(group): echo wheel INTO= /usr/src/local/...
The mk marked command outputs the name of the
expected group owner for all the files in this directory. (In this
case "wheel".) In later releases I may use other submarkers to
provide more meta information.
The -G option to msync
outputs only the recovered group name.
GEN macro and link it
to the __msrc target, the call to
msrc from msync actually
builds the generated files. This is good because that way
msync can check that all the generated files
are built.
But it is not so good when it leaves the generated files in the
directory. The best way to trigger the __clean
target in the control recipe is to provide the -m
option to msync. The best way to do that is
either with a rule in the recipe file, or a marked line.
The mk markup below, when added to the control recipe,
tells msync to
update the msync target rather than doing
the check itself:
This causes# ...Id: makfile,v 1.7 2012/01/15 20:00:00 mjb... # $Msync(target): ${echo:-echo} msync
msync to set a variable
MSYNC_CHECK to "false" which prevents
it from making the check for the markup (so it won't infinity
recurse looking for the target again. That means the target
can call msync with the correct options:
msync: FRC msync -m __msrc:__cleanup
It would be nice if msrc would read the same
markup from the recipe file. Msrc doesn't because
it expects any command-line specification to come from
the recipe file (via an update rule) or from a recipe above.
By adding mk markup processing to
msrc we'd muddy the waters too much.
On the other-hand using mk markup for
the sanity checks msync works because
the information we extract is largely optional (you can always
set the group in the environment, and put the synchronization
update rule in yourself). In fact these are just my local site
policy extended into example code you should tune locally.
If you would rather, you can use your own script and mk
markup. Use a marked line to trigger the correct call to
msync (or your local policy checker):
# ...Id: makefile,v 11.3 2013/02/15 16:19:00 mjs... # $CheckSync: ${msync:-msync} -m source:clean
In any case, I would keep the synchronization information in the control
recipe, not in a separate file. I would only code a replacement for
msync when local policy is
very different from my site policy.
msync for level 3 packagesmsrc from pushing the
directory to another machine. This is accomplished by setting the
INTO make macro
to a multi-word value with underbar prefixes on each word.
For example:
When you try to check the consistency of such a directory with# ...Id: msrc.mk,v agt... # meta makefile for the turbo package... INTO= _This _is _not _cool _to _build _via _msrc. ...
msync you get an error that tells you
"msrc doesn't love this directory."
There is a common idiom to allow the check of the level3 package as if it were a level2 (which is really is for the mechanics of revision control). Add a target to the recipe that looks like this:
That target overrides the value of# Allow an msync check msync: FDIR=$$(mktemp -d /var/tmp/$${USER:-nobody}XXXXXX) && \ MSRC="-y INTO=$${FDIR}" msync $(MSYNC_OPTS); \ rmdir $${FDIR}
INTO for
the calls to msrc that msync
makes, so that we don't fail the macro extraction.
Use the mk markup above,
"Msync(target): msync", to tell
msync to use this spell when it is called
directly from a shell.
The recipe file contains a stanza to collect each directory.
Locally we use rcsvg (see
the source directory) to
extract each of the components. As an example, here is the
recipe that extracts mkcmd's source
from the msrc_base package:
${STAGE}/local/bin/mkcmd: ${STAGE}/local/bin cd ${MSRC}/local/bin/mkcmd && rcsvg ${VGOPTS} -S $@ Eight
Since every stanza that collects a product starts with the same
command prefix (viz. cd ${MSRC}/path && rcsvg ... -S)
we can code a sed script to filter out the
extraction commands and convert them to msync
commands:
sub_sync: FRC tr -s ' \t' ' ' <Makefile | sed -n \ -e 's!^ cd $${MSRC}\([^ ]*\) && rcsvg.*-S $$@ *\([A-Za-z0-9.\\$$]*\)$$!msync \2 ${MSRC}\1!p' \ -e 's!^ rcsvg $${VGOPTS} -S $$[{STAGE}@]* \([A-Za-z0-9.\\$$]*\)!msync \1!p' |\ sh -e
This pulls out the directory and symbolic version from the extraction
commands in the recipe file to build the msync
commands which check for the site policy invariants. But, I'd be the first
person to admit that this is a less than clear example of
sed magic. It looks worse that is actually is
here because make eats dollar-signs, as does the shell.
(The second sed match redundantly matches the
package directory, but forces the correct symbolic release.)
It is better than coding a separate list because the two lists could list different symbolic version specifications or product lists.
To trigger the more detailed scan just used make
to update the sub_sync target.
msync would not
complain about the INTO value, which was a bug.
Now that it is fixed, we need to work around it to check the (few)
level 3 packages.
The msrc
HTML documentation and its
manual page.
$Id: msync.html,v 1.19 2012/04/02 18:32:34 ksb Exp $