To understand this document

This document assumes you are quite familiar with the standard UNIX™ tools make(1), m4(1), and the shell sh(1). In addition, you should have a working understanding of the build process for a program, and some experience configuring multiple hosts under multiple operating systems. Some experience building large software structures would be helpful.

The mmsrc program is largely used to boot-strap the ksb tool-chain. There should be a make recipe file in the root of the distribution you unpacked that told you to come to this directory to install mmsrc. That task is described in the boot-strap HTML document, not in this (more general) document.

Mmsrc is a replacement for the program makeme, which is included in this directory with some modifications. If you use that script at your local site you should install the new one over the top of the old one. Check for any "ports" edits to the file, like setting "HOSTTYPE" to avoid the zsh bug as well.

The command line options to mmsrc

For the purposes of this document, you don't have to understand many of the command-line options to mmsrc, or even why they are included in the program. We present the command-line usage summary below for quick reference (we'll use this later):
mmsrc [-z] [-d flags] [-f makefile] [-m prereq][:postreq] [-u login] [-Pjobs] [-y yoke] [hxmd-opts] [utility]
mmsrc -b [-d flags] [-f makefile] [-m prereq] [-y yoke] [macros]
mmsrc -h
mmsrc -V

After you've installed the real tool-chain and have done a site configuration the hxmd-opts (hxmd options) below become far more useful:

[-B macros] [-C configs] [-D name=value] [-E compares] [-G guard] [-I dirname] [-j m4prep] [-k key] [-o attributes] [-U name] [-X ex-configs] [-Y top] [-Z zero-config]

The flow of a master source build

The master source structure runs the same program on many hosts in a common environment. Think of this as a network-aware version of ssh and some rsync or rdist magic. In the msrc HTML document, you will read about the whole master source structure, but for now you only need the out-line.

Each master source directory has at least 1 file in it: a make recipe file. Said file could even be zero-length, but it must exist. This recipe file (called makefile in the command-line usage) controls which files get sent to the target environment, and which files get left behind. It also may specify that some files should be "mapped" as they are sent to the target through m4.

The key difference between the full version of this program, spelled with 1 "m" (as msrc) and this version spelled with 2 "m"'s (as mmsrc) is that this version is not network aware -- it only works on the local machine and doesn't use any special tools, not even ssh. The other difference is that the real version is way, way more powerful.

Since the whole environment is built from itself, you need this version to get the ball rolling on the first host you touch. The "genesis" process for other local nodes is much simpler (and is covered in another document, which you should read much later).

Using mmsrc as a master source boot-strap tool

Any master source directory with a "Make.host" and a "Distfile" is the older version and still uses "makeme" to build. Others without those 2 files are the 2008 version and need either msrc or mmsrc to operate.

Version 2008 directories have a make recipe file named either "Msrc.mk", "msrc.mk", "makefile", "Makefile", or a script that specifies a different makefile under -f. Usually, I call the recipe file "Makefile" to let it sort to the top of any ls output.

To boot-strap a 2008-style directory in a local environment use mmsrc. To do that, you'll need to pick a local directory where you can make a shadow-copy of the directory. This mocks the normal flow, where we would copy the source to a remote machine, but without the use of another host or the network. Normally, one would use a subdirectory of /tmp. I'm going to use /tmp/boot.me, you might replace me with your login name.

Build the directory to contain the environment: 

$ mkdir -m 0750 /tmp/boot.me
Change your current working directory to the directory you'd like to invoke: 
$ cd /usr/msrc/local/sbin/hxmd

There are macro attributes for a host that you might not have until you build a site configuration, For example, to build a ksb tool we need to know the build-type of the local host (which is normally an attribute in the site configuration). But we'll hope that the directory in question doesn't break (in a bad way) when those are not set, or that you know which -D options to specify on the command-line.

In this example, I'm building one of my tools, so I'm forcing the "HOSTTYPE" to be the one for FreeBSD on the command-line. Run mmsrc to build the local environment under your preferred temporary directory (using -y): 

mmsrc -y INTO=/tmp/boot.me/hxmd -DHOST=localhost -DHOSTTYPE=FREEBSD ls
you should receive a plausible output from ls, which doesn't contain the "Makefile.host". This is because the contents of "Makefile" are the mapped (m4 maco expanded) contents of "Makefile.host". Mapping files, whose names end in ".host", is how the master source structure customizes payloads for each target environment.

At this point, you can build the program in one of two ways: cd to the newly updated directory then run make, or use mmsrc to run the build by replacing the "ls" command with "make all install". If you need to run sudo, op, or su to finish the installation, you might choose to cd there. If you are already able to write in the destination, you might just run the installation command on the end of the msrc line.

After you've built all the stuff you need, you can remove the temporary directory, as it was just built as a shadow-copy of the master copy (and may be re-created at any time): 

rm -rf /tmp/boot.me

This is not the only use of mmsrc

In some cases, it is clever to pass hxmd configuration files on to a remote host (possibly from more than 1 source) to merge them on a neutral or common host. On that host, one might not have the entire ksb tool chain installed, but mmsrc doesn't require the whole chain to build.

In other cases, the full-blown msrc under the -l option, is more versatile; it allows access to the xapply-stack as well. That provides parallel processing and other wrapper tactics that might help you later.

The ENTRY_LOGIN and ENTRY_DEFS attributes

Since there is no actual remote shell started by mmsrc, one might think that the entry attributes are useless. This is not always the case, as they are presented to m4 and may be included in any customized file and used on the local machine to reach across to the target host by some means beyond the grasp of the normal msrc tactics.

To allow better command-line compatibility with msrc, the command-line option -u accepts a default login for ENTRY_LOGIN.

An example of this use would be data collection from the master source host to each target host. A local script would be customized with m4 markup to sample a single host from the target's shadow directory, then report the recovered data in the desired format to stdout.

The -b option

Some versions of make have a command-line option that requests the output of just the value of a macro. Under FreeBSD this option is -V. Since later tools in the chain depend on this feature, and mmsrc has the code to extract macro definitions from make recipe files, we put the functionality in mmsrc.

The command-line option -V is reserved in the ksb tool chains for version, so I moved it to -b.

Under -b, the default prereq value is hard-code to __msync, unless -m is specified (usual default of the program's base-name is ignored).

The default prereq matches msrc

The default prerequisite update target matches the msrc program name (as otherwise we would use our program name prefixed with 2 underscores -- __mmsrc) to make it much easier to use this program as a replacement for msrc. If you would like it use the current program name to generate the prereq, specify the double-colon target. For example: 
$ mmsrc -m :: -V
...
mmsrc: make hook: __mmsrc
...

I'm not sure why you'd want to do this, but you should be able to.

The default HXINCLUDE file doesn't match msrc

Mmsrc mocks actually being msrc in every way except the default file it consults for hxmd options. When no files are specified in the HXINCLUDE macro, msrc looks for the file msrc.hxmd in the directories listed in $MSRC_PATH (which defaults to /usr/local/lib/msrc:/usr/local/lib/hxmd).

Because mmsrc is used on systems were msrc might not even be installed, it searches for the file mmsrc.hxmd. If you want that file to match the msrc default, build a symbolic link to the common one.

Like all of these tools building a link to the program to call it by another name impacts the name of the requested defaults file. And like most of these, it will make sense after you have used it and read everything 3 times and used it for 6 months.

More information

If you read this far and didn't click any links you really need to chain to the msrc HTML document, because this document assumes you know what msrc does. However you may want to start with the xapply HTML document then the hxmd HTML document as those help build a vocabulary you need to read about msrc.

These tools and documents are safe and effective for most application programmers and system admins, but use them only as directed: good luck solving your local configuration management issues.

Kevin S Braunsdorf, Dec 2010 (kevin.braunsdorf at GMail) 
$Id: mmsrc.html,v 1.20 2012/03/29 19:35:06 ksb Exp $