HowTo - Intermezzo project

Make sure you also check out this wiki's FrontPage to see what other Intermezzo informations are gathered here.

The InterSync & InterMezzo HOWTO

Ongoing wiki effort started by the project members v1.9.1, March 2002

Configuration and Operation of the InterSync Daemon on Linux

Acknowledgements

Among the authors of this document are Peter J. Braam <braam clusterfs.com>, Gordon Matzigkeit <gord fig.org>

Disclaimer and License

InterMezzo is a new filesystem middlelayer. It contains kernel code and daemons running with root permissions and can have bugs. Please back up all data when using or experimenting with InterMezzo.

InterMezzo is covered by the GPL. The GPL describes the warranties made to you, and can be found in the file COPYING.

Copyright on InterMezzo kernel components is held by Cluster File Systems, Inc, Peter J. Braam, Stelias Computing, Carnegie Mellon University, Phil Schwan, Los Alamos National Laboratory, and Red Hat Inc., TurboLinux Inc., Tacitus Systems Inc., and Mountain View Data Inc.

InterMezzo is a trademark of Stelias Computing. It may be used freely to refer to the software on the InterMezzo Web Site (http://www.inter-mezzo.org/}.

More Information

There is a detailed description of the InterSync design available on the WWW. Look at http://www.inter-mezzo.org/docs/intersync.pdf. The kernel code is described in http://www.inter-mezzo.org/docs/intermezzo-2001.pdf.

These documents describe how the daemons function and the purpose of special Intermezzo files such as the Kernel Modification Log (KML), which contains the replication log, the Local Modification Log (LML), and the received file (RCVD).

Another article on InterMezzo can be found here: http://www.linuxplanet.com/linuxplanet/reports/4368/1/ The comments on the structure and history of InterMezzo are believed to be accuraet, but please be aware that some of the advice on specifics of configuration is outdated.

Installing and Running InterSync

Overview

The InterSync daemon depends on a kernel that has the InterMezzo file system installed. The InterMezzo filesystem is supported in current 2.4 and 2.6 series kernels. There are also some utilities to make and manage InterMezzo file systems.

Getting the Packages

You should download the following Packages from the web site:

The apache or ghttpd web server. Current version of the InterSync client appear to be configured to use Apache 2.0 rather than Apache 1.3 or ghttpd.
The intersync Package with the InterSync daemon and InterMezzo kernel module in it.

InterSync

InterMezzo

ftp://ftp.inter-mezzo.org/pub/intermezzo/intersync

InterSync is also available from CVS; see the CVS instructions on the InterMezzo web site.

Building InterSync and the InterMezzo Filesystem

In order to build a kernel module for your kernel, you need to have a configured kernel source tree which includes the dependency files generated during the kernel build. Install the kernel and sources as suggested by your distribution:

Proceed by first preparing your kernel sources, and then building InterSync and the module:

cd /usr/src/izo
./autogen.sh
./configure --enable-linuxdir=/lib/modules/2.4.9/build
make install
cd fs24
make install

The first make install will make and install the InterSync and various utilities, documents. The most important part of this step is the InterSync program and the cgi executable files. With default settings, InterSync will be installed to the /sbin directory, while the cgi executable files will be installed to /usr/lib/intermezzo/. The second make install will make the InterMezzo filesystem module and will install it to the compiled kernel's module directory.

Configure should correctly detect if user mode Linux is being used. You can also use your own configured kernel source:

cd /your/source/linux
make distclean
cp your.config  .config
make oldconfig dep

Running InterMezzo

Quick Start

By default, there is almost nothing to configure. First you'll need an intermezzo user and group in your system. TODO: what exactly do they need access to? Then if you want to sync ext3 filesystems on the server and the client. (For other filesystems see below.)

Mount your filesystem on the server and the client e.g:
- On the server: mount -t intermezzo /dev/hdaX /exports/server
- On the client: mount -t intermezzo /dev/hdaY /imports/server
Start InterSync:
- server: intersync /exports/server
- client: intersync --server="server-hostname" /imports/server

Note that there are currently two InterMezzo filesystem types: intermezzo is used for device-based filesystems, vintermezzo is used for running over systems like tmpfs.

Other filesystems than ext3

InterMezzo

-o cache_type=fs-name

The default cache_type is:

intermezzo: default cache_type=ext3.
vintermezzo: default cache_type=tmpfs.

Details

InterSync creates a configuration and cache directory under var/intermezzo-X, where X is the channel number.

In this directory is an apache configuration file and a symbolic link to the mountpoint. Httpd is simply running as: httpd -f /var/intermezzo-X/intersync-httpd.conf. If you are running Apache 1.3, you need to edit the intersync-httpd.conf file, substituting log_config_module for config_log_module .

InterSync uses a library of CGI executable files, which are installed in /usr/lib/intermezzo.

The .intermezzo directory in the mounted directory has a config file, which contains a UUID identifying the InterSync, the fileset name(if no fileset specified in the mount command line, it will be rootfset), the server if InterSync is run with the --server option, the server_port, the local_ip, and the local_port. The server/server_port specifies the intermezzo server and the port it binds; the server_port will be default set to 370. The local_ip/local_port specifies the ip/port the current httpd binds, the default value of the local_ip is "*" which stands for any ip address of the machine, and the default value of the local_port is 370 for the file server and 2432 for the client.These can be overridden in the intersync's command line. Here are two example config files:

On the server side:

  <profile>
    <uuid>e7830cf3-390d-45c1-994c-7ca21d97133e</uuid>
    <fset>testvol</fset>
    <server_port>370</server_port>
    <local_ip>*</local_ip>
    <local_port>370</local_port>
 </profile>

On the client side:

  <profile>
    <uuid>0b146b41-2840-405a-b9cf-c3a054148e4b</uuid>
    <fset>testvol</fset>
    <server>servername</server>
    <server_port>370</server_port>
    <local_ip>*</local_ip>
    <local_port>2432</local_port>
  </profile>

The fileset can be specified in the mount command line. You can build the InterMezzo filesystem using the utility that comes with it - mkizofs. You can get some help information via mkizofs --help. If you are building an InterMezzo filesystem on a block device, you can build the filesystem by using:

mkizofs -t ext3 -r testvol /dev/xxx

If you are builing on a loopback filesystem, you can build the file system by using:

mkizofs -t ext3 -F /DIR/filename

NOTE: mkizofs and httpd both need the intermezzo user/group, so please add it in the system's user/group database. You can manually add it in the /etc/passwd and /etc/group file.

Configuring InterSync

Config Files

The default behaviour of InterMezzo is that InterSync, the file server, when running on the server or cache manager when running on the client generates all configuration files needed.

However, the details are available below:

To ease the mounting of InterMezzo filesets, add one of the following to the /etc/fstab file. For testing and developing, using a loop device as the cache is easiest:

/tmp/cache /izo0 intermezzo loop,fileset=fsetname,mtpt=/mnt/izo0,prestodev=/dev/intermezzo0,cache_type=ext3,noauto 0 0

/tmp/cache is a file associated with a loop device, /izo0 is a mount point (a directory), fsetname is the name of the fileset, and /dev/intermezzo0 is the name of the presto device. The creation of the cache file and the presto device is explained in the examples at the end of this section. The kernel must be configured with loopback device support enabled to do this.

Using a genuine block device is a little easier, because you do not need to set up a loop device. To use the block device /dev/hda9, the /etc/fstab file should contain:

/dev/hda9 /izo0 intermezzo fileset=fsetname,mtpt=/izo0,prestodev=/dev/intermezzo0,cache_type=ext3,noauto 0 0

NOTE: While the lines may wrap in this document, the /etc/fstab entry should be a single line. The same holds for the following examples.

The mountpoint needs to be explicitly passed in the options (future versions of mount will not need this).

Other Files

The file /izo0/.intermezzo/fsetname/kml contains a kernel modification log (aka the KML) which keeps track of all of the changes made in an InterMezzo filesystem. The file /izo0/.intermezzo/fsetname/last_rcvd is the last_rcvd file which keeps track of the distributed synchronization file. In the current release of InterMezzo, the KML and last_rcvd files need to be created (usually by running mkizofs) before first mounting an InterMezzo filesystem.

Symlink /var/intermezzo/cache to the top directory of your mounted InterMezzo cache:

ln -s /izo0 /var/intermezzo/cache

Debugging

To help us find bugs we need logging information. The logs come in two places, from the kernel in /var/log/messages, and from InterSync to /var/log/intersync/.

The kernel debugging log slows things down enormously and is activated with:

echo 8192 > /proc/sys/intermezzo/debug
echo 1 > /proc/sys/intermezzo/trace

The debug mask is one of (or a sum of) the values in this table:

D_SUPER	1
D_INODE	2
D_FILE	4
D_CACHE(cache debugging)	8
D_MALLOC(print malloc, de-alloc information )	16
D_JOURNAL	32
D_UPCALL(up and downcall debugging)	64
D_PSDEV	128
D_PIOCTL	256
D_SPECIAL	512
D_TIMING	1024
D_DOWNCALL	2048
D_KML	4096
D_FSDATA	8192

These can be checked in the fs2x/linux/fsfilter.h or intermezzo_fs.h.

InterSync debugging messages also slow things down enormously, and are activated by passing a --debug X parameter on the command line. X is one of (or a sum of) the values in this table:

D_INFO	1
D_KML	2
D_UPCALL	4
D_HTTP	8
D_REINT	16
D_CHAT	1024

Contact Information

More information about InterMezzo can be found at http://www.inter-mezzo.org/.

General questions about InterMezzo can be sent to mailto:intermezzo-discuss@lists.sourceforge.net.

All InterMezzo related mail lists are archived (follow directions on the InterMezzo web site), so it may be worth checking here to see if your question has already been answered.

Bug reports should be filed on sourceforge. Please include the version of InterMezzo you are using and a description of your system configuration and the problem observed.

Also, please include all relevant logs:

/var/log/messages and /var/log/intersync/ for server and clients.

Back to FrontPage