Sympa Virtual Robot Domains, Apache and Postfix HowTo

Introduction

I have just finished installing Sympa primarily to provide mailing list service for a domain that is not the primary domain of my server.  This required the coordination of three pieces of configuration: While the Sympa documentation provides outlines for what has to be done for each of these tasks, I discovered that there wasn't a document that explained the whole process in one place.  I also struggled with the mechnisms proposed to handle the connection between Postfix and Sympa.  I came to the conclusion that Postfix FAQ documents on the Sympa web site do not provide a good working solution to the problem.

I was fairly naïve about all three tools before I started: in the light of my experiences I thought it would be useful to give a fairly detailed account of getting things running - apologies if this document has got a bit verbose!  This document explains my solution and the additions to the Sympa configuration specification and alias manager that I have made to implement the solution.  It then documents the extra configuration process needed to support Sympa on a (local) server that runs Apache version 2 and Postfix.  The solution is currently implemented for aliases written to text files rather than using a database such as MySQL, but it would be simple enough to incorporate an equivalent solution  in a database driven system.

This dosument is placed into the public domain and may be copied, distributed and modified freely.

Sympa/Postfix Interface Requirements

My requirements for Postfix required it to support multiple domains on a Linux box (a Gentoo distribution but that is more or less irrelevant).  The 'master' domain is associated with local Unix accounts - let us call it master.example.com - and mailing lists can be provided by Sympa using the standard alias mechanism for  Sendmail/Postfix.  The situation that created these requirements requires one (or more) alternative domains which will have a web site hosted on the same machine using Apache vhost capabilities and Sympa-driven mailing lists but there will not be any local Unix accounts associated with mailboxes for the domain.

Supporting such a domain with Postfix is most conveniently done using a Postfix virtual alias domain: the Postfix Virtual Domain Hosting Howto shows the various options and outlines what is needed to support a virtual alias domain.  Virtual alias domains do not necessarily expect to have local mailboxes but instead expect any mail address in the virtual domain to be mapped or aliased to another addrress that could be local or remote, and then delivered or relayed to thjis address.  The main difference that is relevant here is that virtual aliases are placed in different files from ordinary aliases and have slightly different semantics:  in particular virtual aliases cannot directly map to a command 'pipe' unlike standard aliases.

Creating a Sympa mailing list introduces up to six related email addresses: the main address used for submissions to the list (e.g., list@project.example.com) and up to five (depending on the facilities offered by the list) auxiliary 'control' addresses - list-owner@.., list-bounce@..., list-subscribe@..., list-unsubscribe@.. and list-edit@project.example.com.  The extra components (-owner, etc.) are fixed in number and common to all mailing lists and to whatever domain they are related.  All these list addresses have to be redirected to a Sympa process by Postfix (or whatever other SMTP agent is used), and the bounce  addresses go to a different process from all the others.

The inability of Postfix to redirect through virtual aliases to a command pipe means that another means has to be found to pass incoming messages to the Sympa processes. There are various possibilities:
is added to the virtual aliases file.  The Sympa alias_manager.pl script then adds aliases to the main alias file that send the synthesised local addresses to the Sympa processes as described in the Sympa manual Mail Aliases section.  This scheme works but could lead to a lot of bounced messages rather than SMTP 550 error reports if a spammer targets the domain with random email addresses because the virtual alias will match any such address as described in the Sympa Postfix FAQ (I will call this 'the wildcard match problem' - it appears again below).  The bounces could potentially overload both Sympa and Postfix.
This scheme suffers from the same wildcard match problem as the first scheme above:  the regular expressions and aliases match any user part in email addressed to the virtual domain.
The use of Postfix transports does however seem to provide a neat solution to the problem.  Analysing what has to be passed from the incoming email list addresses to the Sympa processes, we see there are three pieces of information:
  1. The (virtual) domain name - potentially infinitely variable.
  2. The mailing list base name - infinitely variable.
  3. The mailing list extension for the auxiliary control addresses or null for the main list address - six fixed values used with each domain/base name pair.
Since transports have to be statically configured in the Postfix main.cf file and the Sympa action associated with each type of auxiliary control address and the base lists are the same whatever the mailing list or domain, it seems appropriate to map the mailing list auxiliary control extension types and the base address type onto six transports.  Whenever an address is to be delivered through a transport, the base and extension addresses are separately available for use in the parameters of the transport. If we can arrange that the mailing list name and the virtual domain name are combined into the user part of the address that is passed to the transport and the transport selected corresponds to the type of auxiliary control address or mailing list base address, then we can have a standard set of  trasnsports that will work for all mailing lists and arbitrary domain names; it will only be necessary to create virtual aliases to do the mapping.  Adding the following six entries to Postfix's master.cf should do the trick: 
# Sympa mailing list manager transports
sympalist        unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}@${extension}
symparequest     unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-request@${extension}
sympaeditor      unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-editor@${extension}
sympasubscribe   unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-subscribe@${extension}
sympaunsubscribe unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-unsubscribe@${extension}
sympabounce      unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/bouncequeue ${user}@${extension}
The transport that Postfix uses to deliver or relay the address that results after it has been transformed by the various rewritngs, including any applicable virtual and standard aliases, is selected by entries in the transport map.  A transport map can be a set of regular expressions and corresponding transport/next hop selections to be used when the transformed ('rewritten' in sendmail speak) address matches the pattern. Since the scheme proposed above expects the transport selected not to depend on the user part of the rewritten address, the transport should be selected according to the domain part of the rewritten address.

I initially assumed that this would mean that I would have to define six 'relay domains' that could be used for this  purpose, but it turns out that Postfix is happy to have dummy domains that are used in the mapping process so long as it has a transport to associate with them.  It isn't necessary to declare them as any particular sort of domain in Postfix's master.cf file.  This means that the virtual alias file maps mailing list addresses as follows
<list_name>-<aux_type>@<virtual_domain> -> <list_name>+<virtual_domain>@<aux_transport_domain> # One of five domains
<list_name>@<virtual_domain>            -> <list_name>+<virtual_domain>@<list_transport_domain>
Exact match expressions for each mailing list defined can then be written to a virtual aliases regular expressions file avoiding the wildcard match problem.  For example:
/^(list_name)-(request|editor|owner|subscribe|unsubscribe)\@project\.example\.com$/
 
$1+project.example.com@sympa$2.
 
/^(list_name)\@project\.example\.com$/
 
$1+project.example.com@sympalist.
The transport map contains fixed regular expressions that map from the dummy domains to the corresponding transports.  The  maps cope with any legitimate mailing list name and are common across any virtual domains that might be defined:
/^.*\@sympalist$/         sympalist:
/^.*\@symparequest$/      symparequest:
/^.*\@sympaeditor$/       sympaeditor:
/^.*\@sympasubscribe$/    sympasubscribe:
/^.*\@sympaunsubscribe$/  sympaunsubscribe:
/^.*\@sympaowner$/        sympabounce:
To use this scheme alias_manager.pl and the alias template, list_aliases.tt2, have to be adapted somewhat.  It is also probably more covenient to keep the virtual aliases  for each virtual domain in a separate file.  I have suggested some changes to the Sympa configuration that will allow the alias_manager.pl to know that a domain is using virtual aliases and optionally name the file to be used for holding the virtual aliases.

That's the theory.  Now on to configuring the various parts.

Order of Events

Since the configurations are somwhat interconnected, it is best to carry out the configurations in the following order:
  1. Install the basic Apache, Sympa and Postfix using a non-virtual host.
  2. Set up the DNS records for the virtual domain.
  3. Configure the virtual host and, if necessary, a suitable Fast CGI server in Apache and get its web site working.
  4. Modify the Sympa code as suggested here.
  5. Configure the virtual robot domain in Sympa and run it up at least once to create an initial virtual aliases file. Test that the web interface starts as expected.
  6. Configure Postfix to use the virtual alias domain.
  7. Test the whole configuration.
  8. Enjoy the fruits of your labour (or panic).

Configuring Apache 2 for Virtual Hosts and Sympa

Apart from the Sympa documentation and the main Apache documentation, you may find the Apache HTTP Server Wiki useful.  In particular it has a handy page that shows how Apache's files are organized in many of the common operating system distributions.  This is useful because configuration for both virtual hosts and non-core modules (such as the Fast CGI modules used by Sympa) is sometimes placed other than in the main httpd.conf file.

Also if you are not an Apache afficionado, there are a couple of useful tutorial courses available on the University of Cambridge Computing Service website:
I'll assume that you have a basic Apache 2 configuration installed and running in what follows.

Mailing List Names, DNS Setup and Web Site Layout

The domain that we are setting up is project.example.com.

Virtual Hosts

A single instance of the Apache 2 web server can be configured to service web sites associated with multiple different domain names.  The web server is then said to be servicing several 'virtual hosts'.  Apache 2 can distinguish which host configuration to use either based on the domain name used in HTTP requests (name-based virtual hosts) or, if the interfaces of the machine running the server have more than one IP address, based on the IP address on which HTTP requests are delivered (IP address-based virtual hosts).  Both types can also be combined with a specific IP port on which Apache is listening.

If the machine only has a single address, then it is simplest to stick to name-based virtual hosts, and this section assumes that we will distinguish between web sites through the domain name in the HTTP request.  Note that simple minded name-based virtual hosting won't work if you want to use SSL/TLS to secure the web site because the domain name is still encrypted at the point where Apache would otherwise choose the virtual host to which the request is directed.  If you want to do secure (HTTPS) hosting on a single IP address machine, you will have to use port numbers to distinguish virtual hosts.  Clients will need to be aware if non-default ports are used (i.e., not port 80 for HTTP and 443 for HTTPS) so they can specify the port in HTTP requests.  This isn't discussed any further here.  The topic of virtual hosts is discussed at length in the Apache documentation at Apache Virtual Host documentation.

Use of name-based virtual hosts is turned on by including one or more NameVirtualHost directives in the Apache configuration file (httpd.conf or a file 'included' into httpd.conf). NameVirtualHost takes a parameter which specfies the IP address(es) and port(s) where requests for name-based virtual hosts will be received, such as:
   NameVirtualHost *:80
This example sets up Apache to treat any request arriving on the standard HTTP port 80 at any IP address as a request for a name-based virtual host.  This has the side effect of stopping Apache from serving any main (non-virtual) host configuration on port 80 (unless you then don't actually define any virtual hosts).  This means that you should probably service all the web sites through virtual hosts if you have at least one virtual host candidate - making things nice and symmetrical if you are configuring Apache from scratch but if your server is already configured to serve a single main domain on port 80 through a non-virtual host, you will probably have to change this domain into a virtual host domain as well.  This is not a big deal.

A NameVirtualHost directive can appear anywhere in the configuration file - it doesn't need to come before the virtual host definition(s).  The NameVirtualHost directive for an (IP address, port) combination should appear just once, even if it applies to several virtual hosts (as with the address wildcard example above).  In practice it doesn't much matter if multiple wildcard NameVirtualHost directives are used - you will just get warnings ("NameVirtualHost *:80 has no VirtualHosts") for the extra directives when the server starts up, but specific (IP address, port) pairs should appear just once.

The configuration specific to each virtual host is then specified enclosed in pairs of <VirtualHost> and </VirtualHost> directives.  The <VirtualHost> directive also takes a parameter that specifies the IP address and port on which requests will be received for this host.  The Apache2 documentation for <VirtualHost> states that this address must exactly match the address given with (one of) the NameVirtualHost directive(s) when using name-based virtual hosts.  Also it is essential that the web server is also told to Listen on these addresses - neither <VirtualHost> or NameVirtualHost forces the server to actually set up open sockets on the specified addresses.  There is a useful article - An In-Depth Discussion of Virtual Host Matching - which discusses how Apache decides which virtual host gets to service an incoming request.

Many Apache directives can be used in the virtual host specific configuration.  You will need at least ServerName to define the host name used by this virtual host and and DocumentRoot ro specify the place in the server directory tree to find files to be served; other frequently used directives include ServerAdmin specifying the webmaster's email address for this virtual host and ServerAlias giving a list of one or more alternative host names for the virtual host.  Here is a basic virtual host configuration.
   NameVirtualHost *:80

   <VirtualHost *:80>
      ServerName   www.project.example.com
      DocumentRoot /home/www/project
      ServerAdmin  webmaster@project.example.com
   </VirtualHost>
Notes on the basic configuration:

Providing a Sympa Web Administration Interface via the Virtual Host

The Sympa administration interface WWSympa is provided by a single CGI server script wwsympa.fcgi.  This is a very large chunk of Perl code and it is best run using FastCGI thereby remaining in memory so that it does not have to be reloaded and recompiled for each client request.  See the WWSympa web page for more details.

The main Sympa documentation for version 5.3.x suggests using the original FastCGI implementation provided by mod_fastcgi for Apache.  The documentation for version 5.4  provides documentation for both mod_fastcgi and the more up-to-date mod_fcgid that was previously covered in the Sympa FAQ .  Although mod_fastcgi can be made to work with Apache 2, it is primarily intended for Apache 1 and the basic code has not been significantly altered since 2003.  Several Linux distributions have deprecated mod_fastcgi because it is barely maintained and also because of the licensing situation.  There are also some reported problems in situations where large numbers of requests are to be handled.  The alternative mod_fcgid is much better supported and is released under the GNU GPL: it provides the functionality and performance needed by the Sympa web interface.

On the other hand, mod_fcgid does not support (AFAICS) the use of external (independently managed) Fast CGI applications that are not managed by Apache's process manager.  So if you need this capability, or if you are still using Apache 1, or if you are using another web server,  you may be stuck with mod_fastcgi. See the configuration notes  below.

Common Configuration for mod_fastcgi and mod_fcgid

Whichever of the Fast CGI modules you use, or if you choose to run with the basic CGI mechanism, the URL that is to select the Sympa web interface starting page needs to be set up to invoke the wwsympa.fcgi Perl script.  The script is stored in the bin directory of the Sympa installation (home/sympa/bin/wwsympa.fcgi for our example).  It has to be run as user sympa rather than as the user id of the Apache server (apache).  In my experience, at least for version 5.3.4 of Sympa, using sudo to set the user executing the script is the only way that works reliably.  I haven't yet tried the C wrapper provided for verion 5.4.  Edit /etc/sudoers (using visudoers while logged in as root) to add the line
   apache ALL=(sympa) NOPASSWD: /home/sympa/bin/wwsympa.fcgi
You should also check that the requiretty and env_reset flags are not set by default in the sudoers configuration file - comment out the default lines as shown here:
   #Defaults    requiretty
   #Defaults    env_reset

With requiretty set, sudo would only run when the user is logged in to a real tty; with env_reset set, most of your environment variables would be ignored... including your server name, the URL requested, etc.

Since the wwsympa.fcgi script is not located under the DocumentRoot for this host, the Apache virtual host configuration for www.project.example.com then needs to map the URL http://www.project.example.com/sympa to the sudo wrapper script wwsympa_sudo_wrapper.pl rather than direct to wwsympa.fcgi.  The configuration needed to pass this script to the Fast CGI module for execution is covered below.  To map the URL add the following directive to the <VirtualHost> sub-directive:

   ScriptAlias /sympa /home/sympa/bin/wwsympa_sudo_wrapper.pl

Ensure that the fitrst argument is /sympa and not /sympa/:  the example given in the Apache documentation for mod_alias uses /cgi-bin/ but this does not work correctly for Sympa.  This is because the Sympa interface accesses both www.project.example.com/sympa and many pseudo-sub-pages such as www.project.example.com/sympa/help rather than just scripts in /sympa. Using ScriptAlias to do the mapping automatically implies that the script is a CGI and so it is not necessary to explicitly specify Option ExecCGI for this location.  If you are not using one of the Fast CGI options this is all the configuration that is needed to get the script executed, but you need to tell Sympa whether you are using Fast CGI or not.

Additionally, you need to map the URLs used by wwsympa.fcgi to access static content to the location in the Sympa installation where this is stored.  Add the following directive to the <VirtualHost> sub-directive:

   Alias /static-sympa /home/sympa/static_content

Using mod_fastcgi

If you are stuck with mod_fastcgi, here are some notes on getting it to work.  Since it is not supported by all distributions, you may meed to compile it manually.  Additional information, including hints on how to compile and install the module, can be gleaned from
 The latter article was written back in May 2007 and is based on mod_fastcgi version 2.4.2 (from 2003).  Since then a new version has been made available that incorporates the necessary patches plus some other fixes targeted mainly at the Debian Linux distribution (YMMV on other distributions!) .  If you are starting with the latest version (2.4.6 as of 1 April 2008) the changes to fcgi.h described in the 'Ruby on Rails' article and elsewhere have already been applied.

The Sympa documentation suggests setting up wwsympa.fcgi as a 'static server' which is essentially continuously running.  Assuming you are using the sudo mechanism, this is done with the directive
   FastCgiServer /home/sympa/bin/wwsympa_sudo_wrapper.pl -processes 2
This directive applies to the whole Apache server - if you are defining multiple virtual hosts using Sympa mailing lists, this directive is needed only once in the main configuration area. Lots of configuration options can be specified with FastCgiServer - they all are given as arguments to the directive.  We just ask for two servers to be started at initialization.

One problem that has beeen reported with mod_fastcgi results in getting 'error 500' from the server because it can't correctly access the Unix Domain socket used to communicate between mod_fastcgi and the main Apache process (see the Sympa FAQ).  This problem is usually related to the permissions on the directory used to store the pseudo-files associated with these sockets.  By default with Apache 2 these are stored in a directory below the server root ${ServerRoot}/logs/fastcgi.  If it doesn't exist mod_fastcgi tries to create it with the right ownership and permissions - owned by the Apache server user and group with permissions 700 (i.e., owner can do anything, group and world can do nothing).  This directory was changed from the original /tmp/fcgi at version 2.4.0  - several documents about this problem refer to the old directory and problems that occur when /tmp is periodically cleared out by a cron job.  Solutions involve either fixing the permissions so that Apache can create the directory or altering the default directory using the FastCgiIpcDir directive to reposition the directory to somewhere where it can create the directory:
   FastCgiIpcDir <runtimedir>/fastcgi
This directive has to be placed between the User and Group directives and the FastCgiServer directive in httpd.conf if it is to have any effect.  Wherever the directory is located mod_fastcgi needs to be able to set the permissions required and must be able to search the whole path to this point.  There are circumstances in which you might want to change this, for example if you are running multiple Apache servers. If you using dynamic servers the sockets will be placed in the dynamic sub-directory of the FastCgiIpcDir which will also be created if it doesn't exist.

Finally, the Apache server needs to be told which CGI handler to use when www.project.example.com/sympa is requested:
  <Location /sympa>
    SetHandler fastcgi-script
  </Location>
The name of the handler is hard coded into the module and must match the Fast CGI mechanism you are using.  Depending on the default permissions for Locations you might need to add an Allow directive as well.  This, or a similar set of directives, is needed for each virtual host, as are the ScriptAlias and Alias directives described in the common configuration section.

So the complete set of additional Apache configurations for using mod_fastcgi and a named virtual host should look something like:
  # Fast CGI module
  LoadModule fastcgi_module modules/mod_fastcgi.so
  # If you have problems with the default socket directory, choose another directory which mod_fastcgi can create/use
  # FastCgiIpcDir logs/fastcgi
  FastCgiServer /home/sympa/bin/wwsympa.fcgi -processes 2

  # Use named virtual host(s) for all requests on port 80
  NameVirtualHost *:80

  # Virtual host setup for www.project.example.com
  <VirtualHost *:80>
     # Basic configuration
     ServerName   www.project.example.com
     DocumentRoot /home/www/project
     ServerAdmin  webmaster@project.example.com
     
     #Use the FastCGI server to process Sympa web interface script
     <Location /sympa>
        SetHandler fastcgi-script
        # Maybe change permissions if need be
        # Order allow,deny
        # Allow from all
     </Location>

     # Aliases to redirect Apache for scripts and documents not in the DocumentRoot directory tree
     ScriptAlias /sympa /home/sympa/bin/wwsympa_sudo_wrapper.pl
     Alias /static-sympa /home/sympa/static_content
  </VirtualHost>

Using mod_fcgid

Using mod_fcgid with Apache to provide the Fast CGI capabilities for the Sympa web interface appears to be preferred route today.  As mentioned previously, some *nix distributions have deprecated mod_fastcgi so it is often easier to build and install mod_fcgid.  On the other hand mod_fcgid is presently Apache version 2 specific and offers only internal, dynamically spawned servers.  This is fine for Sympa with Apache but deployments using other web servers will need to look elsewhere.  Allegedly mod_fcgid's process management is superior but there is also limited evidence that mod_fastcgi can process more requests per second. However, it is not clear if this test is exactly comparing like-for-like, especially as regards to pre-spawned vs. dynamic (the startup time would be a significant part of the documented run-time for mod_fcgid).

As with mod_fastcgi, there are a number of parameters that can be set for the servers created by mod_fcgid. These are documented on the mod_fcgid web pages.  Unlike mod_fastcgi, these parameters are set using separate individual Apache directives.  The values are collected during module initialization and are fixed for all mod_fcgid servers started until the Apache server is next restarted. [Note: It would probably have been nice if the names used for these directives included Fcgid or some such so that it is easy to identify them as mod_fcgid parameters and other module writers would not need to think about clashes! Too late now. :-( ] Sympa again asks for up to two processes - if you have a very heavily loaded server you might increase this figure.  IPCCommTimeout is increased from the default 20 seconds to cater for possible slow initialization or response (wwsympa.fcgi is very large and so may take a while to start). 
   LoadModule fcgid_module modules/mod_fcgid.so
   <IfModule mod_fcgid.c>
      MaxProcessCount 2
      IPCCommTimeout 120
   </IfModule>
The mod_fcgid servers also communicate with Apache via Unix Domain sockets as for mod_fastcgi.  The default directory used for the pseudo-file names is ${ServerRoot}/logs/fcgidsock.   It can be altered using the SocketPath directive.  I believe from code inspection (but haven't attempted to verify on a running system) that  if you change this directory, you will need to manually create the relevant directory and ensure its permissions are correct.   The default directory is owned by the Apache server user, group set to root and with permissions 700 (i.e., owner can do anything, group and world can do nothing). The Apache user must be able to search all levels of this path. The default directory appears to be created during installation.

Finally, the Apache server needs to be told which CGI handler to use when www.project.example.com/sympa is requested:
  <Location /sympa>
    SetHandler fcgid-script
  </Location>
The name of the handler is hard coded into the module and must match the Fast CGI mechanism you are using.  Depending on the default permissions for Locations you might need to add an Allow directive as well.  This, or a similar set of directives, is needed for each virtual host, as are the ScriptAlias and Alias directives described in the common configuration section.

So the complete set of additional Apache configurations for using mod_fcgid and a named virtual host should look something like:
  # Fast CGI module
  LoadModule fcgid_module modules/mod_fcgid.so

  # Set mod_fcgid parameters if loaded OK
  <IfModule mod_fcgid.c>
     MaxProcessCount 2
     IPCCommTimeout 120
     # Move IPC socket location if necessary
     # SocketPath logs/fcgidsock
  </IfModule>

  # Use named virtual host(s) for all requests on port 80
  NameVirtualHost *:80

  # Virtual host setup for www.project.example.com
  <VirtualHost *:80>
     # Basic configuration
     ServerName   www.project.example.com
     DocumentRoot /home/www/project
     ServerAdmin  webmaster@project.example.com
     
     #Use the FastCGI server to process Sympa web interface script
     <Location /sympa>
        SetHandler fcgid-script
        # Maybe change permissions if need be
        # Order allow,deny
        # Allow from all
     </Location>

     # Aliases to redirect Apache for scripts and documents not in the DocumentRoot directory tree
     ScriptAlias /sympa /home/sympa/bin/wwsympa_sudo_wrapper.pl
     Alias /static-sympa /home/sympa/static_content
  </VirtualHost>

Restarting Apache

Apache will need to be restarted for the various configurations to take effect.

Sympa for Virtual Hosts and Modifications for Linking to Postfix

This section is mostly concerned  with the modifications that have to be made to the Sympa scripts to implement my proposal for interfacing to Postfix when using virtual hosts.  The basic Sympa configuration is pretty straightforward and well covered in the documentation:  there are just a few notes on things that were not so obvious to me on first pass.

Enabling Fast CGI

Remember to configure use_fast_cgi in wwsympa.conf.  To enable the proper use of Fast CGI set:
use_fast_cgi 1
If you set this to 0 every request will reinvoke the wwsympa.fcgi script even if you have one of the Fast CGI modules set up in Apache, and performance will be badly impacted.

A Robot on a Virtual Host

Configuring Sympa itself to work with the virtual host is very easy.  Recall that I am setting up mail lists such as <listname>@project.example.com with the interface running on www.project.example .com at www.project.example.com/sympa.  The Sympa documentation contains the basic information on the Virtual host page.

Directory Setup

Working as user sympa create directories named for the robot domain in the etc and expl sub-directories of  the sympa installation directory (typically /home/sympa).  Copy the example robot.conf from the sample sub-directory to the new directory in etc:
su sympa
mkdir /home/sympa/etc/project.example.com
mkdir /home/sympa/expl/project.example.com
cp /home/sympa/sample/robot.conf /home/sympa/etc/project.example.com

Robot Configuration Settings

To connect this to your Apache virtual host, you need to configure at least in the new robot.conf:
 as follows:
host        project.example.com
http_host   www.project.example.com
wwsympa_url http://www.project.example.com/sympa
A limited amount of customisation of the web interface is probably a good idea but most other settings will be inherited from the master sympa.conf.   Setting the title and logo_html_definition will remind you which robot is in use even if you don't do anything more complex, for example:
title Our Project Mailing List Service
logo_html_definition '<a href="http://www.project.example.com"><img style="float: left; margin-top: 7px; margin-left: 37px; 
height: 48px; width: 100px" src="http://www.project.example.com/project_logo_small.gif"  alt="Our Project Logo" />
</a>'

Modifications to Sympa to Use Specific Virtual Aliases and Postfix Transports

As explained in the requirements section, I was seeking to ensure that mail sent to an arbitrary, unrecognized address associated with a virtual host would be rejected with an  SMTP error rather than bounced either by Postfox or Sympa. The standard version of Sympa uses a wildcard matching regular expression in the virtual aliases file together with specific aliases generated by alias_manager.pl in the normal aliases file when using virtual hosts.  To modify this behaviour, alias_manager.pl must be told that it should generate a different form of virtual alias and put it in a separate file.  Because there is extra information associated with each robot domain, I decided that a separate virtual aliases file for each domain would be cleaner and easier to maintain rather than having the virtual aliases for all domains mixed up - although this decision can be overridden by specifying the same file for all virtual aliases and the solution should still work.

The modifications add two configuration items to both the main and robot configurations:
The item virtual_aliases from sympa.conf, which defaults to /etc/mail/sympa_virtual_regexp, can be used as a base name for the virtual alias files assocated with each virtual host. If virtual_aliases is not defined in the robot.conf for a virtual domain, then "_${host}" (e.g., _project.example.com) is appended to the base name so that the default virtual aliases file for our example is /etc/mail/sympa_virtual_regexp_project.example.com.  These files must be creatable and writable by the sympa user - Postfix doesn't need them to be owned either by root or the Postfix user as long as it can read them.

To make these virtual aliases usable by Postfix, they have be converted into a database using the postmap command insread of the newaliases command used for ordinary aliases.  This can be run as the sympa user rather than needing to have a suid wrapper like newaliases.  The path for the postmap command is specified by another new configuration item postmap, which defaults to /usr/sbin/postmap.

To implement the modifications, changes had to be made to three Sympa files
The initial version of the virtual_aliases file is created with a comment associating it with the domain and the special entry needed by Postfix to distinguish a virtual alias domain  (see the Postfix documentation for virtual(5) for more details) :
## This virtual aliases file for domain project.example.com is dedicated to Sympa Mailing List Manager
## You should edit your Postfix main.cf file to declare it
/^project\.example\.com$/            xxx
You will need to add the special entry manually for additional virtual domains after the first if you choose to use a common file rather than separate ones.  The 'xxx'  is a placeholder and its value is irrelevant.
 
It will probably be necessary to manually add some special entries to the virtual alias file to deal with mail to the webmaster and postmaster for the virtual domain, and mail sent to sympa or the Sympa listmaster(s). For example:
/^(root|postmaster|webmaster)\@project\.example\.com$/    $1@
/^(sympa|listmaster)\@n4c.eu$/    $1+n4c.eu@sympalist.
Note the domain name in the right hand side of the last alias is sympalist. ending in a . - this is a really fully qualified domain name.  If the final . is omitted Postfix will try to append the local domain name to what it thinks is an unqualified domain name, which is not what is intended.

The virtual aliases generated for a new mailing list, such as test-list look like this:
#------------------------------ test-list: list alias created 2 April 2008
/^(test-list)-(request|editor|owner|subscribe|unsubscribe)\@project\.example\.com$/    $1+project.example.com@sympa$2.
/^(test-list)\@project\.example\.com$/    $1+project.example.com@sympalist.

The modified files are available here.

Restarting Sympa

Once you have modified the code files and configured the new virtual domain robot, restart Sympa and verify that it has created the initial virtual aliases file with the expected name. Add any manual entries needed to this file once you are happy.

Check that the Sympa web interface starts up as expected according to the URL you have selected for it (e.g., http://www.project.example.com/sympa).

Postfix Configuration for the Virtual Alias Domain

This section discusses the alterations to a pre-existing Postfix configuration set up for a local domain on master.example.com that will allow project.example.com to work as a virtual alias domain primarily to support Sympa mailing lists.  Much helpful information can be found in the 'virtual ALIAS example' section of the Postfix Virtual Domain Hosting Howto. The technique used is to map the different classes of mailing list addresses (basic list and various auxiliary control addresses) to specially 'created' pseudo-domains that are then used to select a Postfix transport that invokes the Sympa processes and feeds the mail to Sympa via a Unix pipe.

The steps required are:
  1. Modify the Postfix master.cf process configuration file to add the Sympa-specific transports associated with the Sympa pseudo-domains.
  2. Create (or modify an existing) transport table containing regular expressions to map the Sympa pseudo-domains to the matching transport.
  3. Create the database representation of the transport table using the postmap command (run with the same owner as was used to create the file - but Postfix must be able to read this file).
  4. Modify the Postfix main.cf file to make Postfix recognize the new virtual alias domain and instruct it to use the new transport table and the virtual aliases table that Sympa has created.
  5. Check that Sympa has generated the corresponding database representation for the virtual aliases file and that Postfix can read this file (this might require associating the Sympa user and the Postfix user in a group).
  6. Reload Postfix with the new configuration using postfix reload command.

Adding Transports to master.cf

Add the following lines to master.cf (typically located in /etc/postfix):
# Sympa mailing list manager transports
sympalist        unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}@${extension}
symparequest     unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-request@${extension}
sympaeditor      unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-editor@${extension}
sympasubscribe   unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-subscribe@${extension}
sympaunsubscribe unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/queue ${user}-unsubscribe@${extension}
sympabounce      unix  -       n       n       -       -       pipe
  flags=RF user=sympa argv=/home/sympa/bin/bouncequeue ${user}@${extension}
Adapt the paths in each line if the Sympa installation is not located in /home/sympa.  The order of lines in master.cf is not significant.

Ensure that the lines starting with 'sympa' start in the first column of the file as they start new logical lines.  The lines starting with 'flags' are continuation lines and must have whitespace in column 1.

Create a Transport Map File

Create a new transport regular expressions file, or alternatively add to an existing transport file (as long as it is using regular expression form).  The file is most conveniently located in the same directory as the other Postix configuration files (typically /etc/postfix) and should be created with the same permissions as the other configuration files - so that Postfix can read it.  The required new lines are:
/^.*\@sympalist$/         sympalist:
/^.*\@symparequest$/      symparequest:
/^.*\@sympaeditor$/       sympaeditor:
/^.*\@sympasubscribe$/    sympasubscribe:
/^.*\@sympaunsubscribe$/  sympaunsubscribe:
/^.*\@sympaowner$/        sympabounce:
The name of the file is at the user's choice - say transport_regexp - the same name has to be written into the Postfix main.cf in due course.

Create the Database Representation for the Transport Map

Run the postmap command to create the database representation for the transport map.  Typically postmap is installed in /usr/sbin:
/usr/sbin/postmap /etc/postfix/transport_regexp

Modify the Postfix Main Configuration File

The Postfix main configuration file main.cf (typically found in /etc/postfix) needs to be updated to declare the virtual alias domain, the transport map and the virtual aliases file.
We also set the recipient limit for each transport to 1 so that messages are delivered individually to the Sympa processes.
 
Add the following lines (adapting paths as necessary) - these lines will only need to be inserted once as the transports are common to all virtual alias domains that are used with Sympa:
# Transport maps for Sympa mailing list manager
transport_maps = regexp:/etc/postfix/transport_regexp

# Set the concurrency for delivery to Sympa
sympalist_destination_recipient_limit = 1
symparequest_destination_recipient_limit = 1
sympaeditor_destination_recipient_limit = 1
sympasubscribe_destination_recipient_limit = 1
sympaunsubscribe_destination_recipient_limit = 1
sympabounce_destination_recipient_limit = 1
Also add the following lines (again adapting paths as necessary) to :
# Specify the virtual alias domain(s)
virtual_alias_domains = project.example.com

# Specify the virtual alias file(s)
virtual_alias_maps = regexp:/etc/mail/sympa_virtual_regexp_project.example.com
If there is more than one virtual alias domain and/or virtual alias file, turn the items into comma separated lists.  The type of virtual alias file (regexp:) has to specified for each file.

WARNING:  DO NOT add the virtual alias domain name to either the mydestination or relay_domains list. The various sympa* 'pseudo-domains' used in the transport maps should not be specified in any of the domain lists

Check Virtual Alias File

Check that Sympa has created the initial state of the virtual alias file and run postmap to create the database representation.  Check that the permissions are such that Postfox will be able to read the database representation (if it is a file).  If you have made manual modifications to the virtual alias file, remember to run postmap again.

The Unix users and groups and directory/file that were used are:
Users:
postfix for Postfix
sympa for Sympa
apache for Apache
Groups:
mailctrl: (contains users) postfix sympa apache + personal ids for webmasters (me + 1)
sympa: (contains users) sympa + personal id of Sympa admin (me)

Directory and File Permissions:
sympa_aliases : is in /etc/mail/sympa_aliases
drwxrwxr-x  3 postfix mailctrl 4096 Mar 25 18:43 /etc/mail

-rw-r--r--  1 sympa sympa 134 Feb 29 17:58 /etc/mail/sympa_aliases
-rw-r--r--  1 sympa sympa 1111 Mar 25 18:38 /etc/mail/sympa_virtual_regexp_<domain_name> #(for virtual aliases)
aliaswrapper:
-rwsr-xr--  1 root  sympa  21057 Jan 19 01:19 /home/sympa/bin/aliaswrapper

Reload Postfix Configuration

Instruct Postfix to re read its configuration:
/usr/sbin/postfix reload
This is recommended rather than stopping and starting Postfix.

Testing the Postfix Mappings

The command 
/usr/sbin/sendmail -bv test-list@project.example.com
is very useful for checking that the Postfix address rewriting and transport mapping is working correctly.

Run from an account on the Postfix machine which is already set up for mail delivery through the non-virtual, it will report on what Postfix would have done with the mail had it been received through a mail message to the user running the command.  The sort of message you would hope to receive looks like this:
This is the Postfix program at host master.example.com.

Enclosed is the mail delivery report that you requested.

			The Postfix program

<test-list+project.example.com@sympaowner> (expanded from <test-list-owner@project.example.com>): delivery
    via sympabounce: delivers to command: /home/sympa/bin/bouncequeue


Reporting-MTA: dns; master.example.com
X-Postfix-Queue-ID: A1CD12904A4
X-Postfix-Sender: rfc822; elwynd@master.example.com
Arrival-Date: Tue, 25 Mar 2008 18:57:15 +0000 (GMT)

Final-Recipient: rfc822; test-list+project.example.com@sympaowner
Original-Recipient: rfc822; test-list-owner@project.example.com
Action: deliverable
Status: 2.0.0
Diagnostic-Code: X-Postfix; delivery via sympabounce: delivers to command:
    /home/sympa/bin/bouncequeue
No message is actually sent to the specified destination - it just tests the sequence of address rewrites that Postfix would have carried out and reports back.

And Finally....

I hope this will prove useful.  Please contact me with comments and suggestions at elwynd@dial.pipex.com.

Elwyn Davies
4 April 2008