sysadmin – Bootlin

Need for notification mailing lists

I found several people looking for a way to implement post-only mailing lists with GNU Mailman. However, I couldn’t find solutions that are described in sufficient detail.

In particular, this type of list is useful for notification mailing lists. In Bootlin’scase, whenever someone pushes commits to our public git trees, a notification e-mail is sent. Sometimes, internal discussions can follow, but we do not wish to make them public. This is why we do not want the list e-mail address to be shown in the messages that are sent. If the list address doesn’t appear in the To, CC or in Reply-To headers, members who are authorized to post messages without moderation won’t post replies to the list by mistake by using the “Reply to all” functionality of their e-mail client.

The problem is that the current version of GNU Mailman doesn’t support this type of list yet, at least with the parameters in the list administration interface. You can turn on the “Full personalization” option, which will send messages to each member individually, so that the list address doesn’t appear in the To header. You can also customize the Reply-To header, to an address that is different from the list address. However, the CC header will still hard-code the list address.

A possibility is to hack the /usr/lib/mailman/Mailman/Handlers/CookHeaders.py file, but this solution would apply to all the lists at once, and the changes you could make may interfere with Mailman updates. A much nice solution is to extend Mailman, to modify its behavior for specific mailing lists.

A working solution

This solution is based on explanations given on the Mailman wiki, and was implemented on Ubuntu 12.04.

First, create a list-test mailing list. Some of the commands below will assume that you named your new list this way. Now, go to its administration interface and enable “Full Personalization” in “Non-digest” options. In “General options”, in the “Reply-To: header munging” section, specify a reply-to address.

If you send a test message to your new list, you will see that the list address is still in the CC header of the message that you receive.

Now, create a RemoveCC.py file in the Handlers directory (/usr/lib/mailman/Mailman/Handlers/RemoveCC.py on Ubuntu 12.04):

# Your comments here

"""Remove CC header in post-only mailing lists

This is to avoid unmoderated members to reply to messages,
making their replies public. Replies should instead go to a private list.
"""

def process(mlist, msg, msgdata):
    del msg['Cc']

This will be yet another filter the list messages will go through. Now compile this file in the directory where you put it:

pycompile RemoveCC.py

The next thing to do is to modify the default filter pipeline for your new list. You can do it by creating a /var/lib/mailman/lists/list-test/extend.py file with the below contents:

import copy
from Mailman import mm_cfg
def extend(mlist):
    mlist.pipeline = copy.copy(mm_cfg.GLOBAL_PIPELINE)
    # The next line inserts MyHandler after CookHeaders.
    mlist.pipeline.insert(mlist.pipeline.index('CookHeaders') + 1, 'RemoveCC')

This will add your new filter right after the CookHeaders one. To enable this, you have to run:

/usr/sbin/config_list -i /var/lib/mailman/lists/list-test/extend.py list-test

You can now send a new test message, and you will see that the CC header is now gone.

Notes

Of course, you can reuse the same extend.py file for multiple mailing lists. However, the solution doesn’t work if you don’t put the file inside /var/lib/mailman/lists/list-name (distributions other than Ubuntu 12.04 may have different paths).
I didn’t manage to undo this change. The Mailman wiki gives a solution based on creating a file containing del mlist.pipeline and running /usr/sbin/config_list -i this-file list-name, but it didn’t work for me. Please post a comment below if you find a way to implement this, and return to “factory” settings.
Don’t hesitate to share other ways of implementing this kind of functionality!

GNU mailman logo Bootlin is not in the system administration business (we offer free and open-source solutions for embedded systems), but we do our best to share whatever experience we acquire, and whatever code we produce.

We configured a KVM virtual machine to run our mailing lists, and we used Ubuntu 10.04, the long term support (LTS) version. Here are instructions based on this experience. You could also use the same instructions to install Ubuntu 10.04 on a tiny, low power ARM board. And if you wish to use a more recent version of Ubuntu, I expect the steps to be very similar.

Here we assume that your domain is example.com and that you install your mailing list software on lists.example.com. It is indeed a good idea to install your mailing list software on a different server. This way, you won’t mess up with your main web and mail servers. Not having CGI scripts running on it will also keep your main mail server more secure.

Install packages

Install the Apache web server. It will allow administrators to configure and manage the lists, and users to subscribe, unsubscribe and tune their subscription settings.

sudo apt-get install apache2

For e-mail delivery, I chose the Postfix MTA (Mail Transport Agent). Exim would have been a good solution too, but I am more familiar with Postfix, which is already in use on our main mail server.

sudo apt-get install postfix

Choose the Internet site option.

Now install the mailman package:

sudo apt-get install mailman

Select the languages that you want your mailing list interfaces to support. I chose English and French for the moment. To add more languages later, run:

sudo dpkg-reconfigure mailman

Configure the Apache web server

First copy the sample Apache configuration file provided by the mailman package:

cp /etc/mailman/apache.conf /etc/apache2/sites-available/mailman

Now, in /etc/apache2/sites-available/mailman, enable short URLs by enabling:

ScriptAlias /mailman/ /usr/lib/cgi-bin/mailman/

Also modify /etc/mailman/mm_cfg.py:

DEFAULT_URL_PATTERN = 'http://%s/mailman/'

The last step is to enable your mailman site in Apache:

sudo a2ensite mailman
sudo /etc/init.d/apache2 restart

You should now have a new symbolic link in /etc/apache2/sites-enabled/.

Check that the Mailman website works by opening your mailing lists home page: http://lists.example.com/mailman/listinfo. It should look like https://lists.bootlin.com/mailman/listinfo.

Configure postfix

Enable the following line in /etc/mailman/mm_cfg.py:

MTA='Postfix'

Once the MTA is configured, generate Mailman specific aliases for Postfix:

sudo /usr/lib/mailman/bin/genaliases

Now, you need to configure Postfix through its main.cf file. A convenient way to do this is to run the below commands:

sudo postconf -e 'relay_domains = lists.example.com'
sudo postconf -e 'transport_maps = hash:/etc/postfix/transport'
sudo postconf -e 'mailman_destination_recipient_limit = 1'
sudo postconf -e 'alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases'

Also add the following line to /etc/postfix/transport:

lists.bootlin.com      mailman:

and run:

sudo postmap -v /etc/postfix/transport

You won’t have any mail delivery if you forget. I struggled for a few hours before I realized I forgot this setting.

Now, set correct file ownership:

sudo chown root:list /var/lib/mailman/data/aliases
sudo chown root:list /etc/aliases

For logging and debugging e-mail delivery, I recommend to install the sysklogd package. Without it, you won’t have any mail.info, mail.warn and mail.err files in /var/mail/.

To install this package, enable the universe repository if needed (uncomment the lines with universe in the /etc/apt/sources.list file), and run:

sudo apt-get update
sudo apt-get install sysklogd

To avoid having two mail.info and mail.log files with identical contents, edit /etc/syslog.conf and remove the below line:

mail.*                          -/var/log/mail.log

Also create a /etc/logrotate.d/mail file to rotate logs, as in the below example:

/var/log/mail.* {
        daily
        size 10M
        rotate 4
        compress
        missingok
        notifempty
        create 640 root adm
}

Last but not least, restart Postfix:

/etc/init.d/postfix restart

At this point, a good idea is to check that mail delivery works:

sudo apt-get install bsd-mailx
mailx alice@example.com
Subject: test
test
.
Cc:

Note: that’s the line containing only a dot character that allows to terminate the message.

If the receipient doesn’t receive this message, there is an issue in the way your mail server is configured. This could be because the firewall doesn’t allow connections to outside machines through tcp port 25. Anyway, look at the logs in /var/log/mail.* to get a clue. There is no point going on in this howto until you get this fixed.

Creating the mailman site list

Mailman needs a so-called “site list”, which is the list from which password reminders and such are sent out from. The default name for this list list mailman, though you can change this through the MAILMAN_SITE_LIST setting in /etc/mailman/mm_cfg.py.

To create this list, run:

sudo newlist mailman

You will have to answer a few questions like:

Enter the email of the person running the list: postmaster@example.com
Initial newsletter password: xxx
Hit enter to notify newsletter owner...

Choose the password carefully, as crackers will be able to highjack your mailing list if it is too easy to guess.

The next required step is to add the list aliases to /etc/aliases:

# mailman mailing list
mailman:              "|/var/lib/mailman/mail/mailman post mailman"
mailman-admin:        "|/var/lib/mailman/mail/mailman admin mailman"
mailman-bounces:      "|/var/lib/mailman/mail/mailman bounces mailman"
mailman-confirm:      "|/var/lib/mailman/mail/mailman confirm mailman"
mailman-join:         "|/var/lib/mailman/mail/mailman join mailman"
mailman-leave:        "|/var/lib/mailman/mail/mailman leave mailman"
mailman-owner:        "|/var/lib/mailman/mail/mailman owner mailman"
mailman-request:      "|/var/lib/mailman/mail/mailman request mailman"
mailman-subscribe:    "|/var/lib/mailman/mail/mailman subscribe mailman"
mailman-unsubscribe:  "|/var/lib/mailman/mail/mailman unsubscribe mailman"

In addition to delivering e-mail to the mailing lists, these aliases also allow to subscribe and unsubscribe by writing to special e-mail addresses.

You also need to run the newaliases command, without which there is no e-mail delivery:

sudo newaliases
sudo /etc/init.d/postfix restart
sudo /etc/init.d/mailman restart

Look at the /var/log/mailman/error file for potential issues.

Create regular mailing lists

Regular mailing lists are created in the same way as above: Assuming you want to create a newsletter mailing list. You will need run:

sudo newlist newsletter

Add your new mailing list to /etc/aliases:

# newletter mailing list
newsletter:              "|/var/lib/mailman/mail/mailman post newsletter"
newsletter-admin:        "|/var/lib/mailman/mail/mailman admin newsletter"
newsletter-bounces:      "|/var/lib/mailman/mail/mailman bounces newsletter"
newsletter-confirm:      "|/var/lib/mailman/mail/mailman confirm newsletter"
newsletter-join:         "|/var/lib/mailman/mail/mailman join newsletter"
newsletter-leave:        "|/var/lib/mailman/mail/mailman leave newsletter"
newsletter-owner:        "|/var/lib/mailman/mail/mailman owner newsletter"
newsletter-request:      "|/var/lib/mailman/mail/mailman request newsletter"
newsletter-subscribe:    "|/var/lib/mailman/mail/mailman subscribe newsletter"
newsletter-unsubscribe:  "|/var/lib/mailman/mail/mailman unsubscribe newsletter"

Then, run the usual commands:

sudo newaliases
sudo /etc/init.d/postfix restart
sudo /etc/init.d/mailman restart

Configuring your lists

The easiest way to configure your lists and add members is to open the http://lists.example.com.com/mailman/listinfo URL with a browser.

A few things are also possible from the command line. For example, you can add a member as follows:

echo "alice@example.com" > /tmp/foo
sudo add_members -r /tmp/foo newsletter

Enabling archives

Default file permissions are not completely ready to support mailing list archives:

sudo chown -R root:list /var/lib/mailman/archives
sudo chmod o+rX /var/lib/mailman/archives/private

The second line allows the webserver to access the archives. Note that these settings were forgotten in the official Ubuntu documentation.

Settings for newsletter mailing lists

At Bootlin, we also use Mailman to deliver our newsletters. Mailman provides recipients with an easy mechanism to subscribe by themselves and unsubscribe whenever they want.

To make it even easier to unsubscribe from a newsletter, Mailman can add a special footer to each recipient, with a custom URL that allows to unsubscribe without having to remember one’s password.

To enable this feature, you have to enable the following line in /etc/mailman/mm_cfg.py:

# Extra options
# Allow to personalize each message
# (useful to provide a password-less unsubscribe link)
OWNERS_CAN_ENABLE_PERSONALIZATION = 1

Restart Mailman (/etc/init.d/mailman restart) and go to the administrative interface for your list. Under Non digest options, you will then be able to set the Should Mailman personalize each non-digest delivery? option to Full Personalization.

Then, you can set a custom footer for each recipient in the Footer added to mail sent to regular list members option. Here is an example:

_______________________________________________
Bootlin quarterly newsletter
Unsubscribe: %(user_optionsurl)s?password=%(user_password)s
Archives: https://lists.bootlin.com/pipermail/newsletter/

Beware that sending custom e-mails to each recipient will increase the load on your server. You may not want to do this on mailing lists with great numbers of subscribers.