Some months back, I discovered the Notmuch mail reader. It’s a pretty sweet piece of software based on Xapian to index and tag your mail. As of today, the email setup described here has displaced my Google Apps hosted Gmail for my personal domain. My goal here is to document how I’ve setup my mail workflow.
As it says in the name, Notmuch is not much of a mail client. It doesn’t get any mail, send any mail, or even display any mail. All it does is to provide an interface to a Xapian index for your mail. We need to provide other software that will help us handle the other pieces.
On one of my computers I’m using Arch Linux. I’ve tried just about every Linux distribution in the past fourteen years and have found Arch to be a pretty good sweet spot for a workstation. The distro has stabilized in recent years and a system upgrade seems to be less likely to completely bork your computer. I prefer debian stable on the server since it’s so well tested. The instructions here might be a bit Arch-Centric. I’m going to discuss sending and receiving mail before discussing how to configure Notmuch. Sending and receiving mail are critical for just about every GNU/Linux mail client and most of this should be generally applicable.
Getting the mail
My mail lives on an IMAP server somewhere in cloud-land. I’ve had an account on this machine for several years, but I’ve recently forwarded all of the mail for my domains to this server. There are several good IMAP clients these days that will synchronize the mail from your local computer with your mail on the IMAP server.
Among the ones that I’ve tried in the past are:
My mail setup requires the use of STARTTLS, so Getmail and OfflineIMAP are out of the question. They both support STARTTLS, but they’re using a Python library that doesn’t perform any verification of the certificate. This leaves me open to a MITM attack where someone upstream can present a fake certificate and proxy my traffic. I ended up using MBSync to fetch my mail from the server.
The documentation for MBSync is pretty terse, but there’s a nice page at the ArchLinux wiki.. I’ve included a configuration file that works for my needs.
=~/.mbsyncrc= ## # Address to connect to Host mail.servername.info Port 143 User BostonEnginerd PassCmd "gnome-keyring-query get servername\\_mail" SSLType starttls SSLVersions TLSv1.2 SystemCertificates no # The following line should work. If get certificate errors, uncomment the two following lines and read the "Troubleshooting" section. CertificateFile ~/.cert/mail.servername.pem IMAPStore servername-remote Account servername MaildirStore servername-local # The trailing "/" is important Path ~/.mail/servername/ Inbox ~/.mail/servernamw/Inbox Channel servername Master :servername-remote: Slave :servername-local: Patterns * # Automatically create missing mailboxes, both locally and on the server Create Both # Save the synchronization state files in the relevant directory SyncState *
You will need to get the ssl certificate from your server. If it’s a “real” SSL certificate signed by a CA, including the CA certificate should be sufficient. If it’s self-signed, it’s easiest to get the certificate from the server and save it to a file. The commandline openssl command can do that for you:
openssl s_client -starttls imap -connect some.imap.server:port -showcerts
I just copied and pasted from the ——-BEGIN CERTIFICATE—– to the END CERTIFICATE line into a file and referred to that in the configuration.
I’m using the gnome-keyring-daemon to store the mail password so it’s
not sitting on my hard disk in plaintext. There’s a program in the AUR
gnome-keyring-query which allows you to interact with the
keyring on the command line. The
get command gets the requested key.
There is a
set command as well for storing a password. One of these
days, I might try to setup something that uses a certificate to
authenticate the client.
Sending the Mail
The second most important thing that a mail client can do is send mail.
I’m using the SMTP server on my host, again using STARTTLS. To send the
mail, I decided to use a program called
msmtp which implements a small
send-only smtp client. The ArchLinux wiki has a
nice tutorial on using
After installing msmtp, you need to configure it. A working config file is below:
# Set default values for all following accounts. defaults tls on tls_starttls on tls_fingerprint AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA: logfile ~/.msmtp.log # A freemail service account myserver host my.server.tld port 9000 from bostonenginerd@myserver auth on user myname passwordeval gnome-keyring-query get myserver.smtp # A second mail address at the same freemail service account myotheraccountname : myserver from myname@myotherdomain # Set a default account account default : myserver
I get the TLS fingerprint from the server using the
Again, I’m using the gnome-keyring to store the passwords encrypted on
Once msmtp is working you can send mail when your computer is online.
A nice feature would be to queue messages when the network connection is
down and send them when you come back online. Fortunately recent
versions of msmtp come with a queue management script. On Arch, there
are two included. I’m using the one called
msmtpqueue which includes
three scripts -
msmtp-runqueue.sh. These scripts pretty much do what they say. Copy
them to somewhere in your path.
Using the scripts requires that you have a directory called
~/.msmtpqueue. The email will be stored in this directory when you use
the msmtp-enqueue.sh command in your mail client. To send all the mail,
you can use the msmtp-runqueue.sh script when you’re connected to the
internet. In the past, I’ve had problems sending too many messages too
quickly. If you encounter this, you can put a sleep command in the
Sending and Receiving Automatically
Now that we’re able to receive and send mail from the command line, it
would be good to automate the process. I put together a hacked up script
that checks if we’re connected to the Internet, sends all of the mail
sitting in the queue and pulls in new messages.
nmcli interacts with
the NetworkManager daemon to query the state of the internet connection.
Possible answers are
full. We only
want to go through the trouble of trying to send messages when we have
#!/bin/sh STATE=`nmcli networking connectivity` if [ $STATE = 'full' ] then /usr/local/bin/msmtp-runqueue.sh mbsync -qq accountname exit 0 fi echo "No internet connection." exit 0
Now that we have this together, it would be awfully nice if my computer would run this script periodically. There are two ways to do this now. One way is to use cron. This is the standard way to accomplish something like this.
I thought that I would take the opportunity to play with what seems to be the new init system for major Linux distributions – systemd. I haven’t really been following all the drama surrounding this project, but it’s the new init system for Arch, Debian and Fedora. I’ve noticed that computers with systemd tend to boot substantially faster than the initV based systems. I was inspired to do this based on a blog post that Joey Hess wrote to document building himself an alarm clock with systemd.
I wanted to check for new mail and send old mail every fifteen minutes.
To do this requires that I create two files
checkmail.timer in the
[Unit] Description=check mail RefuseManualStart=no RefuseManualStop=yes [Service] Type=oneshot ExecStart=/home/bostonenginerd/scripts/checkmail.sh
This file defines what the service does.
Type=oneshot indicates that
this is not a daemon and will just run and exit. Setting the
RefuseManualStop=yes variable will make
systemctl unable to stop the
service. In this case, that’s fine because the script will teriminate
when it’s done running. Now we need to setup the schedule for running
[Unit] Description=Check Mail every fifteen minutes RefuseManualStart=no RefuseManualStop=no [Timer] Persistent=false OnBootSec= 5min OnUnitActiveSec= 15min Unit=checkmail.service [Install] WantedBy=default.target
In this file, we’re setting
RefuseManualStop=no because this will be a
long running service which we will can terminate at some point. In the
[Timer] section, we set
OnBootSec to five minutes. This means that
the checkmail.service job will not run until five minutes after the
computer boots up. I put this in to wait some time for a network
connection to come up before trying to get the mail. The
OnUnitActiveSec variable control the time delay in between repeats of
the job. Here it’s set to 15 minutes, but could be set for any interval.
To start the timer, you should type
systemctl --user start checkmail.timer. Using the command
systemctl --user status checkmail.timer will allow you to see if the
timer was successfully started. To start the timer automatically, issue
systemctl --user start checkmail.timer command.
Another neat command is
systemctl --user list-timers. This will list
all of the timers that you have active and how long until they’re
triggered again. Neat stuff. Systemd is pretty clever.
Ideally we could simplify our script and add a systemd dependency to the service that requires that the internet be active before running the script. I’d love to hear any ideas on how to do that, or how to do this more efficiently.
End Part 1
I’ll finish up the Notmuch specific portions of my email setup at a later date.