Some months back, I discovered the Notmuch mail reader. It’s a pretty sweet piece of software based on Xapian to index and tag your mail. As of today, the email setup described here has displaced my Google Apps hosted Gmail for my personal domain. My goal here is to document how I’ve setup my mail workflow.
As it says in the name, Notmuch is not much of a mail client. It doesn’t get any mail, send any mail, or even display any mail. All it does is to provide an interface to a Xapian index for your mail. We need to provide other software that will help us handle the other pieces.
On one of my computers I’m using Arch Linux. I’ve tried just about every Linux distribution in the past fourteen years and have found Arch to be a pretty good sweet spot for a workstation. The distro has stabilized in recent years and a system upgrade seems to be less likely to completely bork your computer. I prefer debian stable on the server since it’s so well tested. The instructions here might be a bit Arch-Centric. I’m going to discuss sending and receiving mail before discussing how to configure Notmuch. Sending and receiving mail are critical for just about every GNU/Linux mail client and most of this should be generally applicable.
Getting the mail
My mail lives on an IMAP server somewhere in cloud-land. I’ve had an account on this machine for several years, but I’ve recently forwarded all of the mail for my domains to this server. There are several good IMAP clients these days that will synchronize the mail from your local computer with your mail on the IMAP server.
Among the ones that I’ve tried in the past are:
My mail setup requires the use of STARTTLS, so Getmail and OfflineIMAP are out of the question. They both support STARTTLS, but they’re using a Python library that doesn’t perform any verification of the certificate. This leaves me open to a MITM attack where someone upstream can present a fake certificate and proxy my traffic. I ended up using MBSync to fetch my mail from the server.
The documentation for MBSync is pretty terse, but there’s a nice page at the ArchLinux wiki.. I’ve included a configuration file that works for my needs.
# Address to connect to Host mail.servername.info Port 143 User BostonEnginerd PassCmd "gnome-keyring-query get servername_mail" SSLType starttls SSLVersions TLSv1.2 SystemCertificates no # The following line should work. If get certificate errors, uncomment the two following lines and read the "Troubleshooting" section. CertificateFile ~/.cert/mail.servername.pem IMAPStore servername-remote Account servername MaildirStore servername-local # The trailing "/" is important Path ~/.mail/servername/ Inbox ~/.mail/servernamw/Inbox Channel servername Master :servername-remote: Slave :servername-local: Patterns * # Automatically create missing mailboxes, both locally and on the server Create Both # Save the synchronization state files in the relevant directory SyncState *
You will need to get the ssl certificate from your server. If it’s a “real” SSL certificate signed by a CA, including the CA certificate should be sufficient. If it’s self-signed, it’s easiest to get the certificate from the server and save it to a file. The commandline openssl command can do that for you:
openssl s_client -starttls imap -connect some.imap.server:port -showcerts
I just copied and pasted from the ———-BEGIN CERTIFICATE——- to the END CERTIFICATE line into a file and referred to that in the configuration.
I’m using the gnome-keyring-daemon to store the mail password so it’s
not sitting on my hard disk in plaintext. There’s a program in the AUR
gnome-keyring-query which allows you to interact with the
keyring on the command line. The
get command gets the requested
key. There is a
set command as well for storing a password. One of
these days, I might try to setup something that uses a certificate to
authenticate the client.
Sending the Mail
The second most important thing that a mail client can do is send
mail. I’m using the SMTP server on my host, again using STARTTLS. To
send the mail, I decided to use a program called
implements a small send-only smtp client. The ArchLinux wiki has a
nice tutorial on using this also.
After installing msmtp, you need to configure it. A working config file is below:
# Set default values for all following accounts. defaults tls on tls_starttls on tls_fingerprint AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA: logfile ~/.msmtp.log # A freemail service account myserver host my.server.tld port 9000 from bostonenginerd@myserver auth on user myname passwordeval gnome-keyring-query get myserver.smtp # A second mail address at the same freemail service account myotheraccountname : myserver from myname@myotherdomain # Set a default account account default : myserver
I get the TLS fingerprint from the server using the
s_client. Again, I’m using the gnome-keyring to store the passwords
encrypted on the disk.
Once msmtp is working you can send mail when your computer is online.
A nice feature would be to queue messages when the network connection
is down and send them when you come back online. Fortunately recent
versions of msmtp come with a queue management script. On Arch, there
are two included. I’m using the one called
msmtpqueue which includes
three scripts -
msmtp-runqueue.sh. These scripts pretty much do what they say. Copy
them to somewhere in your path.
Using the scripts requires that you have a directory called
~/.msmtpqueue. The email will be stored in this directory when you
use the msmtp-enqueue.sh command in your mail client. To send all the
mail, you can use the msmtp-runqueue.sh script when you’re connected
to the internet. In the past, I’ve had problems sending too many
messages too quickly. If you encounter this, you can put a sleep
command in the msmtp-runqueue.sh script.
Sending and Receiving Automatically
Now that we’re able to receive and send mail from the command line, it
would be good to automate the process. I put together a hacked up
script that checks if we’re connected to the Internet, sends all of
the mail sitting in the queue and pulls in new messages.
interacts with the NetworkManager daemon to query the state of the
internet connection. Possible answers are
full. We only want to go through the trouble of trying to send
messages when we have full connectivity.
1 2 3 4 5 6 7 8 9 10 11 12
#!/bin/sh STATE=`nmcli networking connectivity` if [ $STATE = 'full' ] then /usr/local/bin/msmtp-runqueue.sh mbsync -qq accountname exit 0 fi echo "No internet connection." exit 0
Now that we have this together, it would be awfully nice if my computer would run this script periodically. There are two ways to do this now. One way is to use cron. This is the standard way to accomplish something like this.
I thought that I would take the opportunity to play with what seems to be the new init system for major Linux distributions — systemd. I haven’t really been following all the drama surrounding this project, but it’s the new init system for Arch, Debian and Fedora. I’ve noticed that computers with systemd tend to boot substantially faster than the initV based systems. I was inspired to do this based on a blog post that Joey Hess wrote to document building himself an alarm clock with systemd.
I wanted to check for new mail and send old mail every fifteen
minutes. To do this requires that I create two files
checkmail.timer in the
[Unit] Description=check mail RefuseManualStart=no RefuseManualStop=yes [Service] Type=oneshot ExecStart=/home/bostonenginerd/scripts/checkmail.sh
This file defines what the service does.
Type=oneshot indicates that
this is not a daemon and will just run and exit. Setting the
RefuseManualStop=yes variable will make
systemctl unable to stop
the service. In this case, that’s fine because the script will
teriminate when it’s done running. Now we need to setup the schedule
for running the job.
[Unit] Description=Check Mail every fifteen minutes RefuseManualStart=no RefuseManualStop=no [Timer] Persistent=false OnBootSec= 5min OnUnitActiveSec= 15min Unit=checkmail.service [Install] WantedBy=default.target
In this file, we’re setting
RefuseManualStop=no because this will be
a long running service which we will can terminate at some point. In
[Timer] section, we set
OnBootSec to five minutes. This means
that the checkmail.service job will not run until five minutes after
the computer boots up. I put this in to wait some time for a network
connection to come up before trying to get the mail. The
OnUnitActiveSec variable control the time delay in between repeats
of the job. Here it’s set to 15 minutes, but could be set for any interval.
To start the timer, you should type
systemctl --user start
checkmail.timer. Using the command
systemctl --user status
checkmail.timer will allow you to see if the timer was successfully
started. To start the timer automatically, issue the
start checkmail.timer command.
Another neat command is
systemctl --user list-timers. This will list
all of the timers that you have active and how long until they’re
triggered again. Neat stuff. Systemd is pretty clever.
Ideally we could simplify our script and add a systemd dependency to the service that requires that the internet be active before running the script. I’d love to hear any ideas on how to do that, or how to do this more efficiently.
End Part 1
I’ll finish up the Notmuch specific portions of my email setup at a later date.