SMTP Server with LDAP Authentication
Table of Contents
- 1. Introduction
- 2. Send email only from trusted hosts
- 3. Forward-Only email accounts with LDAP authentication
- 4. WKD+WKS server
1. Introduction
This mail server allows you to send email only if you authenticate with a valid username and password, which are stored on a LDAP server. If someone sends email to one of the mail domains supported by the server, the recipient address will be checked on the LDAP directory whether it is valid, before the mail is accepted. The received emails are not stored locally, but are forwarded instead to an external email address that belongs to the user (and is recorded in the LDAP directory).
This setup could be useful for organizations or companies. It allows them to have branded email addresses and to use their own mail server for sending emails. However it spares their users from having multiple email accounts, allowing them to receive all the emails in a single mailbox, on their preferred mail provider (for example gmail.com). At the same time, it relieves the postmaster of the organization from having to manage mailboxes and backups, scanning emails for viruses and spam, etc. These are not easy tasks and they can be done much more professionally from the email providers.
To build a mail server you need to have access to the DNS configuration of the mail domain.
We will build the mail server in these major steps:
- First, we will build a mail server that sends emails only from trusted hosts (the local host, docker containers on the host, and specific IP addresses/networks that are listed on a configuration file).
- Then we will install an OpenLDAP server with user accounts (username, password, and external email address) and will enable LDAP authentication on the mail server.
- As a bonus, we will also install a WKD+WKS server, so that our users can publish the GnuPG public key that is related to their email address.
2. Send email only from trusted hosts
Allowing only the trusted hosts to send emails prevents any spammers from abusing the mail server. This setup can be useful for sending email notifications from web applications (like Gitea, NextCloud, Moodle, etc.).
2.1. Minimal DNS configuration
In order to build a mail server you need to own a domain (say
example.org
) and be able to customize its DNS records.
For each email domain you need something like this on the DNS configuration:
smtp.example.org. IN A 10.11.12.13 example.org. IN MX 10 smtp.example.org. example.org. IN MX 20 smtp.example.org. example.org. IN TXT "v=spf1 mx -all"
Note: Yes, both MX records point to the same server, it is not a typo. It is done because the configuration of the server enables postscreen deep protocol tests in order to block spambots (for more details see this article). This means that the server disconnects new/unknown SMTP clients when they connect for the first time, even if they are legitimate. A legitimate client usually goes on to try the second mailserver in the list, and the second time they try to connect they are accepted.
The last line basically tells to the other SMTP servers that only this
server is allowed to send emails on behalf of this domain, and no
other servers. This is done to prevent spammers from faking your email
addresses. If a spammer tries to send a mail as if it is coming from
your domain, the SMTP server that is getting this email will consult
this DNS recorda nd will figure out that the server of the spammer is
not allowed to send emails on behalf of example.org
.
Depending on your DNS server, DNS changes may take from a few minutes
to a couple of days to propagate. You can use dig
to verify that
these DNS records have been activated:
$ dig MX example.org +short 10 smtp.example.org. 20 smtp.example.org. $ dig A smtp.example.org +short 10.11.12.13 $ dig TXT example.org +short "v=spf1 mx -all"
2.2. Build a postfix container with docker-scripts
We need to install fist docker-scripts and wsproxy.
2.2.1. Install docker-scripts
sudo su apt install m4 make git git clone https://gitlab.com/docker-scripts/ds /opt/docker-scripts/ds cd /opt/docker-scripts/ds/ make install
2.2.2. Install web-server proxy
We need wsproxy
to get and manage letsencrypt SSL certificates for
the postfix
container.
- Get the scripts:
ds pull wsproxy
- Initialize a directory:
ds init wsproxy @wsproxy
- Fix the settings:
cd /var/ds/wsproxy/; vim settings.sh
- Make the container:
ds make
2.2.3. Install postfix
- Get the scripts:
ds pull postfix
- Initialize a directory:
ds init postfix @smtp.example.org
- Fix the settings:
cd /var/ds/smtp.example.org/ ; vim settings.sh
- Make the container:
ds make
Check the installation:
cd /var/ds/smtp.example.org/ ls sslcert/ ls /etc/cron.d/ ls config/ cat config/trusted_hosts cat config/virtual_alias_maps cat config/virtual_alias_maps.regexp ls config/dkim-keys/ ds shell ps ax systemctl status postfix ls /etc/postfix/ postconf -nf | less tail /var/log/mail.log exit
2.3. Make the mail server trustworthy
To increase the chances that the other mail servers take seriously the emails comming from our mailserver we have to do some extra DNS configurations. Otherwise the mails that are sent may end up being classified as spam and most probably will not reach the recipient.
2.3.1. Activate a DKIM key
DKIM keys are used by a mail server to sign the emails that it sends, so that the emails cannot be changed in transit, and so that the receiver can verify that this server is authorized to send emails for a domain. It is an important tool against spams and faked emails. If an smtp server signs the messages that it sends, it is less likely that they will be classified as spam.
Installation scripts generate a DKIM key as well, which is on
config/dkim-keys/example.org/
. To activate it you need to add a
record like this on the DNS configuration of the domain:
mail._domainkey.example.org. IN TXT "v=DKIM1; h=sha256; k=rsa;" " p=MIIBIjANBgkqhkiG9w0BAQE....kMJdAwIDAQAB"
You can find the content of the public key on the file:
config/dkim-keys/example.org/mail.txt
.
To check whether it has been activated or not, try the command:
dig TXT mail._domainkey.example.org +short
2.3.2. Create a DMARC record
DMARC is a standard that allows you to set policies on who can send email for your domain based on DKIM and SPF.
You can add a DMARC Record on DNS that will allow you to get weekly reports from major ISPs about the usage of your email domain.
- Go to http://dmarc.postmarkapp.com/ and add your email address where
you want to receive reports, and email domain name (
example.org
). On the DNS configuration of the domain add a TXT record like this:
_dmarc.example.org. IN TXT "v=DMARC1; p=none; pct=100; " "rua=mailto:re+x2i0yw1hoq7@dmarc.postmarkapp.com; sp=none; aspf=r;"
The value of this TXT record is the one generated by the website above.
To check that it has been activated, try the command:
dig TXT _dmarc.example.org. +short
2.4. Test the SMTP server
For a quick automated test of the basic functionality of the server
try: ds test1
. Following are some tests that can be done manually.
2.4.1. Send test emails from inside the container
cd /var/ds/smtp.example.org/ ds shell swaks --from noreply@example.org --to info@example.org tail /var/log/mail.log cat /host/config/virtual_alias_maps cat /host/config/virtual_alias_maps.regexp
This test succeeds because the recipient (info@example.org
) matches
the line /^info@/
on the virtual_alias_maps.regexp
. The sender can
be anything (in this case noreply
) and the email will still be sent,
since we are sending from localhost, which is a trusted host.
This email is actually forwarded to the address that is listed on
virtual_alias_maps.regexp
(check also the spam folder if you cannot
find it on the inbox).
swaks --from info@example.org --to noreply@example.org tail /var/log/mail.log
This test fails because the recipient (noreply@example.org
) does not
match any entry on virtual_alias_maps
or
virtual_alias_maps.regexp
, so the mailserver does not know what to
do with this email.
Note: If you edit virtual_alias_maps
and add a line like this:
noreply@example.org user@mail.com
then the second test will succeed as well and the email will be forwarded to the given address. But first you will have to update the map with:
postmap /host/config/virtual_alias_maps
2.4.2. Send test emails to a gmail account
swaks --from info@example.org --to username@gmail.com -tlso tail /var/log/mail.log swaks --from noreply@example.org --to username@gmail.com -tlso tail /var/log/mail.log
Both of these tests will succeed. The option -tlso
enables the TLS
encryption of the connection.
On gmail use "Show original" from the menu, to see the source of the received email.
You can also send email from an external account to info
, but you
cannot send to noreply
.
2.4.3. Send a test email from the host (outside the container)
cd /var/ds/smtp.example.org/ apt install swaks swaks --from info@example.org --to admin@example.org -tlso ds exec tail /var/log/mail.log
It may fail, because the IP of the host might not be on the list of
the trusted hosts. Add it on config/trusted_hosts
and then run ds
inject update.sh
. Verify that now it works:
swaks --from info@example.org --to admin@example.org -tlso ds exec tail /var/log/mail.log
2.4.4. Add a new recipient address
Try to send email to test@example.org
:
swaks --from info@example.org --to test@example.org -tlso ... <** 550 5.1.1 <test@example.org>: Recipient address rejected: User unknown in virtual alias table ...
It will fail because the recipient does not exist on the alias table.
On config/virtual_alias_maps
add a line like this:
test@example.org username@gmail.com
Then update the alias db: ds exec postmap
/host/config/virtual_alias_maps
(or ds inject update.sh
). Verify
that now you can send email to this address:
swaks --from info@example.org --to test@example.org -tlso ds exec tail /var/log/mail.log
2.4.5. Send email to the ports 587
and 465
swaks --server smtp.example.org:587 \ --from info@example.org --to admin@example.org swaks --server smtp.example.org --port 465 \ --from info@example.org --to admin@example.org
2.5. Check the health of the mail server
Send an email to
check-auth@verifier.port25.com
:swaks --server smtp.example.org -tlso \ --from info@example.org \ --to check-auth@verifier.port25.com
The automatic reply will give you important information about the status and health of your email server (for example whether the mails sent from it pass the SPF and DKIM checks, whether they are considered spam or not, etc.)
Go to https://www.mail-tester.com/ and send a message to the email address displayed there, like this:
swaks --server smtp.example.org -tlso \ --from info@example.org \ --to test-1p4f6@mail-tester.com
Then click the button for checking the score.
- There are lots of other tools and websites that help to check the configuration of a mail server (DNS settings, configuration, security features, etc.) For example:
2.6. Add another email domain
This smtp server can support more than one mail domains. If we want to
add another mail domain, for example example.COM
, we have to do
these:
Set DNS configurations like this:
; mail for example.com smtp.example.com. IN CNAME smtp.example.ORG. example.com. IN MX 10 smtp.example.com. example.com. IN MX 20 smtp.example.com. example.com. IN TXT "v=spf1 mx -all"
Note that the first record (
CNAME
) redirects the new smtp domain to the first one.You can check these DNS configurations like this:
dig +short smtp.example.com. A dig +short example.com. MX dig +short example.com. TXT
Edit
settings.sh
and add the new domain onVIRTUAL_DOMAINS
:VIRTUAL_DOMAINS="example.org example.com"
- Run
ds make
to rebuild the container. The rebuild is needed because we need to request a new SSL cert from letsencrypt, which covers all the domains (including the new one). - Go to http://dmarc.postmarkapp.com/ and generate a DMARC record for the new domain.
Update the DNS configuration with records like these:
mail._domainkey.example.com. IN TXT "v=DKIM1; h=sha256; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQE....kMJdAwIDAQAB" _dmarc.example.com. IN TXT "v=DMARC1; p=none; pct=100; rua=mailto:re+x2i0yw1hoq7@dmarc.postmarkapp.com; sp=none; aspf=r;"
Note that:
- The value of the key for the DKIM record can be found on the file:
config/dkim-keys/example.com/mail.txt
- The value of the DMARC record is the one obtained on the previous step.
You can check them like this:
dig +short mail._domainkey.example.com. TXT dig +short _dmarc.example.com. TXT
- The value of the key for the DKIM record can be found on the file:
3. Forward-Only email accounts with LDAP authentication
We want a mail server that:
- Allows you to send email if you authenticate with a valid username and password, which are stored on an LDAP server.
- If someone sends email to one of the mail domains supported by the server, the recipient address will be checked on the LDAP directory whether it is valid, before the mail is accepted.
- The received emails are not stored locally, but are forwarded instead to an external email address that belongs to the user (and is recorded in the LDAP directory).
For this setup we obviously need to have an LDAP server that contains the user details (username, password and forward email). So, first of all we will see how to install such a server. Then we will enable LDAP authentication on the mail server that we built on the previous section.
3.1. Setup an LDAP server
3.1.1. Install OpenLDAP container
Let's assume that you have already installed docker-scripts and wsproxy from the previous sections. Follow these steps to install OpenLDAP:
- Get the scripts:
ds pull openldap
- Initialize a directory:
ds init openldap @ldap.example.org
- Customize settings:
cd /var/ds/ldap.example.org/ ; vim settings.sh
- Make the container:
ds make
Check the container:
cd /var/ds/ldap.example.org/ ds shell hostname # must be equal to $HOSTNAME ps ax systemctl status slapd slapcat slapcat -n0 # cn=config database ds exit
3.1.2. Add data to LDAP
Add organizational units
users
andapps
:ds ou ds ou add users ds ou add apps ds ou ls
The unit
users
is for the email users, andapps
is for the mail server and other applications that need to read the user data from LDAP.Add some test users:
ds user ds user add users user1@example.org abc@mail.com pass123 ds user add users user2@example.org xyz@mail.com ds set-passwd pass234 uid=user2@example.org,ou=users,dc=example,dc=org ds user ls
Note: The users that are created with
ds user add
have a minimal data structure that looks like this:dn: uid=user1@example.org,ou=users,dc=example,dc=org uid: user1@example.org objectClass: top objectClass: inetOrgPerson sn: user1@example.org cn: user1@example.org mail: abc@mail.com userPassword:: e1NTSEF9OUU2VWhqUXNNY2wvcGdicTExR3FBNXRnOGduUmxhMmM=
This is sufficient for the purpose of keeping forward email accounts. But if you have to customize it, make a local copy of
cmd/user.sh
and modify it as needed:cd /var/ds/ldap.example.org/ mkdir -p cmd/ cp /opt/docker-scripts/openldap/cmd/user.sh cmd/ vim cmd/user.sh
Note: The users can change or recover their password with a web interface at https://ldap.example.org/. The admin can reset user passwords at https://ldap.example.org/service/.
Create an
apps
user for the mail server, so that it can read the data from LDAP:ds app ds app add mail1 pass123 ds app ls
Check that the app user
cn=mail1,ou=apps,dc=example,dc=org
can read data from the server:apt install ldap-utils ldapsearch -xLLL \ -H ldap://ldap.example.org/ \ -D cn=mail1,ou=apps,dc=example,dc=org -w pass123 \ -b dc=example,dc=org "(uid=user1@example.org)" ldapsearch -xLLL -ZZ \ -H ldap://ldap.example.org/ \ -D cn=mail1,ou=apps,dc=example,dc=org -w pass123 \ -b dc=example,dc=org "(uid=user1@example.org)"
The option
-ZZ
is for using a TLS connection.If there is any problem, add the option
-d9
for debugging.
3.2. Enable LDAP authentication on the SMTP server
- Go to SMTP app directory:
cd /var/ds/smtp.example.org/
Edit
settings.sh
and uncomment the LDAP settings, giving them proper values:LDAP_SERVER_HOST="ldap.example.org" LDAP_SEARCH_BASE="ou=users,dc=example,dc=org" LDAP_BIND_DN="cn=mail1,ou=apps,dc=example,dc=org" LDAP_BIND_PW="pass123"
- Re-make the container:
ds make
3.3. Testing
For a quick automated test of the basic functionality of the server
try: ds test2
. It assumes that the test users (user1@example.org,
pass1) and (user2@example.org, pass2) exist on the LDAP
database. They can be added like this:
cd /var/ds/ldap.example.org/ ds ou add users ds user ds user add users user1@example.org forward-address@mail.com pass1 ds user add users user2@example.org forward-address@mail.com pass2
We can also try the manual tests described in the following sections.
3.3.1. Test the LDAP queries
cat config/smtpd_sender_login_maps.ldap cat config/virtual_alias_maps.ldap ds shell postconf -f virtual_alias_maps postconf -f smtpd_sender_login_maps postmap -q user1@example.org \ ldap:/host/config/virtual_alias_maps.ldap postmap -q user1@example.org \ ldap:/host/config/smtpd_sender_login_maps.ldap
If the postmap queries don't return the expected result, append the
line debuglevel = -1
on smtpd_sender_login_maps.ldap
and
virtual_alias_maps.ldap
and try to identify the problem. You can
also try with ldapsearch
, like this:
apt install ldap-utils ldapsearch -x -LLL -ZZ \ -H ldap://ldap.example.org/ \ -D cn=mail1,ou=apps,dc=example,dc=org -w pass123 \ -b dc=example,dc=org \ "(uid=user1@example.org)"
3.3.2. Test SASL authentication
ds shell cat /etc/saslauthd.conf cat /etc/postfix/sasl/smtpd.conf cat /etc/default/saslauthd | grep MECHANISMS systemctl status saslauthd testsaslauthd \ -f /var/spool/postfix/var/run/saslauthd/mux \ -u user1 -r example.org -p pass123 postconf smtpd_sasl_auth_enable
3.3.3. Test sending and forwarding emails
These test commands are supposed to run on a computer other than the
server, which has a public IP that is NOT on the list of trusted hosts
(config/trusted_hosts
). While running them, it may be useful to check
the logs on the mail server, on a separate tab/terminal:
cd /var/ds/smtp.example.org/ ds shell tail /var/log/mail.log -f
3.3.3.1. Try to send message without authentication
Send a test message from user1@example.org
to a gmail account:
swaks --server smtp.example.org \ --from user1@example.org --to user@gmail.com [. . .] <** 450 4.3.2 Service currently unavailable [. . .] swaks --server smtp.example.org \ --from user1@example.org --to user@gmail.com [. . .] <** 554 5.7.1 <user@gmail.com>: Relay access denied [. . .]
The first time you try to send an email from a new client the service is automatically refused (in order to avoid spambots). The second time it fails because the sender is not authenticated.
Note: If you succeed to send email without authentication, make sure
that your public IP is not on config/trusted_hosts
. If you modify
this file you should also run ds inject update.sh
.
3.3.3.2. Send a message with authentication
swaks --server smtp.example.org \ --from user1@example.org \ --to user@gmail.com \ --auth-user user1@example.org \ --auth-password pass123 [. . .] <- 235 2.7.0 Authentication successful [. . .]
This time it should succeed, if the password is correct.
3.3.3.3. Send a message from the wrong address
Let's try to send a message from the same user as before, but with
--from user2@example.org
:
swaks --server smtp.example.org \ --auth-user user1@example.org \ --auth-password pass123 \ --from user2@example.org \ --to user@gmail.com [. . .] <- 235 2.7.0 Authentication successful [. . .] <** 553 5.7.1 <user2@example.org>: Sender address rejected: not owned by user user1@example.org [. . .]
This --from
address does not belong to the authenticated user, so
the server refuses to send the mail.
3.3.3.4. Test forwarding emails
Let's send an email from user1
to user2
. It should be forwarded to
the external email address of user2
:
swaks --server smtp.example.org \ --auth-user user1@example.org \ --auth-password pass123 \ --from user1@example.org \ --to user2@example.org \
Try also to send an email from an external email address to
user1@example.org
and user2@example.org
.
4. WKD+WKS server
Web Key Directories provide an easy way to discover public keys through HTTPS. They provide an important piece of infrastructure for improving the user experience about exchanging secure emails and files.
A Web Key Directory can be built and maintained even manually, however for a large organization it is recommended to set up a Web Key Service. A WKS is usually implemented as part of your mail server, however it is also possible to implement it as a separate component that cooperates with your mail server. This is how we will install it.
4.1. Install the WebKey container
Let's assume that you have already installed docker-scripts and wsproxy from the previous sections. Follow these steps to install the WebKey container:
- Get the scripts:
ds pull webkey
- Create a directory for the container:
ds init webkey @wks.example.org
Fix the settings:
cd /var/ds/wks.example.org/ ; vim settings.sh
. OnWEBKEY_DOMAINS
list the mail domains that will be supported by this container, like this:WEBKEY_DOMAINS="example.org example.com"
- Make the container:
ds make
- Test it:
ds play tests/test1.sh
4.2. Configuration on the mail server
We need to make these configurations on the mail server in order to integrate it with the WKD+WKS server:
First of all, we need to add a record like this on the DNS configuration of the mail server:
openpgpkey.example.org. IN CNAME wks.example.org.
It is needed to tell the mail clients that the WKD of the domain
example.org
is served bywks.example.org
. We should do the same thing for the domainexample.com
as well.Then we should also add a WKS server with the command
ds wks
, like this:cd /var/ds/smtp.example.org/ ds wks ds wks add wks.example.org 25 example.org example.com
In this case we assume that the WebKey container is in the same docker network as the mail server, so we are using directly the port
25
ofwks.example.org
. If this was not the case, we should have used instead the port10025
, which is forwarded by docker to the port25
of the WebKey container.
If we had to make the configurations on the second step manually
(without using ds wks
), we could do them like this:
Add a virtual alias from
keys@example.org
tokeys@wks.example.org
. It might be done like this:postconf 'virtual_alias_maps = hash:/host/config/virtual_alias_maps' cat << EOF >> /host/config/virtual_alias_maps keys@example.org keys@wks.example.org EOF postmap /host/config/virtual_alias_maps postfix reload
This ensures that key submission emails and submission confirmation emails are forwarded to the WKS server.
Tell the mailserver to send (transport) the mails for the domain
wks.example.org
tosmtp:[wks.example.org]:10025
. It may be done like this:postconf 'transport_maps = hash:/host/config/transport_maps' cat << EOF > /host/config/transport_maps wks.example.org smtp:[wks.example.org]:10025 EOF postmap /host/config/transport_maps postfix reload
Allow the WKS server to send replies (emails) through the mailserver, without authentication, by adding its IP to the list of
mynetworks
. It might be done like this:postconf 'mynetworks = /host/config/trusted_hosts' cat << EOF >> /host/config/trusted_hosts 12.34.56.78 EOF postfix reload
The envelope of the emails (replies) sent from the WKS server will have a sender address like
webkey@wks.example.org
. This may confuse the SPF check of mail providers (in case the emails of the clients are forwarded to external addresses). As a result these emails may be considered less trusty and may end up as spam.To prevent this we should configure the mailserver to rewrite this address to
keys@example.org
. It might be done like this:postconf 'smtp_generic_maps = hash:/host/config/smtp_generic_maps' cat << EOF > /host/config/smtp_generic_maps webkey@wks.example.org keys@example.org EOF postmap /host/config/smtp_generic_maps postfix reload
4.3. Testing WKD
For some basic testing of WKD we can use the tool gpg-wks-client
:
Make sure that it is installed:
apt install gpg-wks-client /usr/lib/gnupg/gpg-wks-client -h
Get the WKD url of the key submission address and check that it exists:
alias wkcl='/usr/lib/gnupg/gpg-wks-client -v' wkcl --print-wkd-url test@example.org wget -qO- https://openpgpkey.example.org/.well-known/ openpgpkey/example.org/submission-address submission_address=$(wget -qO- https://openpgpkey.example.org/ .well-known/openpgpkey/example.org/submission-address) wkcl --print-wkd-url $submission_address wkcl --check test@example.org wkcl --check $submission_address
4.4. Publishing a GnuPG key manually
Let's assume that we want to publish the GnuPG public key for
test1@example.org
. We will use the command gpg-wks-client
, but we
also need a tool for sending emails from the command line; msmtp
is
a good one.
- Make sure that
msmtp
is installed:apt install msmtp
Send a key publishing request like this:
gpg --list-keys test1@example.org /usr/lib/gnupg/gpg-wks-client \ --create \ AB97233AD0EB0180882D1227799020EF6FF16876 \ test1@example.org \ | msmtp \ --read-envelope-from --read-recipients \ --tls=on --auth=on \ --host=smtp.example.org --port=587 \ --user=test1@example.org --passwordeval="echo pass1"
The command
gpg-wks-client --create
creates a request email in the format that is required by the WKS, andmsmtp
sends it through the mailserver, with authentication.- Soon WKS will send as followup an email with subject Confirm your key publication.
Save it as a text file:
Confirm-your-key-publication.eml
Send the confirmation email with the command
gpg-wks-client --receive
, like this:cat Confirm-your-key-publication.eml \ | /usr/lib/gnupg/gpg-wks-client --receive \ | msmtp \ --read-envelope-from --read-recipients \ --tls=on --auth=on \ --host=smtp.example.org --port=587 \ --user=test1@example.org --passwordeval="echo pass1"
Note: Key publishing process was automated by Thunderbird+Enigmail. However Thunderbird 78 dropped Enigmail. Its functionality is supposed to be merged to Thunderbird, but not all of it is already merged. In particular the features that facilitate and automate the interaction with a WKS server are missing. Let’s hope that they will be added back soon.