Running your own non-bouncing Zimbra server
Despite having configured SPF, DKIM and DMARC for my email domain, I recently noticed that the mail queue of my Zimbra server was going through the roof with thousands of undelivered and postponed email messages, mostly with sender “mailer-daemon” and some recipient I did not know. If you run a Zimbra server and have this problem or perhaps even better; if you don’t have it yet and don’t want to have it, read on.
Prerequisites
A few of us still like to run our own email servers and refuse to admit defeat to the increasing complexity this involves. And it is admittedly getting complicated. I have used Zimbra as email server platform of choice for many years, mostly without problems. Recent years have of course seen a steady stream of junk email, but also a stream of initiatives designed to combat this at the server and infrastructure level. The technologies that seem to be here to stay are Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). Both techniques are based on the DNS-system, more specifically on publishing special records in your email domain’s zone, thus allowing other email servers to verify that email coming from your server is valid and authentic. SPF allows email servers to look up which servers you have permitted to send email from your domain. DKIM allow email servers to lookup the public cryptographic keys for your domain which you also have published in DNS zone, and verify that email messages have in fact been signed using the matching private key and not been tampered with. Information about your implementation of SPF and DKIM may be published in your DNS zone as well, in a standardized format known as Domain-based Message Authentication, Reporting and Conformance (DMARC), which also specifies how mail failing to meet your policies should be handled.
The backscatter problem
Configuring Zimbra (or Postfix for that matter, which is the MTA Zimbra uses) for SPF, DKIM and DMARC is mostly pretty well-documented. However you may find yourself in a situation, where you have done all the required work, sucessfully tested your setup using the excellent MXtoolbox, but still find yourself unable to deliver email to other email servers, specifically you may find it hard to deliver email to Google’s Gmail servers.
First step is to check your log files. Start with “/var/log/mail.log”.
If you find error messages in you mail log containing something along the lines of:
“550 5.7.1 Our system has detected an unusual amount of unsolicited mail originating from your IP address. To protect our users from spam, mail sent from your IP address4 has been rejected/temporarily blocked.”
it is possible, that your email domain is a favourite of spammers. It is also very possible that they are using bogus email addresses ending in your domain name. SPF is designed to eliminate this by checking if the spammer’s email server is in fact allowed to send email with addresses ending in your domain name. Not everyone has implemented SPF, and a lot of the spam being sent is getting accepted by the receiving email servers. Some of it is then categorized as spam or it is determined that the recipient does not exist, and the poor email is bounced back to the apparent sender – an email address ending in your domain name. This is called backscatter. There is not much you can do about it, except make sure your have SPF, DKIM and DMARC set up correctly and then ignore it. The problem I, and I suspect other Zimbra users, had was that our Zimbra mail server was not ignoring this backscatter, but in fact bouncing it back. This of course only aggravates the original problem of too much junk email bouncing around the Internet. Sigh…
The non-bouncing email server
Letting all incoming email through the door, and then bouncing troublemakers was probably a good idea at some point in Internet history, but today you are usually much better off just not opening the door to anyone you don’t know. Let’s try to make our Zimbra server non-bouncing.
For starters, you should follow these instructions, in order to have Zimbra reject emails to non-existing recipients, instead of accepting them and then bouncing them back afterwards. This does in fact make Zimbra reject email to unknow accounts rather than queuing and bouncing. Problem solved I thought. Except a lot of “status bounced” messages kept appearing in “/var/log/mail.log”. I noticed that the messages getting accepted and bounced were mostly being tagged as spam as well. It turns out that Zimbra’s built-in utility for handling spam filters – Amavisd was getting in the way. Mail flagged as spam was not being rejected outright, only non-spam emails for non-existing users were being rejected. That was only a small portion of the backscatter it turned out. The fix was to reconfigure Amavisd. I do not know if this can be done using “zmprov” – I edited “/opt/zimbra/conf/amavisd.conf.in” and added the following lines:
$warnspamsender = 0;
$warnbadhsender = 0;
$warnvirussender = 0;
$warnbannedsender = 0;
right under “# COMMONLY ADJUSTED SETTINGS:”. After adding the lines, I did a “zmamavisdctl stop” and “zmamavisdctl start” as the zimbra user. Now messages tagged as spam for non-existing users was also getting rejected, which is what I wanted. Problem really solved I thought.
Except I still noticed some “status bounced” messages in the log files. I also noticed that the “status=bounced” lines in the log file all were preceded by a line containing a “orig_to=<some_address@some.domain>”, where “some.domain” was a Zimbra alias domain. This time apparently Zimbra’s nice alias domain feature was getting in the way. Oh well, can’t have it all, I guess. So I fired up the Zimbra admin UI, deleted the alias domain, and added aliases for the indidual email addresses I wanted aliases for instead. For now it looks like I have a non-bouncing Zimbra server. Will update this post if more bounces show up.
robin | Jun 3,2019
great tutorial and write up. helped me a lot.
Joe Ashley | Aug 29,2022
I know this is a little old but what you described in the first paragraph is just like what I have done. I’ve been running Zimbra and Zextras for about 12 years. I recently threw my email server behind pfSense so I could give the router back to my ISP that I was paying for every damn month. Since then it seems my ISP (upstream SMTP server) start blocking me due to excessive SPAM. Nothing else changed so I started looking. My family sends about 30 email per week and yet my dailyemailreport shows over 1000 some days. I have NOT been able to get that number down, nor can I even confirm that number by watching the queues. (monitored with Zabbix)
Any hints would be appreciated.
Origo | Aug 30,2022
Well, sure sounds like you’re bouncing. I would need a bit more info, in order to provide any useful hints. First of all – “/var/log/mail.log” surely provides some hints as to what’s going on…?
/chr
Joe Ashley | Aug 30,2022
I have tons of info. If you would like to have the discussion here I am fine with that but if somewhere else is better let me know.
Joe Ashley | Aug 30,2022
I just tried to leave more information and it didn’t stick