Since I installed Varnish I lost the capability of identifying client IP address, both on log and on error files. This prevents fail2ban from properly blocking offenders
I searched the net for a solution, and most of the solutions were too complicated and didn't work, so I did my own homework.
This applies to fail2ban 0.9.3-1, Apache 2.4.29-2, Varnish 5.2.1-1.
There is a simple solution.
Install mod_rpaf with the command:
# sudo apt-get install libapache2-mod-rpaf
Now enable it in Apache
# sudo a2enmod rpaf
Check that you have the log_config module loaded:
# sudo apachectl -M |grep log_config
You should get an answer containing: log_config_module (static)
If not:
# sudo a2enmod log_config
Now edit the apache configuration file:
# sudo nano /etc/apache2/apache2.conf
Locate the LogFormat directives and after those insert this line:
ErrorLogFormat "%{X-Forwarded-For}i [%t] [:%l] [pid %P] [client %a] %M"
Now save and restart Apache
# sudo service apache2 restart
Check your access and error logs, those will have client IP at start and your error log will be in the usual format
fail2ban has problems if you enable the apache-fakegooglebot jail, it will block googlebot, which is not the wanted result. Edit the jail configuration:
# sudo nano /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot
and replace the first line #!/usr/bin/python by #!/usr/bin/python3 then restart fail2ban:
# sudo service fail2ban restart
If you use logrotate to rotate your apache logs (they take huge space with time), as as I do are hosting multiple sites, you may get permission problems with fail2ban or awstats. You may solve it by editing /etc/logrotate.d/apache2 and insering this block, assuming you have a logs directory on each site root and want to rotate weekly:
/
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 775 www-data www-data
sharedscripts
postrotate
if invoke-rc.d apache2 status > /dev/null 2>&1; then \
invoke-rc.d apache2 reload > /dev/null 2>&1; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}
Installation of the above is entirely on your risk.