Reverse Proxy Notes

Notes for a reverse proxy server based on CentOS 7, utilizing HAProxy with Letsencrypt certificates and fail2ban.

HAProxy

HAProxy is super easy to setup. Most settings are default. Here is the core config for my frontend's and backend's in /etc/haproxy/haproxy.cfg:

frontend  www-http
    bind *:80
    reqadd X-Forwarded-Proto:\ http

    acl host_www   hdr(host) -i    www.e5i5o.com
    acl host_www   hdr(host) -i    photos.e5i5o.com

    acl letsencrypt-acl path_beg /.well-known/acme-challenge/

    use_backend letsencrypt-backend if letsencrypt-acl
    use_backend www if host_www
    default_backend             default

frontend www-https
    bind *:443 ssl crt /etc/haproxy/certs/combined.pem
    reqadd X-Forwarded-Proto:\ https

    acl host_www      hdr(host) -i    www.e5i5o.com
    acl host_www      hdr(host) -i    photos.e5i5o.com
    acl host_mail     hdr(host) -i    mail.e5i5o.com

    acl letsencrypt-acl path_beg /.well-known/acme-challenge/

    use_backend letsencrypt-backend if letsencrypt-acl
    use_backend www if host_www
    use_backend mail if host_mail
    default_backend             www

backend default
    server  default 192.168.0.10:80

backend www
    redirect scheme https if !{ ssl_fc }
    server  www 192.168.0.11:80

backend mail
    redirect scheme https if !{ ssl_fc }
    server  mail 192.168.0.12:80

backend letsencrypt-backend
   server letsencrypt 127.0.0.1:54321

To silence a warning I add the following to the Global section:

tune.ssl.default-dh-param 2048

To add a monitoring page, add the following. Make sure to change UserName and Password:

listen  stats   *:8080
        mode            http
        log             global

        maxconn 10

        timeout connect 100s
        timeout client  100s
        timeout server  100s
        timeout queue   100s

        stats enable
        stats hide-version
        stats refresh 30s
        stats show-node
        stats auth <UserName>:<Password>
        stats uri  /haproxy?stats

Letsencrypt

We are going to terminate SSL at the proxy server so lets install Letsencrypt:

yum install epel-release
yum install python-certbot-apache

Prepare a cert directory:

mkdir -p /etc/haproxy/certs

Grab some certs:

certbot certonly --standalone --preferred-challenges http --http-01-port 54321 -d e5i5o.com -d www.e5i5o.com -d photos.e5i5o.com -d mail.e5i5o.com

Note: Notice --http-01-port 54321. Letsencrypt will listen to this port when authorizing certs. HAProxy has a backend that will forward to this port. Externally everything is still standard HTTP ports.

Combine fullchain.pem and privkey.pem for haproxy:

cat /etc/letsencrypt/live/e5i5o.com/fullchain.pem /etc/letsencrypt/live/e5i5o.com/privkey.pem > /etc/haproxy/certs/combined.pem
chmod -R go-rwx /etc/haproxy/certs

Renewals every 90 days

certbot renew

Apache

Since my Apache server is now behind a proxy, Apache logs show all traffic coming from a local address. To fix this I installed mod_rpaf on my web server.

Note: I'm running CentOS 6 for my web server. If I were running CentOS 7 I would use mod_remoteip.

Install:

yum localinstall http://y-ken.github.com/package/centos/6/x86_64/mod_rpaf-fork-0.6-5.el6.x86_64.rpm

Edit /etc/httpd/conf.d/rpaf.conf for your environment:

# mod_rpaf-fork is an Apache-2.2 module for reverse proxy.
# Set the header for REMOTE_ADDR, HTTPS, and HTTP_PORT from upstream proxy environment variables.
# Documentation at http://github.com/y-ken/mod_rpaf
#
LoadModule rpaf_module modules/mod_rpaf-2.0.so
RPAFenable On
RPAFproxy_ips 192.168.0.XXX
RPAFheader X-Forwarded-For
RPAFsethostname On
RPAFsethttps Off
RPAFsetport Off

Restart Apache:

/etc/init.d/httpd restart

Now if you watch /var/log/httpd/access_log you should see real IP's.

Fail2ban

Fail2ban is used here to monitor and limit logon attempts for ssh.

Installation

Before installing Fail2ban, make sure you have the EPEL repository installed.

yum install epel-release

Now install fail2ban:

yum install fail2ban

Enable:

systemctl enable fail2ban

Configuration

Fail2ban will pull it's configuration from /etc/fail2ban/jail.conf by default. To avoid editing the default file we can create /etc/fail2ban/jail.local. Any settings in this file will override the defaults.

vi /etc/fail2ban/jail.local

Add the following to enable sshd monitoring with a ban of 1 hour:

[DEFAULT]
# Ban hosts for one hour:
bantime = 3600

# Override /etc/fail2ban/jail.d/00-firewalld.conf:
banaction = iptables-multiport

[sshd]
enabled = true

Now restart fail2ban:

systemctl restart fail2ban

Monitoring

Monitor all jails:

# fail2ban-client status

Status
|- Number of jail:   1
`- Jail list:   sshd

Monitor sshd jail:

# fail2ban-client status sshd

Status for the jail: sshd
|- Filter
|  |- Currently failed:   1
|  |- Total failed:   87
|  `- Journal matches:   _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
   |- Currently banned:   2
   |- Total banned:   15
   `- Banned IP list:   218.65.30.38 114.113.234.66