Showing posts with label Ubuntu. Show all posts
Showing posts with label Ubuntu. Show all posts

Tuesday, November 13, 2012

Howto : Fail2ban, Secure your Network services with Fail2ban and ban malicious hosts





Last Update: 11/13/2012


The Goal: 

Fail2Ban is a very flexible and powerful tool to help you improving your security by temporarily banishing hosts IPs trying to have malicious activity against your services. (brute force attack...)

Easy to install and configure, Fail2Ban will permanently read your files logs and look for suspicious connection attempts or activity, based on provided / custom rules.

Depending on your configuration, Fail2Ban will ban the remote IP using Netfilter rules (iptables) and then unban it depending on your settings.

This a very great tool you MUST have in your server if you opened any service like ssh, http/https with basic authentication, smtp...and much more.


Major sources: 

Official Fail2ban Wiki:
http://www.fail2ban.org/wiki/index.php/Main_Page

Various articles:
http://www.fanatips.org/Apache_%2F%2F_Nginx_tips/Secure_Nginx_with_fail2ban
http://snippets.aktagon.com/snippets/554-How-to-Secure-an-nginx-Server-with-Fail2Ban



Summary of steps: 

Step 1: Requirements
Step 2: Installation and base configuration
Step 3: Enabling services
Step 4: Test your configuration
Step 5: Adding new customized services, example with Nginx Web Server

FAQ & Issues



Step 1: Requirements

Fail2ban is quite easy to install and setup, still depending on your needs you have some requirements:

Iptables:

Fail2ban will require iptables to banish suspicious hosts, it is not a requirement to setup your iptables Firewall configuration (even if recommended).

Fail2ban will insert a drop line, whenever you are using iptables or not.

It is probably installed by defaut (use the command "iptables" to check), if not:
$ sudo apt-get install iptables

MTA configuration for mail alerting:

If you want Fail2ban to able to send you alerting mails in case of action done (banish, unbanish...), then you need to setup your MTA.

Your system probably only be a client of a mail central hub (eg. your ISP), if this is the case i recommend to install "ssmtp" wich is very light and simple to setup, but you may prefer to use sendmail, postfix or anything else.

If your system is a mail server itself, i guess you already know how to configure it :-)

Installation: 
$ sudo apt-get install ssmtp

Configuration (real simple!): 

Edit "/etc/ssmtp/ssmtp.conf" and set:

- mailhub: Define here the smtp server of your local ISP (eg. myisp.smtp.com)
- rewriteDomain: if you want the sender domain to be rewritten with the value of your choice

For most of cases, this will be the only modifications required, in case of need see ssmtp configuration pages. (ask Google ^^)

Quick send test:
$ echo "Testing" | mail -s Subject dest@domain.com

If you receive the mail then you're done. (check /var/log/mail* if required)


Step 2: Installation and base configuration

Installation:

Very simple, For Debian/Ubuntu based systems:
$ sudo apt-get install fail2ban

Initial Configuration:

Edit main Fail2Ban configuration file: "/etc/fail2ban/jail.conf" and define:

1. Local hosts to ignore:

If you want to ignore some local hosts, add them to your configuration, example with 192.168.0.0/24 local network:
# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1/8 192.168.0.0/24
bantime  = 600
maxretry = 3

2. Mail alert recipient:

If you want to send alter when an action has been done, set the recipient
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = admin@mydomain.com

3. MTA:

Sendmail is defined as default command called for sending mails, both should work but i prefer to call standard "mail" command:
# email action. Since 0.8.1 upstream fail2ban uses sendmail
# MTA for the mailing. Change mta configuration parameter to mail
# if you want to revert to conventional 'mail'.
mta = mail

4. Default action when a host has banished

In default configuration, when a host has been banned, the information is just logged and no mail report will be sent.

If you want a report to be sent by mail (associated with a whois request report for the concerned IP), set:
action = %(action_mw)s

If you want a report to be sent by mail with a log extract (associated with a whois request report for the concerned IP), set:
action = %(action_mwl)s



Step 3: Enabling services

SSH example:

The next step will be to define which Network services you will want to supervise, in default configuration Fail2Ban will only monitor SSH for both login failed and DDOS attacks, extract:
[ssh]

enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 6

The import thing here will be the enabled value set to "true" (^^) but also the port, in case of host banishing the iptables rules will be created using this port.

Fail2ban will create multiport iptables rules, so you can add as many ports as you require.

Example if your SSH is running on several port (example 22 for internal and OTHER for external purposes) then you can set:
[ssh]

enabled  = true
port     = ssh,OTHER
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 6


Other services important vars values:

In each service section, you can overwrite default vars values or set specific ones for your needs.
Here are some you may need to set.

1. Protocol type:

The default protocol if not set will be tcp, you can set it to "udp" or "all" if you need to block bother tcp/udp ports.
protocol = all

2. Ban time:

You may want to overwrite the default bantime, then you just need to it in the service concerned section, time is set in seconds, example for 10 minutes banishment:
bantime = 600 # 10 minutes

3. Ports:

As seen before with the SSH example, you can set as many ports as you required in the port section.
You can call them by names if they are present in /etc/services or by the port number.

Each port must be separated by a coma.

4. Action:

Defaults actions in response of matched events are defined in jail.conf, but you can overwrite values for each specific network service.

In the following example, let's say our default configuration sends emails alert with log extract ("action = %(action_mwl)s") but for a specific Network service we just it to be banned without any altering.

Then you will overwrite the action value in your service section:
action_ = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s


Enable your services:

Depending on your configuration, you need to enable Network services as you require.
Follow each section to enable if required.

If you need to create your own service, follow the example of Nginx as above.



Step 4: Testing your configuration


Testing your configuration is very simple :-)

Just test accessing your SSH host outside of your local network (using your Smartphone or whatever you want!) and put false credentials.

Upon the max retry failures, your host should have been banished, you will see in fail2ban logs:

/var/log/fail2ban.log
2012-11-13 10:41:43,487 fail2ban.actions: WARNING [ss] Ban xxx.xxx.xxx.xxx

Locally, a new iptables rule has been created inside the jail dedicated iptables section (automatically created by fail2ban at boot time):
Chain fail2ban-ssh (1 references)
target     prot opt source               destination         
DROP       all  --  37.160.52.0          anywhere            
RETURN     all  --  anywhere             anywhere            


And you should (if you set it) have received a mail with all required information.

Depending on your configuration, XX minutes/days or whatever after having been ban, the host will be unban and the iptables drop rule will be removed.


Powerful, simple, beautiful :-)



Step 5: Adding new customized services, example with nginx Web Server


One of very great things with Fail2ban is that you can easily add any customized service you would require, you just add to define regex expression that will be used to match patterns and define Network section.

Here comes a full example for Nginx Web Server, we will monitor basic authentication failures, badbots, prevent script execution... and off course this can be easily customized to your needs.

Nginx Web Server Fail2Ban integration:

1. Create Nginx filters

Create each required file with its content:

/etc/fail2ban/filter.d/nginx-auth.conf
#
# Auth filter /etc/fail2ban/filter.d/nginx-auth.conf:
#
# Blocks IPs that fail to authenticate using basic authentication
#
[INCLUDES]                                                                                                                    
                                                                                                                              
# Read common prefixes. If any customizations available -- read them from                                                     
# common.local                                                                                                                
before = common.conf 

[Definition]
 
failregex = no user/password was provided for basic authentication.*client: <HOST>
            user .* was not found in.*client: <HOST>
            user .* password mismatch.*client: <HOST>
  
ignoreregex =

/etc/fail2ban/filter.d/nginx-login.conf
#
# Login filter /etc/fail2ban/filter.d/nginx-login.conf:
#
# Blocks IPs that fail to authenticate using web application's log in page
#
# Scan access log for HTTP 200 + POST /sessions => failed log in
[INCLUDES]                                                                                                                    
                                                                                                                              
# Read common prefixes. If any customizations available -- read them from                                                     
# common.local                                                                                                                
before = common.conf 

[Definition]
failregex = ^<HOST> -.*POST /sessions HTTP/1\.." 200
ignoreregex =

/etc/fail2ban/filter.d/nginx-noscript.conf
# Noscript filter /etc/fail2ban/filter.d/nginx-noscript.conf:
#
# Block IPs trying to execute scripts such as .php, .pl, .exe and other funny scripts.
#
# Matches e.g.
# 192.168.1.1 - - "GET /something.php
#
[INCLUDES]                                                                                                                    
                                                                                                                              
# Read common prefixes. If any customizations available -- read them from                                                     
# common.local                                                                                                                
before = common.conf 

[Definition]
failregex = ^<HOST> -.*GET.*(\.php|\.asp|\.exe|\.pl|\.cgi|\scgi)
ignoreregex =

/etc/fail2ban/filter.d/nginx-proxy.conf
# Proxy filter /etc/fail2ban/filter.d/nginx-proxy.conf:
#
# Block IPs trying to use server as proxy.
#
# Matches e.g.
# 192.168.1.1 - - "GET http://www.something.com/
#
[INCLUDES]                                                                                                                    
                                                                                                                              
# Read common prefixes. If any customizations available -- read them from                                                     
# common.local                                                                                                                
before = common.conf 

[Definition]
failregex = ^<HOST> -.*GET http.*
ignoreregex =


2. Create Nginx configuration

Edit Fail2Ban configuration  "/etc/fail2ban/jail.conf" and add:

Note: 

If Nginx is listening to non standard http/https port, think to adapt to your configuration
Also, nginx-noscript will prevent any script execution on your sites, if this is not what you need just comment out this section.

/etc/fail2ban/jail.conf
[nginx-auth]
enabled = true
filter = nginx-auth
port = http,https
logpath = /var/log/nginx*/*error*.log
bantime = 600 # 10 minutes
maxretry = 6
 
[nginx-login]
enabled = true
filter = nginx-login
port = http,https
logpath = /var/log/nginx*/*access*.log
bantime = 600 # 10 minutes
maxretry = 6
  
[nginx-badbots]
enabled  = true
filter = apache-badbots
port = http,https
logpath = /var/log/nginx*/*access*.log
bantime = 86400 # 1 day
maxretry = 1
  
[nginx-noscript]
enabled = true
port = http,https
filter = nginx-noscript
logpath = /var/log/nginx*/*access*.log
axretry = 6
bantime  = 86400 # 1 day
  
[nginx-proxy]
enabled = true
port = http,https
filter = nginx-proxy
logpath = /var/log/nginx*/*access*.log
maxretry = 0
bantime  = 86400 # 1 day



Finally, restart fail2ban and you're done! (sudo service fail2ban restart)

To test your configuration, just try to access to your basic authentification protected Web Site with bad credentials, as for the SSH example your host will be ban and you'll get the report :-)



FAQ & Issues


  • How to prevent Fail2ban from sending alerts mails when stopped/started

When email alerting is configured, fail2ban will send you one mail per jail configured (one per Network service) for each action (stop/start).

This can be as usefull as boring :-)

If you want to disable this behavior:

- Go in "/etc/fail2ban/actions.d/"

- You will find 6 configurations files related to sendmail & mail, depending on what you set (sendmail vs mail) you just have to comment out actionstart and actionstop:

"action_": comment "actionstart" & "actionstop" in action.d/sendmail.conf
"action_mw": comment "actionstart" & "actionstop" in action.d/sendmail-whois.conf
"action_mwl": comment "actionstart" & "actionstop" in action.d/sendmail-whois-lines.conf

"action_": comment "actionstart" & "actionstop" in action.d/mail.conf
"action_mw": comment "actionstart" & "actionstop" in action.d/mail-whois.conf
"action_mwl": comment "actionstart" & "actionstop" in action.d/mail-whois-lines.conf

- Restart fail2ban, you're done!













Sunday, September 9, 2012

Howto : Linux - Setup a simple and powerful DNLA server (UPnP protocol) in 5 minutes with minidlna




The Goal:

With "minidlna" you will create within a few minutes a real simple but powerful DLNA Server.

UPnP is a protocol (and DLNA a certification based on this protocol) being more and more used by many manufacturers, it allow you to share your multimedia files between all your DLNA compliant devices.

As a simple example, you have a computer acting as your DLNA server and sharing multimedia files through your network to your Home Cinema.

Minidlna is the perfect DNLA server, simple, light and powerful.

Let's begin!



1 - Install minidna


On Ubuntu based systems, just do:

$ sudo apt-get install minidlna


2 - Configure minidlna


Edit main minidlna configuration file "/etc/minidlna.conf" and do same changes as follows:

- Change port to "49200"

- Add your media sources, you can add as many sources as you want like this:
media_dir=V,<MY VIDEO SOURCE 1>
media_dir=V,<MY VIDEO SOURCE 2>
media_dir=A,<MY MUSIC SOURCE>

- uncomment "friendly_name" (delete "#") and set it to the value you want, it will be the displayed name of your DLNA server

- uncomment "log_dir" (delete #) to ensure main log will be available in /var/log

You case also manually setup the location of the database file. (called files.db)

3 - Start minidlna and begin the scan of your library


Minidlna has been started automatically when installed, but we want it to take in charge our modifications and begin to scan our sources.

First, stop minidlna:
$ sudo /etc/init.d/minidlna stop

Delete existing database:
$ sudo rm /var/lib/minidlna/files.db

Start minidlna:
$ sudo /etc/init.d/minidlna start

And finally take a look at the log files to see what is doing minidlna process:
$ tail -20f /var/log/minidlna.log

Depending on number of files in your sources, minidlna will need a few time to scan, you should see a message like this:
[2012/09/08 17:08:25] scanner.c:719: warn: Scanning /media/FREECOM-HDD/Multimedia/Films_HD-DVD
[2012/09/08 17:08:57] scanner.c:790: warn: Scanning /media/FREECOM-HDD/Multimedia/Films_HD-DVD finished (232 files)!


4 - Access to your DLNA device and navigate to your DLNA Server



Depending on your device, go the your menu and navigate into your DLNA server.

Here is one example screenshot using my LG Home Cinema:





And this works very well, i can play a big big mkv in 1024p (around 45 GB size, native Blu-ray) using local Network (my LG Home Cinema has a 100 Mbps local network connection) with absolutely no issues!

Real great!

Keep in mind that this may also depends on your DLNA device capacity...

Finally, with minidlna note a few last more things:

1. In case of trouble, do not hesitate to delete the database file (see steps above) and restart minidlna, it will automatically recreate the database

2. minidlna sometimes does not like specific characters, you should avoid this in your folder names

3. Some file format may not be playable, it depends on your DLNA device and minidlna.
In my experience, prefer a mkv file, almost all recent devices will support mkv

4. Any added media file will be automatically available in minidlna




Saturday, September 8, 2012

Linux TIP : Run an application in a different Language


Sometimes you may need to run an application in different languages, real easy to do with Linux, won't be permanent so you can get back to your initial language.

Here's how:

1. First locate the language locale you want, let's say you want US English, use the command "locale -av" and locate the section you need:


# locale -av


Output example:


locale: en_US.utf8      archive: /usr/lib/locale/locale-archive
-------------------------------------------------------------------------------
    title | English locale for the USA
   source | Free Software Foundation, Inc.
  address | 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA
    email | bug-glibc-locales@gnu.org
 language | English
territory | USA
 revision | 1.0
     date | 2000-06-24
  codeset | UTF-8

2. Open a terminal, you will open your application within this terminal, that way all of your system won't affected but just the application you want


3. Change your locale environnement var  "LC_MESSAGES" to the locale value, in our example:


$ export LC_MESSAGES=en_US.UTF-8


4. Open your application, language should be US

5. To revert, nothing to do, just close your terminal an re-open the application

Note: For this to work on some application, you may have the install the language associated package.
If this does not work, you may lack a language package.

You can also have to deal with "LANG" var, just test.




Thursday, June 28, 2012

VPN - Protect your personal Internet Traffic with a VPN Provider on your Home Linux Server (with auto check and reconnect shell script)

Protect your personal Internet Traffic with a VPN Provider on your Home Linux Server (with auto check and reconnect shell script)

The Goal:

I will not explain in details what's the benefit of a VPN (Virtual Private Network), a quick search on the Internet will easily answer to that question :)

Anonymity, confidentiality, accessing to Internet services closed to your own country, protecting your Internet traffic from being inspected by anyone or even your ISP... There may be thousands of good reasons to need a personal VPN !

Please do not hesitate to comment if you like, hate... or see any error, update required ! :-) 

What you need:

First of all, you need a VPN provider.
You will find a lot of provider offers on Internet, ensure enough servers and countries are provided and also if you expect to use on a Linux Home server, ensure openvpn protocol is provided. (you may also use pptp or l2tp but i not recommend it)

Take a look on quick google research and you will what you look for, i would personally recommend :
  • vpntunnel.se
  • vyprvpn
  • hidemass (my great favorite, serious, stable and performer!=)


Step 1: Prepare your system


First, you need to install openvpn and openvpn additional requirements.
On Debian and derived (Ubuntu...) systems, this will be simply achieved:

sudo apt-get install openvpn

Step 2: Get your VPN provider configuration files

You will have to follow your provider's specific Howto to get needed configuration files.

In the case of Hidemyass as for an example, you will be able to choose between TCP or UDP configuration, i advise you to choose UDP because of the protocol itself you'll be expecting better inbound and outbound rates.

In the case of hidemyass, i recommend you to check their nice Wiki page:

http://wiki.hidemyass.com/Main_Page

And specific Linux page:

http://wiki.hidemyass.com/Linux_OpenVPN

Step 3: Initiate your first connection

  • In "/etc/openvpn" create a text file "passwd.txt" that will contain credential information in 2 lines:
login
password

This file will contain your password in clear, and as far as i know it's possible to hash it...

  • For hidemyass (and probably any provider), you will have to extract all configuration files (including certificate files) to /etc/openvpn
  • Prepare each openvpn configuration file to include access to your password file:
cd /etc/openvpn
for i in `ls *.ovpn`; do echo "" >> $i && echo "auth-user-pass /etc/openvpn/passwd.txt" >> $i; done
NB: You will be able to connect without having to submit credentials
  • Choose a country configuration file, and create symbolic link to openvpn.conf:
Example:
ln -s NL.ROTTERDAM4-UDP.ovpn openvpn.conf
  • Initiate the connection:
sudo /etc/init.d/openvpn start
  • Check connection status:
If your connection is functional, you should have a new device called "tun0"

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet adr:10.200.5.1  P-t-P:10.200.5.1  Masque:255.255.252.0
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          Packets reçus:19 erreurs:0 :0 overruns:0 frame:0
          TX packets:38 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:100 
          Octets reçus:6751 (6.7 KB) Octets transmis:10242 (10.2 KB)
  • Test your Internet access


Step 3: Implement auto connection check and auto reconnect

These 2 scripts will automatically initiate the VPN connection, check its reliability by an icmp request to google.com, in case of failure will stop it and try to reconnect.
  • Copy "watch_openvpn" to /etc/init.d/watch_openvpn
  • Attribute correct owner and rights:
sudo chown root:root /etc/init.d/watch_openvpn
sudo chmod 755 /etc/init.d/watch_openvpn
  • Copy watch_vpn.sh to /etc/openvpn/watch_vpn.sh
  • Attribute correct owner and rights:
sudo chown root:root /etc/openvpn/watch_vpn.sh
sudo chmod 755 /etc/openvpn/watch_vpn.sh
  • Add it to run levels:
sudo update-rc.d watch_openvpn defaults
  • Add some alias to manage your connection:
Edit ~/.bashrc and add:

alias vpnstart="sudo /etc/init.d/watch_openvpn start_vpn"
alias vpnstop="sudo /etc/init.d/watch_openvpn stop_vpn"
alias vpnstatus="sudo /etc/init.d/watch_openvpn status_vpn"
alias watchstart="sudo /etc/init.d/watch_openvpn start"
alias watchstop="sudo /etc/init.d/watch_openvpn stop"
alias watchstatus="sudo /etc/init.d/watch_openvpn status"
  • Optional: allow your user to manage VPN connection without providing credentials:
sudo visudo

Adapt your username and add:
user ALL=(ALL) NOPASSWD: /etc/init.d/watch_openvpn *

Logfile will be available in /var/log/openvpn.log

Do not hesitate to adapt these scripts to your needs!

Enjoy :-)

FAQ, issues and advices:


  • When my Linux home server is connected to my VPN, i can't access anymore to any hosted service! (eg. ssh remote access, web traffic, etc...)

Be aware that all your outgoing Internet traffic will be routed to the VPN, that means that any outside request to your server to a service port you have opened will be lost in the VPN tunnel.

There is no "easy" solution, in my opinion the better solution is to create a virtual machine in your host (using Virtualbox as for an example) that will use a bridge connection.

Your virtual machine will be considered as a normal host on the network and any outside connection will be correctly routed to your virtual host.

Moreover, this will improve your security by avoiding any direct connection to your server and jailing it into a virtual host.

You will be able to use your virtual host as an SSH gateway and finally connect to your real host.

Also, if you need to host any web service, you can simply use apache as a reverse proxy (on your virtual host) that will request your physical host.

 Example of a apache reverse proxy configuration with SSL:
<VirtualHost *:PORT>
  ServerName XXXXXXXXXXX
  ProxyRequests Off
  ProxyVia Off
    <Proxy *>
     Order deny,allow
     Allow from all
    </Proxy>
  ProxyPass / http://MY PHYSICAL HOST:PORT
  ProxyPassReverse / http://MY PHYSICAL HOST:PORT
  <Location />
    Order allow,deny
    Allow from all
    AuthName "Access Restricted"
    AuthType Basic
    AuthUserFile "/etc/apache2/.htpasswd"
    Require valid-user
  </Location>
  LogLevel info
  CustomLog /var/log/apache2/access_xxxxxx.log combined
  ErrorLog /var/log/apache2/error_xxxxxxx.log
  SSLEngine on
  SSLCertificateFile /etc/apache2/server.crt
  SSLCertificateKeyFile /etc/apache2/server.key
</VirtualHost>

  • How to find the better VPN server of my VPN provider with better download and upload rates ?
Hidemyass provides a Software with the capacity to give better servers for you...but unfortunately not for Linux !

You can use a simple shell script that will connect to each *.ovpn config file in your openvpn directory, then download a sample file and log download rates:

NB: Adapt it to your needs if required

!/bin/bash
# Speed test, launch as root

cd /etc/openvpn

> test_result.txt

for i in `ls *.ovpn`; do

echo "Speed test using VPN Server : $i "
echo

echo "Stoping VPN"

/etc/init.d/watch_openvpn stop_vpn
sleep 2
rm openvpn.conf
ln -s $i openvpn.conf

echo "Starting VPN using $i"
/etc/init.d/watch_openvpn start_vpn
echo "sleeping 10 seconds..."

sleep 30

echo "Starting Test !"
echo "Testing VPN server $i :" >> test_result.txt
wget -o /tmp/wget.tmp -O /dev/null http://cachefly.cachefly.net/10mb.test
grep '/dev/null' /tmp/wget.tmp >> test_result.txt
echo "Test Done." >> test_result.txt

done

echo "Test terminated !"

Then compare results and make your choice :)