Category: IDS

Snorby in CentOS 6.4

19 Comments

25/02/2014: Hi, I’ve created a new post for CentOS 6.5 but it may work also for CentOS 6.4, please visit this new post before. I’ll keep this article as an archived version if you need to compile wkhtmltopdf for any reason.

 

If I want to have fun with Suricata IDS I think it will be useful to have a monitoring tool to track possible alerts. I’ve chosen Snorby as it seems to have a nice and intuitive GUI. Snorby uses ruby on rails and a mysql database. After many attempts I’ve finally got it running so I want to share with you the steps I’ve followed.

According to its web page, Snorby requires:

  • Ruby > 1.9.2
  • ImageMagick > 6.6.4
  • Rails > 3.0.0
  • Wkhtmltopdf

Unfortunately, packages in CentOS 6.4 repositories have older versions, maybe you can find newer versions in other repositories but time I’ll stick with the official repositories. Compilation fun! Warning: if you use the following commands check if newer versions of downloaded packages exist and change directories and names accordingly.

  1. We’ll start installing some packages using yum 
    yum groupinstall "Development Tools"
yum install openssl-devel readline-devel libxml2-devel libxslt-devel mysql mysql-devel mysql-libs mysql-server urw-fonts libX11-devel libXext-devel qconf fontconfig-devel libXrender-devel unzip wget
  2. Let’s compile ImageMagick. 
    cd /opt
wget http://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/ImageMagick-6.8.5-9.tar.gz
  3. tar xvfz ImageMagick-6.8.5-9.tar.gz
cd ImageMagick-6.8.5-9
./configure
make
make install
ldconfig /usr/local/lib
  4. Time for Wkhtmltopdf. I’ve downloaded the source files because I had problems with the static versions of Wkhtmltopdf. I’ve used the readme file (README_WKHTMLTOPDF) which comes with wkhtmltopdf as a guide, but notice that the gitorious repository is not found.Warning: this step is going to take a loooong time so if you don’t need pdf reporting skip it.Warning: if you see squares in your pdf report instead of text install with yum the urw-fonts package. This information is provided thanks to this stackoverflow question. 
    cd /opt
git clone git://github.com/jcsalterego/wkhtmltopdf-qt.git wkhtmltopdf-qt
cd wkhtmltopdf-qt
  5. // Edit the mkspecs/linux-g++-64/qmake.conf file and change the following lines:
QMAKE_LIBDIR_X11 = /usr/lib64
QMAKE_LIBDIR_OPENGL = /usr/lib64
  6. ./configure -nomake tools,examples,demos,docs,translations -opensource -prefix ../wkqt
make -j3
make install

cd /opt
wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.11.0_rc1.tar.bz2
tar jxvf wkhtmltopdf-0.11.0_rc1.tar.bz2
cd wkhtmltopdf-0.11.0_rc1
../wkqt/bin/qmake
make
ldconfig
ln -s /opt/wkhtmltopdf-0.11.0_rc1/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
ln -s /opt/wkhtmltopdf-0.11.0_rc1/bin/libwkhtmltox.so.0 /usr/lib64/libwkhtmltox.so.0
  7. MySQL!. Start the service and if it’s the first time you install it use the mysql_secure_installation to set root’s password and remove unnecessary tables. 
    service mysqld start
mysql_secure_installation
chkconfig mysqld on
  8. And now we need Ruby and RubyGems. I’m installing ruby 1.9.3 from ruby’s webpage. 
    cd /opt
wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p448.tar.gz
tar xvfz ruby-1.9.3-p448.tar.gz
cd ruby-1.9.3-p448
./configure
make
make install

cd /opt
wget http://production.cf.rubygems.org/rubygems/rubygems-2.0.4.tgz
tar xvfz rubygems-2.0.4.tgz
cd rubygems-2.0.4
ruby setup.rb
  9. We now have gem installed and we’re going to install bundler gem which is needed by Snorby setup. 
    cd /opt
gem install bundler
  10. OK. I want to use Snorby so I need to download it!. I’ve had problems with the latest git version of snorby so I had to use the zip with the stable version (which is linked from Snorby’s webpage). 
    wget -O snorby.zip --no-check-certificate https://github.com/Snorby/snorby/archive/master.zip
unzip snorby.zip
cd snorby-master
  11. Now, pay attention! I’ve found many problems trying to use bundle with ruby 9.3 and I spent several hours finding out what to do. This is what I’ve done, I can guarantee that it’ll work with a newer ruby or snorby version but at least if you find the same problem you won’t suffer. If you find any error please contact me maybe I can help you and update the post so it’s useful for other users. 
    //Edit the Gemfile file and change this line: 
//gem 'rake', '0.9.2'   to:  gem 'rake', '> 0.9.2'
//unless you want this error to show when using bundler: "error: /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.1/lib/bundler/fetcher.rb:112:in `specs': undefined method `each' //for nil:NilClass (NoMethodError)"

//Edit the Gemfile file and add this line: 
//gem 'orm_adapter' after the line: gem netaddr, 
//unless you want this error when using bundler: 'orm_adapter' file not found

//Edit Gemfile.lock and change rake (0.9.2) to rake(0.9.2.2) 
//so rake setup does not complain about a different rake version

//Create a snorby_config.yml file. Edit the production section and set your domain
cp config/snorby_config.yml.example config/snorby_config.yml
// Create a MySQL database and a user for snorby
mysql> create database snorby;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on snorby.* to snorby@localhost identified by 'snorby';

// Create a database.yml config file. 
// Edit the file and set the root password and MySQL server location
cp config/database.yml.example config/database.yml
  12. OK! Now let’s install Snorby. And don’t forget to read the README.md file provided by Snorby’s developers 
    // We are in the snorby directory 
bundle install
rake snorby:setup

// This is the command's output
ERROR 1007 (HY000) at line 1: Can't create database 'snorby'; database exists
[datamapper] Finished auto_upgrade! for :default repository 'snorby'
[~] Adding `index_timestamp_cid_sid` index to the event table
[~] Adding `id` to the event table
[~] Building `aggregated_events` database view
[~] Building `events_with_join` database view
* Removing old jobs
* Starting the Snorby worker process.
* Adding jobs to the queue
  13. Snorby is installed woohoo! Before launching it let’s create an iptables rule (TCP 3000 is the default port) 
    iptables -I INPUT -p tcp --dport 3000 -m state --state=NEW,ESTABLISHED,RELATED -j ACCEPT
  14. And now finally let’s start Snorby in my CentOS 6.4 server: 
    rails server -e production
Booting WEBrick Rails 3.1.12 application starting in production on http://0.0.0.0:300
Call with -d to detach 
Ctrl-C to shutdown server
  15. If all is fine, open a web browser http://x.x.x.x:3000.  Here, two screenshots so you can check that I haven’t lied to you.
    snorby_newsnorby_new_2

Snorby default credentials are: snorby@snorby.org and password snorby.

I’ve had several issues with QT compilation and these links helped me:

Enjoy!

Installing Suricata IDS from source – CentOS 6.4

1 Comment

This post has been updated: 31/05/2013. I’ve tested the installation steps with CentOS 6.4. It also works with CentOS 6.3

Today I’ll compile Suricata in a clean CentOS 6.4 server. I’m not compiling it with PFRING support (it would increase performance). Ok, hands on:

  1. You’ll need the EPEL repository, see the step 2 from this post.
  2. I’ll install Development Tools group and some packages needed by Suricata. 
    yum groupinstall "Development Tools"
yum install pcre-devel libyaml-devel libnet-devel libpcap-devel libcap-ng-devel file-devel zlib-devel
  3. Download Suricata from its web page. Move the tar.gz file to a suitable directory, in my case I’ve chosen /opt directory.
  4. Uncompress it (I’m compiling 1.4.3 version) and configure the compilation. I’ve set some prefixes and directories and added the  –disable-gccmarch as I was having problems (Illegal Instruction) when executing Suricata on my QEMU/KVM virtual machine (the post that helped me). 
    tar xvfz suricata-1.4.3.tar.gz
cd suricata-1.4.3
./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ --disable-gccmarch-native
  5. Ok. Now let’s use make, make install and if you want Suricata to create a config file and download rules from Emerging Threats use make install-full. 
    make
make install
make install-full
ldconfig
  6. And finally let’s try to execute suricata command. 
    [root@sherlock ~]# suricata -V
This is Suricata version 1.4.3 RELEASE

I read in ntop’s web page that virtual pf_ring would improve performance dramatically for virtualization environments like KVM but I have no money now to pay for the fee (if you want to donate let me know :-D) so I’ll try to use it for a few minutes as they suggest for evaluation purposes.

As always I appreciate any comments to improve the quality of this post. Enjoy!

Snorby on CentOS 6.3

13 Comments

Update: 02/06/2013. I’ve created a new post for CentOS 6.4 ig you have problems with CentOS 6.3 try to check this post first.

If I want to have fun with Suricata IDS I think it will be useful to have a monitoring tool to track possible alerts. I’ve chosen Snorby as it seems to have a nice and intuitive GUI. Snorby uses ruby on rails and a mysql database. After many attempts I’ve finally got it running so I want to share with you the steps I’ve followed.

According to its web page, Snorby requires:

  • Ruby > 0.9.2
  • ImageMagick > 6.6.4
  • Rails > 3.0.0
  • Wkhtmltopdf

Unfortunately, packages in CentOS 6.3 repositories have older versions, maybe you can find newer versions in other repositories but time I’ll stick with the official repositories. Compilation fun! Warning: if you use the following commands check if newer versions of downloaded packages exist and change directories and names accordingly.

  1. We’ll start installing some packages using yum  
    yum groupinstall "Development Tools"
yum install openssl-devel readline-devel libxml2-devel libxslt-devel mysql mysql-devel mysql-libs mysql-server urw-fonts
  2. Let’s compile ImageMagick. 
    cd /opt
wget ftp://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/ImageMagick-6.8.3-7.tar.gz
cd ImageMagick-6.8.3-7
./configure
make
make install
ldconfig /usr/local/lib
  3. Time for Wkhtmltopdf. I’ve downloaded the source files because I had problems with the static versions of Wkhtmltopdf. I’ve used the readme file (README_WKHTMLTOPDF) which comes with wkhtmltopdf as a guide, but notice that the gitorious repository is not found.Warning: this step is going to take a loooong time so if you don’t need pdf reporting skip it.

    Warning: if you see squares in your pdf report instead of text install with yum the urw-fonts package. This information is provided thanks to this stackoverflow question.

    cd /opt
git clone git://github.com/jcsalterego/wkhtmltopdf-qt.git wkhtmltopdf-qt
cd wkhtmltopdf-qt
./configure -nomake tools,examples,demos,docs,translations -opensource -prefix ../wkqt
make -j3
make install

cd /opt
wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.11.0_rc1.tar.bz2
tar jxvf wkhtmltopdf-0.11.0_rc1.tar.bz2
cd wkhtmltopdf-0.11.0_rc1
../wkqt/bin/qmake
make
ldconfig
ln -s /opt/wkhtmltopdf-0.11.0_rc1/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
  4. MySQL!. Start the service and if it’s the first time you install it use the mysql_secure_installation to set root’s password and remove unnecessary tables. 
    service mysqld start
mysql_secure_installation
chkconfig mysqld on
  5. And now we need Ruby and RubyGems. I’m installing ruby 9.3 from ruby’s webpage. 
    cd /opt
wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p392.tar.gz
tar xvfz ruby-1.9.3-p392.tar.gz
cd ruby-1.9.3-p392
./configure
make
make install

cd /opt
wget http://production.cf.rubygems.org/rubygems/rubygems-2.0.2.tgz
tar xvfz rubygems-2.0.2.tgz
cd rubygems-2.0.2
ruby setup.rb
  6. We now have gem installed and we’re going to install bundler gem which is needed by Snorby setup. 
    cd /opt
gem install bundler
  7. OK. I want to use Snorby so I need to download it!. I’ve had problems with the latest git version of snorby so I had to use the zip with the stable version (which is linked from Snorby’s webpage). 
    wget -O snorby.zip https://github.com/Snorby/snorby/zipball/v2.5.6
unzip snorby.zip
cd Snorby-snorby-42dd6d5
  8. Now, pay attention! I’ve found many problems trying to use bundle with ruby 9.3 and I spent several hours finding out what to do. This is what I’ve done, I can guarantee that it’ll work with a newer ruby or snorby version but at least if you find the same problem you won’t suffer. If you find any error please contact me maybe I can help you and update the post so it’s useful for other users. 
    Edit the Gemfile file and change this line: gem 'rake', '0.9.2'   to:  gem 'rake', '> 0.9.2' unless you want this error to show when using bundler: "error: /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.1/lib/bundler/fetcher.rb:112:in `specs': undefined method `each' for nil:NilClass (NoMethodError)"

Edit the Gemfile file and add this line: gem 'orm_adapter' after the line: gem netaddr , unless you want this error when using bundler: 'orm_adapter' file not found

Edit Gemfile.lock and change rake (0.9.2) to rake(0.9.2.2) so rake setup does not complain about a different rake version.

    //Create a snorby_config.yml file. Edit the production section and set your domain
cp config/snorby_config.yml.example config/snorby_config.yml

// Create a database.yml config file. Edit the file and set the root password and MySQL server location
cp config/database.yml.example config/database.yml
  9. OK! Now let’s install Snorby. And don’t forget to read the README.md file provided by Snorby’s developers 
    // We are in the snorby directory //
bundle install
rake snorby:setup

// This is the command's output //
[datamapper] Created database 'snorby'
[datamapper] Finished auto_upgrade! for :default repository 'snorby'
[~] Adding `index_timestamp_cid_sid` index to the event table
[~] Adding `id` to the event table
[~] Building `aggregated_events` database view
[~] Building `events_with_join` database view
* Removing old jobs
* Starting the Snorby worker process.
* Adding jobs to the queue
  10. Snorby is installed woohoo! Before launching it let’s create an iptables rule (TCP 3000 is the default port) 
    iptables -I INPUT -p tcp --dport 3000 -m state --state=NEW,ESTABLISHED,RELATED -j ACCEP
  11. And now finally let’s start Snorby in my CentOS 6.3 server: 
    rails server -e production

Booting WEBrick
=> Rails 3.1.10 application starting in production on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2013-03-08 19:01:18] INFO WEBrick 1.3.1
[2013-03-08 19:01:18] INFO ruby 1.9.3 (2013-02-22) [x86_64-linux]
[2013-03-08 19:01:18] INFO WEBrick::HTTPServer#start: pid=1347 port=3000
  12. If all is fine, open a web browser http://x.x.x.x:3000 and use the default user/password: snorby@snorby.org / snorby. Here, two screenshots so you can check that I haven’t lied to you.snorby_loginsnorby_administrator

Ok. Soon I’ll update this post or create another one explaining how to configure other important things (mail, sensor…)

P.S: You should create a snorby user in your mysql server and change it in the database.yml config file so the root user is not used. e.g: grant all privileges on snorby.* to snorby@localhost identified by ‘password’;

P.S 2 (2013-03-10): PDF reporting problem with wkhtmltopdf has been solved after compiling wkhtmltopdf and QT (be patient you’ll need time). This is a Snorby Report sample.

Tcpdump fun! What kind of traffic is coming from Internet?

Leave a comment

A short post. Just before using Suricata IDS and a honeypot in my virtual lab, I wanted to check that port mirroring was working fine. Also I was curious about what kind of traffic would I receive from Internet so I run tcpdump and this is the result (3 hours listening):

  • 88% of the traffic was ssh login attempts from a China host: user root, oracle, nagios and postgres.
  • 3% of the traffic was bittorrent (lost packages?).
  • 3% of the traffic is against TCP 10021 and TCP 34900 ports.
  • 3% of the traffic is SSDP protocol (trying to configure and detect wireless devices?)
  • The rest of traffic is a mixture of mysql, telnet, ms-sql and http connection attempts.

What will I find when I start the honeypot? What will Suricata detect?

OpenVswitch – Port mirroring

13 Comments

I’m preparing a virtual laboratory for IDS and Honeypot testing that will be running on my OpenNebula private cloud. In the following diagram you can see that I want my IDS (sherlock) to inspect traffic from/to my Vyatta firewall so I’ll do that configuring port mirroring, thanks to openvswitch impressive functionalities. (I’ve used Dia for creating this diagram, what a great tool!).

n40lab_ids_honeypot_ovswitch

My N40L server has only one physical ethernet interface (eth0) and I’ve already created an openvswitch bridge called br0 which is associated to eth0.

In OpenNebula I’ve configured one virtual network called inet which is assigned to br0. Watson (my vyatta router) and Sherlock (my OSSIM machine full of sensors) have one interface attached to inet/br0: vnet2 and vnet0 respectively.

#ovs-vsctl show
...
Bridge "br0"
    Port "vnet2"
        Interface "vnet2"
    Port "eth0"
        Interface "eth0"
    Port "vnet0"
        Interface "vnet0"
    Port "br0"
        Interface "br0"
          type: internal

OK. Now, according to ovs-vsctl man page, if I want the vnet2’s traffic to be mirrored onto vnet0 I’ll run this command:


#ovs-vsctl -- set Bridge br0 mirrors=@m \
 -- --id=@vnet0 get Port vnet0 \
 -- --id=@vnet2 get Port vnet2 \
 -- --id=@m create Mirror name=mymirror select-dst-port=@vnet0 select-src-port=@vnet0 output-port=@vnet2
b74eaf43-92ca-48f5-9fe6-9a561a9937dd // the mirror id

#ovs-vsctl list Bridge br0
_uuid : e3e03141-9754-489b-b4b8-17de2f063b44
controller : []
datapath_id : "0000009c029752bd"
datapath_type : ""
external_ids : {}
fail_mode : []
flood_vlans : []
flow_tables : {}
mirrors : [b74eaf43-92ca-48f5-9fe6-9a561a9937dd] // The mirror id is the one shown after the previous command<strong>
</strong>

I checked that as soon as the mirror was set, my tcpdump started showing traffic sent to my firewall so that’s what I exactly wanted 😀

If you want to stop the mirror you can use:

# ovs-vsctl clear Bridge br0 mirrors

Well, ovs-vsctl man page is full of info so if I find more interesting things I’ll let you know.