Tuesday, 1 January 2013

Installing Snorby on Ubuntu 12.04

Prerequisites:

This guide assumes you have setup a clean Ubuntu Server 12.04 image with only SSH installed
Let's get started:

# sudo su -
# apt-get update
# apt-get upgrade

# apt-get install mysql-server
# apt-get install apache2
# apt-get install libyaml-dev git-core default-jre imagemagick libmagickwand-dev wkhtmltopdf gcc g++ build-essential libssl-dev libreadline-gplv2-dev zlib1g-dev linux-headers-generic libsqlite3-dev libxslt1-dev libxml2-dev libmysqlclient-dev libmysql++-dev apache2-prefork-dev libcurl4-openssl-dev -y

Install Ruby & Friends

# apt-get install ruby1.9.3 ruby-text-format
# gem install bundler 
# gem install rails
# gem install rake --version=0.9.2

Install Snorby

# cd /var/www/
# git clone http://github.com/Snorby/snorby.git


# cd /var/www/snorby/config/ 
# cp database.yml.example database.yml
# cp snorby_config.yml.example snorby_config.yml
# sed -i s/"\/usr\/local\/bin\/wkhtmltopdf"/"\/usr\/bin\/wkhtmltopdf"/g /var/www/snorby/config/snorby_config.yml

Configure snorby database username and password

# vi database.yml

Edit the settings to match your MySQL installation

# cd /var/www/snorby/
# bundle install --deployment
# rake snorby:setup

Setup Apache

# gem install passenger
# passenger-install-apache2-module

Copy the lines provided at the end of the installation script

 # vi /etc/apache2/apache2.conf

The lines should look something like this at the end of the file

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.
   LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
   PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.18
   PassengerRuby /usr/bin/ruby1.9.1

# Include generic snippets of statements
Include conf.d/

# Include the virtual host configurations:
Include sites-enabled/



Now add the website as follows:

# vi /etc/apache2/sites-available/snorby

Add the following lines (change the ServerAdmin and Servername to something of your choice)

<VirtualHost *:80>
        ServerAdmin admin@localhost
        ServerName snorby.localhost
        DocumentRoot /var/www/snorby/public

        <Directory "/var/www/snorby/public">
                AllowOverride all
                Order deny,allow
                Allow from all
                Options -MultiViews
        </Directory>
</VirtualHost>

Enable the new website and disable the default site

# a2dissite default
# a2ensite snorby

# service apache2 reload

Browse to the new website and login with the following default credentials:

Username: snorby@snorby.org
Password: snorby

Be sure to change the default credentials after your first login

You should see a page that looks something like this


Be sure to start the Snorby worker in the interface by clicking on Administration -> Worker Options -> Start Worker

Mysql and Snort Sensor Setup

Unfortunately our work is not done. Depending on your setup, you still need to get the snort sensors to log to your Snorby server. In my case I have seperate, remote snort sensors so to get them to log to snorby I have to do the following:

We first need to get MySQL to listen on for remote connections, as this is turned off by default these days

# vi /etc/mysql/my.cnf

Uncomment the bind-address line like so

#bind-address           = 127.0.0.1

Restart MySQL

#service mysql restart

Now we need to create users for the various snort sensors like so. The below assumes your snort sensor is running on 10.0.0.10, so adjust accordingly

# mysql
mysql> CREATE USER 'snort'@'10.0.0.10' IDENTIFIED BY 'snortsnort';
mysql> GRANT ALL PRIVILEGES ON snorby.*TO 'snort'@'10.0.0.10';
mysql> FLUSH PRIVILEGES;

Now on your snort sensor, either in your snort.conf file or barnyard.conf file, whichever you are using, you need to have a line that looks something like this (again, change to match your situation)

output database: log, mysql, user=snort password=snortsnort dbname=snorby host=10.0.0.11

Troubleshooting

After installation my Snorby worker did not want to start through the web interface, so I started it manually like so

# rails c production
Loading production environment (Rails 3.1.0)
irb(main):001:0> Snorby::Worker.stop
=> ""
irb(main):002:0> Snorby::Jobs.clear_cache
=> nil
irb(main):003:0> Snorby::Worker.start
=> ""
irb(main):004:0> exit

Fine Tuning Snorby

Once the events start rolling in, you will probably see a lot of events that are classified as High that you would to change to Medium or Low


I normally do this by getting the signature id from the interface first and then I change the severity in the database directly like so

# mysql snorby
mysql> select * from signature where sig_sid = "100000230";
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
| sig_id | sig_class_id | sig_name                                          | sig_priority | sig_rev | sig_sid   | sig_gid | events_count |
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
|      6 |            1 | GPL CHAT MISC Jabber/Google Talk Outgoing Traffic |            1 |       2 | 100000230 |       1 |         2667 |
+--------+--------------+---------------------------------------------------+--------------+---------+-----------+---------+--------------+
1 row in set (0.00 sec)

mysql> update signature set sig_priority = 3 where sig_sid = "100000230";
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0


Feature Request! It would be nice to be able to do this directly from the interface

25 comments:

  1. i had to remove rake-10.0.3 to get snorby to create the database.

    my error
    -------------
    /var/www/snorby# rake snorby:setup
    rake aborted!
    You have already activated rake 10.0.3, but your Gemfile requires rake 0.9.2. Using bundle exec may solve this.
    my fix
    -------
    gem uninstall rake
    i removed the 10.0.3 version and things worked fine


    ReplyDelete
  2. I had an error after running rake snorby:setup too. I did what you suggested and had to uninstall version 10.04 then it built the DB ok

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. I tried to run it as 'bundle exec rake snorby:setup' and it completed but would not start the Snorby worker process. I followed the "gem uninstall rake" to get rid of the 10.0.4 I had and re-ran "rake snorby:setup" and it still didn't work.

    * Removing old jobs
    * Starting the Snorby worker process.
    /var/lib/gems/1.9.1/gems/bundler-1.3.4/lib/bundler/shared_helpers.rb:2:in `require': no such file to load -- rubygems (LoadError)
    from /var/lib/gems/1.9.1/gems/bundler-1.3.4/lib/bundler/shared_helpers.rb:2
    from /var/lib/gems/1.9.1/gems/bundler-1.3.4/lib/bundler/setup.rb:1:in `require'
    from /var/lib/gems/1.9.1/gems/bundler-1.3.4/lib/bundler/setup.rb:1
    [X] Error: Unable to start the Snorby worker process.

    ReplyDelete
    Replies
    1. You need to rename /usr/bin/ruby1.9.3 to /usr/bin/ruby so that you use the correct version with gems installed.

      Delete
  5. gem not working. How to fix?

    gem install bundler
    ERROR: While executing gem...(Zlib::GzipFile::Error) not in gzip format
    gem sources
    http://rubygems.org/
    gem env
    RUBYGEMS VERSION 1.8.11
    RUBY VERSION 1.9.3 (2011-10-30 patchlevel 0)[x86_64-linux]
    INSTALLATION DIRECTORY: /var/lib/gems/1.9.1
    EXECUTABLE DIRECTORY: /usr/local/bin
    RUBYGEMS PLATFORMS:
    -ruby
    -x86_64-linux
    GEM PATHS:
    -/var/lib/gems/1.9.1
    -/root/.gem/ruby/1.9.1
    GEM CONFIGURATION:
    -:update_courses => true
    -:verbose => true
    -: benchmark => false
    -:backrace => false
    -bulk_threshold => 1000
    REMOTE SOURCES:
    - http://rubygems.org/


    ReplyDelete
  6. Try to ping rubygems.org and see if you can even get to it.

    ReplyDelete
  7. ping rubygems.org returns an IP address, but does not reply to the ping.

    ReplyDelete
  8. root@ubuntu:~# gem install blunder
    ERROR: While executing gem ... (Zlib::GzipFile::Error)
    not in gzip format
    root@ubuntu:~# ping rubygems.org
    PING rubygems.org (54.245.255.174) 56(84) bytes of data.

    ReplyDelete
    Replies
    1. You put gem install BLUNDER instead of BUNDLER.

      Delete
  9. I seem to be able to get to rubygems.org, but still get the ERROR.
    root@ubuntu:~# wget http://production.cf.rubygems.org/rubygems/rubygems-2.0.3.tgz
    --2013-04-09 20:19:34-- http://production.cf.rubygems.org/rubygems/rubygems-2.0.3.tgz
    Resolving production.cf.rubygems.org (production.cf.rubygems.org)... 216.137.37.193, 216.137.37.217, 216.137.37.45, ...
    Connecting to production.cf.rubygems.org (production.cf.rubygems.org)|216.137.37.193|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 334203 (326K) [application/x-tar]
    Saving to: `rubygems-2.0.3.tgz.1'

    100%[======================================>] 334,203 444K/s in 0.7s

    2013-04-09 20:19:36 (444 KB/s) - `rubygems-2.0.3.tgz.1' saved [334203/334203]

    root@ubuntu:~#

    ReplyDelete
  10. I decided to abandon Snorby and Ubuntu and install snort on debian. Unable to get the former two to work.

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. Finally got it all working! Boy what a headache.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  13. I'm having some trouble getting Barnyard2 to start on the pfsense box, from what I can tell everything went fine on snorby the install. i went through the guide without too much trouble, but once I put in the snort sensor info in the barnyard2 tab nothing happens and it wont start. Anyone have any suggestions?

    ReplyDelete
    Replies
    1. Finally got it to function, it was a problem with the mySQL permissions. Snorby is up and running, my only problem now is that the dashboard it not updating. Its receiving the events but it just wont change the dashboard. Any ideas?

      Delete
  14. Did the install on Ubuntu Server 13.04 without a hitch except for the line "rake snorby:setup". I had to run "bundle exec rake snorby:setup" for it to work.

    I also noticed that the Snorby worker process don't start after a reboot so I added the following to /etc/rc.local

    #Make sure Snorby is running
    echo "Starting Snorby Worker...."
    cd /var/www/snorby; /usr/bin/ruby script/delayed_job start


    ReplyDelete
  15. Are these instructions, assuming, that the sensor is installed on another machine?

    ReplyDelete
  16. Hi seem to be stuck at the following error in production.log when trying to load the snorby dashboard:

    [datamapper] Setting up the "production" environment:
    [datamapper] Setting up :default repository: 'snorby' on mysql
    [datamapper] Setting up the "production" environment:
    [datamapper] Setting up :default repository: 'snorby' on mysql


    Started GET "/snorby" for 37.153.219.1 at 2013-10-03 09:57:12 +0200

    ActionController::RoutingError (uninitialized constant SnorbyController):

    I followed each step above on a 12.04.3 installation.

    Any pointers are more then welcome.

    Thanks

    ReplyDelete
  17. Problem: "Too many Redirections"
    Concern: Using Ruby 2.0.0-p247 yet seeing 1.9.1 in in bundler's paths.

    I am inexperienced with ruby programming and am attempting to install Snorby per this tutorial, on Ubuntu 13.10, with Ruby 2.0.0-p247 installed with OpenSSL using RVM.

    When I execute:
    sudo bundle install --no-deployment --verbose,

    I end up with "Too Many redirections" (complete output is provided below).
    In addition, I am wondering whether ruby 1.9.1, th
    Can someone advise me what to do next?

    ========================
    sudo bundle install --no-deployment --verbose

    Updating http://github.com/postmodern/dm-is-read_only.git
    Fetching from: http://rubygems.org/api/v1/dependencies
    HTTP Redirection
    .
    .
    .

    Fetching from: http://rubygems.global.ssl.fastly.net/quick/Marshal.4.8/json-1.7.7-java.gemspec.rz
    HTTP Redirection

    Too many redirects
    Bundler::HTTPError: Too many redirects
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:197:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:222:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:222:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:222:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:222:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:222:in `fetch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/fetcher.rb:92:in `fetch_spec'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/remote_specification.rb:46:in `_remote_specification'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/remote_specification.rb:51:in `method_missing'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/resolver.rb:101:in `block in __dependencies'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/resolver.rb:98:in `each'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/resolver.rb:98:in `__dependencies'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/resolver.rb:68:in .
    .
    .
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/installer.rb:83:in `run'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/installer.rb:14:in `install'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/cli.rb:247:in `install'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/task.rb:27:in `run'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/invocation.rb:120:in `invoke_task'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor.rb:344:in `dispatch'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/vendor/thor/base.rb:434:in `start'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/bin/bundle:20:in `block in '
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/friendly_errors.rb:3:in `with_friendly_errors'
    /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.3.5/bin/bundle:20:in `'
    /usr/local/bin/bundle:19:in `load'
    /usr/local/bin/bundle:19:in `'
    ========================

    ReplyDelete
  18. My initial comment/inquiry had a partial sentence that i want to complete here.
    I am wondering whether the the rubygem path, accessed by the bundler is correct for the ruby version (2.0.0) that I installed using RVM.

    ReplyDelete
  19. If you have problems installing snorby on ubuntu 12.04.3 LTS and it comes up with this error: 'ArgumentError: wrong number of arguments (1 for 0)
    An error occurred while installing delayed_job_data_mapper (1.0.0.rc), and
    Bundler cannot continue.'

    use this gemfile, i edited it and it worked for me:

    http://pastebin.com/22XiMt15

    ReplyDelete
  20. also do 'bundle install' instead of 'bundle install --deployment'


    also #2 if you cant start the snorby worker process and get the error described above, use this command: sudo update-alternatives --config ruby

    then select the latest version of ruby.

    ReplyDelete
  21. When I try to run the setup I get an error with rake-0.9.2. Any ideas? I installed it.

    sudo bundle exec rake snorby:setup
    No time_zone specified in snorby_config.yml; detected time_zone: US/Pacific-New
    60fa58ac588cb78172318d32cda1a42cdf118edaa9ea8855ddacc57f6878a0db9264ae5d4122b52ed13292aa06dc4cb00c5fba366ca579f5f2f9ef18c88cdcec
    ERROR 1007 (HY000) at line 1: Can't create database 'snorby'; database exists
    [datamapper] Finished auto_upgrade! for :default repository 'snorby'
    * Removing old jobs
    * Starting the Snorby worker process.
    /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/spec_set.rb:92:in `materialize': Could not find rake-0.9.2 in any of the sources (Bundler::GemNotFound)
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/spec_set.rb:85:in `map!'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/spec_set.rb:85:in `materialize'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/definition.rb:133:in `specs'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/definition.rb:178:in `specs_for'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/definition.rb:167:in `requested_specs'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/environment.rb:18:in `requested_specs'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/runtime.rb:13:in `setup'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler.rb:120:in `setup'
    from /var/lib/gems/1.9.1/gems/bundler-1.6.2/lib/bundler/setup.rb:17
    [X] Error: Unable to start the Snorby worker process.

    ReplyDelete