Sandfly Security

Sandfly Security Documentation

Welcome to the Sandfly Security documentation hub. Sandfly is an agentless compromise and intrusion detection system for Linux.

Sandfly automates security investigation and forensic evidence collection on Linux. Sandfly constantly searches for intruders 24 hours a day on your network without needing to load any agents on your endpoints.

Get Started

Server Install

Installing the Sandfly Server

Server Installation

The Sandfly server hosts the User Interface (UI), REST API, and database. A server instance must always be installed and running for Sandfly to work.

Please follow these instructions exactly and you will be up and going in no time.

Download Setup Scripts

The Sandfly setup scripts are located on Github. Clone them onto your system. You will need to have git installed on your system with one of the following commands on CentOS or Ubuntu:

sudo yum -y install git

OR

sudo apt install git

Now use git to download the install scripts:


git clone https://github.com/sandflysecurity/sandfly-setup.git

There should be a directory named sandfly-setup. This is where all the operations below will take place.

Install Docker

Ubuntu and CentOS Repositories Are Too Old for Docker

Some Linux distributions (such a CentOS) contain old versions of Docker and are not compatible with Sandfly. Please install using the newest versions of Docker from the scripts provided.

Sandfly uses Docker as everything runs inside a container for security and performance reasons. It is important to use the latest version of Docker.

Use the install scripts below to install the latest versions.

Centos 7 Docker Install


~/sandfly-setup/setup/install_docker_centos7.sh

Ubuntu 17 and Older Docker Install


~/sandfly-setup/setup/install_docker_ubuntu.sh

Ubuntu 18 and Newer Docker Install


~/sandfly-setup/setup/install_docker_ubuntu18.sh

Debian 9 and Newer Docker Install


~/sandfly-setup/setup/install_docker_debian.sh

Start Docker

Make sure the Docker daemon starts automatically or you can start it manually on Linux with the following command:

 
 service docker start
 

Login With Your Docker ID

Docker Login Required

Before you can access the Sandfly Docker repository, you will need a valid Docker account and this account will need to be added to read the repository by Sandfly.

Please setup a Docker account and let us know what the Docker ID is so we can add it so you can perform the install.

Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to hub.docker.com to create one.

To login with your Docker ID, you will need to do the following:

docker login
Username: exampleuser
Password:
Login Succeeded

Run Server Setup Script

This script will start Elasticsearch and a management container and will initialize the DB with the tables and user information for logging in. The script will output the secret data into a special directory that we will also use later to start the server and scanning nodes.

Go to the setup directory.


cd ~/sandfly-setup/setup

Run server install script.


./install.sh

Agree to erase your entire database.

You will see a warning that we are about to delete the entire DB. If this is a new system then simply type in YES DELETE ENTIRE DB exactly as it says. If you have data in an existing Elasticsearch database previously used by Sandfly on this host, then backup the data first.

ALL DATA WILL BE PERMANENTLY DESTROYED

Any data in the DB will be permanently destroyed if it exists. If the DB does not exist then it will be created.

Example:


Are you sure you want to do this (type YES DELETE ENTIRE DB)? YES DELETE ENTIRE DB


Supply the hostname/IP address where the API server will reside.

The API server is the same as the main server you connect to to use the UI. Put in the IP address or hostname of the system that is hosting the DB which is the same as the API.

Example:


Please supply the server API hostname or IP address here: example.sandflysecurity.com


Supply the RabbitMQ hostname where the messaging queue server will be located.

Again, this is the same box as the server and DB. Just use the same entry as the above step. If you need to locate this Rabbit server on another host, then please contact us for assistance.

Example:


Please supply the rabbit server hostname or IP address here: example.sandflysecurity.com


The script will have scrolled by a lot of data on the DB being initialized. If you get a DB error, then please go and make sure that Elastic is running as stated in step 3 above. If you are still getting errors, then make a copy of what you see and contact Sandfly for help.

SSL Setup

This install script will generate self-signed SSL keys for use by the scanning nodes and server. If you want a signed key, go to the step below where we describe how to make a signed key after setup completes.

When generating keys we also generate Diffie Hellman parameters. This can take a while depending on the system and other factors. As long as the screen is moving, then it's generating the keys fine.

Hit the enter key when asked for export passwords. We are not using passwords for these keys.

You will be asked next for items like Country Name, etc. You can just hit enter here, BUT, do not hit enter when you get to Common Name.

At Common Name, enter the HOSTNAME ONLY of the system where the server is located. The Common Name must match what the host thinks it is called or you will get SSL errors.

ENTER HOSTNAME ONLY!

Do not enter the fully qualified domain name (FQDN) here. HOSTNAME ONLY!

If you enter the fully qualified domain name for the Common Name you will get SSL errors later. The Rabbit messaging service used by Sandfly uses an extremely long password, plus a client certificate for authentication over TLS/SSL. Entering a FQDN can cause errors if your nodes are not resolvable and is not needed for security purposes with these certificates.

RabbitMQ will reject the SSL certificate if you use the FQDN here. Just use the hostname and it will be signed by the Sandfly CA and work as expected.

Example: For example.sandflysecurity.com we do the following:

Country Name (2 letter code) []:

State or Province Name (full name) []:

Locality Name (eg, city) []:

Organization Name (eg, company) []:

Organizational Unit Name (eg, section) []:

Common Name (eg, fully qualified host name) []:example <<<< HOSTNAME ONLY. NOT FQDN!**

Email Address []:


Hit enter through the rest of the prompts for passwords. They are not needed.

Optional: Generate a Signed SSL Certificate

At the end of the install script you will be asked to generate signed SSL certificates. This is recommended, but if you are running Sandfly on internal systems it may not be possible to use the built-in script to do this and you can skip this step.

However, if the system has access to the Internet and can be reached on port 80, you can use the built-in script to make signed certs for you. You can generate a signed certificate for the server using EFF's Let's Encrypt service. This will stop you from having SSL errors when connecting to the UI.

Port 80 Must Be Visible From The Internet During Signing!

Make sure the server you are using has a legitimate hostname that is reachable from the Internet and resolves correctly. Port 80 will need to be open for the EFF server to validate the host.**

Yes, the server needs to be reachable by the EFF certificate generation process on the HTTP port (80).

You can block this port again after you receive your certificate from Let’s Encrypt, but it must be open during the generation process.

Make sure, again, that the hostname you put in is legitimate and port 80 can be reached from the Internet. The Let's Encrypt service will not sign any certificate for servers that are not reachable on the Internet.

**********************************
*      Make Signed SSL Key?      *
**********************************

Generate signed SSL keys (type YES)? YES

Answer the questions:


What is your fully qualified hostname for the signed SSL cert? example.sandflysecurity.com


Next you will be asked for a contact e-mail. We recommend you put in a valid e-mail in case there is a security alert about the certificates. You can opt in or out of the EFF mailing list.

Signed keys are put under setup_data as:

server_cert_signed.b64
server_key_signed.b64

When starting the server, the scripts look for the signed versions first before trying to use the unsigned versions if present.

If all is well, when you connect to the UI you will not get any warnings from your browser about invalid certificates.

If you are using an internal server to host Sandfly, then you probably can't use this method. You'll have to find another way to get the server certificate signed. If you are fine using a self-signed certificate and just telling your browser to accept it manually, then skip this step.

If you have a way to generate signed keys with your own internal CA or other method, you will want to place the cert and key file for or the server under setup_data. They will need to be base64 encoded and named the following:

server_cert_signed.b64
server_key_signed.b64

Setup Complete

When the install script runs you will see the following at the end. You are now ready to either generate an optional pre-login screen password, or to install the node.

**********************************
*         Setup Complete!        *
**********************************

Optional: Pre-login Screen

You can enable a pre-login password for the webserver that will ask for a password before presenting the actual Sandfly login screen. This password can help keep nosy webscrapers and people away and not reveal that Sandfly UI is running on the host.

If you want to enable this feature, you can simply add a plain text password to the file located under ~/sandfly-setup/setup/setup_data/login.screen.password.txt

This file is read on server startup and you will receive a generic web server login prompt for this password. This password is different from the admin password and cannot be used to login to the main system. It is only used to keep casual people away that may be browsing around.


echo "example_login_password" > ~/sandfly-setup/setup/setup_data/login.screen.password.txt


Start the server

After the install script runs, we are ready to start the server. This is done by running the start scripts below in the order listed.


cd ~/sandfly-setup/start_scripts


First we start the Elasticsearch server in case it is not running. If it is still running from the setup process, you can skip this step. If you run this script with Elasticsearch already running you will get an error, but it is not fatal so you can just continue.

Normal Error if Elasticsearch is Already Running

If Elastic is still running from the main setup scripts (likely), then when you run the command below you'll get an error like the following:

docker: Error response from daemon: Conflict. The container name "/elasticsearch" is already in use by container "6bcd3761a75ef9c9240034309239f0817b31afbf0bfb6d77cfc6abf22aea954a". You have to remove (or rename) that container to be able to reuse that name.

This is fine and just means Elastic is still running from before. Just continue with the install.


./start_elastic.sh


Next we start the Rabbit messaging server.


./start_rabbit.sh


Then we start the server:


./start_server.sh


When you start the server after install the node secret key will be present until we copy and delete it as detailed under the node install instructions. In this case you will receive this warning when you first run the server:

********* WARNING ***********

The node secret key ../setup/setup_data/node.sec.asc.b64 is present on the server. This key must be deleted from
the server to fully protect the SSH keys stored in the database. It should only be on the nodes.


********* WARNING ***********


Are you sure you want to start the server with the node secret key present? (YES) YES

For now, type YES, and hit enter. After we copy the keys under the Node Install instructions we will delete the node secret key and you should not see this warning any more.

Check if Elasticsearch, RabbitMQ and the server are running:

root@example:~/sandfly-setup/setup# docker ps

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                    NAMES
be5e5caf816b        sandfly/sandfly-server:latest                         "/usr/local/sandfly/…"   10 seconds ago      Up 9 seconds        0.0.0.0:443->8443/tcp    sandfly-server
db7a5567a8f1        sandfly/sandfly-rabbit:latest                         "/bin/sh -c /usr/loc…"   29 seconds ago      Up 28 seconds       0.0.0.0:5673->5673/tcp   sandfly-rabbit
b5ba80831a5d        docker.elastic.co/elasticsearch/elasticsearch:6.3.1   "/usr/local/bin/dock…"   36 seconds ago      Up 34 seconds       9200/tcp, 9300/tcp       elasticsearch

Now copy the random password generated for the admin user so we can test logging in:

cat ~/sandfly-setup/setup/setup_data/admin.password.txt

<long random password here>

Go to the Server with Your Browser

Go to your new webserver address for Sandfly:

https://example.sandflysecurity.com

If you did not sign your certificate with a valid certificate authority, you will receive a warning. You can just ignore this for now. We recommend you use a signed certificate though for operational purposes.

After you enter the optional login screen password, you will be presented with the real login screen.

Put in the username admin and the random password generated by Sandfly above.

You should see the UI appear.

The server is now ready. Let's go load the node and connect it all together.

Server Install


Installing the Sandfly Server

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.