Server Install - Docker Image

Installing the Sandfly Server

Server Installation

The Sandfly server hosts the User Interface (UI), REST API, and optional database. A server instance must always be installed and running for Sandfly to work.

Please follow these instructions exactly and you will be up and going in no time.

Download Setup Scripts

The Sandfly setup scripts are located on Github. Please visit the URL below to obtain the latest version:

https://github.com/sandflysecurity/sandfly-setup/releases

The version format is X.X.X (e.g. 2.9.0)

wget https://github.com/sandflysecurity/sandfly-setup/releases/download/vX.X.X/sandfly-setup-X.X.X.tgz

tar -xzvf sandfly-setup-X.X.X.tgz

There should be a directory named sandfly-setup after you decompress the image. This is where all the operations below will take place.

Install Docker

❗️

CentOS Repositories Are Too Old for Docker

Some Linux distributions (such a CentOS) contain old versions of Docker and are not compatible with Sandfly. Please install using the newest versions of Docker from the scripts provided.

Sandfly uses Docker as everything runs inside a container for security and performance reasons. It is important to use the latest version of Docker.

Use the install scripts below to install the latest versions.

Centos 7 Docker Install

~/sandfly-setup/setup/install_docker_centos7.sh

Ubuntu 18 Docker Install

~/sandfly-setup/setup/install_docker_ubuntu18.sh

Ubuntu 20 Docker Install

Ubuntu 20 has a recent version of Docker and can be setup using the standard package install command below.

apt install docker.io -y

Optionally you can use the install script supplied:

~/sandfly-setup/setup/install_docker_ubuntu20.sh

Debian 9 and Newer Docker Install

~/sandfly-setup/setup/install_docker_debian.sh

Start Docker

Make sure the Docker daemon starts automatically or you can start it manually on Linux with the following command:

service docker start

Run Server Setup Script

This script will start Elasticsearch and a management container and will initialize the DB with the tables and user information for logging in. The script will output the secret data into a special directory that we will also use later to start the server and scanning nodes.

Go to the setup directory.

cd ~/sandfly-setup/setup

Run server install script.

./install.sh

Prepare the Database

We now need to setup what kind of database to use with Sandfly.

The default is to use the internally routed Elasticsearch version. This version is fine for smaller installations. However, if you wish to use an external Elasticsearch cluster for larger deployments you'll need to have a username/password to enter into the URL.

Installing Sandfly server version <version>.

Copyright (c)2016-2021 Sandfly Security Ltd.

Welcome to the Sandfly <version> server setup.


...

Optional Elasticsearch URL (Default: http://elasticsearch:9200):

If you want to use the default, then just hit enter to continue.

If you wish to use an externally defined Elasticsearch cluster, you will need to enter the full URL along with the username/password in the following format:

https://username:[email protected]:9200

In the above the following are required:

username - Username for Elasticsearch (usually "elastic" by default).
password - Password for the user.
servername - FQDN of the server.
port - Port number (default is 9200)

Note that the remote server must have a valid signed key for Sandfly to send data to it. If you are using a self-signed key, you will need to add the certificate file manually. This is detailed further in the instructions.

If you are using a signed certificate from a well-known certificate authority, Sandfly will use its internal list of certificates and likely will recognize it for you so you will not need to do anything else.

Delete the Database

After the database starts, or if the URL supplied for the external database is valid, you will see a warning that we are about to delete the entire DB. If this is a new system then simply type in YES DELETE ENTIRE DB exactly as it says. If you have data in an existing Elasticsearch database previously used by Sandfly on this host, then backup the data first.

❗️

ALL DATA WILL BE PERMANENTLY DESTROYED

Any data in the DB will be permanently destroyed if it exists. If the DB does not exist then it will be created.

Example:

...

Are you sure you want to do this (Type: YES DELETE ENTIRE DB)?

After you agree to delete the entire database, the install script will ask you a few more questions before doing the actual initialization.

Add the Server API Hostname/IP Address

Supply the hostname/IP address where the API server will reside.

The API server is the same as the main server you connect to to use the UI. Put in the IP address or hostname of the system that is hosting the DB which is the same as the API.

If your host is not resolvable in DNS, you can put in the external interface IP address. Otherwise, put in your resolvable DNS fully qualified domain name.

Do not put in localhost (127.0.0.1) as your server address. The system will not work with localhost as your address. It must be a valid external interface such as the IP address or fully qualified domain name the system uses for connectivity.

❗️

Do Not Use Localhost (127.0.0.1) As Your Server Address

Do not put in localhost (127.0.0.1) as your server address. The system will not work with localhost as your address. It must be a valid external interface such as the IP address or fully qualified domain name the system uses for connectivity.

Example:

...

Please supply the server API hostname or IP address here 
(NOT localhost): example.sandflysecurity.com

Or you can put in your externally reachable IP address if DNS is not available for your host:

...

Please supply the server API hostname or IP address here 
(NOT localhost): example.sandflysecurity.com

Add the RabbitMQ Hostname/IP Address

Supply the RabbitMQ hostname where the messaging queue server will be located. Most of the time this will be the same system address as the Server API above. The install will use this as the default if you hit enter.

Again, if your host is not resolvable in DNS, you can put in the external interface IP address. Otherwise, put in your resolvable DNS fully qualified domain name.

Like the server address, do not use localhost (127.0.0.1) as your address or the system will not work. It must be a valid IP address or hostname.

❗️

Do Not Use Localhost (127.0.0.1) As Your Server Address

Do not put in localhost (127.0.0.1) as your server address. The system will not work with localhost as your address. It must be a valid external interface such as the IP address or fully qualified domain name the system uses for connectivity.

Example:

...

Please supply the rabbit server hostname or IP address 
here (Enter: example.sandflysecurity.com): <enter>

Optional: Elasticsearch Results Replication

Sandfly has the ability to replicate results from the server to a centralized Elasticsearch database cluster. This allows you to run multiple Sandfly servers across different segments, but have all the results go to a central location for storage and analysis.

If you want to use this feature, you will need to enter the URL of the Elasticsearch database along with the username and password. Like the previous section on remote Elasticsearch database setup, if you are using a self-signed certificate you will need to add it later. Otherwise if the certificate was signed by a well-known certificate authority, chances are Sandfly will recognize it and not require anything else.

...

Please supply optional Elasticsearch URL for replication 
of results (Enter: None):

If you are not using this feature, just hit enter.

The script will have scrolled by a lot of data on the DB being initialized. If you get a DB error, then please go and make sure that Elastic is running as stated in step 3 above. If you are still getting errors, then make a copy of what you see and contact Sandfly for help.

SSL Setup

This install script will generate self-signed SSL keys for use by the scanning nodes and server. You will have a chance to make a signed key after setup completes.

When generating keys we also generate Diffie Hellman parameters. This can take a while depending on the system and other factors. As long as the screen is moving, then it's generating the keys fine.

You will be asked to enter your hostname only during key generation for internal Sandfly keys. Please enter your hostname only for the server and not the FQDN.

If you are installing to an IP address and not an FQDN, then enter the IP address here.

❗️

ENTER HOSTNAME ONLY!

Do not enter the fully qualified domain name (FQDN) here. HOSTNAME ONLY!

If you are installing to an IP address and not an FQDN, then enter the IP address here.

If you enter the fully qualified domain name for the Common Name you will get SSL errors later.

RabbitMQ will reject the SSL certificate if you use the FQDN here. Just use the hostname or IP address and it will be signed by the Sandfly CA and work as expected.

Example: For example.sandflysecurity.com we do the following:

Enter server hostname ONLY or IP address if not 
resolvable: example

Or if using externally reachable IP address for servers 
as setup above:

Enter server hostname ONLY or IP address if not 
resolvable: 192.168.1.10

Optional: Generate a Signed SSL Certificate

At the end of the install script you will be asked to generate signed SSL certificates. This is recommended, but if you are running Sandfly on internal systems it may not be possible to use the built-in script to do this and you can skip this step.

If the system has access to the Internet and can be reached on port 80, you can use the built-in script to make signed certs for you. You can generate a signed certificate for the server using EFF's Let's Encrypt service. This will stop you from having SSL errors when connecting to the UI with a modern browser.

❗️

Port 80 Must Be Visible From The Internet During Signing!

Make sure the server you are using has a legitimate hostname that is reachable from the Internet and resolves correctly. Port 80 will need to be open for the EFF server to validate the host.**

Yes, the server needs to be reachable by the EFF certificate generation process on the HTTP port (80).

You can block this port again after you receive your certificate from Let’s Encrypt, but it must be open during the generation process.

Make sure, again, that the hostname you put in is legitimate and port 80 can be reached from the Internet. The Let's Encrypt service will not sign any certificate for servers that are not reachable on the Internet.

...

Generate signed SSL keys (type YES)?

Answer the questions:

What is your fully qualified hostname for the signed SSL 
cert? example.sandflysecurity.com

Next you will be asked for a contact e-mail. We recommend you put in a valid e-mail in case there is a security alert about the certificates. You can opt in or out of the EFF mailing list.

When starting the server, the scripts look for the signed versions first before trying to use the unsigned versions if present.

If all is well, when you connect to the UI you will not get any warnings from your browser about invalid certificates.

If you are using an internal server to host Sandfly, then you probably can't use this method. You'll have to find another way to get the server certificate signed. If you are fine using a self-signed certificate and just telling your browser to accept it manually, then skip this step.

If you have a way to generate signed keys with your own CA, you will want to base64 encode the certificate and key and place them in the fields in the config.server.json file located under setup_data:

server.ssl.server.cert_signed
server.ssl.server.private_key_signed

Setup Complete

When the install script runs you will see the following at the end. You are now ready to install the node.

******************************************************************
Setup Complete!

Your setup is complete. Please see below for the path 
to the admin password to login.

You will need to go to /root/sandfly-setup/setup/start_scripts 
and run the following to start the server:

./start_sandfly.sh

Your randomly generated password for the admin account 
is is located under:

/root/sandfly-setup/setup/setup_data/admin.password.txt

******************************************************************

The admin password is a randomly generated diceware string of words. You can change it once you login and the generated password will no longer work:

[email protected]:~/sandfly-setup/setup$ cat ~/sandfly-setup/setup/setup_data/admin.password.txt

serrated-heroics-proofing-hardening-utility-shell-frisk

Start the server

After the install script runs, we are ready to start the server. Go to the start_scripts directory and run the script start_sandfly.sh.

cd ~/sandfly-setup/start_scripts/
./start_sandfly.sh

*** Starting ElasticSearch.
...
*** Starting RabbitMQ server.
...
Waiting for RabbitMQ to configure and start. This will take about 45 seconds.
......
*** Starting Sandfly Server.
...

<server is started>

📘

Normal Error if Elasticsearch is Already Running

If Elastic is still running from the main setup scripts (likely), then when you run the command below you'll get an error like the following:

docker: Error response from daemon: Conflict. The container name "/elasticsearch" is already in use by container "6bcd3761a75ef9c9240034309239f0817b31afbf0bfb6d77cfc6abf22aea954a". You have to remove (or rename) that container to be able to reuse that name.

This is fine and just means Elastic is still running from before. Just continue with the install.

When you start the server after install the node config will be present until we copy and delete it as detailed under the node install instructions. In this case you will receive this warning when you first run the server:

********* WARNING ***********

The node config data (../setup/setup_data/config.node.json) is present on 
the server. This file must be deleted from the server 
to fully protect the SSH keys stored in the database. 
It should only be on the nodes.


********* WARNING ***********


Are you sure you want to start the server with the node config data present? Type YES if you're sure. (NO): YES

For now, type YES, and hit enter. After we copy the config under the Node Install instructions we will delete the node config JSON and you should not see this warning any more.

Standard Security Install Ignore Warning

If you are running the Standard Security mode with the server and node on the same system, you can ignore this warning about the node config file present.

Check Containers are Running

Check if Elasticsearch, RabbitMQ and the server are running:

[email protected]:~/sandfly-setup/setup# docker ps

CONTAINER ID        IMAGE                                                  COMMAND                  CREATED             STATUS              PORTS                                         NAMES
3585a98ed648        sandfly/sandfly-server:2.9.0                           "/usr/local/sandfly/…"   21 seconds ago      Up 15 seconds       0.0.0.0:80->8000/tcp, 0.0.0.0:443->8443/tcp   sandfly-server
97bff4440116        sandfly/sandfly-rabbit:2.9.0                           "/bin/sh -c /usr/loc…"   53 seconds ago      Up 51 seconds       0.0.0.0:5673->5673/tcp                        sandfly-rabbit
00aa764a3b65        docker.elastic.co/elasticsearch/elasticsearch:7.10.2   "/tini -- /usr/local…"   15 minutes ago      Up 15 minutes       9200/tcp, 9300/tcp

Now copy the diceware password generated for the admin user so we can test logging in:

cat ~/sandfly-setup/setup/setup_data/admin.password.txt

<diceware password here>

Go to the Server with Your Browser

Go to your new webserver address for Sandfly:

https://example.sandflysecurity.com

If you did not sign your certificate with a valid certificate authority, you will receive a warning. You can just ignore this for now. We recommend you use a signed certificate though for operational purposes.

Put in the username admin and the diceware password generated by Sandfly above.

You should see the UI appear.

The server is now ready. Let's go load the node and connect it all together.


Did this page help you?