Sandfly Security

Sandfly Security Documentation

Welcome to the Sandfly Security documentation hub. Sandfly is an agentless compromise and intrusion detection system for Linux.

Sandfly automates security investigation and forensic evidence collection on Linux. Sandfly constantly searches for intruders 24 hours a day on your network without needing to load any agents on your endpoints.

Get Started

Node Install - Docker Image

Sandfly Node Installation

Sandfly Node Install

Sandfly nodes allow the system to connect to Linux hosts to do agentless investigations and forensic analysis. You need at least one node running at all times, but it is recommended you start multiple nodes for redundancy and performance.

Referring to the diagram below, the nodes are the workhorse of Sandfly. The nodes connect to the protected hosts over SSH to do investigations and report back the results to the server. Each node has 500 threads and can easily scan many times this number of hosts during operation.

Sandfly High-Level Overview

We recommend you start more than one node container. Each new node container provides 500 more scanning threads so it is very easy to build massive capability with Sandfly to protect many hosts even in large network deployments. The nodes will connect to the server and handle scanning requests on demand with automatic load balancing. You don't need to do anything to the nodes except ensure they have SSH access to the hosts they are required to protect.

The containers can all run on the same Virtual Machine (VM), but we recommend that this VM not be the same one used to host the server for security reasons.

The rest of these instructions will get the host VM ready to run the node containers. The only limit to how many node containers you can run is the CPU and RAM of the VM hosting them.

Standard Security vs. Maximum Security Installation

The section on Standard Security vs. Maximum Security installation goes over the differences in how to deploy Sandfly for your environment. If you are running very small deployment, or testing the product, you may want to use the Standard Security mode. For customers with resources to do so, we highly recommend the Maximum Security installation of running the server and nodes on separate VMs.

I Want to Use the Standard Security Install

If you are happy running the server and scanning node containers on the same VM, you can skip most of the instructions here. You can simply go to the end and start your scanning node on the same system with the server and proceed with logging in and using Sandfly.

I Want to Use the Maximum Security Install

If you want to use the recommended separate VMs for running the server and scanning nodes, you'll need to do the following steps outlined here.

Download Setup Scripts

The Sandfly setup scripts are located on Github. Please visit the URL below to obtain the latest version:

https://github.com/sandflysecurity/sandfly-setup/releases

The version format is X.X.X (e.g. 2.9.0)

wget https://github.com/sandflysecurity/sandfly-setup/releases/download/vX.X.X/sandfly-setup-X.X.X.tgz

tar -xzvf sandfly-setup-X.X.X.tgz

There should be a directory named sandfly-setup after you decompress the image. This is where all the operations below will take place.

Install Docker

Sandfly uses Docker to run its containers. Please install Docker on your host. It is important to use the latest version of Docker. Ubuntu and Centos repositories contain very old versions of Docker and are not compatible with Sandfly. Please use the install scripts below to install the latest versions:

❗️

Ubuntu and CentOS Repositories Are Too Old for Docker

Some Linux distributions (such a CentOS) contain old versions of Docker and are not compatible with Sandfly. Please install using the newest versions of Docker from the scripts provided.

Centos Docker Install

~/sandfly-setup/setup/install_docker_centos7.sh

Ubuntu 18 Docker Install

~/sandfly-setup/setup/install_docker_ubuntu18.sh

Ubuntu 20 Docker Install

Ubuntu 20 has a recent version of Docker and can be setup using the standard package install command below.

apt install docker.io -y

Debian 9 and Newer Docker Install

~/sandfly-setup/setup/install_docker_debian.sh

Make sure the Docker daemon starts automatically or you can start it manually on Linux with the following command:

service docker start

Copy Over Config JSON to Node

We now need to copy over the generated node config JSON file from the server. This file is populated with all cryptographic keys and related setup information for the node to automatically connect to the server and operate.

You will want to open two terminal windows. One will need to be connected to the server, and the other to the node. You could also use scp to copy the file or any other method you want as long as it is secure.

Go to the sandfly-setup/setup_data directory on the server:

Server

cd ~/sandfly-setup/setup/setup_data

Go to the sandfly-setup/setup_data directory on the NODE:

Node

cd ~/sandfly-setup/setup/setup_data

Copy Over Config JSON from the Server to the Node

You can use scp, or cut and paste between screens. But the file config.node.json must be copied under the setup_data directory under the Sandfly setup area above.

cd ~/sandfly-setup/setup/setup_data

cat config.node.json

<copy contents>

Then on the node paste the file under the setup_data directory:

cd ~/sandfly-setup/setup/setup_data

cat > config.node.json

<paste contents>


CTRL-D

The entire config.node.json file must be copied with all keys intact. Most of these values should not be altered unless advised to do so by Sandfly Security.

DELETE THE NODE CONFIG FILE

Sandfly uses high performance elliptic curve cryptography to secure SSH keys in the server database. To ensure these SSH keys are safe in the event of server compromise, the secret keys used to decrypt them are only stored on the scanning nodes.

Because of the above, we don't want the server to have both public and private keys for the nodes. After you copy the node config JSON to your nodes, we want to remove it from the server.

Go into the server setup_data directory and delete the config.node.json file. The server only needs the config.server.json and config.rabbit.json files present.

You can use a secure delete on the node config files if available as shown below:

ON SERVER:

shred -u ~/sandfly-setup/setup/setup_data/config.node.json

Or standard delete:

ON SERVER:

rm ~/sandfly-setup/setup/setup_data/config.node.json

❗️

DELETE THE SECRET KEY

You must delete the node config (config.node.json) from the server to ensure full security of your SSH credentials with Sandfly.

Start One Node Container

Once you deleted the secret key from the server, then you can start the node. Docker images will be pulled over and the node will start if the keys above were copied over correctly.

cd ~/sandfly-setup/start_scripts
./start_node.sh

Start Multiple Node Containers

You can start multiple node containers on the same system to get more performance and redundancy by simply running the start_node.sh script repeatedly (or start_node_unsigned_cert.sh). Make sure your host instance has the RAM to run multiple node containers before doing this.

👍

We Recommend Running Multiple Containers

We recommend you run multiple node containers. You can run multiple containers on a single host instance or on individual hosts. Running multiple containers provides much higher performance and redundancy if a container exits unexpectedly.

Each node container runs 500 scanning threads. So for each node container you add onto the system you expand scanning capacity by 500 threads.

Running 5 nodes for instance gives you 2500 scanning threads. This means that you can scan 2500 hosts concurrently. It also means that if one container should die unexpectedly, you will still have capacity for scanning to continue uninterrupted.

[email protected]:~/sandfly-setup/start_scripts# ./start_node.sh
0106c87dbfd304b3f6fff847702a41f603eb5e625c7b6194ba5fd30019533421

[email protected]:~/sandfly-setup/start_scripts# ./start_node.sh
9ecc25cdaae72589d4792b01989ab73001bcf400da05cfd436a54e9defc38be9

[email protected]:~/sandfly-setup/start_scripts# ./start_node.sh
a8c3b80228c47a7feabf0dfbee89cbd6a2d5abbe80ec7b2a61fc86ed246bfbd7

You can run the command below to see all the nodes containers running.

docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS               NAMES
a8c3b80228c4        sandfly/sandfly-node:latest   "/usr/local/sandfly/…"   3 seconds ago       Up 1 second                             gallant_neumann
9ecc25cdaae7        sandfly/sandfly-node:latest   "/usr/local/sandfly/…"   4 seconds ago       Up 3 seconds                            confident_knuth
0106c87dbfd3        sandfly/sandfly-node:latest   "/usr/local/sandfly/…"   6 seconds ago       Up 4 seconds                            ecstatic_shockley
fb25ff348c30        sandfly/sandfly-node:latest   "/usr/local/sandfly/…"   7 seconds ago       Up 6 seconds                            elegant_lovelace

❗️

Node Container RAM and CPU

Make sure your host instance for the node containers has enough RAM before running many containers and a couple CPUs to make sure there are no performance issues.

A 2GB instance can run 4 containers comfortably. A 4GB instance can run around 10 node containers or perhaps more.

If you want to run many node containers on a single instance you will need to scale up RAM and CPU accordingly.

If you want, you can view the log of the node to make sure it is connected properly. Do this by finding out what the Docker log is called for output after you run the above.

Use the docker name for the container to find what the unique log name is for that container instance:

docker inspect gallant_neumann | grep log
        "LogPath": "/var/lib/docker/containers/a8c3b80228c47a7feabf0dfbee89cbd6a2d5abbe80ec7b2a61fc86ed246bfbd7/a8c3b80228c47a7feabf0dfbee89cbd6a2d5abbe80ec7b2a61fc86ed246bfbd7-json.log",

With the above, you can then view the log and watch information scroll by:

tail -f /var/lib/docker/containers/a8c3b80228c47a7feabf0dfbee89cbd6a2d5abbe80ec7b2a61fc86ed246bfbd7/a8c3b80228c47a7feabf0dfbee89cbd6a2d5abbe80ec7b2a61fc86ed246bfbd7-json.log

{"log":"Adding certificate for server.\n","stream":"stdout","time":"2018-05-31T03:52:58.445303074Z"}
{"log":"Adding key for server.\n","stream":"stdout","time":"2018-05-31T03:52:58.450253392Z"}
{"log":"Changing rabbit and API passwords in conf files to supplied value.\n","stream":"stdout","time":"2018-05-31T03:52:58.467539835Z"}
{"log":"Changing server URL to supplied value.\n","stream":"stdout","time":"2018-05-31T03:52:58.482353609Z"}
{"log":"Starting Celery as lone node\n","stream":"stdout","time":"2018-05-31T03:52:58.484690779Z"}
{"log":"[2018-05-31 03:53:02,275: INFO/MainProcess] Connected to amqp://node:**@example.sandflysecurity.com:5673//\n","stream":"stderr","time":"2018-05-31T03:53:02.279679495Z"}
{"log":"[2018-05-31 03:53:02,380: INFO/MainProcess] mingle: searching for neighbors\n","stream":"stderr","time":"2018-05-31T03:53:02.381229883Z"}
{"log":"[2018-05-31 03:53:03,486: INFO/MainProcess] mingle: sync with 2 nodes\n","stream":"stderr","time":"2018-05-31T03:53:03.487267081Z"}
{"log":"[2018-05-31 03:53:03,489: INFO/MainProcess] mingle: sync complete\n","stream":"stderr","time":"2018-05-31T03:53:03.489423504Z"}
{"log":"[2018-05-31 03:53:03,529: INFO/MainProcess] [email protected] ready.\n","stream":"stderr","time":"2018-05-31T03:53:03.530126446Z"}

Once this is done, you can go back to the server UI and initiate a scan and you will see the node messages scroll by (very fast).

Updated 2 days ago


What's Next

Server Install

Login Screen

Node Install - Docker Image


Sandfly Node Installation

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.