Located under Settings > Integrations > Elasticsearch, the Elasticsearch replication page enables the configuration of result data replication to an external Elasticsearch database for independent, long-term storage and data analysis. The Configuration details card shows the current Elasticsearch replication settings.

If you also want to ingest host asset and SSH key data, and have more control over ingestion, consider our third-party Elastic Connector.

Elasticsearch Replication Form

To change these settings, open the Configuration details card's edit form. The form contains the following fields:

• Elastic replication enabled - Select Enabled to start Elasticsearch replication and make the other fields available.
• URL - (Required when replication is enabled.) The URL of the external Elasticsearch server, for example https://my-elastic-server.example.com:9200/.
• Server CA certificate (optional) - If the external Elasticsearch server uses a certificate from a private CA or is self-signed, provide the trusted certificate in PEM format in this field. Sandfly trusts public certificate authorities by default. The server rejects a value that is not a valid PEM certificate.
• Username (optional) - If using username/password authentication, enter the Elasticsearch username. If using API Key authentication, you must enter the literal value api_key in this field.
• Set password - Turn on to enter a new password. When it is off, Sandfly keeps the saved password.
• Password (optional) - If username/password authentication is used, enter the external Elasticsearch password associated with the specified username. If API Key authentication is used, enter the Base64-encoded API key (the encoded id:api_key value) as the Password. This value is encrypted when saved and is not shown again.

Under Replication options:

• Summary results (optional) - Enable to send the result summary JSON data rather than the full JSON data. Note that the summary JSON data only includes basic information about the alert and does not include most of the details included in the full JSON data. This setting applies to all results that Sandfly sends, both new and changed, except duplicate alerts.
• Duplicate alerts (optional) - Enable to send duplicate alerts when the alert last seen time is updated. Note: this only applies to alerts and not pass results. The last seen time is updated when an existing alert is triggered and updated with new information. Each duplicate alert replaces the existing document for that alert in the index. This increases the amount of data that Sandfly sends, not the number of documents, since the alert may be triggered each time a scan is performed on the target system.
• Summary duplicate alerts (optional) - Enable to send the result summary JSON data for duplicate alerts rather than the full JSON data. Note that the summary JSON data only includes basic information about the alert and does not include most of the details included in the full JSON data. This setting only applies to duplicate Alerts and does not affect new Alerts or new Pass results. Available only when Duplicate alerts is selected. It is selected automatically when Summary results and Duplicate alerts are both selected.

With correct settings and replication enabled, the Sandfly Server will automatically create and maintain an index within Elasticsearch called sandfly_results. No additional configuration in Elasticsearch is required. If the index does not exist, Sandfly creates it with the required field mapping. The configured account must have permission to create and write to the sandfly_results index.

When you enable replication, Sandfly sends only results that are created or changed after that time. Existing results are not sent. Sandfly sends new data every 60 seconds. Elasticsearch 7.6 or later is required; with an older version, replication fails and the error is recorded in the audit log. Replication results and errors are recorded in the audit log.