Sandfly supports the replication of results data to an external Sentinel database for independent, long-term storage and analysis.

ℹ️

INFO: Upgrade Feature - Sentinel Replication

The ability to configure and use Sentinel Replication requires an upgraded plan. Please see https://www.sandflysecurity.com/get-sandfly/ for details.

Configure Sentinel for Result Replication

Prepare the Azure Workspace

To send events from Sandfly to Microsoft Sentinel, there are four main steps required to configure the Azure Monitor.

1. Create Credentials via Azure App Registration
   • Application (client) ID
   • Directory (tenant) ID
   • Configure Access Credentials
     • Application Secret Value, or
     • Certificate and Private Key
2. Configure the Role Assignment for the Application
3. Create a Data Collection Endpoint (DCE)
   • Logs ingestion endpoint URL
4. Create Custom Tables for Sandfly Events
   • Data Collection Rule (DCR)
   • Custom Table Stream Name
     • Results Table
     • Host Assets Table (optional)
     • SSH Keys Table (optional)

Once all the steps are complete, there will be 6-8 artifacts required to configure the Sandfly Server connection to Azure Sentinel.

1. Application (client) ID
2. Directory (tenant) ID
3. Application Secret Value or Certificate and Private Key
4. Logs Ingestion Endpoint URL
5. Data Collection Rule (DCR) Immutable ID
6. Results Custom Table Stream Name
7. Host Assets Custom Table Stream Name (optional)
8. SSH Keys Custom Table Stream Name (optional)

Create Credentials via Azure App Registration

1. Navigate to the Azure Portal Home as an Administrator
2. Search for and navigate to the portal App registrations section
3. Click on New registration
   • Name the application, for example: app-SandflyReplication
   • Select Accounts in this organizational directory only
   • Ignore the Redirect URI
   • Click on Register
4. On the Overview blade
   • Save the Application (client) ID
   • Save the Directory (tenant) ID
5. On the Overview blade, click on Add a certificate or secret
6. To use Client secrets, click on New client secret
   • Enter a Description
   • Select the Expires value (Recommended default is 180 days)
   • NOTE: Save the Value now as you will not be able to view it later
7. To use Certificates, you will need to generate a certificate and an unencrypted private key
   1. Generate a certificate and private key that has the following configuration
      1. Certificate format: x509
      2. Hash algorithm: SHA-256
      3. Encryption algorithm: RSA 2048
      4. You will need to remove the pass phrase from the private key for use in Sandfly
   2. Click on Upload certificate
      1. Select your certificate file (can be of type .cer, .pem, .crt)
      2. Enter a Description
      3. Click Add

NOTE: Make sure you have saved the following values from this section:

• Application (client) ID
• Directory (tenant) ID
• Application Secret Value or Certificate and Unencrypted Private Key

Configure the Role Assignment for the Application

1. Navigate to the Azure Portal Home
2. Open Subscriptions and select the Subscription Name
3. Click on Access Control (IAM)
4. Select Add > Add role assignment
5. Search for and select Monitoring Metrics Publisher
6. Click Next
7. On the Add role assignment blade, verify the role is Monitoring Metrics Publisher
8. Click Select Members
9. In the Search by name or email address type the name of the Application (app-SandflyReplication)
10. Click on the Application Name
11. Verify the Application is listed under Selected members
12. Click Select
13. Verify the Application is listed under the Members section
14. Click Review + assign and verify the Application is listed under the Members section
15. Click Review + assign again
16. Select the Role assignments tab, click Refresh and verify the Application is listed under Monitoring Metrics Publisher

Create a Data Collection Endpoint

1. In the Azure Portal, search for and navigate to Data collection endpoints
2. Click Create
   • Enter the Endpoint Name, for example: dce-SandflyReplication
   • Select the Subscription
   • Select or Create new Resource Group, for example: rg-SandflyReplication
   • Select the Region
   • Click Review + create
   • Review the details and click Create
3. Click Refresh to verify the new DCE has been created
4. Select the newly created DCE
5. On the Overview blade
   • Save the Logs Ingestion endpoint URL (e.g., https://dce-sandflyreplication-abcd.eastus2-1.ingest.monitor.azure.com)
   • Endpoint URL Format: https://<Endpoint-Name>-<Identifier>.<Region>.ingest.monitor.azure.com

NOTE: make sure you have saved the following values from this section:

• Logs Ingestion Endpoint URL

Create a Custom Table for Sandfly Results

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click on Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflyResults (Note: do not add the _CL extension, it is added automatically)
5. Enter the (Optional) Description
6. Click Create a new data collection rule
   • Select the Subscription and Resource group
   • Enter the Name, for example: dcr-SandflyReplication
   • Click Done
7. Select the Data collection endpoint created previously
8. Click Next
9. On the Schema and transformation blade
   • Select Upload sample file and browse for the sentinel_template.json file, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
10. Verify the fields and data are uploaded correctly
11. Click Next
12. Click Create
13. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (may need to refresh the page)
14. Select the Table and click on the three dots in the far right column and select Manage table
15. Click on the Data Collection Rule created in Step 6 above
16. On the DCR Overview blade
    • Save the DCR Immutable Id (e.g., dcr-15f5913c29774ff18b2ab926829741bf)
    • Click on JSON View and save the streamDeclarations custom table stream name (e.g., Custom-SandflyResults_CL)

NOTE: make sure you have saved the following values from this section:

• Data Collection Rule Immutable ID
• Custom Table Stream Name (Results)

Create a Custom Table for Sandfly Host Assets (optional)

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click on Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflyHostAssets (Note: do not add the _CL extension, it is added automatically)
5. Enter the (Optional) Description
6. Select the Data collection rule created in the above section for the Results.
7. Click Next
8. On the Schema and transformation blade
   • Select Upload sample file and browse for the JSON file sandfly_template_hosts.json file, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
9. Verify the fields and data are uploaded correctly
10. Click Next
11. Click Create
12. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (may need to refresh the page)
13. Select the Table and click on the three dots in the far right column and select Manage table
14. Click on the Data Collection Rule
15. On the DCR Overview blade, click on JSON View
    • Scroll down until you find the table in the streamDeclarations section, and save the new custom table stream name (e.g., Custom-SandflyHostAssets_CL) (may need to refresh the page)

NOTE: make sure you have saved the following values from this section:

• Custom Table Stream Name (Host Assets)

Create a Custom Table for Sandfly SSH Keys (optional)

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click on Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflySSHKeys (Note: do not add the _CL extension, it is added automatically)
5. Enter the (Optional) Description
6. Select the Data collection rule created in the above section for the Results.
7. Click Next
8. On the Schema and transformation blade
   • Select Upload sample file and browse for the JSON file _sandfly_template_sshkey.json file, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
9. Verify the fields and data are uploaded correctly
10. Click Next
11. Click Create
12. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (may need to refresh the page)
13. Select the Table and click on the three dots in the far right column and select Manage table
14. Click on the Data Collection Rule
15. On the DCR Overview blade, click on JSON View
    • Scroll down until you find the table in the streamDeclarations section, and save the new custom table stream name (e.g., Custom-SandflySSHKeys_CL) (may need to refresh the page)

NOTE: make sure you have saved the following values from this section:

• Custom Table Stream Name (SSH Keys)

Sentinel Replication - Sandfly Server Settings

The Sentinel Replication tab, available via the Settings > Server Configuration menu, contains the following settings that can be edited:

• Microsoft Sentinel Replication Enabled - Set it to True to enable access to the Sentinel Replication settings and activate the replication service.
• Application (Client) ID- Enter the value from the Sentinel configuration steps.
• Directory (Tenant) ID - Enter the value from the Sentinel configuration steps.
• Logs Ingestion Endpoint URL - The replication URL of the external Sentinel server with the general format of https://<Endpoint-Name>-<Identifier>.<Region>.ingest.monitor.azure.com similar to the reference image.
• Data Collection Rule (DCR) Immutable ID - Enter the value from the Sentinel configuration steps.
• Custom Table Stream Name - Enter the value from the Sentinel configuration steps for the Results custom table.
• SSH Enabled (optional) - Toggle to enable SSH Key replication.
• SSH Stream Name (optional) - When SSH Enabled is enabled, enter the value from the Sentinel configuration steps for the SSH Keys custom table.
• Host Enabled (optional) - Toggle to enable Host Assets replication.
• Host Stream Name (optional) - When Host Enabled is enabled, enter the value from the Sentinel configuration steps for the Host Assets custom table.
• Authentication - Toggle Change Authentication to change the authentication method and/or update the authentication tokens.
• Authentication Method - Select the desired authentication method, either Client Secret or Certificate Credentials
• When Client Secret is selected, enter the Application Secret Value saved from the Sentinel configuration steps.
• When Certificate Credentials is selected, enter the Private Key in unencrypted PEM format, and enter the Certificate in PEM format.

Using OpenSSL to Generate a Self-Signed Certificate

In order to use a Certificate as credentials for the Microsoft Azure App Registration, you will need to generate a Certificate and an Unencrypted Private Key. The Certificate is uploaded into Microsoft Azure and both the Certificate and Unencrypted Private Key are uploaded into the Sandfly Server Sentinel Replication configuration.

This example uses OpenSSL to generate a self-signed certificate with the following configuration

• Certificate Format: x509
• Hash Algorithm: SHA-256
• Encryption Algorithm: RSA 2048
• Valid for 1 year

Run the following command to generate the self-signed certificate:

openssl req -x509 -sha256 -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt

• Enter the PEM pass phrase and verify the pass phrase
• Enter the information requested, including Country Name, State or Province Name, Locality Name, Organization Name, Organizational Unit Name, Common Name, and Email Address. Some fields can be left blank or you can use the default value.
• This will generate two files certificate.crt and privateKey.key
  • The file certificate.crt is uploaded into Microsoft Azure and is also used in the Sandfly Server Sentinel Replication configuration
  • The file privateKey.key is the private key that is encrypted with the pass phrase entered above

Run the following command to remove the pass phrase from the private key:

openssl rsa -in privateKey.key -out privateKeyNoPass.key

• Enter the pass phrase for the privateKey.key file
• This will generate a file privateKeyNoPass.key with the unencrypted private key that is uploaded into the Sandfly Server Sentinel Replication configuration