Located under Settings > Integrations > Sentinel, the Microsoft Sentinel replication page enables the configuration of data replication to an external Microsoft Sentinel workspace for independent, long-term storage and data analysis.

Configure Sentinel for Result Replication

Prepare the Azure Workspace

To send events from Sandfly to Microsoft Sentinel, there are four main steps required to configure the Azure Monitor.

1. Create Credentials via Azure App Registration
   • Application (client) ID
   • Directory (tenant) ID
   • Configure Access Credentials
     • Application Secret Value, or
     • Certificate and Private Key
2. Configure the Role Assignment for the Application
3. Create a Data Collection Endpoint (DCE)
   • Logs ingestion endpoint URL
4. Create Custom Tables for Sandfly Events
   • Data Collection Rule (DCR)
   • Custom Table Stream Name
     • Results Table
     • Host Assets Table (optional)
     • SSH Keys Table (optional)

After you complete all the steps, you have six to eight values to configure the Sandfly Server connection to Microsoft Sentinel.

1. Application (client) ID
2. Directory (tenant) ID
3. Application Secret Value or Certificate and Private Key
4. Logs Ingestion Endpoint URL
5. Data Collection Rule (DCR) Immutable ID
6. Results Custom Table Stream Name
7. Host Assets Custom Table Stream Name (optional)
8. SSH Keys Custom Table Stream Name (optional)

Create Credentials via Azure App Registration

1. Navigate to the Azure Portal Home as an Administrator
2. Search for and navigate to the portal App registrations section
3. Click New registration
   • Name the application, for example: app-SandflyReplication
   • Select Accounts in this organizational directory only
   • Ignore the Redirect URI
   • Click Register
4. On the Overview blade
   • Save the Application (client) ID
   • Save the Directory (tenant) ID
5. On the Overview blade, click Add a certificate or secret
6. To use Client secrets, click New client secret
   • Enter a Description
   • Select the Expires value (the recommended default is 180 days)
   • NOTE: Save the Value now, as you will not be able to view it later
7. To use Certificates, you will need to generate a certificate and an unencrypted private key
   1. Generate a certificate and private key that have the following configuration
      1. Certificate format: x509
      2. Hash algorithm: SHA-256
      3. Encryption algorithm: RSA 2048
      4. You will need to remove the pass phrase from the private key for use in Sandfly
   2. Click Upload certificate
      1. Select your certificate file (can be of type .cer, .pem, or .crt)
      2. Enter a Description
      3. Click Add

NOTE: Make sure you have saved the following values from this section:

• Application (client) ID
• Directory (tenant) ID
• Application Secret Value or Certificate and Unencrypted Private Key

Configure the Role Assignment for the Application

1. Navigate to the Azure Portal Home
2. Open Subscriptions and select the Subscription Name
3. Click Access Control (IAM)
4. Select Add > Add role assignment
5. Search for and select Monitoring Metrics Publisher
6. Click Next
7. On the Add role assignment blade, verify the role is Monitoring Metrics Publisher
8. Click Select Members
9. In the Search by name or email address box, type the name of the Application (app-SandflyReplication)
10. Click the Application Name
11. Verify the Application is listed under Selected members
12. Click Select
13. Verify the Application is listed under the Members section
14. Click Review + assign and verify the Application is listed under the Members section
15. Click Review + assign again
16. Select the Role assignments tab, click Refresh, and verify the Application is listed under Monitoring Metrics Publisher

Create a Data Collection Endpoint

1. In the Azure Portal, search for and navigate to Data collection endpoints
2. Click Create
   • Enter the Endpoint Name, for example: dce-SandflyReplication
   • Select the Subscription
   • Select or create a new Resource Group, for example: rg-SandflyReplication
   • Select the Region
   • Click Review + create
   • Review the details and click Create
3. Click Refresh to verify the new DCE has been created
4. Select the newly created DCE
5. On the Overview blade
   • Save the Logs Ingestion endpoint URL (e.g., https://dce-sandflyreplication-abcd.eastus2-1.ingest.monitor.azure.com)
   • Endpoint URL Format: https://<Endpoint-Name>-<Identifier>.<Region>.ingest.monitor.azure.com

NOTE: Make sure you have saved the following values from this section:

• Logs Ingestion Endpoint URL

Create a Custom Table for Sandfly Results

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflyResults (Note: do not add the _CL extension; it is added automatically) Use only letters and digits in the table name, 45 characters maximum. Sandfly does not accept other characters in the stream name.
5. Enter the Description (optional)
6. Click Create a new data collection rule
   • Select the Subscription and Resource group
   • Enter the Name, for example: dcr-SandflyReplication
   • Click Done
7. Select the Data collection endpoint created previously
8. Click Next
9. On the Schema and transformation blade
   • Select Upload sample file and browse for the sentinel_template.json file, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
10. Verify the fields and data are uploaded correctly
11. Click Next
12. Click Create
13. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (you may need to refresh the page)
14. Select the Table, click the three dots in the far right column, and select Manage table
15. Click the Data Collection Rule created in Step 6 above
16. On the DCR Overview blade
    • Save the DCR Immutable Id (e.g., dcr-15f5913c29774ff18b2ab926829741bf)
    • Click JSON View and save the streamDeclarations custom table stream name (e.g., Custom-SandflyResults_CL)

NOTE: Make sure you have saved the following values from this section:

• Data Collection Rule Immutable ID
• Custom Table Stream Name (Results)

Create a Custom Table for Sandfly Host Assets (optional)

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflyHostAssets (Note: do not add the _CL extension; it is added automatically) Use only letters and digits in the table name, 45 characters maximum. Sandfly does not accept other characters in the stream name.
5. Enter the Description (optional)
6. Select the Data collection rule created in the above section for the Results.
7. Click Next
8. On the Schema and transformation blade
   • Select Upload sample file and browse for the JSON file sentinel_template_hosts.json, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
9. Verify the fields and data are uploaded correctly
10. Click Next
11. Click Create
12. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (you may need to refresh the page)
13. Select the Table, click the three dots in the far right column, and select Manage table
14. Click the Data Collection Rule
15. On the DCR Overview blade, click JSON View
    • Scroll down until you find the table in the streamDeclarations section, and save the new custom table stream name (e.g., Custom-SandflyHostAssets_CL) (you may need to refresh the page)

NOTE: Make sure you have saved the following values from this section:

• Custom Table Stream Name (Host Assets)

Create a Custom Table for Sandfly SSH Keys (optional)

1. In the Azure Portal, search for and navigate to the Log Analytics workspaces and select your workspace
2. Expand the Settings section and select Tables
3. Click Create > New custom log (DCR-based)
4. Enter the Table Name, for example: SandflySSHKeys (Note: do not add the _CL extension; it is added automatically) Use only letters and digits in the table name, 45 characters maximum. Sandfly does not accept other characters in the stream name.
5. Enter the Description (optional)
6. Select the Data collection rule created in the above section for the Results.
7. Click Next
8. On the Schema and transformation blade
   • Select Upload sample file and browse for the JSON file sentinel_template_sshkey.json, which is included in the sandfly-setup bundle in the following directory: sandfly-setup/integrations/sentinel/
9. Verify the fields and data are uploaded correctly
10. Click Next
11. Click Create
12. In the Filter by name box, type _CL to verify the new Custom Table appears in the Tables list (you may need to refresh the page)
13. Select the Table, click the three dots in the far right column, and select Manage table
14. Click the Data Collection Rule
15. On the DCR Overview blade, click JSON View
    • Scroll down until you find the table in the streamDeclarations section, and save the new custom table stream name (e.g., Custom-SandflySSHKeys_CL) (you may need to refresh the page)

NOTE: Make sure you have saved the following values from this section:

• Custom Table Stream Name (SSH Keys)

Configuring Sandfly

In the Sandfly UI, in the sidebar menu, select Settings > Integrations > Sentinel.

Microsoft Sentinel Replication Form

To change these settings, open the Configuration details card's edit form. The form contains the following fields:

• Microsoft Sentinel replication enabled - Select Enabled to start Microsoft Sentinel replication and make the other fields available.
• Application (client) ID - Enter the value from the Sentinel configuration steps.
• Directory (tenant) ID - Enter the value from the Sentinel configuration steps.
• Logs ingestion endpoint URL - Enter the Logs Ingestion endpoint URL of the data collection endpoint (DCE) from the Sentinel configuration steps. The URL must use https and has the format https://<Endpoint-Name>-<Identifier>.<Region>.ingest.monitor.azure.com.
• Data collection rule (DCR) immutable ID - Enter the value from the Sentinel configuration steps. The ID must start with dcr- followed by 32 hexadecimal characters.
• Custom table stream name - Enter the value from the Sentinel configuration steps for the Results custom table.
• SSH enabled (optional) - Turn on to enable SSH key replication.
• SSH stream name - Required when SSH enabled is on. Enter the stream name of the SSH Keys custom table.
• Host enabled (optional) - Turn on to enable host asset replication.
• Host stream name - Required when Host enabled is on. Enter the stream name of the Host Assets custom table.
• Result data options - Select Send all results to send alert, error, and pass results. Select Send only alerts to send alert and error results only. The default is Send only alerts.
• Change authentication - Turn on to change the authentication method or to enter new credentials. The authentication fields are disabled until you turn on this switch.
• Authentication method - Select the desired authentication method, either Client secret or Certificate credentials.
• When Client secret is selected, enter the Application secret value saved from the Sentinel configuration steps.
• When Certificate credentials is selected, enter the RSA Private key in unencrypted PEM format (PKCS#1 or PKCS#8), and enter the Certificate in PEM format.
• Sandfly encrypts the application secret value and the private key when they are saved and does not show them again.

Under Replication options:

• Summary results (optional) - Enable to send the result summary JSON data rather than the full JSON data. Note that the summary JSON data only includes basic information about the alert and does not include most of the details included in the full JSON data. This setting applies to all results that Sandfly sends, including results that Sandfly sends again when an alert is resolved, but not to duplicate alerts.
• Duplicate alerts (optional) - Enable to send duplicate alerts when the alert last seen time is updated. Note: this only applies to Alerts and not Pass results. The last seen time is updated when an existing alert is triggered and updated with new information. This will result in a larger volume of data since the alert may be triggered each time a scan is performed on the target system.
• Summary duplicate alerts (optional) - Enable to send the result summary JSON data for duplicate alerts rather than the full JSON data. Note that the summary JSON data only includes basic information about the alert and does not include most of the details included in the full JSON data. This setting only applies to duplicate Alerts and does not affect new Alerts or new Pass results. This option is available only when Duplicate alerts is on. When Summary results and Duplicate alerts are both on, Sandfly turns on this option automatically.

Sandfly sends only results created after you enable replication. When an alert is resolved, Sandfly sends the result again with its new status. Sandfly does not send a result larger than 1 MB. If you use certificate credentials, Sandfly writes a warning to the audit log 30 days before the certificate expires.

Using OpenSSL to Generate a Self-Signed Certificate

In order to use a Certificate as credentials for the Microsoft Azure App Registration, you will need to generate a Certificate and an Unencrypted Private Key. The Certificate is uploaded into Microsoft Azure. You paste the contents of the Certificate and the Unencrypted Private Key into the Certificate and Private key fields of the Sandfly Microsoft Sentinel replication form.

This example uses OpenSSL to generate a self-signed certificate with the following configuration:

• Certificate Format: x509
• Hash Algorithm: SHA-256
• Encryption Algorithm: RSA 2048
• Valid for 1 year

Run the following command to generate the self-signed certificate:

openssl req -x509 -sha256 -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt

• Enter the PEM pass phrase and verify the pass phrase
• Enter the information requested, including Country Name, State or Province Name, Locality Name, Organization Name, Organizational Unit Name, Common Name, and Email Address. Some fields can be left blank, or you can use the default value.
• This will generate two files: certificate.crt and privateKey.key
  • The file certificate.crt is uploaded into Microsoft Azure, and its contents are pasted into the Certificate field in Sandfly
  • The file privateKey.key is the private key that is encrypted with the pass phrase entered above

Run the following command to remove the pass phrase from the private key:

openssl rsa -in privateKey.key -out privateKeyNoPass.key

• Enter the pass phrase for the privateKey.key file
• This will generate a file privateKeyNoPass.key with the unencrypted private key, whose contents are pasted into the Private key field in Sandfly