Both PaaS and IaaS services hosted in Azure generate a large amount of data in security logs. These logs contain vital information that can provide intelligence and powerful insights into policy violations, internal and external threats, regulatory compliance issues, and anomalies in network, host, and user activity.

This ability to get raw logs from your Azure resources into your Security Information and Event Management (SIEM) systems provides a unified dashboard for all your assets, on-premises or in the cloud, so that you can aggregate, correlate, analyze and alert for security events associated with your applications. Azure Log Integration enables you to integrate these logs from assets deployed in Azure to on-premises Security Information Event Management (SIEM) systems . 

High level architecture: 

What logs can I integrate?

Azure produces extensive logging for every service. These logs are categorized by two main types:

• Control/Management logs – Control/Management logs give visibility into the create, update and delete operations that goes through Azure Resource Manager. Azure Audit logs contain these logs

Data Plane logs – Data plane logs give visibility into the events raised using the usage of the Azure resource. An example of this are the Windows event system, security and application logs in a Virtual machine

Get Started with Azure Log Integration

Download the package from the Microsoft Download Center  and install Azure Log integration

Note: The Azure Log integration service collects telemetry data from the machine on which it is installed. Please uncheck the option if you would not like to allow Microsoft to collect the telemetry data.

Telemetry data collected –

• Exception information that happens during execution of Azure log integration
• Metrics about # of queries made and # of events processed
• Usage statistics about which Azlog.exe command line option is being used

Integrate Azure VM Logs from your WAD (Windows Azure Diagnostics) Storage accounts

1. Ensure that your WAD storage account is collecting the logs before continuing on the Azure log integration
2. Open command prompt, and cd into c:Program FilesMicrosoft Azure Log Integration
3. Run the command:  azlog source add WAD
   – This is the Azure storage account configured to receive Diagnostics events from your Virtual Machine
    Example: azlog source add azlogtest WAD azlog9414 fxxxFxxxxxxxxywoEJK2xxxxxxxxxixxxJ+xVJx6m/X5SQDYc4Wpjpli9S9Mm+vXS2RVYtp1mes0t9H5cuqXEw==
   Optionally, you can append the subscription ID to the friendly name if you would like the subscription id to show up in the event XML. 
   azlog source add . WAD
4. To view the events that are pulled from the storage account, Open Event Viewer –>Windows Event log–> Forwarded Events on the Azlog Integrator
5. Make sure your standard SIEM connector (e.g. Splunk Universal Forwarder or ArcSight Windows Event Smart Collector or QRadar WinCollect) installed on the machine is configured to pick events from forwarded events folder and pipe them to SIEM instance. Review the SIEM specific information to ensure that you are integrating the Azure VM logs.

Integrate Azure Audit logs and Azure Security Center Alerts

1. Open command prompt, and cd into c:Program FilesMicrosoft Azure Log Integration
2. Run the command:  azlog createazureid  This command will prompt for your Azure Login and creates an Azure Active Directory Service Principal in the Azure AD Tenants that host the Azure subscriptions in which the logged in user is a Co-Administrator or owner. The command will fail if the logged in user is only a Guest user in the Azure AD Tenant.Authentication to azure is done through Azure AD.  Creating a service principal for Azlog Integration will create the Azure AD identity that will be given access to read from Azure subscriptions.
3. Run the command:  azlog authorize
   The azlog authorize command assigns reader access on the subscription to the service principal created in step # If you don’t specify a SubscriptionID , then the service principal will be assigned the reader role to all subscriptions to which you have any access.(Note: You may see some warnings if you run the authorize command immediately after createazureid. The reason for this is that there is some latency between the Azure Active Directory account creation and the account being available for use. If you wait about 10 seconds after running createazureid and then run authorize, then you should not see these warnings)
4. Check the following folders to confirm Audit log JSON files exist in them:
   C:UsersazlogAzureResourceManagerJson
   C:UsersazlogAzureResourceManagerJsonLD
   The tool generates both pretty printed and line delimited JSON.
5. Check the following folders to confirm that Azure Security Center alerts exist in them:
   C:Usersazlog AzureSecurityCenterJson
   C:UsersazlogAzureSecurityCenterJsonLD
6. Point the standard SIEM file forwarder connector to the appropriate folder to pipe the data to SIEM instance. You may need some field mappings based on SIEM product you are using.
   To learn more about Azure Audit logs and property definitions, please see:
   https://msdn.microsoft.com/library/azure/dn931934.aspx
   https://azure.microsoft.com/en-us/documentation/articles/resource-group-audit/
   To learn about Azure security center alerts, please visit
   https://azure.microsoft.com/en-us/documentation/articles/security-center-managing-and-responding-alerts/