Storage Analytics Logging - How to Enable and Where to Find the logs.
Logging Overview
Storage Analytics can log detailed information for authenticated requests and successful anonymous requests to a storage service.
Logging is enabled and disabled at the service level, not the storage account level. For example, to enable logging for the Blob, Table, and Queue services, you must make a separate request to each service: one to the Blob service, one to the Table service, and one to the Queue service.
How To: Configure Logging
The following article talks about enabling the storage analytics.
https://www.windowsazure.com/en-us/manage/services/storage/how-to-monitor-a-storage-account/
For each of the storage services available with your storage account (Blob, Table, and Queue), you can save diagnostics logs for Read Requests, Write Requests, and/or Delete Requests, and can set the data retention policy for each of the services.
- In the Management Portal, click Storage, and then click the name of the storage account to open the dashboard.
- Click Configure, and use the Down arrow on the keyboard to scroll down to logging (shown below).
- For each service (Blob, Table, and Queue), configure the following:
The types of request to log: Read Requests, Write Requests, and Delete Requests
The number of days to retain the logged data. Enter zero is if you do not want to set a retention policy. If you do not set a retention policy, it is up to you to delete the logs. - Click Save.
How Logs are Stored
All logs are stored in block blobs in a container named $logs, which is automatically created when Storage Analytics is enabled for a storage account.
The $logs container is located in the blob namespace of the storage account, for example: https://<accountname>.blob.core.windows.net/$logs.
This container cannot be deleted once Storage Analytics has been enabled, though its contents can be deleted.
All data in this container can be accessed by using the Blob service APIs.
A storage account administrator can read and delete logs, but cannot create or update them.
Additionally, the $logs container cannot be deleted, but its contents can be deleted.
As requests are logged, Storage Analytics will upload intermediate results as blocks. Periodically, Storage Analytics will commit these blocks and make them available as a blob.
For information about accessing the $logs container, see About Storage Analytics Logging.
Here is a snapshot of the $logs folder using Azure Storage Explorer(note: you need to use the version5 Preview1 and above to look at the $logs folder. You can download it from https://azurestorageexplorer.codeplex.com/releases/view/89713)
Storage Analytics Log Format
Each log entry conforms to a standard log format that is determined by the version of Storage Analytics Logging.
All fields, populated or empty, will be separated by a semicolon “;”, so it’s easier to look at this log file using Excel and delimiting by “;”
Open the log file using from Excel (Excelà File à Open).
This should open the “Text Import Wizard”. Select “Delimited” option on step 1,
Select “SemiColon” from the list of Delimiters on Step2 and click Finish.
Attached is an excel sheet(Analytics.xlsx) with the column headers. Copy this row and and paste it as the first row in the excel worksheet above.
(Right click on First Row --> Paste Special: Match destination formatting: Formats the pasted data to match the formatting already existing in the new location). See https://support.microsoft.com/kb/291358 for details on this.
Now you can set a filter on the columns as shown above.
For performance related issues, one quick thing to look for in the logs is the difference between the 6th and 7th columns to see if the delay is on the server side or on the client/network.
6.E2ELatency (duration): The total time in milliseconds taken to complete a request in the Windows Azure Storage Service. This value includes the required processing time within Windows Azure Storage to read the request, send the response, and receive acknowledgement of the response.
7.Server Latency (duration): The total processing time in milliseconds taken by the Windows Azure Storage Service to process a request. This value does not include the network latency specified in E2E Latency.