How to maintain Azure SQL Indexes and Statistics


[2017-07-16]Note: New version is now available, added option to log operation to table including automatic retention of 3 last operation (can be modified) 

There are a lot of work that Azure SQL saves from you, and most of the users we meet seem to believe that maintain the database indexes and statistics is one the missions you can forget when you migrate to Azure SQL.

Statistics objects, like in the box version of SQL have by default "Auto update" set to ON.

the condition to run the auto update is updating for at least 20% + 500 rows in the table. if the condition is met, the update will occur when the optimizer needs to generate execution plan against that table. so except that the update will take some time and use IO resources (which is very important information if you are on the cloud PAAS database) the update will read only 30% of the rows to generate the updated statistics, therefore, the statistic might not be 100% accurate.

  • You can use QDS to force the best plan for the statistics, but this is your responsibility to keep it up to date.

so, this maintenance still needs to be done by the DBA on a regular basis, and if this has not been done, your database can suffer from poor performance just because of that.

I created a maintenance solution (Yes, I was inspired by Ola Hallengren's solution) just that this one is lighter and suitable with Azure SQL DB (it also compatible with SQL Server 2012 and higher)

Now, all you have to do is download the T-SQL script, execute it on your Azure SQL DB and execute the maintenance procedure.

A quick remark about the options you have:

exec  AzureSQLMaintenance @operation,@mode

@operation: {all, index, statistics} (no default)

statistics: will run only statistics update

index: will run only index maintenance

@mode: {smart, dummy} (Default: smart)

smart: will touch only modified statistics and choose index maintenance by % of fragmentation

dummy: will run through all statistics or indexes.

@logtotable: {0, 1} (Default: 0)

0: this feature is off.

1: will log the operation to table [AzureSQLMaintenanceLog] (if the table does not exist it will be created automatically) the log will update ongoing with any progress so you can monitor the progress with this log. for every operation, you can find detailed information about the object before it was maintained (fragmentation percent for indexes, and modification counter for statistics) by default only 3 last operations will be kept in the log table, older execution log will be automatically removed, this can be changed in the procedure code.

Download procedure code:

AzureSQLMaintenance

 

Scheduling and automation:

one of the easiest way to implement automation for this maintenance is to use Azure Functions that is now on GA

Here is example of how to do that: functions scenario database table cleanup

 

! updated: 2016-10-27 - fixed performance issue while updating statistics.

! updated: 2016-11-14 - fixed issue with the double rebuild of indexes. (thank you JPelttari for the comment)

! updated: 2016-11-29 - adding information about scheduling and automation.

! updated: 2017-07-16 - adding an option to log operation to table with a rotation of 3 last operations.

Comments (18)

  1. JPelttari says:

    If there is multiple fragmented indexes in table, this script rebuilds indexes on that table multiple times as it rebuilds them with ALL identifier

    1. Aman [MSFT] says:

      Thank you JPelttari for the comment. you absolutely right.
      It’s now fixed.

  2. timvdh says:

    Very helpful script, thank you. I somehow thought that index fragmentation is no big issue on Azure SQL, maybe because maintenance functionality is absent. But running this script actually made a big difference for a clustered index of a table where 80% of the records were deleted before.

    1. Aman [MSFT] says:

      Thank you for the feedback 🙂

  3. syfre says:

    Thank you, it is really helpfull, works also on an on premise server (at least SQL 2012)
    I suggest you add an option to check constraint to retrust foreign keys and constraints, it will be usefull after bcp upload of data

    1. Aman [MSFT] says:

      Thank you for the feedback.
      about constraints and foreign keys this is a different subject that is not handled by regular maintenance task as you first need to disable them before you run the bcp load, and this probably will be more selective on the database.
      if you meant to something else – please reply on this comment.

  4. LitalSQLForums says:

    like

  5. Bogdan Mart says:

    Thank you, Yochanan!
    You saved my day. Suddenly SQL performance degraded to expreme. Index maintenance helped alot!

  6. Antonio says:

    Hello Yochanan and thanks for your article!!

    I just tried it with a test database in Azure. It looks that the store procedure execution stops after several minutes. Maybe it is Azure functions or any other security mechanism killing the job.

    Do you know any solution to workaround this? Any other suggestions?

    My other choice would be to use Azure Automation and Runbooks to schedule the index and statistics maintenance. As described here:
    https://geeks.ms/davidjrh/2015/10/08/rebuilding-sql-database-indexes-using-azure-automation/

    Thanks!!

  7. ABlake says:

    Thank you for developing this! Is there a good rule of thumb for how often should this be run on the database?

    1. Hi ABlake,

      it really depend on your database activity pattern and usage.
      I can tell that the most often you can run this maintenance is the better.
      say, you are using local application, so you can run it at night when the application is idle
      for global application that being used 24 hours, weekend might be also a good option.

      Regards,
      Yochanan.

  8. Babji Talluri says:

    Hello Yochanan thanks for your article,

    I have a question,Why are were updating the statistics with FULLSCAN,Is it not ideal to let the SAMPLE decide by Azure database to updatestats,My understanding was when omitted, SQL Server uses sampling to create the statistics, and determines the sample size that is required to create a high quality query plan.

    Thanks in advance,
    Babji Talluri

    1. Babji Talluri says:

      Any thoughts please?

      1. sampling is good but not as good as full scan to get a full picture of the data distribution.
        for instance, when auto create statistics create a new statistics object it will scan sample set in order to quickly get to an execution plan.
        but when we run maintenance task we would like to bring the database to the best figure which we can get by updating the statistics with a full scan.

        1. Babji Talluri says:

          Thanks for the clarification Yochanan,When i run update stats with full scan it consuming lot of eDTU’s and causing the application to slow down every night,any thoughts on how to get around this?

          Thanks in advance,
          Babji

          1. Updating statistics and rebuilding indexes are intensive IO bound operations, therefore we recommend to run the maintenance while the application is idle (or less used by application users). conceptual workaround for this is to scale up the database before the maintenance takes place, and then when finished to scale down to the original service tier. but it might be too many efforts (and money) to get a tiny benefit (note that there is no SLA for scale operation and it can be done instantly and up to 6 hours depend on the data size)
            I hope this information helps.

  9. John Langston says:

    We’ve implemented it across 350+ databases with the actual execution being kicked off by an automation account runbook. Very nice.

    1. I’m very happy to hear that it helped you get better performance.
      thank you so much for this comment.

Skip to main content