Azure SQL Database enables you to directly load files stored on Azure Blob Storage using the BULK INSERT T-SQL command and OPENROWSET function…. Read more
SQL Server 2016 and Azure SQL Database enable you to parse JSON text and transform it into tabular format. In this post, you might see that JSON functions can handle very large JSON text – up to 4GB…. Read more
IIS generates logs where are recorded many information about HTTP requests such as what Url was called, when the request happened, what is the origin, etc. If you want to analyze information from log files you can use use text search, regular expressions, or some log analysis tools; however, this might be tedious job. SQL Server… Read more
Line-delimited JSON is one common format used to exchange data between systems and for streaming JSON data. SQL Server can be used to export content of tables into line-delimited JSON format…. Read more
Azure File Storage supports SMB protocol, so you can map your local virtual drive to the Azure File storage share using the following procedure: Create file storage account (e.g. mystorage), file share (e.g. logs), and folder using Azure portal or Azure PowerShell SDK. Create firewall outbound rule in Windows Firewall on your computer that allows… Read more
Loading Line-Delimited JSON files in SQL Server One of the problem with JSON is the fact that you cannot continuously append JSON messages. If you want to have valid array of JSON objects you will need to surround them with brackets. Once you add final bracket, you cannot add new data. Line delimited JSON (aka. JSON lines or Newline-delimited… Read more
In the previous blog https://blogs.msdn.microsoft.com/sqlserverstorageengine/2016/01/10/clustered-columnstore-index-data-load-optimizations-minimal-logging/ , we looked at minimal-logging when bulk importing data into a table with clustered columnstore index. In this blog, we will look at parallel bulk import. Recall that on rowstore tables (i.e. the tables organized as rows not as columnstore), SQL Server requires you to specify TABLOCK for parallel bulk… Read more
When bulk importing large number of rows, you want to make sure it can be done efficiently with minimal resource usage so that that you can finish the data load quickly and start running your analytics. SQL Server provides following three key optimizations during bulk import Minimal logging Reduced overhead of locks Concurrent Inserts Just… Read more
Introduction Data load is one of the most important processes in databases. However, relational databases are not optimal for loading complex relational data. JSON can be used to improve performance and reduce complexity in data load process if you serialize some entities as JSON collections. In this post we will see how you can use… Read more
In my earlier blog http://blogs.msdn.com/b/sqlserverstorageengine/archive/2008/10/24/new-update-on-minimal-logging-for-sql-server-2008.aspx, we discussued how we can get minimal logging using TF-610 when using TSQL INSERT statement. Interestingly, you can get minimal logging when using MERGE statement as well. Here is a simple example to illustrate it. use bulktestgo alter database bulktest set recovery SIMPLE — enable the trace flag 610dbcc traceon… Read more