Azure Blob Storage operations with Storage Python SDK
This blog describes how to perform the basic operations on blobs using the Python API. We’ll be using Python API provided in Azure SDK to achieve the following functionalities.
- Create a container
- Upload a blob into a container
- Download blobs
- List the blobs in a container
- Delete a blob
Installing the SDK:
My machine is a 64-bit windows machine, so I downloaded the "Windows x86-64 executable installer" available underthe latest version of Python for windows from here.
Then I downloaded Azure storage SDK for Python. In order to do that, we need to follow any of the below 3 steps.
Option 1: Via PyPi
To install via the Python Package Index (PyPI), type:
pip install azure-storage Option 2: Source Via Git
To get the source code of the SDK via git just type:
git clone git://github.com/Azure/azure-storage-python.git
cd ./azure-storage-python
python setup.py install Option 3: Source Zip
Download a zip of the code via GitHub or PyPi. Then, type:
cd ./azure-storage-python
python setup.py install
Create a Container:
Every blob in Azure storage must reside in a container. Please make sure that the following naming convention has been followed for a container.
- Container names must start with a letter or number, and can contain only letters, numbers, and the dash (-) character.
- Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names.
- All letters in a container name must be lowercase. Please note that the name of a container must always be lowercase. If you include an upper-case letter in a container name, or otherwise violate the container naming rules, you may receive a 400 error (Bad Request).
- Container names must be from 3 through 63 characters long.
By default, the new container that is created is private. "public_access=PublicAccess.Container" is used in the code below to set the container access policy to "Container". Alternatively, you can modify a container after you have created it using "set_container_acl" method in "blob_service" object.
Code used to create a container:
from azure.storage.blob import BlockBlobService
from azure.storage.blob import PublicAccess
# Setting Parameters
ACCOUNT_NAME = "<Your Storage account name>"
ACCOUNT_KEY = "<Your storage account key>"
CONTAINER_NAME = "<Container name you want to create>"
# Code block to create a container
blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
try:
container_status = blob_service.create_container(CONTAINER_NAME, public_access=PublicAccess.Container)
print("Container %s"%CONTAINER_NAME + " creation success status: %s"%container_status)
except:
print("Container %s"%CONTAINER_NAME + " creation failed")
Screenshot:
Upload a blob into a container:
To create a block blob and upload data, use the create_blob_from_path, create_blob_from_stream, create_blob_from_bytes or create_blob_from_text methods. They are high-level methods that perform the necessary chunking when the size of the data exceeds 64 MB. create_blob_from_path uploads the contents of a file from the specified path, and create_blob_from_stream uploads the contents from an already opened file/stream. create_blob_from_bytes uploads an array of bytes, and create_blob_from_text uploads the specified text value using the specified encoding (defaults to UTF-8).
Code used to upload blob into a container:
from azure.storage.blob import BlockBlobService
from os import listdir
from os.path import isfile, join
# Setting Parameters
ACCOUNT_NAME = "<Your Storage account name>"
ACCOUNT_KEY = "<Your storage account key>"
CONTAINER_NAME = "<Your storage container name>"
LOCAL_DIRECT = r"<Local directory containing files>"
# Code block to upload blob to Blob storage
blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
local_file_list = [f for f in listdir(LOCAL_DIRECT) if isfile(join(LOCAL_DIRECT, f))]
file_num = len(local_file_list)
for i in range(file_num):
local_file = join(LOCAL_DIRECT, local_file_list[i])
blob_name = local_file_list[i]
try:
blob_service.create_blob_from_path(CONTAINER_NAME, blob_name, local_file)
print("File upload successful %s"%blob_name)
except:
print ("Something went wrong while uploading the files %s"%blob_name)
Screenshot:
Download Blobs:
To download data from a blob, use get_blob_to_path, get_blob_to_stream, get_blob_to_bytes, or get_blob_to_text. There are high-level methods that perform the necessary chunking when the size of the data exceeds 64 MB.
Code used to download blob to a local directory:
from azure.storage.blob import BlockBlobService
# Setting Parameters
ACCOUNT_NAME = "<Your Storage account name>"
ACCOUNT_KEY = "<Your storage account key>"
CONTAINER_NAME = "<Your storage container name>"
BLOB_TO_BE_DOWNLOADED = "<Your Blob Name>"
# Code block to download blobs
blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
try:
blob_service.get_blob_to_path(CONTAINER_NAME, BLOB_TO_BE_DOWNLOADED , file_path = "<Local Path>\<File Name>.<extension of the original blob>")
print("Blob has been downloaded")
except:
print("Blob download failed")
Screenshot:
List the blobs in a container:
To list the blobs in a container, use the list_blobs method.
Code used to list blobs in a container:
from azure.storage.blob import BlockBlobService
# Setting Parameters
ACCOUNT_NAME = "<Your Storage account name>"
ACCOUNT_KEY = "<Your storage account key>"
CONTAINER_NAME = "<Your storage container name>"
# Code block to list the blobs present in a container
blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
try:
blob_names = blob_service.list_blobs(CONTAINER_NAME)
for blob in blob_names:
print(blob.name)
except:
print("Blob listing failed")
Screenshot:
Delete a blob:
To delete a blob, we leverage delete_blob method.
Code used to delete a blob:
from azure.storage.blob import BlockBlobService
# Setting Parameters
ACCOUNT_NAME = "<Your Storage account name>"
ACCOUNT_KEY = "<Your storage account key>"
CONTAINER_NAME = "<Your storage container name>"
BLOB_TO_BE_DELETED = "<Blob name with extension>"
# Code block to delete a blob
blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
try:
blob_service.delete_blob(CONTAINER_NAME, BLOB_TO_BE_DELETED)
print("Blob has been deleted successfully")
except:
print("Blob deletion failed")
Screenshot:
Articles referred:
Introduction on azure storage with Python: /en-us/azure/storage/storage-python-how-to-use-blob-storage
Azure Storage SDK for Python: https://github.com/Azure/azure-storage-python