Installing Python packages in Azure App Services is little tricky using pip. In this blog, I would provide best practice to do that.
Pip Install on Azure App Services might fail because
- It may simply be that the package is not available on the Python Package Index.
- It could be that a compiler is missing (Azure App Service is a sandbox environment and does not have all the modules/compilers installed inside it).
Below is one such common scenario
building 'Crypto.Random.OSRNG.winrandom' extension
warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from https://aka.ms/vcpython27
It’s recommend to use wheels for installing Python dependencies. Many of the popular modules already provide wheel files listed in PyPI.
Let’s say if I want to install below list of Python modules
django<2
pycrypto==2.6.1
pandas==0.18.1
numpy==1.11.1
Creating Wheel Files in Local Environment
I’m using Python 3.5.2 x64
for this blog. So i will create wheel files for Python35 x64.
Note : You don’t have to create wheels for all the Python modules. Do this only for modules which have trouble installing in Azure App Services
- I have
python35 x64
installed in my local environment @ C:\Python35 folder - Create a requirements.txt file and add above modules inside it.
- Install wheel module using below command (we would use this module to create wheels later)
> C:\Python35\python.exe -m pip install wheel
- Use Below Command to create wheel files inside wheelhouse folder in Local environment
> C:\Python35\python.exe -m pip wheel -r requirements.txt -w wheelhouse
Above step would have created a wheelhouse folder with wheel files for modules listed in requirements.txt file (In my cases it created wheel files for Python35 arch as you can see in file names)
- Alternatively, You can also Download wheel files available online @ http://www.lfd.uci.edu/~gohlke/pythonlibs/
Upgrade Python version on Web Apps
Default Python/Pip versions on Azure Web Apps are little old (as of 01/17 -This might change in future). Older version of Python/Pip have few known issues during deployment/run-time.
Follow Below steps to update Python version on your web app.
- Navigate to Azure portal
- Click on App Service blade of Web App, select Extensions and then Add.
- From the list of extensions, scroll down until you spot the Python logos, then choose the version you need
For this blog I’m choosing Python 3.5.2 x64
, It would install new version of python @ D:\home\Python35
Install requirements on Azure App Services
- While moving your code to Azure App services make sure to include below line of code as first line in requirements.txt file
--find-links wheelhouse
Install Manually using Kudu
- Navigate to kudu console of your app (https://
Your_app_name
.scm.azurewebsites.net/DebugConsole) - Navigate to the folder where you have requirements.txt file and run bellow command
> D:\home\Python35\python.exe -m pip install --upgrade -r requirements.txt
- After running above command, all the Python modules listed in requirements.txt file would be installed @
D:\home\Python35\Lib\
folder
You might have different path for python.exe depending on python version you are using.
Install using Deployment Script
If you are using Deployment script, Include below line of code in deploy.cmd
:: 2. Install packages
echo Pip install requirements.
D:\home\Python35\python.exe -m pip install --upgrade -r requirements.txt
IF !ERRORLEVEL! NEQ 0 goto error
For more details refer Django app with HttpPlatformHandler in Azure App Services – Windows
Install modules in non-default folder
As I have mentioned earlier, running above command would install modules
@ D:\home\Python35\Lib\
folder.
- Use -t option to install dependencies in non-default folder
D:\home\Python35\python.exe -m pip install --upgrade -r requirements.txt -t D:\home\site\wwwroot\pymodules
Above Command would install Python modules @ D:\home\site\wwwroot\pymodules
folder
- Set PYTHONPATH in App settings at Azure portal for web app
I have seen few wheel files/Python modules having trouble installing with older version of Pip/Python. Try with newer version’s of Python/pip if you have any issues.
Thanks for the post. Saved me few hours 🙂
I have been struggling with –find-links for an entire day, and I will be very grateful if sb could help me out here.
I have been developing using Azure Storage( the most recent version) and it requires cryptograph, which requires cffi, idna, etc…
However, when I try to test it against Azure Webapp, the deployment failes, saying ‘error : unable to find vcvarsall.bat’
With some research, I figured putting –find-links wheelhouse at the top of my requirements.txt and have wheels(cffi-1.8.2-cp26-cp26m-win32.whl (md5) and cryptography-1.5-cp26-cp26m-win32.whl (md5)) located at wheelhouse folder in the root. This was not helping at all, and I was running into same problems.
I tried –no-index and it gives “Could not find any downloads that satisfy the requirement cffi==1.8.2”. Somebody says if I want to use –no-index, then I should have all wheels located in wheelhouse; otherwise, i will get that error.
With this, I would like to use my wheels for cffi and cryptograph and the rest download from pypi. Anyone have any clue…? HELP!
Hi Jang,
Your issue could most probably because you are using older version of pip in web apps. to use newer version refer below blog link
https://blogs.msdn.microsoft.com/azureossds/2016/08/25/deploying-django-app-to-azure-app-services-using-git-and-new-version-of-python/
Thanks for the post – but I ran into an issue. First, the .whl I got wasn’t named “pycrypto-2.6.1-cp27-none-win32.whl” like yours, it was named “pycrypto-2.6.1-cp27-cp27m-win32.whl”. At first it didn’t work, but by renaming it it did the trick, installed it and said “deployment successful”, no errors, all requirements satisfied – including pycrypto.
I can indeed find a pycrypto-2.6.1.dist-info folder inside wwwroot\env\Lib\site-packages. It’s filled with files such as “WHEEL”, “RECORD”, “METADATA”, which is unusual but I’m guessing it’s binary.
The problem is, this is what my python code tells me:
“ImportError: No module named Crypto.PublicKey”
Check if your python app is using virtual env… that could be reason for import error
– pip on azure is very old.. I will update this blog soon on how to use newer version of pip/python..
Meanwhile, below blog has details on how to do that..
https://blogs.msdn.microsoft.com/azureossds/2016/08/25/deploying-django-app-to-azure-app-services-using-git-and-new-version-of-python/
hi When I try to use D:\home\Python35\python.exe -m pip install –upgrade -r requirements.txt this command it says access is denied in azure console. How can I sort this out? Thanks
Can you share your requirements.txt ? it should technically work.
I am getting the same error as the user above. I am just trying to do a GET request in Python Flask – so I need either urllib or requests I think.
requirements.txt is:
–find-links wheelhouse
requests<2.11.0
Flaskpython.exe -m pip install –upgrade -r D:\home\site\wwwroot\requirements.txt
Downloading/unpacking requests<2.11.0 (from -r D:\home\site\wwwroot\requirements.txt (line 2))
Downloading/unpacking Flask=0.21 (from Flask-r D:\home\site\wwwroot\requirements.txt (line 3))
Running setup.py (path:d:\local\temp\pip_build_RD0003FF93497A$\itsdangerous\setup.py) egg_info for package itsdangerous
warning: no previously-included files matching ‘*’ found under directory ‘docs\_build’
Downloading/unpacking Werkzeug>=0.7 (from Flask-r D:\home\site\wwwroot\requirements.txt (line 3))
Downloading/unpacking Jinja2>=2.4 (from Flask-r D:\home\site\wwwroot\requirements.txt (line 3))
Downloading/unpacking click>=2.0 (from Flask-r D:\home\site\wwwroot\requirements.txt (line 3))
Downloading/unpacking MarkupSafe>=0.23 (from Jinja2>=2.4->Flask-r D:\home\site\wwwroot\requirements.txt (line 3))
Downloading MarkupSafe-1.0.tar.gz
Running setup.py (path:d:\local\temp\pip_build_RD0003FF93497A$\MarkupSafe\setup.py) egg_info for package MarkupSafe
Installing collected packages: requests, Flask, itsdangerous, Werkzeug, Jinja2, click, MarkupSafe
Cleaning up…
Exception:
Traceback (most recent call last):
File “D:\Python27\lib\site-packages\pip\basecommand.py”, line 122, in main
status = self.run(options, args)
File “D:\Python27\lib\site-packages\pip\commands\install.py”, line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File “D:\Python27\lib\site-packages\pip\req.py”, line 1435, in install
requirement.install(install_options, global_options, *args, **kwargs)
File “D:\Python27\lib\site-packages\pip\req.py”, line 671, in install
self.move_wheel_files(self.source_dir, root=root)
File “D:\Python27\lib\site-packages\pip\req.py”, line 901, in move_wheel_files
pycompile=self.pycompile,
File “D:\Python27\lib\site-packages\pip\wheel.py”, line 215, in move_wheel_files
clobber(source, lib_dir, True)
File “D:\Python27\lib\site-packages\pip\wheel.py”, line 205, in clobber
os.makedirs(destdir)
File “D:\Python27\lib\os.py”, line 157, in makedirs
mkdir(name, mode)
WindowsError: [Error 5] Access is denied: ‘D:\\Python27\\Lib\\site-packages\\requests’
Storing debug log for failure in D:\home\pip\pip.log
Thank you in advance for any help you could provide!
**Sorry! My message got chopped or something – maybe due to the less than brackets or length. Here it goes again**
I am getting the same error as the user above. I am just trying to do a GET request in Python Flask – so I need either urllib or requests I think.
requirements.txt is:
–find-links wheelhouse
requests[lessthan]2.11.0
Flask[lessthan]1
python.exe -m pip install –upgrade -r D:\home\site\wwwroot\requirements.txt
…
Cleaning up…
Exception:
Traceback (most recent call last):
File “D:\Python27\lib\site-packages\pip\basecommand.py”, line 122, in main
status = self.run(options, args)
File “D:\Python27\lib\site-packages\pip\commands\install.py”, line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File “D:\Python27\lib\site-packages\pip\req.py”, line 1435, in install
requirement.install(install_options, global_options, *args, **kwargs)
File “D:\Python27\lib\site-packages\pip\req.py”, line 671, in install
self.move_wheel_files(self.source_dir, root=root)
File “D:\Python27\lib\site-packages\pip\req.py”, line 901, in move_wheel_files
pycompile=self.pycompile,
File “D:\Python27\lib\site-packages\pip\wheel.py”, line 215, in move_wheel_files
clobber(source, lib_dir, True)
File “D:\Python27\lib\site-packages\pip\wheel.py”, line 205, in clobber
os.makedirs(destdir)
File “D:\Python27\lib\os.py”, line 157, in makedirs
mkdir(name, mode)
WindowsError: [Error 5] Access is denied: ‘D:\\Python27\\Lib\\site-packages\\requests’
Storing debug log for failure in D:\home\pip\pip.log
Thank you in advance for any help you could provide!
UPDATE:
I was able to resolve my issue. I didn’t need to do imports at all. Azure web apps already come with some packages available – for example urllib and urllib2 – at least for Python 2.7. So that others can benefit, you can see all of the available packages (ones that you can do a simple “import XXX” for in your app) by running this on the Azure DebugConsole:
ls D:\Python27\Lib
Note that paths seem to have changed since this tutorial was published, so what was once “D:\home\Python27\Lib” is now “D:\Python27\Lib”. The same command above would work for Python 3.5 (“ls D:\Python35\Lib”).
I hope this helps!
Hi Augustine C,
Sorry that you have been that issue..
By default azure has
D:\Python27 and
D:\Python34
but these are very old.. you have to use site extensions to install newer versions of python ..
From the error message you had.. looks like you are trying to use the default python versions in azure.. you don’t have access to that folder..
WindowsError: [Error 5] Access is denied: ‘D:\\Python27\\Lib\\site-packages\\requests’
Below link has details on how to upgrade python version on app services… check if it helps
https://prmadi.com/running-flask-app-with-httpplatformhandler-in-azure-app-services/
Hi, i ran into some problems, maybe you can help me?
https://social.msdn.microsoft.com/Forums/azure/en-US/313d4b2e-78bb-45e0-9805-063185b48685
Thanks in advance.
Done. I have also updated blog to show details on “Upgrading Python Version using Site Extension”.
I’m struggling to make the Extensions for an app. The code has dependency for Exifread. Followed this article and see all the libs in the environment. It still keeps loading version 2.7 instead of version 3.6.1…
I couldn’t find a place to update PYTHONPATH on the portal. Tried it in web.config and it yielded same result…. the last mile for me to complete the app. Appreciate any help.
Python Path:
[u’D:\\home\\site\\wwwroot\\env\\Lib\\site-packages’,
‘.’,
‘D:\\Windows\\SYSTEM32\\python27.zip’,
‘D:\\Python27\\DLLs’,
‘D:\\Python27\\lib’,
‘D:\\Python27\\lib\\plat-win’,
‘D:\\Python27\\lib\\lib-tk’,
‘D:\\Python27’,
‘D:\\Python27\\lib\\site-packages’,
‘D:\\home\\site\\wwwroot\\env’]
Hi Jay,
I feel that something else I going wrong here. looks like the Virtual env you have created is based on python27.
I would prefer not to use virtual env. Check below links for details
Flask app Using HTTP platform Handler – https://prmadi.com/running-flask-app-with-httpplatformhandler-in-azure-app-services/
Django app Using Http Platform handler -https://prmadi.com/django-app-with-httpplatformhandler-in-azure-app-services-windows/
You can set PYTHONPATH using application settings in azure portal