I am trying to run a python script with Azure functions.
I had success updating the python version and installing modules on Azure functions under the App Services plan but I need to use it under the Consumption plan as my script will only execute once everyday and for only a few minutes, so I want to pay only for the time of execution. See: https://azure.microsoft.com/en-au/services/functions/
Now I'm still new to this but from my understanding the consumption plan spins up the vm and terminates it after your script has been executed unlike the App Service plan which is always on.
I am not sure why this would mean that I can't have install anything on it. I thought that would just mean I have to install it every time I spin it up.
I have tried installing modules through the python script itself and the kudu command line with no success.
While under the app service plan it was simple, following this tutorial: https://prmadi.com/running-python-code-on-azure-functions-app/
On Functions Comsumption plan, Kudu extensions are not available. However, you can update pip to be able to install all your dependencies correctly:
Create your Python script on Functions (let's say NameOfMyFunction/run.py)
Open a Kudu console
Go to the folder of your script (should be d:/home/site/wwwroot/NameOfMyFunction)
Create a virtualenv in this folder (python -m virtualenv myvenv)
Load this venv (cd myenv/Scripts and call activate.bat)
Your shell should be now prefixed by (myvenv)
Update pip (python -m pip install -U pip)
Install what you need (python -m pip install flask)
Now in the Azure Portal, in your script, update the sys.path to add this venv:
import sys, os.path
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), 'myvenv/Lib/site-packages')))
You should be able to start what you want now.
(Reference: https://github.com/Azure/azure-sdk-for-python/issues/1044)
Edit: reading previous comment, it seems you need numpy. I just tested right now and I was able to install 1.12.1 with no issues.
You may upload the modules for the Python version of your choice in Consumption Plan. Kindly refer to the instructions at this link: https://github.com/Azure/azure-webjobs-sdk-script/wiki/Using-a-custom-version-of-Python
This is what worked for me:
Dislaimer: I use C# Function that includes Python script execution, using command line with System.Diagnostics.Process class.
Add relevant Python extension for the Azure Function from Azure Portal:
Platform Features -> Development Tools -> Extensions
It installed python to D:\home\python364x86 (as seen from Kudu console)
Add an application setting called WEBSITE_USE_PLACEHOLDER and set its value to 0. This is necessary to work around an Azure Functions issue that causes the Python extension to stop working after the function app is unloaded.
See: Using Python 3 in Azure Functions question.
Install the packages from Kudu CMD line console using pip install ...
(in my case it was pip install pandas)
Related
When running a Python script in Python, Python packages are sometimes missing from the python environment being used:
How can I configure PyCharm so that it automatically installs any missing Python package?
As others have said - wise to avoid automatic installs
Run deployments in PyCharm via a CLI rather than the Studio. This will facilitate construction of batch scripts to drive the install (in your preferred OS interpreter). The output of installs are then easily spooled to a log file. Then a couple of commands filtering the log file for (e.g.) "No module named ' "
Weeding out these lines - it is then possible to construct an script to install the missing dependencies
At work we have a Windows machine with a lot of power. It runs different programs, software etc., but as a Python user I would also like to be able to run scripts, write code etc. on that machine as well to take advantage of the power.
As of now we have gotten Python installed. The issue arises when I log onto the server with my account, then when I do a e.g. pip install numpy it installs this package on my account/user folder. So basically that means that every person logging in needs to download every package from the beginning if they want to use it etc. Somewhat not what we want to do.
So my question is: How do we enable global installation from all users via pip ?
Maybe install python to the machine instead of installing it for a specific user?
In "Customize installation->Next":
Select install for all users when installing python.
Create a public package installation folder, like "C:\Users\Public\site-packages"
Add or set public installation folder to environment variable PYTHONPATH and PYTHONUSERBASE
Execute pip config set global.target [YOUR PUBLIC FOLDER] in command line.
Then pip will install package to the public folder and also python could find it by PYTHONPATH and PYTHONUSERBASE.
I'm a mobile dev, no experience with backend environment. I'd appreciate some steps to config my Windows to run python projects, like this one: https://github.com/avilash/TikTokAPI-Python
I can't figure out how to config, run, and test those methods that are described. I assume it should be an IDE for python, but not sure which one is good for Windows, and how ton config it to run a project like this.
On mobile it's a lot easier, just download project, import in the IDE and run it. Any best practices for experienced devs are appreciated!
install python on your pc, then do pip3 install -r requirement.txt . I personally prefer vscode. Its the best with required extension installed, or if you are the newbie and you want to learn python, then don't install extension.
Download the standard Python distro from here https://www.python.org/downloads/. I would stick with versions 3.6 or 3.7 for now for stability and compatibility reasons.
Run the installer, the default installation path will be something like C:\Program Files\Python3X\. Since you're installing it for the first time, it will be your only installation, so you can choose during the install to set your environment variables (typically including PATH and PYTHONPATH) to point to this installation as the default one.
Create your project folder anywhere you like, where you can put your .py scripts. You can then cd into the folder in the command line and run any scipt like this python myscript.py (or if you haven't set the default env vars C:\Program Files\Python3X\python.exe myscript.py).
This is the bare minimum you need to be able to run Python projects.
To install the package you mentioned, in the command line type pip install git+https://github.com/avilash/TikTokAPI-Python (or C:\Program Files\Python3X\Scripts\pip.exe install git+https://github.com/avilash/TikTokAPI-Python). This will work because the project has a setup script (setup.py). Note that this will install it into the Python installation folder and you will NOT be able to modify it. If you want to be able to do development on it, clone it into a separate folder, navigate into it and install in 'dev' mode from the current folder using pip install -e . (note the dot!)
For the IDE, you can use VS Code if you're already using it, just install the Python extension. If you're more into IntelliJ, you can then use PyCharm which will have the familiar interface but it's a separate IDE. Both are similarly good in terms of features and maturity.
Other notes you can find useful at the beginning:
If you install multiple Python installation in different places, they are pretty much self-contained and defined by their installation path, but of course the one for which you will select to set the environment vars during installation will be the default one. If you want to run the pip command for a specific installation, then you will have to use C:\Program Files\Python3X\python.exe -m pip install ....
pip is Python package manager, typically you can install anything available on https://pypi.org/ with just pip install pandas for example.
You can check all packages installed using pip list. Since pip itself is also just a Python package, it will always be in the list as it's installed by default when you install Python.
I need to run the python scripts on Azure web jobs but i am getting the below error. I tried all the possible ways like scripts with virtualenv and append the path but none of them is working.
[10/08/2018 11:27:27 > ca6024: ERR ] ImportError: No module named request
Can you please help me to fix?
The script used in the file is,
import urllib.request
print('success')
according to
https://docs.python.org/2/library/urllib.html
you can check your python version. it's different between python2 and python3.
in python2.7, use :
urllib.urlopen()
instead of :
urllib.request.urlopen()
Please refer to below steps which I uploaded Python script into Webjobs previously.
1: Use the virtualenv component to create an independent Python runtime environment in your system.If you don't have it, just install it first with command pip install virtualenv
If you installed it successfully, you could see it in your python/Scripts file.
2 : Run the command to create independent Python runtime environment.
3: Then go into the created directory's Scripts folder and activate it (this step is important , don't miss it)
Please don't close this command window and use pip install <your libraryname> to download external libraries in this command window. Such as pip install request for you.
4：Keep the Sample.py uniformly compressed into a folder with the libs packages in the Libs/site-packages folder that you rely on.
5: Create webjob in Web app service and upload the zip file, then you could execute your Web Job and check the log
You could also refer to the SO thread :Options for running Python scripts in Azure
I have a bunch of Python scripts that I want to deploy to other machines. Thing is, I want to have everything self-contained and not depend on the other machines' libraries. For example I don't want to request users to have virtual environment and pip as installed in order for my app to work.
On my local machine I use virtual environment with --no-site-packages and pip install -r requirements.txt to get everything in place.
The bad news is virtualenvironment's activate script has my local path hardcoded into it and using the --relocatable option does not help with this situation so I suppose virtualenvironment is out of the question?
What I would like to have is something similar to this:
base_app_dir:
- main_app_dir
- my_init_script.py
- bin(includes python binary)
- lib(includes pip installed packages and python libraries)
so that I can instruct the end user to just cd into base_app_dir and do a ./bin/python -m my_init_script.py but that means I now need to instruct Python to look into my ./lib folder when importing packages.
I've tried setting os.path.insert(1, 'base_app_dir/lib/site-packages') but this work on per module basis.
Also how about lookup for default Python modules? Right now for example when import hashlib it tries to get it from /usr/lib/python2.7/hashlib.py. I would like to deploy these default Python modules as well and instruct the app to import them from my custom location.
Py2exe or creating a .deb file is not an option right now so please try to address my specific question.