I'm attempting to write an Azure function which converts an html input to pdf and either writes this to a blob and/or returns the pdf to the client. I'm using the pdfkit python library. This requires the wkhtmltopdf executable to be available.
To test this locally on my windows machine, I installed the windows version of wkhtmltopdf and this works completely fine.
When I deployed this function on a Linux app service on Azure, I could still execute the function successfully only after I execute the sudo command on kudo tools to install wkhtmltopdf on the app service.
sudo apt-get install wkhtmltopdf
I'm also aware that I can write this start up script on the app service itself.
My question is : Is there something I can do on my local windows machine so I can just deploy the the azure function along with the linux version of wkhtmltopdf directly from my vscode without having to execute another script on the app service itself?
By setting the below commands in the App configuration will work.
Thanks to #pamelafox for the comments.
Commands
PRE_BUILD_COMMAND or POST_BUILD_COMMAND
The following process is applied for each build.
Run custom command or script if specified by PRE_BUILD_COMMAND or PRE_BUILD_SCRIPT_PATH.
Create python virtual environment if specified by VIRTUALENV_NAME.
Run python -m pip install --cache-dir /usr/local/share/pip-cache --prefer-binary -r requirements.txt if requirements.txt exists in the root of repo or specified by CUSTOM_REQUIREMENTSTXT_PATH.
Run python setup.py install if setup.py exists.
Run python package commands and determine python package wheel.
If manage.py is found in the root of the repo manage.py collectstatic is run. However, if DISABLE_COLLECTSTATIC is set to true this step is skipped.
Compress virtual environment folder if specified by compress_virtualenv property key.
Run custom command or script if specified by POST_BUILD_COMMAND or POST_BUILD_SCRIPT_PATH.
Build Conda environment and Python JupyterNotebook
The following process is applied for each build.
Run custom command or script if specified by PRE_BUILD_COMMAND or PRE_BUILD_SCRIPT_PATH.
Set up Conda virtual environemnt conda env create --file $envFile.
If requirment.txt exists in the root of repo or specified by CUSTOM_REQUIREMENTSTXT_PATH, activate environemnt conda activate $environmentPrefix and run pip install --no-cache-dir -r requirements.txt.
Run custom command or script if specified by POST_BUILD_COMMAND or POST_BUILD_SCRIPT_PATH.
Package manager
The latest version of pip is used to install dependencies.
Run
The below process is applied to know how to start an app.
If user has specified a start script, run it.
Else, find a WSGI module and run with gunicorn.
Look for and run a directory containing a wsgi.py file (for Django).
Look for the following files in the root of the repo and an app class within them (for Flask and other WSGI frameworks).
application.py
app.py
index.py
server.py
Gunicorn multiple workers support
To enable running gunicorn with multiple workers strategy and fully utilize the cores to improve performance and prevent potential timeout/blocks from sync workers, add and set the environment variable PYTHON_ENABLE_GUNICORN_MULTIWORKERS=true into the app settings.
In Azure Web Apps the version of the Python runtime which runs your app is determined by the value of LinuxFxVersion in your site config. See ../base_images.md for how to modify this.
References taken from
Python runtime on App Service
Related
I am trying to deploy a python Webapp which is basically an interface for an excel file. The application is based on python.
I've build and ran the container on my local machine and everything is smooth. But when I try to deploy on Azure Web Apps I face an issue which says that my file is missing.
The Dockerfile looks like this
FROM python:3.8-slim-buster AS base
WORKDIR /home/mydashboard
RUN python -m pip install --upgrade pip
RUN pip3 install pipenv
COPY Pipfile /home/mydashboard/Pipfile
COPY Pipfile.lock /home/mydashboard/Pipfile.lock
COPY dataset.csv /home/mydashboard/dataset.csv
COPY src/ /home/mydashboard/src
RUN pipenv install --system
WORKDIR /home/mydashboard/src
EXPOSE 8000
CMD gunicorn -b :8000 dashboard_app.main:server
Weirdly enough when this runs on the Azure App Service I receive a message which was that "dataset.csv" does not exist.
I printed the files inside of /home/dashboard/ it seems that it was an empty folder!!
Who does this happen??
It runs perfectly on my personal computer but it seems that it just run on Azure.
Based on:
Your error message is dataset.csv does not exist
It works on your local machine
Fails in Azure
Probably dataset.csv exists in the copy from location on your local machine, but is missing in Azure.
I am experimenting with running jenkins on kubernetes cluster. I have achieved running jenkins on the cluster using helm chart. However, I'm unable to run any test cases since my code base requires python, mongodb
In my JenkinsFile, I have tried the following
1.
withPythonEnv('python3.9') {
pysh 'pip3 install pytest'
}
stage('Test') {
sh 'python --version'
}
But it says java.io.IOException: error=2, No such file or directory.
It is not feasible to always run the python install command and have it hardcoded into the JenkinsFile. After some research I found out that I have to declare kube to install python while the pod is being provisioned but there seems to be no PreStart hook/lifecycle for the pod, there is only PostStart and PreStop.
I'm not sure how to install python and mongodb use it as a template for kube pods.
This is the default YAML file that I used for the helm chart - jenkins-values.yaml
Also I'm not sure if I need to use helm.
You should create a new container image with the packages installed. In this case, the Dockerfile could look something like this:
FROM jenkins/jenkins
RUN apt install -y appname
Then build the container, push it to a container registry, and replace the "Image: jenkins/jenkins" in your helm chart with the name of the container image you built plus the container registry you uploaded it to. With this, your applications are installed on your container every time it runs.
The second way, which works but isn't perfect, is to run environment commands, with something like what is described here:
https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
the issue with this method is that some deployments already use the startup commands, and by redefining the entrypoint, you can stop the starting command of the container from ever running, thus causing the container to fail.
(This should work if added to the helm chart in the deployment section, as they should share roughly the same format)
Otherwise, there's a really improper way of installing programs in a running pod - use kubectl exec -it deployment.apps/jenkins -- bash then run your installation commands in the pod itself.
That being said, it's a poor idea to do this because if the pod restarts, it will revert back to the original image without the required applications installed. If you build a new container image, your apps will remain installed each time the pod restarts. This should basically never be used, unless it is a temporary pod as a testing environment.
Trying to host a python http server and works fine.
FROM python:latest
COPY index.html /
CMD python3 -m http.server
But when trying with python virtualenv, facing issues.
FROM python:3
COPY index.html .
RUN pip install virtualenv
RUN virtualenv --python="python3" .virtualenv
RUN .virtualenv/bin/pip install boto3
RUN python3 -m http.server &
CMD ["/bin/bash"]
Please help.
I just want to point up that using virtualenv within docker container might be redundant.
With docker, you are encapsulating your one specific application along with its dependencies (libraries, frameworks, boto3 in your case), as opposed to your local machine where you might have several applications being developed, each with different dependencies.
Thus, I would recommend considering again the necessity of virtualenv within docker.
Second, running the command:
RUN python3 -m http.server &
in the background is also bad practice here. You want to run it with the CMD command in the foreground, so it will run as the first process (PID 1). Then it will receive all docker signals, and start automatically with the container start:
CMD ["virtualenv/bin/python3", "-m", "http.server"]
This following codes are inside a uWSGI configuration file called flask1.ini:
[uwsgi]
socket = /tmp/flask1.sock
chmod-socket = 777
evn = PRODUCTION=TRUE
module = indy
callable = app
processes = 4
threads = 2
logto = /var/indylog
The production server is set up on ubuntu 14.04 using uWSGI and nginx for Flask application.
I wrote a new module that uses Python 2.7 and it runs without any error on my local ubuntu 14.04 virtualenv (Flask development server) and the same nginx and uWSGI set up as the production environment. However, when I deployed the same code live on production server, it gives a bunch of syntax errors, I am trying to figure out why this is the case.
I run python --version on my local desktop and production server, they are both Python 2.7.6.
My questions: with the above uWSGI configuration on production server, which Python is being used? The machine Python or virtualenv Python?
To be precise, neither. uwsgi does not actually run the Python binary, it uses libpython directly. It just follows your system's LD_LIBRARY_PATH to find the corresponding libpython library and this is normally not affected by virtualenv.
What is affected by virtualenv, however, is the location from which uwsgi will load your packages. You will still need to add a line in your uwsgi.ini to specify the path your virtualenv like this:
virtualenv = /path/to/your/virtualenv
If you wish to configure uwsgi to use different versions of libpython, you will need to build the corresponding plugin for each version and specify it in uwsgi.ini. You can find more information about this here
At first, you have to create Python 3 environment for your source code:
virtualenv -p /usr/bin/python3 path_to_your_project/env
And install packets required:
cd path_to_your_project
source env/bin/activate
# you can use pip to install packets required, e.g:
pip install -r requirements.txt
Finally, add virtualenv to your uwsgi.ini file:
virtualenv = path_to_your_project/env
Install uwsgi in the virtualenv to use whichever Python version the env is configured with. /path/to/env/bin/uwsgi --ini /path/to/flask.ini. Instead of global uwsgi path/to/your/flask.ini, which would use the Python version that the system installed.
Linux newbie question: I have a personal PiCloud environment and can install my own Python extensions. But I would like to use a pre-compiled C shared library (mylib.so), i.e., place it in /user/lib. Is that possible? If I have to build it on the PiCloud environment server, how do I upload the source?
It's possible that you could simply copy mylib.so to your environment's /usr/lib. But, it's preferred that you compile mylib.so on the setup server to ensure that all the dependencies are available on the server, and that the correct architecture is used (AMD64).
Here are the steps:
Create an environment, and put it in modification mode.
You will need to copy your files to the setup server for the environment. If you're on Linux, it'll be easiest to use scp. If you're on Windows, you'll need to use something like Tunnelier. On either OS, you'll need to click on the key icon, and download the SSH Identity file you'll need to authenticate with the setup server for copying files.
$ scp -i picloud_rsa mylib.tar.gz picloud#setup-server-hostname.com:~/
Once the files are on the server, you can either SSH into the setup server, or use the web browser console (new feature!). From there, run your compile scripts. You can copy your .so file to /usr/lib. Don't forget to use "sudo".
$ sudo cp mylib.so /usr/lib
You should run whatever program depends on mylib.so on the setup server to ensure it's working properly. If you're going to run a test, you'll need to run "ldconfig" so that your shared library is in the library cache.
$ sudo ldconfig
$ ./run_your_program