This following codes are inside a uWSGI configuration file called flask1.ini:
[uwsgi]
socket = /tmp/flask1.sock
chmod-socket = 777
evn = PRODUCTION=TRUE
module = indy
callable = app
processes = 4
threads = 2
logto = /var/indylog
The production server is set up on ubuntu 14.04 using uWSGI and nginx for Flask application.
I wrote a new module that uses Python 2.7 and it runs without any error on my local ubuntu 14.04 virtualenv (Flask development server) and the same nginx and uWSGI set up as the production environment. However, when I deployed the same code live on production server, it gives a bunch of syntax errors, I am trying to figure out why this is the case.
I run python --version on my local desktop and production server, they are both Python 2.7.6.
My questions: with the above uWSGI configuration on production server, which Python is being used? The machine Python or virtualenv Python?
To be precise, neither. uwsgi does not actually run the Python binary, it uses libpython directly. It just follows your system's LD_LIBRARY_PATH to find the corresponding libpython library and this is normally not affected by virtualenv.
What is affected by virtualenv, however, is the location from which uwsgi will load your packages. You will still need to add a line in your uwsgi.ini to specify the path your virtualenv like this:
virtualenv = /path/to/your/virtualenv
If you wish to configure uwsgi to use different versions of libpython, you will need to build the corresponding plugin for each version and specify it in uwsgi.ini. You can find more information about this here
At first, you have to create Python 3 environment for your source code:
virtualenv -p /usr/bin/python3 path_to_your_project/env
And install packets required:
cd path_to_your_project
source env/bin/activate
# you can use pip to install packets required, e.g:
pip install -r requirements.txt
Finally, add virtualenv to your uwsgi.ini file:
virtualenv = path_to_your_project/env
Install uwsgi in the virtualenv to use whichever Python version the env is configured with. /path/to/env/bin/uwsgi --ini /path/to/flask.ini. Instead of global uwsgi path/to/your/flask.ini, which would use the Python version that the system installed.
Related
I'm attempting to write an Azure function which converts an html input to pdf and either writes this to a blob and/or returns the pdf to the client. I'm using the pdfkit python library. This requires the wkhtmltopdf executable to be available.
To test this locally on my windows machine, I installed the windows version of wkhtmltopdf and this works completely fine.
When I deployed this function on a Linux app service on Azure, I could still execute the function successfully only after I execute the sudo command on kudo tools to install wkhtmltopdf on the app service.
sudo apt-get install wkhtmltopdf
I'm also aware that I can write this start up script on the app service itself.
My question is : Is there something I can do on my local windows machine so I can just deploy the the azure function along with the linux version of wkhtmltopdf directly from my vscode without having to execute another script on the app service itself?
By setting the below commands in the App configuration will work.
Thanks to #pamelafox for the comments.
Commands
PRE_BUILD_COMMAND or POST_BUILD_COMMAND
The following process is applied for each build.
Run custom command or script if specified by PRE_BUILD_COMMAND or PRE_BUILD_SCRIPT_PATH.
Create python virtual environment if specified by VIRTUALENV_NAME.
Run python -m pip install --cache-dir /usr/local/share/pip-cache --prefer-binary -r requirements.txt if requirements.txt exists in the root of repo or specified by CUSTOM_REQUIREMENTSTXT_PATH.
Run python setup.py install if setup.py exists.
Run python package commands and determine python package wheel.
If manage.py is found in the root of the repo manage.py collectstatic is run. However, if DISABLE_COLLECTSTATIC is set to true this step is skipped.
Compress virtual environment folder if specified by compress_virtualenv property key.
Run custom command or script if specified by POST_BUILD_COMMAND or POST_BUILD_SCRIPT_PATH.
Build Conda environment and Python JupyterNotebook
The following process is applied for each build.
Run custom command or script if specified by PRE_BUILD_COMMAND or PRE_BUILD_SCRIPT_PATH.
Set up Conda virtual environemnt conda env create --file $envFile.
If requirment.txt exists in the root of repo or specified by CUSTOM_REQUIREMENTSTXT_PATH, activate environemnt conda activate $environmentPrefix and run pip install --no-cache-dir -r requirements.txt.
Run custom command or script if specified by POST_BUILD_COMMAND or POST_BUILD_SCRIPT_PATH.
Package manager
The latest version of pip is used to install dependencies.
Run
The below process is applied to know how to start an app.
If user has specified a start script, run it.
Else, find a WSGI module and run with gunicorn.
Look for and run a directory containing a wsgi.py file (for Django).
Look for the following files in the root of the repo and an app class within them (for Flask and other WSGI frameworks).
application.py
app.py
index.py
server.py
Gunicorn multiple workers support
To enable running gunicorn with multiple workers strategy and fully utilize the cores to improve performance and prevent potential timeout/blocks from sync workers, add and set the environment variable PYTHON_ENABLE_GUNICORN_MULTIWORKERS=true into the app settings.
In Azure Web Apps the version of the Python runtime which runs your app is determined by the value of LinuxFxVersion in your site config. See ../base_images.md for how to modify this.
References taken from
Python runtime on App Service
I have an Elastic Beanstalk environment running Python 3.6 on AWS Linux 1, and I want to switch it to Python 3.8 on Amazon Linux 2.
I know that I can upgrade environments using the aws CLI update-environment command:
aws elasticbeanstalk update-environment --environment-name <ENV_NAME> --solution-stack-name "64bit Amazon Linux 2 v3.3.7 running Python 3.8"
However, AWS Linux 2 uses different configuration parameters. I can't deploy the AWS Linux 2 config because it's invalid on AWS Linux 1 and I can't upgrade to AWS Linux 2 because my config is invalid.
How do I do the upgrade, and is there a way to do it in-place?
Differences in Configuration
AWS Linux 2 has changed a lot of how elastic beanstalk works and how it is configured. Regardless of whether you are doing an in-place upgrade or spinning up a new environment, here is a list of things that will be different to run through before making the upgrade. Most of the items here are things that are different in Elastic Beanstalk config that live in .ebextensions.
There are differences in sub-package dependencies between Python 3.6 and 3.8. You should test your requirements file on Python 3.8 and make sure it's compatible, especially if you use a generated requirements.txt.
AWS Linux 2 no longer allows you to write Apache config using a file directive in .ebextensions. These modifications now need to live in .platform/httpd/conf.
The virtual environment is no longer active while running container_commands. Any container commands that use your code need to have source $PYTHONPATH/activate run first.
Generated files now get wiped on config changes, so commands like django's collectstatic need to get moved to hooks.
Postgres client is no longer available normally though yum. To install it, you need to do:
packages:
yum:
amazon-linux-extras: []
commands:
01_postgres_activate:
command: sudo amazon-linux-extras enable postgresql10
02_postgres_install:
command: sudo yum install -y postgresql-devel
Apache is no longer the default web server (it is Nginx). To continue using it, you need to specify that as an option on your environment, such as:
option_settings:
aws:elasticbeanstalk:environment:proxy:
ProxyServer: apache
Modwsgi has been replaced with Gunicorn. Any modwsgi customizations you have will no longer work, and the WSGI path has a different format:
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: config.wsgi:application
Static files config has a different format:
option_settings:
aws:elasticbeanstalk:environment:proxy:staticfiles:
/static: staticfiles
You will get opted-in to advanced health reporting. Adding an Elastic Beanstalk health check is strongly recommended:
option_settings:
aws:elasticbeanstalk:application:
Application Healthcheck URL: /health-check/
The application is now run on port 8000 on the server via Gunicorn and Apache/Nginx are just proxying requests to Gunicorn. This matters if you are doing apache customizations such as encrypting traffic between the load balancers and applications servers.
Apache is now run through systemctl rather than supervisord. If you are trying to restart Apache, the command is now sudo systemctl restart httpd
If you want to load your environment variables when sshed into the server, you need to parse them differently:
The environment variables live in a different place and have a different format. To get access to them when SSHed in, you need to add jq: [] to your yum installs. Then, either run the following commands or add them to the bashrc of the server (using a file directive in .ebextensions) to load environment variables and activate the python virtual environment:
source <(/opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""')
source $PYTHONPATH/activate
cd /var/app/current
Upgrading by Launching a New Environment
To take this upgrade path, you need to not be using the Elastic Beanstalk data tier (i.e. you launched your RDS instance yourself, rather than through Elastic Beanstalk).
Create a code branch with your AWS Linux 2 config
Launch a new Elastic Beanstalk environment on AWS Linux 2.
Copy the environment variables from your previous environment.
Allow access to your database from the new environment (add the new environment's server security group as the target of an ingress rule on the database's security group)
Set up SSL on the new environment.
Deploy the AWS Linux 2 code branch to the new environment.
Test this new environment, ignoring browser certificate warnings (or set up a temporary DNS entry to test it).
Switch the DNS entry to point to your new environment or use AWS's CNAME swap feature on the two environemnts.
After your new environment has been running without problems for sufficient time, terminate your old environment.
Upgrading In-Place
There is a way to do the upgrade in-place, though there will be a few minutes where your site says "502 Bad Gateway". In order to do this, you need EB config that is compatible with both the AWS Linux 1 and AWS Linux 2 environments.
For Python, you can do this with a small Flask app and a four part deploy.
Part 1: deploy placeholder app that is compatible with both platforms
Add flask to your requirements.txt (if it's not already there).
Delete all files in .ebextensions
Make .ebextensions/01.config:
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: wsgi_shim.py
make wsgi_shim.py:
from flask import Flask
application = Flask(__name__)
#application.route("/")
#application.route("/<path:path>/")
def hello_world(path=None):
return "This site is currently down for maintenance"
[If using load balancer to application server encryption, change the load balancer to send all traffic to the server via HTTP.]
eb deploy
Part 2: upgrade platform to AWS Linux 2
[If you have any static routes configured in elastic beanstalk delete them.]
Upgrade your eb environment
# Get list of solution stacks
aws elasticbeanstalk list-available-solution-stacks --output=json --query 'SolutionStacks' --region us-east-1
# Use one of the above options here
aws elasticbeanstalk update-environment --environment-name <ENV_NAME> --solution-stack-name "64bit Amazon Linux 2 v3.3.7 running Python 3.8"
Part 3: deploy your main application to AWS Linux 2
Replace .ebextensions/01.config with your new AWS Linux 2 config.
Add .platform/httpd/conf.d/ssl_rewrite.conf:
RewriteEngine On
<If "-n '%{HTTP:X-Forwarded-Proto}' && %{HTTP:X-Forwarded-Proto} != 'https'">
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
eb deploy
Part 4: cleanup
[If using load balancer to application server encryption, change the load balancer back to sending traffic to the server via HTTPS.]
Delete wsgi_shim.py and remove flask from requirements.txt (unless it's a flask project).
eb deploy
I have a django website setup and configured in a python virtual environment (venv) on Ubuntu and all is working fine. Now in order to to run my server on port80 I need to use "sudo" which does not execute in the context of the virtual environment, raising errors (i.e no module named django ...)
Is there a way to get "sudo" to execute in the context of the python virtual environment?!
No, you don't need to do this. You shouldn't be trying to run the development server on port 80; if you're setting up a production environment, use a proper server.
As #DanielRoseman said you should not be using the Django development server in production.
But if you need to run the development server on port 80 you have to reference the use the virtual environment python executable directly.
sudo ../bin/python manage.py runserver localhost:80
This should be the solution.
Even tho I really don't recomment doing this. If you need sudo within Python you are probably on a wrong way.
this did the trick:
$ sudo -- sh -c './venv-bin-path/activate; gunicorn <params> -b 0.0.0.0:80'
I am trying to setup Vagrant in my machine(ubuntu 15.10 64bit). and I followed the steps mentioned here link
I am getting error as no Flask found when I run app.py
Am i missing something here? Its mentioned that all packages from requirements will be installed automatically. But I am not able to make it work.
Steps are as follows:
Getting started
Install Vagrant
Clone this repo as your project name:
git clone git#github.com:paste/fvang.git NEW-PROJECT-NAME
Configure project name and host name in ansible/roles/common/vars/main.yml:
project_name: "fvang"
host_name: "fvang.local"
Modify your local /etc/hosts:
192.168.33.11 fvang.local
Build your Vagrant VM:
vagrant up
Log into the VM via SSH:
vagrant ssh
Start Flask development server:
cd ~/fvang
python app/app.py
I am the author of the FVANG repo, but I don't have the rep to join your chat. I posted a response on the github issue, see here:
https://github.com/paste/fvang/issues/2
I think the Ansible provisioning script failed to complete due to changes in Ansible 2.0. (otherwise Flask would have been installed from requirements.txt). You can check which version of Ansible was installed by running ansible --version. I will be upgrading the scripts to 2.0 shortly.
Edit --
I just updated the repo to work with Ansible 2.0 and simplified a few things. Everything should work as expected now, give it a shot. You'll probably want to just vagrant destroy and vagrant up again.
A vagrant machine as new as a new operating system. You need to install each and every software you need. try this
sudo pip install Flask
After installation if you need to run the app, then you need to uncomment vagrant's ip
(In Vagrantfile) before accessing vagrant's localhost, it turns out to be 192.168.33.10 generally and port 5000
I have written a python script on my local laptop which uses several third party packages. I now want to run my script regularly (via a cron job) on an external server.
The external server most likely does not have all the dependencies installed, is there is a way to package and deploy my python script and dependencies in order to ensure that it will run?
I have already tried to package the script as an exe, but failed to do so.
Not clear what kind of third party packages you have, but for those that were installed with pip, you can do this in your dev environment:
$ pip freeze > requirements.txt
And then you can install these packages in your production environment:
$ pip install requirements.txt
Ideally, you will already have a virtualenv on your production box. If not, it may be well worth reading about these before deploying your script.
Just turn your computer into a server. Simply set up your router for port forwarding so that your server's content's will display when the router's IP is entered. You can of course purchase a DNS domain to give that IP a human readable URL.