I have a node js Dyno on Heroku and I've changed the script so it uses the npm python-shell module to communicate with a Python program I've written. I'd rather not translate the Python code to js as it is performance-sensitive and works best with certain python modules.
Is it possible to place the Python file in my deployment folder, push it to the Dyno and have it work somehow? I couldn't find much online for this.
Thanks a lot!
You can set up a node.js and Python environment on the same dyno by following Heroku's advice on using multiple build packs:
You can type this into the Heroku shell:
heroku buildpacks:add --index 1 heroku/python
You then need to add the relevant Python start command into your Procfile so that both your Python and Node applications run side by side.
Related
I'm trying to deploy a Node.js application with a child process that runs a machine learning algorithm. I can use this locally, but when I try to run at the Heroku server I recive some messages calling that are some libraries missing, like bellow:
ModuleNotFoundError: No module named 'pandas'
I tried to create manually the requirements.txt and put the necessary libraries there:
pandas
pymongo
dnspython
scikit-learn
scipy
selenium
webdriver-manager
textblob
But it doesn't work. Do I need to do some extra configuration?
Thank you so much for your help!
The way your Heroku dynos run your software is through something called a buildpack.
When you deploy an application to Heroku, it looks at your code and tries to figure out which programming language you are using, then based on that, will run your app using the corresponding buildpack.
For example, if you deploy an app to Heroku and the app has a package.json file in the root of your project directory, Heroku will assume your app is a JavaScript app and use the Node.js buildpack.
Buildpacks contain a number of pre-installed dependencies. For example, the Node.js buildpack contains node (so you can run your JavaScript code) as well as a number of Linux dependencies so that your app will be able to install common libraries/tools that might rely on them.
But... One downside of this buildpack strategy is that if you're deploying a Node.js app, for example, the default Node.js building will NOT come with Python and the various Python library dependencies installed. This is because Heroku supports a lot of different programming environments, and it would be slow/complex if there was just a single buildpack that had EVERYTHING installed. It'd be crazy!
So what you need to do, in your case, is use multiple buildpacks! Heroku has a way for you to enable multiple buildpacks for your app so that your app can have the Node.js dependencies as well as the Python dependencies, for example!
This article on Heroku's documentation site explains how to use multiple buildpacks for a given app.
Here are the specific instructions for simplicity's sake:
# This command will set your default buildpack to Node.js
$ heroku buildpacks:set heroku/nodejs
# This command will set it up so that the Heroku Python buildpack will run first
$ heroku buildpacks:add --index 1 heroku/python
By doing the above, you'll be able to have Heroku install your Python dependencies via a traditional requirements.txt file like you would with any normal Python application.
I am new to the deployment process of python, I have been reading about things like Docker (I know a little bit about Docker) but I haven't really found any concrete guides to deploying python scripts. (I see a lot of examples for flask applications and django applications) My exact use case is a Kafka Consumer that needs to run 24/7. Would I just drop my project on an EC2 and run the docker command or the python script from the command line? if not how can I deploy my python script?
edit: I have tried writing a Dockerfile that just runs my script. Which means I would just drop my project on an EC2 and run the docker file? This sounds wrong to me, but im not entirely sure.
My web host does not have python and I am trying to build a machine learning application. I know that heroku lets you use python. I was wondering if I could use heroku as a python server? As in I would let heroku do all of the python processing for me and use my regular domain for everything else.
Yes, and it may be a pain at first but once it is set I would say Heroku is the easiest platform to continually deploy to. However, it is not intuitive - don't try and just 'take a stab' at it; follow a tutorial and try and understand why Heroku works the way it does.
Following the docs is a good bet; Heroku has great documentation for the most part.
Here's the generalized workflow for deploying to Heroku:
Locally, create your project and use virtualenv to install/manage
libraries.
Initialize a git repository in the base dir for your
Python project; create a heroku remote (heroku create)
Create a
procfile for Heroku to use when starting gunicorn (or see
the options for using waitress/etc); this is used by Heroku to start your process
cd to your base dir; freeze
your virtualenv (pip freeze > requirements.txt) and add/commit
requirements.txt. This tells Heroku what packages need to be installed, a requirement for your deployment to work. If you are trying to run a Python project and there are required packages missing, the app will be unable to start and Heroku will display an Internal Server Error.
Whenever changes are made, git commit your changes and git push heroku master to push all commits to Heroku. This will cause Heroku to restart the server application with your updated deployment. If there's a failure, you can use heroku rollback to just return to your last deployment.
In reality, it's not a pain in the ass, just particular. Knowing the rules of Heroku, you are able to manage your deployment with command-line git commands with ease.
One caveat - If deploying Django, Flask applications etc there are peculiarities to account for; specifically, non-project files (including assets) should NOT be stored on Heroku as Heroku periodically restarts your 'dyno' (server instance(s)), loading the whole project from the latest push to Heroku. With Django and Flask, this typically means serving assets/static/media files from an Amazon S3 bucket.
That being said, if you use virtualenv properly, provision your databases, and follow Heroku practices for serving files and commiting updates, it is (imho) the absolute best platform out there for ease of use, reliable uptime, and well-oiled rolling deployments.
One last tip - if you are creating a Django app, I'd suggest starting your project out of this boilerplate. I have a custom one I use for new projects and can start and publish a project in minutes.
Yes, you can use Heroku as a python server. I put a Python Flask server on Heroku but it was a pain: Heroku seemed to have some difficulties, and there were lots of conflicting advice on getting around those. I eventually got it working, can't remember what web page had the ultimate answer but you might look at this one: http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xviii-deployment-on-the-heroku-cloud
Have you done your Python Server on Heroku by using twisted?
I don't know if this can help you.
I see the doc 'Getting Started on Heroku with Python' is about the Django.
It is sure that Heroku can use Twisted from docs
Pure Python applications, such as headless processes and evented web frameworks like Twisted, are fully supported.
django-twisted-server has twisted in django but it isn't on Heroku.
I was referring to this site as i am learning python/flask and trying to use heroku.
http://ryaneshea.com/lightweight-python-apps-with-flask-twitter-bootstrap-and-heroku
Let me explain what all I did, so that any one who is stuck like me can get the picture.
I am using Linux Backtrack command line .
1. I started virtualenv and then installed flask
virtualenv --distribute
pip install flask
Then, connected heroku and github, created a repo also in the github.
Wrote a simple script and saved it in app.py
Then, asked to create a procfile!
web: python app.py
Questions.
1. What is procfile in layman terms?
2. When i type web: python app.py, it says 'web:: command not found'
Please elaborate how this works?
I have one more doubt, sudo is supreme user right?
We are not supposed to use it in virtualenv?
And for what exactly are we using virtualenv? A simple example.
Questions are pretty basic. DO bare.
the Procfile tells Heroku what commands should be run (https://devcenter.heroku.com/articles/procfile).
You are able to define difference process types, such as web (the only one which will autostart by default), workers, etc...
So basically a Procfile containing
web: python app.py
is telling Heroku to started a named process called web, and to run python app.py when it starts.
There is Python specific documentation for Heroku at https://devcenter.heroku.com/articles/getting-started-with-python#declare-process-types-with-procfile
I made a simple application that makes use of Python CGI scripts. I have a working local version (works fine with lighttpd), but now I'd like to upload it to Heroku. The application consists of 2 or 3 scripts that make operations on a file and print information back to the browser, so I don't think I'll need any module other than CGI.
But the Heroku documentation only explains how to upload Python applications with fancy web frameworks, and I'm not using any of those.
I want to know if it's possible to run CGI scripts on Heroku, and if so, how to do it.
Heroku Cedar is centered around self-hosting web applications, so you need to be able to bundle your application together and run it as a single command.
I think the easiest way would be to port your application to Flask. It isn't very complicated, especially if it is only 2 or 3 scripts.
Another option (depending on your performance requirements) would be to use the simple CGI server in the Python standard library and the Python buildpack. I think you would need to bundle up your scripts in a ./cgi-bin directory and start the server (in the procfile) with:
web: bin/python -m CGIHTTPServer $PORT
The most complex way would be to bundle lighttpd and your scripts together and write a shell script to start it all up. You would have to make sure your compiled binaries are compatible with Heroku. I would look at the PHP buildpack as a starting point.
I inquired with Heroku support about a cgi application that I tried to serve on Heroku's platform and here's the response:
Hello,
Unfortunately, we don't support CGI-style applications, only pure-Python ones. You may have some luck playing around with the Python CGIHTTPServer module some more, but if it doesn't suit your needs, you may be out of luck.
To add to the top answer, for Python 3 the command in the Procfile should be this.
web: python -m http.server --cgi $PORT