I deployed a python worker in Heroku cloud with an app. My Procfile is as follows.
worker: python worker.py
Worker will read a text file, do some changes and writes to another text file. Both text files are in app root directory. This is happening line by line. Approximately, this will take around 6hrs. But once I deploy it, within 1min or so it will crash. Error is logged as
State changed from up to crashed
Process exited with status 0
I have no clue what is causing this.
Related
I have a Django app running on Heroku, which consist in a web process and a worker process (running Django background tasks library). My Procfile looks like this:
web: gunicorn search.wsgi --log-file -
worker: python manage.py process_tasks
I need to run a Python management command when worker starts(and not when the web process starts), to check some issues related to the daily dyno restart of Heroku. Specifically, when worker restarts, I need to run python manage.py somescript.py, and then python manage.py process_taks (the command that starts the worker as needed by DJB library).
How can I achieve this? Is there any way to run two or more commands per process in the procfile? Thanks in advance!
I am hosting a Discord.py Bot on Heroku.
I am getting error code H 14
My procfile is below:
worker: python "Main Script.py"
My requirements file contains:
discord
I tried running the following command:
heroku ps:scale web=1
but it just says couldn't find the web process type.
There are a few issues here:
Your Procfile must be called Procfile exactly. You spell it all lowercase in your quesiton, but it needs a capital P.
Your Procfile only defines a worker process. I'm not sure why you are trying to scale up a web process. Try
heroku ps:scale worker=1
instead.
Or, if your Main Script.py is supposed to listen for HTTP requests, define it as a web process in your `Procfile:
web: python "Main Script.py"
In this case, make sure you're binding to the port Heroku assigns you via the PORT environment variable.
I suggest renaming your file as well, e.g. to main.py, but that shouldn't cause anything to break.
I'm a beginner with discord bot on Python and I'd like to host one on Heroku's free tier. Just a simple one, so I can upgrade to something more complex afterwards.
So I upload this from Github and create the app (I don't use the discord_bot part at the moment). The build finishes without an error, but when I start the application it crashes and i get this log :
2018-09-22T21:37:30.267582+00:00 heroku[worker.1]: Starting process with command `: python3 test_discord.py`
2018-09-22T21:37:31.001035+00:00 heroku[worker.1]: State changed from starting to up
2018-09-22T21:37:32.859278+00:00 heroku[worker.1]: Process exited with status 0
2018-09-22T21:37:32.877853+00:00 heroku[worker.1]: State changed from up to crashed
And... That's all. No error number, no detailed log, nothing i can look for in the heroku documentation.
The most intriguing part is the "status 0". Because from what I've learned looking on stack overflow, this means that everything is... fine !?
So I'm a little lost right here.
This part of the log file looks suspicious
Starting process with command `: python3 test_discord.py`
^^
The colon shouldn't be part of the command.
The reason for that colon is your Procfile, it has a surplus space:
worker : python3 testdiscord.py
^
It needs to be:
worker: python3 testdiscord.py
The exit code is 0 because a colon in shell does nothing and always returns with status 0.
I decided to use python's subprocess where I'm invoking a particular command line program. It was working before but after starting my ec2 instance (from being shutdown), this is showing up in the command line:
[INFO] Handling signal: ttou
Here are the programs I've used.
nginx and gunicorn for my webserver, flask for my api, python 2.7.
Here's the "now problematic" line of code that used to work.
query = subprocess.check_output(couch_cmd, shell=True)
this is the value for couch_cmd:
couch_cmd = "/opt/couchbase/bin/cbq --script=\"select * from `user` where email='" + email + "'\""
This used to work before but after I stopped and started my ec2 instance, this keeps on appearing in the logs when I call my api.
[2017-10-18 02:13:39 +0000] [3324] [INFO] Handling signal: ttou
Note: I've also used the command above and executed it in the python shell. And it's working! I've already changed the appropriate config in nginx to point to my dns and also used proxy_pass. I think my nginx config is fine cause the request gets routed to my webserver. I think there's something in gunicorn that messes up the invocation of subprocess.check_output(). Please help!
I've read this article and realized my mistake:
https://www.computerhope.com/unix/signals.htm
The TTIN and TTOU signals are sent to a process when it attempts to
read or write respectively from the tty while in the background.
Typically, this signal can be received only by processes under job
control; daemons do not have controlling terminals and should never
receive this signal.
I've realized that I was starting gunicorn in the background. This must have prompted gunicorn to intercept the output of check_output() when I tried to log its result to the terminal, thereby intercepting the output and making my request timeout. I was using the output to in my response payload.
How I started gunicorn:
gunicorn app:app -b 127.0.0.1:8000 &
Solution:
gunicorn app:app -b 127.0.0.1:8000
As Heroku has read-only file system except two directories (log and tmp) I wanted to dump my logs from python app to one of them.
The git repository pushed to heroku server app contains both of the folders created by my (checked twice, even downloaded app after push to check if both dirs are there).
While running "heroku run bash" I am able to see only the "tmp" folder - "log" is not visible using "ls -la" or even to the app as I receive errors regarding missing location for the .log files.
2013-08-05T13:10:41.170434+00:00 heroku[web.1]: Starting process with command `python runserver.py`
2013-08-05T13:10:43.132418+00:00 app[web.1]: Traceback (most recent call last):
2013-08-05T13:10:43.609980+00:00 app[web.1]: File "runserver.py", line 2, in <module>
2013-08-05T13:10:43.725134+00:00 app[web.1]: from app_name import app
2013-08-05T13:10:43.850738+00:00 app[web.1]: File "/app/app_name/__init__.py", line 13, in <module>
2013-08-05T13:10:43.968714+00:00 app[web.1]: from logger import flask_debug
2013-08-05T13:10:44.081900+00:00 app[web.1]: File "/app/logger.py", line 8, in <module>
2013-08-05T13:10:44.194540+00:00 app[web.1]: logging.config.dictConfig(CONFIG)
2013-08-05T13:10:44.306174+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/logging/config.py", line 797, in dictConfig
2013-08-05T13:10:44.425589+00:00 app[web.1]: dictConfigClass(config).configure()
2013-08-05T13:10:44.535392+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/logging/config.py", line 579, in configure
2013-08-05T13:10:44.274067+00:00 heroku[web.1]: Process exited with status 1
2013-08-05T13:10:44.287252+00:00 heroku[web.1]: State changed from starting to crashed
So quick thinking I wanted to check if I can place the logs in tmp folder. The app starts, everything looks fine... but nothing is dumped from the app.
I am lost actually as I've looked for the solution quite a time.
Maybe someone will tell me:
Why "log" directory under /app_name folder is not visible?
Why "tmp" directory is not receiving logs?
http://speedy.sh/VaR2w/logger.conf - here's the *.conf file for my loggers
http://speedy.sh/c3QNs/logger.py - here are loggers
PS. Logs are working for the console for the "tmp" folder configuration.
Two better ways:
1) Using Heroku's logdrain feature: https://devcenter.heroku.com/articles/logging#syslog-drains
2) Using one of the addons: Papertrail and Logly come to mind, both have free plans.
I use #1 above: I have an ec2 instance I set up that aggregates all the logs from all of my dynos into a specific set of files, which I use logrotate to manage. I then use simple grep to search thru them, and tail -f to follow, if I feel like it. My rsyslogd configuration on said ec2 instance is:
----- 8< ----- cut here ----- 8< ----- cut here ----- 8< ----- cut here -----
$ModLoad imtcp
$InputTCPServerRun 5514
# vi /etc/rsyslog.d/01-heroku.conf
if $syslogtag startswith 'app[postgres]' then /matchspot-logs/postgres
& ~
if $syslogtag startswith 'app[pgbackups]' then /matchspot-logs/postgres
& ~
if $syslogtag startswith 'heroku[' then /matchspot-logs/heroku
& ~
if $syslogtag startswith 'app[' then /matchspot-logs/app
& ~
when you heroku run, it spins up a new dyno so as not to degrade your web server's performance... think about it like this, if you had 2 web dynos, which one would you get when you heroku run? :)
the approaches suggested by other people here should work way better for you
also, you should be able to add free addons without a confirmed account