This article teaches how to run background jobs in heroku.
I have my clock.py file, in the same directory where Procfile is.
Following is my clock.py file:
from apscheduler.scheduler import Scheduler
from subscription.views import send_subs_mail
sched = Scheduler()
#sched.cron_schedule(day_of_week='sat', hour=23)
def scheduled_job():
print 'This job is run every saturday at 11pm.'
send_subs_mail()
sched.start()
while True:
pass
I have updated my Procfile, to look as follows:
web: newrelic-admin run-program gunicorn hellodjango.wsgi -b 0.0.0.0:$PORT -w 3
clock: python clock.py
Earlier it used to look like:
web: newrelic-admin run-program gunicorn hellodjango.wsgi -b 0.0.0.0:$PORT -w 3
Having done all this, I do the following in my terminal:
heroku ps:scale clock=1
And I get the following error:
Scaling clock processes... failed ! No such type as clock.
I have updated my requirements file, as mentioned in the article.
and here is the heroku logs
2013-05-22T15:52:08.200587+00:00 heroku[web.1]: Starting process with command `newrelic-admin run-program gunicorn hellodjango.wsgi -b 0.0.0.0:48070 -w 3 clock: python clock.py`
2013-05-22T15:52:09.081883+00:00 heroku[web.1]: Process exited with status 0
2013-05-22T15:52:10.691985+00:00 app[web.1]: gunicorn: error: No application module specified.
2013-05-22T15:52:10.683457+00:00 app[web.1]: Usage: gunicorn [OPTIONS] APP_MODULE
2013-05-22T15:52:10.691843+00:00 app[web.1]:
2013-05-22T15:52:12.504514+00:00 heroku[web.1]: Process exited with status 2
2013-05-22T15:52:12.525765+00:00 heroku[web.1]: State changed from crashed to starting
2013-05-22T15:52:12.525765+00:00 heroku[web.1]: State changed from starting to crashed
2013-05-22T15:52:16.198417+00:00 heroku[web.1]: Starting process with command `newrelic-admin run-program gunicorn hellodjango.wsgi -b 0.0.0.0:55149 -w 3 clock: python clock.py`
2013-05-22T15:52:17.343513+00:00 app[web.1]: Usage: gunicorn [OPTIONS] APP_MODULE
2013-05-22T15:52:17.343513+00:00 app[web.1]: gunicorn: error: No application module specified.
2013-05-22T15:52:17.343513+00:00 app[web.1]:
2013-05-22T15:52:18.557818+00:00 heroku[web.1]: State changed from starting to crashed
2013-05-22T15:52:18.542409+00:00 heroku[web.1]: Process exited with status 2
What is wrong?
Each process type should be on it's own line, like so:
web: newrelic-admin run-program gunicorn hellodjango.wsgi -b 0.0.0.0:$PORT -w 3
clock: python clock.py
Hope that helps!
Related
I'm deploying torchserve on heroku free dyno. Deploy works fine but the app isn't running properly.
LOG1:
2022-03-17T02:11:10.352655+00:00 heroku[web.1]: Starting process with command `torchserve --start --ncs --model-store model_store --models squeezenet1_1.mar`
2022-03-17T02:11:11.558957+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2022-03-17T02:11:11.701395+00:00 app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx300m -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8
2022-03-17T02:11:11.896962+00:00 heroku[web.1]: Process exited with status 0
2022-03-17T02:11:12.059922+00:00 heroku[web.1]: State changed from starting to crashed
LOG2:
2022-03-17T05:17:29.472219+00:00 heroku[web.1]: Starting process with command `torchserve --start --ncs --model-store model_store --models squeezenet1_1.mar`
2022-03-17T05:17:30.854303+00:00 app[web.1]: Setting JAVA_TOOL_OPTIONS defaults based on dyno size. Custom settings will override them.
2022-03-17T05:17:31.304897+00:00 heroku[web.1]: Process exited with status 0
2022-03-17T05:17:31.364160+00:00 heroku[web.1]: State changed from starting to crashed
Also, my requirements.txt:
-f https://download.pytorch.org/whl/cpu/torch_stable.html
future
psutil
requests
captum
packaging
numpy
cython
wheel
pillow==9.0.1
torch==1.10.0+cpu
torchvision==0.11.1+cpu
torchaudio==0.10.0+cpu
torchtext==0.11.0
torchserve==0.5.3
runtime.txt
python-3.8.12
system.properties
java.runtime.version=1.8.0_322
Problem was torchserve was running in background mode by default.
Adding --foreground to the torchserve command in Procfile fixed this.
(Also, java runtime 11.0.14.1 is required by torchserve)
I am attempting to deploy a django project to Heroku and I'm getting slapped left and right with errors but hopefully this is the last one.
2022-03-07T22:03:09.363036+00:00 heroku[release.9772]: Starting process with command `/bin/sh -c 'if curl https://heroku-release-output.s3.amazonaws.com/log-stream?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJ3LIQ2SWG7V76SVQ%2F20220307%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220307T220304Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=ddb760b1c36e913b7c0cc984428e3b853cdac67755e4cfa6038cef77a37b6a25 --silent --connect-timeout 10 --retry 3 --retry-delay 1 >/tmp/log-stream; then
2022-03-07T22:03:10.031451+00:00 heroku[release.9772]: State changed from starting to up
2022-03-07T22:03:10.527868+00:00 app[release.9772]: python: can't open file '/app/backend.manage.py': [Errno 2] No such file or directory
2022-03-07T22:03:10.706087+00:00 heroku[release.9772]: Process exited with status 2
2022-03-07T22:03:10.901301+00:00 heroku[release.9772]: State changed from up to complete
2022-03-07T22:02:37.000000+00:00 app[api]: Build started by user email#protonmail.com
2022-03-07T22:03:03.734406+00:00 app[api]: Deploy 922870fe by user email#protonmail.com
2022-03-07T22:03:03.734406+00:00 app[api]: Running release v26 commands by user email#protonmail.com
2022-03-07T22:03:05.039171+00:00 app[api]: Starting process with command `/bin/sh -c 'if curl $HEROKU_RELEASE_LOG_STREAM --silent --connect-timeout 10 --retry 3 --retry-delay 1 >/tmp/log-stream; then
2022-03-07T22:03:05.039171+00:00 app[api]: chmod u+x /tmp/log-stream
2022-03-07T22:03:05.039171+00:00 app[api]: /tmp/log-stream /bin/sh -c '"'"'python backend.manage.py makemigrations - - no-input'"'"'
2022-03-07T22:03:05.039171+00:00 app[api]: else
2022-03-07T22:03:05.039171+00:00 app[api]: python backend.manage.py makemigrations - - no-input
2022-03-07T22:03:05.039171+00:00 app[api]: fi'` by user email#protonmail.com
2022-03-07T22:03:12.919074+00:00 app[api]: Release v26 command failed by user email#protonmail.com
2022-03-07T22:03:15.000000+00:00 app[api]: Build succeeded
And when I try to run the program in a virtual environment on my local machine using the command heroku local:
6:06:35 PM web.1 | [2022-03-07 18:06:35 -0400] [33412] [INFO] Starting gunicorn 20.1.0
6:06:35 PM web.1 | [2022-03-07 18:06:35 -0400] [33412] [ERROR] Connection in use: ('0.0.0.0', 5000)
6:06:35 PM web.1 | [2022-03-07 18:06:35 -0400] [33412] [ERROR] Retrying in 1 second.
6:06:36 PM web.1 | [2022-03-07 18:06:36 -0400] [33412] [ERROR] Connection in use: ('0.0.0.0', 5000)
And so on and so on.
I suspect this has to do with my Procfile. And admittedly, I'm not too sure on how to set it up. I currently have in my Procfile:
web: gunicorn --pythonpath backend.feh backend.feh.wsgi --log-file -
release: python backend.manage.py makemigrations - - no-input
And my directory structure is like:
runtime.txt
requirements.txt
Procfile
backend/
|-- __init__.py
api/
feh/
|-- __init__.py
errorlog
settings.py
urls.py
wsgi.py
static/
db.sqlite3
manage.py
scraper.py
I'm positive my Procfile is the cause for this issue but I don't know how to go about debugging this. At the very least, the project runs fine on my venv. Thank you in advance!
Neither of the processes in your Procfile is specified correctly.
The web command's --pythonpath should just be backend and it should receive the dotted path feh.wsgi as an argument:
web: gunicorn --pythonpath backend feh.wsgi --log-file -
The release command refers to a file called backend.manage.py:
release: python backend.manage.py makemigrations - - no-input
That argument should be a path to a file, not a dotted Python module, and - - no-input should be the single argument --no-input. Try this instead:
release: python backend/manage.py makemigrations --no-input
I have a python flask app deployed to heroku.
After the build completes, this is the log.
2021-05-15T17:58:00.000000+00:00 app[api]: Build started by user xxx
2021-05-15T17:58:39.006534+00:00 app[api]: Deploy 23e180a8 by user xxx
2021-05-15T17:58:39.006534+00:00 app[api]: Release v29 created by xxx
2021-05-15T17:58:39.356233+00:00 heroku[web.1]: State changed from crashed to starting
2021-05-15T17:58:44.631213+00:00 heroku[web.1]: Starting process with command `gunicorn app:app`
2021-05-15T17:58:46.642208+00:00 heroku[web.1]: Process exited with status 126
2021-05-15T17:58:46.707103+00:00 heroku[web.1]: State changed from starting to crashed
2021-05-15T17:58:46.591536+00:00 app[web.1]: Error: No such file or directory
2021-05-15T17:58:49.000000+00:00 app[api]: Build succeeded
These are the contents in my Procfile
web: gunicorn app:app
I have the main file name as app.py
And these are the requirements.txt
app-version==1.0.1
APScheduler==3.6.3
certifi==2020.12.5
chardet==3.0.4
click==8.0.0
Werkzeug==2.0.0
Flask==2.0.0
gunicorn==20.1.0
idna==2.8
Jinja2==3.0.0
MarkupSafe==2.0.0
notify==0.3.1
notify-run==0.0.13
PyQRCode==1.2.1
pytz==2021.1
requests==2.22.0
selenium==3.141.0
six==1.16.0
tornado==6.1
tzlocal==2.1
urllib3==1.25.11
validate-email==1.3
Added these using pip freeze. Any idea whats going wrong?
https://docs.gunicorn.org/en/latest/run.html
Format for Procfile if you using gunicorn:
gunicorn [OPTIONS] [WSGI_APP]
"web: gunicorn app:app" - it is mean file app.py has function app that run application, and file app.py saved on root of project. Check patch.
I am working on a Django project which I have pushed on Heroku, for background tasking I have used Celery. Although Celery works fine locally, but on the Heroku server I have observed that celery worker is getting crashed. I have set CLOUDAMQP_URL properly in settings.py and configured worker configuration in Procfile, but still worker is getting crashed.
Procfile
web: gunicorn my_django_app.wsgi --log-file -
worker: python manage.py celery worker --loglevel=info
Settings.py
...
# Celery
BROKER_URL = os.environ.get("CLOUDAMQP_URL", "django://")
#CELERY_BROKER_URL = 'amqp://localhost'
BROKER_POOL_LIMIT = 1
BROKER_CONNECTION_MAX_RETRIES = 100
CELERY_TASK_SERIALIZER="json"
CELERY_RESULT_SERIALIZER="json"
CELERY_RESULT_BACKEND = "amqp://"
Logs
2019-08-05T15:03:51.296563+00:00 heroku[worker.1]: State changed from crashed to starting
2019-08-05T15:04:05.370900+00:00 heroku[worker.1]: Starting process with command `python manage.py celery worker --loglevel=info`
2019-08-05T15:04:06.173210+00:00 heroku[worker.1]: State changed from starting to up
2019-08-05T15:04:09.067794+00:00 heroku[worker.1]: State changed from up to crashed
2019-08-05T15:04:08.778426+00:00 app[worker.1]: Unknown command: 'celery'
2019-08-05T15:04:08.778447+00:00 app[worker.1]: Type 'manage.py help' for usage.
2019-08-05T15:04:09.048404+00:00 heroku[worker.1]: Process exited with status 1
manage.py
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_django_app.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
I made the following changes in Procfile and error got resolved
web: gunicorn my_django_app.wsgi --log-file -
worker: celery -A my_django_app worker -l info
I Launch my Django app in a docker container, using a startup script as per the below:-
#!/bin/bash
python manage.py collectstatic --noinput
python manage.py makemigrations
python manage.py migrate
/etc/init.d/celeryd start
/etc/init.d/celerybeat start
exec gunicorn itapp.wsgi -b 0.0.0.0:8000
Unicorn and ngnix every now and then give me a 502 error, so I went to look for logs and it looks by default gunicorn does not log any? im on version 19.7.1
so I added the to the exec guncicorn command:
exec gunicorn itapp.wsgi -b 0.0.0.0:8000 --error-logfile /var/log/gunicorn/errors.log , --log-file /var/log/gunicorn/access.log
Which I goy form the gunicorn documentation. however now Gunicorn fails to launch with the following error:-
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: ,
How can I create the gunicorn logs to debug these errors?
Thanks
Remove comma and you should be fine:
exec gunicorn itapp.wsgi -b 0.0.0.0:8000 --error-logfile /var/log/gunicorn/errors.log --log-file /var/log/gunicorn/access.log