Django AWS Eslastic Beanktalk not deploying - Internal Server Error - python

I'm following this tutorial:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html
I'm on Step 6. I already created my AWS Elastic Beanstalk Python Application and now I want to link it to my Django application.
This is my directory:
~/Documents/myapp
myapp manage.py requirements.txt .ebextensions .elasticbeanstalk .git .gitignore
Inside my .ebextensions is myapp.config:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- option_name: AWS_SECRET_KEY
value: myUsersSecretKey
- option_name: AWS_ACCESS_KEY_ID
value: myUsersAcessKey
Inside
~/Documents/myapp/myapp
is my settings.py, views.py etc. I made the database in my settings.py to:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
}
as it said in the tutorial. I then did
git add .
git commit -m "eb configuration"
git aws.push
it said that the environment update initiated successfully. After I waited for the status to be ready, I went to the URL and it is still the AWS Elastic Beanstalk Python success page and is not my django application. Any idea why?
Edit: I then tried following this tutorial:
https://www.youtube.com/watch?v=YJoOnKiSYws (pause at 7:44)
and changed my config file to:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- namespace: aws:autoscaling:launchconfiguration
option_name: EC2KeyName
value: myKeyPairName
and now when I refreshed the page after the status was ready, it gave an internal server error saying:
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the sever administrator, root#localhost and inform them of the time and error occured, and anything you might have done that may have acused the error.
More information about this error may be available in the server error log.
EDIT 2:
Is my wsgi.py file supposed to look a certain way when dealing with AWS elastic beanstalk? I left the wsgi.py file how it was when it was first created because the tutorial did as well. This is my wsgi.py:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "ayflare.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

I think there are a couple of things here,
You might want to try using "python manage.py syncdb --noinput" in
the config instead of using "django-admin.py". I seem to remember
encountering some issues until I switched to "python manage.py".
Another thing that might be worthwhile doing is enabling DEBUG in
your django settings file to see if you can get a more detailed error
message about the issue.
Sometimes, if you mess up the deployment is EB, rebuilding the
environment can help smooth out issues once you sort them in your
code/config files. You can do this in the Elastic Beanstalk page in
the AWS Management Console.
Your first myapp.config file looks correct. The updated one that you got from the youtube video doesn't seem correct if you are following the official Amazon Docs.

Related

ERROR Invalid option specification - AWS Elasticbeanstalk - Django

I am trying to set up my Django project using the AWS CLI service but I get the following error
ERROR Invalid option specification (Namespace: 'aws:elasticbeanstalk:container:python', OptionName: 'StaticFiles'): Unknown configuration setting.
I work on Windows and follow these instructions: https://www.1strategy.com/blog/2017/05/23/tutorial-django-elastic-beanstalk/. This is from 2017 and I have already made some adjustments, as the AWS services is constantly being updated
But now something seems to be wrong in the python.config file.
Maybe the name of the command has been changed. I couldn't find much about that anyway.
This is what my python.config file looks like:
container_commands:
01_migrate:
command: "python manage.py migrate"
leader_only: true
02_collectstatic:
command: "python manage.py collectstatic --noinput"
option_settings:
"aws:elasticbeanstalk:application:environment":
DJANGO_SETTINGS_MODULE: "Whisky.settings"
PYTHONPATH: "$PYTHONPATH"
"aws:elasticbeanstalk:container:python":
WSGIPath: "Whisky/wsgi.py"
StaticFiles: "/static/=www/static/"
packages:
yum:
postgresql95-devel: []
I hope you can help me
The StaticFiles as well as the tutorial linked are for Amazon Linux 1 (AL1). However, the current versions of EB use Amazon Linux 2 (AL2). AL2 is likely what you are using now.
In AL2, such option is invalid and new options for static files should be used as shown here and here.
Please not that the tutorial is for AL1, and other things may also not work. There are many differences between AL1 and AL2.

How to deploy / migrate an existing django app / project to a production server on Heroku?

I have a basic django app (Newsdiffs)that runs just fine at localhost:8000 with python website/manage.py runserver but I'd like to migrate it to Heroku and I can't figure out what my next step is.
I thought getting it running locally would translate to running it on Heroku, but I'm realizing that python website/manage.py runserver is launching the dev settings and I'm not sure how to tell it to use the main settings.
All that is in my Procfile is this:
web: python website/manage.py runserver
Locally, that works fine, though it launches it at http://127.0.0.1:8000/ which is probably not what I want on Heroku. So how do I figure out where to set the hostname and port? I don't see either in the app anyplace.
I have just drawn this list for myself two days ago.
It was put together after having followed the steps described in Heroku's help pages for python.
It's by no means definitive nor perfect, and it will change, but it's a valid trace, since I was able to put the site online.
Some issues remain, to be checked thoroughly, e.g. the location of the media/ directory where files are uploaded should/could live outside your project for security reasons (now it works, but I have noticed if the dyno sleeps then the files are not reached/displayed by the template later).
The same goes for the staticfiles/ directory (although this one seems to work fine).
Also, you might want to set django's debug mode to false.
So here it is:
My first steps to deploy an EXISTING django application to Heroku
ASSUMPTIONS:
a) your django project is in a virtual environment already
b) you have already collected all your project's required packages with
pip freeze > requirements.txt
and committed it to git
git add requirements.txt
git commit -m 'my prj requirements'
0) Activate your project's virtual environment
workon xyz #using virtualenvwrapper
then go to your django project's directory (DPD for short) if not already taken there
cd ~/prj/xyz (or cdproject with virtualenvwrapper if setup properly)
and create a new git branch for heroku twiddling to prevent messing things up
git checkout -b he
1) Create the app on heroku
heroku create xyz
that also adds heroku as a remote of your repo
2) Add the needed packages to requirements.txt
vi requirements.txt
add
dj-database-url==0.3.0
django-postgrespool==0.3.0
gunicorn==19.3.0
psycopg2==2.6
django-toolbelt==0.0.1
static3==0.5.1
whitenoise==2.0.3
3) Install all dependencies in the local venv
pip install -r requirements.txt --allow-all-external
4) Setup the heroku django settings
cd xyz
create a copy
cp setting.py settings_heroku.py
and edit it
vi settings_heroku.py
import os
import dj_database_url
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'), )
MEDIA_ROOT = os.path.join(BASE_DIR, "media")
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
SECRET_KEY = os.environ["DJANGO_SECRET_KEY"]
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
replace django's std db cfg with
DATABASES['default'] = dj_database_url.config()
DATABASES['default']['ENGINE'] = 'django_postgrespool'
and
WSGI_APPLICATION = 'xyz.wsgi_heroku.application'
5) Configure the necessary environment variables (heroku configs)
edit the .env file
vi .env
e.g.
DJANGO_SECRET_KEY=whatever
EMAIL_HOST_USER=youruser#gmail.com
EMAIL_HOST_PASSWORD=whateveritis
and/or set them manually if needed (in my case .env had no effect, wasn't loaded apparently, and had to set the vars manually for now)
heroku config:set DJANGO_SECRET_KEY=whatever
heroku config:set EMAIL_HOST_USER=youruser#gmail.com
heroku config:set EMAIL_HOST_PASSWORD=whateveritis
6) Create a separate wsgi file for heroku
cd xyx
cp wsgi.py wsgi_heroku.py
and edit it to make it point to the right settings
vi wsgi_heroku.py
from whitenoise.django import DjangoWhiteNoise
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "xyz.settings_heroku")
application = get_wsgi_application()
application = DjangoWhiteNoise(application)
7) Make sure all the templates use
{% load staticfiles %}
8) Define the Procfile file so that it points to the right wsgi
e.g.
cd ~/prj/xyz (DPD)
vi Procfile
add
web: gunicorn xyz.wsgi_heroku --log-file -
9) Collect all static content into DPD/staticfiles/
locally, make sure django points to the right wsgi settings
export WSGI_APPLICATION=blogger.wsgi_heroku.application
python manage.py collectstatic
10) add the changes to the local git repo (he branch)
git add --all .
git commit -m 'first 4 heroku'
11) check the whole thing works locally
heroku local # in heroku's help they also add `web`, not needed?!
12) push your code to heroku
git push heroku he:master
13) make sure a instance of the app is running
heroku ps:scale web=1
14) create the tables on the heroku DB
heroku run python manage.py migrate
Note: if you see a message that says, “You just installed Django’s auth system, which means you don’t have any superusers defined. Would you like to create one now?”, type no.
15) add the superuser to the heroku DB
heroku run bash
python manage.py createsuperuser
and fill in the details, as usual
16) Populate the DB with the necessary fixtures
heroku run python manage.py loaddata yourfile.json
17) Visit the website page on heroku's webserver
heroku open
or go to
https://xyz.herokuapp.com/
and the admin
https://xyz.herokuapp.com/admin/
and the DB
https://xyz.herokuapp.com/db
Useful commands:
View the app's logs
heroku logs [--tail]
List add-ons deployed
heroku addons
and use one:
heroku addons:open <add-on-name>
Run a command on heroku (the remote env, where you are deploying)
heroku run python manage.py shell
heroku run bash
Set a config var on Heroku
heroku config:set VARNAME=whatever
View the config vars that are set (including the DB's)
heroku config
View postgres DB details
heroku pg
If you know some python and have a lot of experience building web apps in other languages but don't totally understand where Heroku fits, I highly recommend Discover Flask, which patched a lot of the holes in my understanding of how these pieces all fit together.
Some of the things that I worked out:
you really do need an isolated virtual environment if you're going to deploy to Heroku, because Heroku installs Python modules from the requirements.txt file.
Gunicorn is a web server, and you definitely need to run your app under Gunicorn or it won't run on Heroku.
The "Procfile" doesn't just give the command you use to run the app locally. And Heroku requires it. So if you've got an app that was built to run on Heroku and it doesn't include a Procfile, they left something out.
You don't tell Heroku what your hostname is. When you run heroku create it should tell you what your domain name is going to be. And every time you run git push heroku master (or whatever branch you're pushing, maybe it isn't master), Heroku will (try to) restart your app.
Heroku doesn't support sqlite. You have to run your Production DB in Postgres.
This doesn't directly answer my question, but it does fill in some of the missing pieces that were making it hard for me to even ask the right question. RTFM notwithstanding. :)

Running Django with gunicorn: Bad Request (400)

I am following this tutorial to set up a Django application and serve it with Gunicorn on a Debian DigitalOcean server: http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/
I have got as far as the section starting "Now that you have gunicorn, you can test whether it can serve your Django application by running the following command". Now I'm stuck.
In other words, I can successfully run the application using python manage.py runserver, but not by using gunicorn.
I've successfully accessed my app with:
$ python manage.py runserver xx.xx.xx.xx:8000
Now from the same directory, I'm trying to run:
$ gunicorn my_django.wsgi:application --bind xx.xx.xx.xx:8001
It appears to start OK, but when I go to http://xx.xx.xx.xx:8001, I see:
Bad Request (400)
I'm not sure how to debug this: there's nothing in /var/log/gunicorn/.
I have set ALLOWED_HOSTS=['xx.xx.xx.xx'] in my settings file.
UPDATE: Being an idiot: gunicorn was looking in production settings file, not local settings file. Setting the ALLOWED_HOSTS in production settings fixed it.
I'd still really like to know how to debug problems like this though.
The answer: gunicorn was looking in production settings file, not local settings file. Setting the ALLOWED_HOSTS in production settings fixed it.

"Command hooks failed" when deploying Django on AWS

i'm trying to deploy a Django app on Amazon Elastic Beanstalk. Following amazon's guide, i changed my database settings to RDS and wrote a config file.
This is in django's settings.py:
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
and this is my config file for Elastic Beanstalk:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
02_createadmin:
command: "scripts/createadmin.py"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- namespace: aws:elasticbeanstalk:container:python:staticfiles
option_name: /static/
value: static/
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- option_name: AWS_SECRET_KEY
value: *my secret key*
- option_name: AWS_ACCESS_KEY_ID
value: *my access key*
I'm using eb to deploy. With "eb init" and "eb start" i can successfully create the app and it's env and with git aws.push i'm pushing my last commit to the server.
However, after it updates i got this error:
2014-07-03 13:41:27 UTC-0300 ERROR [Instance: i-4b1aef3e Module: AWSEBAutoScalingGroup ConfigSet: null] Command failed on instance. Return code: 1 Output: Error occurred during build: Command hooks failed .
2014-07-03 13:41:25 UTC-0300 ERROR Script /opt/elasticbeanstalk/hooks/appdeploy/pre/01new.py failed with returncode 1
2014-07-03 13:41:25 UTC-0300 ERROR Application version failed to deploy.
What i'm doing wrong?
UPDATE:
This is the log of a deploy
You should check if you are trying to push: git aws.push from your virtualenv or where you have python installed. Also, you should try restarting the EC2 machine that powers your ElasticBeanstalk application.

Heroku Django db postgres error

I am trying to get my first django app up, but I'm running into this db error:
OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
This happens whenever I try to syncdb, migrate, or createsuperuser
I am configuring my DATABASES variable like:
DATABASES = {'default' : dj_database_url.config(default=os.environ["HEROKU_POSTGRESQL_OLIVE_URL"]) }
Is there something else I need to configure or am doing wrong?
EDIT (SOLVED):
Thanks for helping me narrow down the problem, I've found the solution.
Since this was the first time I deployed to heroku and the first time I used the two scoops django directory format. I thought doing something like
python manage.py syncdb # would be okay
instead beause my settings folder looks like
.../settings
base.py
local.py
production.py
demo.py
# ...
I need to do
python manage.py syncdb --app.settings.demo
Your syntax seems right, try using DATABASE_URL after verifying you promoted HEROKU_POSTGRESQL_OLIVE_URL, although it should work even when not promoted.
$ heroku pg:promote HEROKU_POSTGRESQL_OLIVE_URL
and then:
import dj_database_url
DATABASES = {
'default': dj_database_url.config(default=os.getenv('DATABASE_URL'))
}
This setup should be working. Else, check that you are working on the right settings file. You can verify this by running:
$ heroku run python manage.py shell
and then:
>>> from django.conf import settings
>>> print settings.DATABASES['default']
and verify result.
I had an error in .gitignore and my local settings were getting set by Heroku by mistake.
This will also cause this error message message.

Categories