i'm trying to deploy a Django app on Amazon Elastic Beanstalk. Following amazon's guide, i changed my database settings to RDS and wrote a config file.
This is in django's settings.py:
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
and this is my config file for Elastic Beanstalk:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
02_createadmin:
command: "scripts/createadmin.py"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- namespace: aws:elasticbeanstalk:container:python:staticfiles
option_name: /static/
value: static/
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- option_name: AWS_SECRET_KEY
value: *my secret key*
- option_name: AWS_ACCESS_KEY_ID
value: *my access key*
I'm using eb to deploy. With "eb init" and "eb start" i can successfully create the app and it's env and with git aws.push i'm pushing my last commit to the server.
However, after it updates i got this error:
2014-07-03 13:41:27 UTC-0300 ERROR [Instance: i-4b1aef3e Module: AWSEBAutoScalingGroup ConfigSet: null] Command failed on instance. Return code: 1 Output: Error occurred during build: Command hooks failed .
2014-07-03 13:41:25 UTC-0300 ERROR Script /opt/elasticbeanstalk/hooks/appdeploy/pre/01new.py failed with returncode 1
2014-07-03 13:41:25 UTC-0300 ERROR Application version failed to deploy.
What i'm doing wrong?
UPDATE:
This is the log of a deploy
You should check if you are trying to push: git aws.push from your virtualenv or where you have python installed. Also, you should try restarting the EC2 machine that powers your ElasticBeanstalk application.
Related
I have a Django application that uses Python 2.7.10 and Django 1.10.3. My Dockerfile contains the following:
FROM python:2.7.10
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
My requirements.txt contains the following:
Django>=1.10,<2.0
Mysql-python
My docker-compose.yml contains the following:
version: '2'
services:
localhost:
image: mysql
restart: always
environment:
MYSQL_DATABASE: 'test'
MYSQL_USER: 'test'
MYSQL_PASSWORD: 'password'
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- localhost
settings.py contains the following for the database:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'test',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '3306',
}
I first do a "% docker-compose build", then I do a "% docker-compose up" and get the following error:
web_1 | django.db.utils.OperationalError: (2002, "Can't connect
to local MySQL server through socket '/var/run/mysqld/mysqld.sock'
I've looked and tried the solutions available on Stackoverflow with no luck. Any help would be much appreciated.
Any help would be much appreciated.
In regards to MySql and Django App relation you have three possible cases:
MySql is residing on same container as Django App (localhost)
MySql is residing on separate container from Django App (hostname is different, can be handled on different topology - same subnet or different etc)
MySql is residing outside of docker ecosystem (bare metal, cloud, whatever...)
Judging from your settings.py and docker-compose.yml you are mixing first two approaches. Now, since you can reference external MySql server from settings.py I'll consider that case trivial, next, cramming MySql on same container as Django App is usually bad idea (if you ever want to scale them separately) so this answer will focus on second case (MySql in separate container) and, for simplicity, on same docker-compose file (you can separate them as well).
Important excerpt from docker-compose.yml is:
version: '2'
services:
# The Database
database:
image: mysql:5.6
volumes:
- dbdata:/var/lib/mysql
environment:
- "MYSQL_DATABASE=test_db"
- "MYSQL_USER=test_user"
- "MYSQL_PASSWORD=some_password"
- "MYSQL_ROOT_PASSWORD=some_root_password"
ports:
- "33061:3306"
container_name: "mysql_database"
# Django App (taken from your example, added env vars as hint)
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
environment:
- "DB_PORT=3306"
- "DB_HOST=database"
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- database
volumes:
dbdata:
With such docker-compose you would have following setting.py config (note that HOST is taking value of docker-compose service entry for database container, port is internal to container not external to docker-host):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test_db',
'USER': 'test_user',
'PASSWORD': 'some_password',
'HOST': 'database',
'PORT': '3306',
}
As a sidenote, we usually run database with separate docker-compose.yml from Django App file since in usual dev cycle it is far more frequent to tweak/restart/crash something on Django App container than on database one and we can handle it separately that way, plus such a separately defined database can handle several different DjangoApps each with its own container.
I am trying to connect a Django app running inside of a docker container to a PostgreSQL database running on another computer (lets say with an IP address of 192.168.1.22). Thus far in my experimentation, I always get the following error when attempting to connect to the PostgreSQL instance from a docker container (my code works fine outside of the docker container):
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I've made sure that I allow all inbound connections to the PostgreSQL database (by changing the config file for my PostgreSQL server as recommended here.
Here is my settings code in my Django app for the database connection:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'xxxxx',
'USER': 'xxxxxxx',
'PASSWORD': 'xxxxxxxxxxx',
'HOST': '192.168.1.22',
'PORT': '5432',
}
}
Here is my Dockerfile:
FROM python:3.6
ENV PYTHONUNBUFFERED 1
# Install dependencies
RUN mkdir /config
ADD /config/requirements.pip /config/
RUN pip install -r /config/requirements.pip
# Install source code
RUN mkdir /src
WORKDIR /src
COPY ./src/ /src/
ENV UWSGI_WSGI_FILE=/src/xxxxxxxxx/wsgi.py UWSGI_HTTP=:8000 UWSGI_MASTER=1 UWSGI_WORKERS=2 UWSGI_THREADS=8 UWSGI_UID=1000 UWSGI_GID=2000 UWSGI_LAZY_APPS=1 UWSGI_WSGI_ENV_BEHAVIOR=holy
RUN python /src/manage.py collectstatic --noinput
EXPOSE 8000
CMD ["uwsgi", "--http-auto-chunked", "--http-keepalive"]
Here is the script I use to build (& run) my docker container:
#!/bin/bash
docker build .
docker run -i -t -p 8000:8000 xxxx/xxxxxxx
I've also tried using the add-host option for Docker to add the database as a host, then reference the host name from within the settings file of my project. In all scenarios I end up with the same error above.
Thank you for the help in advance!
I fix this problem using extra_hosts and Docker-compose
For example
web:
image: my-web-app
build: .
command: bash -c "uwsgi --http-auto-chunked --http-keepalive"
ports:
- "8000:8000"
extra_hosts:
- "postgresql:192.168.1.22"
In your Django configuration
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'xxxxx',
'USER': 'xxxxxxx',
'PASSWORD': 'xxxxxxxxxxx',
'HOST': 'postgresql',
'PORT': '5432',
}
To run your app use docker-compose up
I am trying to setup continuous integration for my django project. Since I am using postgresql als my database, I want to setup the same in travis. This is my travis.yml file:
language: python
services:
- postgresql
python:
- 3.4
env:
- DJANGO=1.8 DB=postgres
before_install:
- export DJANGO_SETTINGS_MODULE=ledenbestand.settings.local
install:
- pip install -r travis/requirements.txt
before_script:
- psql -c 'create database ledenbestand;' -U postgres
script:
- python manage.py test
notifications:
email:
recipients:
- random#email.com
on_success: never
on_failure: always
The problem is that this will fail because my local.py settings also gives a password when connecting to the database, the postgres user on travis doesn't have a password:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'ledenbestand',
'USER': 'postgres',
**'PASSWORD': 'password',**
'HOST': 'localhost',
'PORT': '5432',
}
}
The travis documentation says the following:
Should your local test setup use different credentials or settings to access the local test database, we recommend putting these settings in a database.yml.travis in your repository and copying that over as part of your build:
before_script:
- cp config/database.yml.travis config/database.yml
I however have no clue how this could work, how can this line of code overwrite my local.py settings?
Fixed this by adding an If statement in my config file:
if 'TRAVIS' in os.environ:`
I'm getting an error when I query MySQL on my Django app hosted on elastic beanstalk. The error says:
OperationalError at /admin/login
(1045, "Access denied for user 'adminDB'#'172.30.23.5' (using password: YES)")
Here is my .config file:
container_commands:
01_migrate:
command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput"
leader_only: true
option_settings:
"aws:elasticbeanstalk:application:environment":
DJANGO_SETTINGS_MODULE: "mysite.settings"
"PYTHONPATH": "/opt/python/current/app/mysite:$PYTHONPATH"
"ALLOWED_HOSTS": ".elasticbeanstalk.com"
"aws:elasticbeanstalk:container:python":
WSGIPath: mysite/wsgi.py
NumProcesses: 3
NumThreads: 20
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "www/static/"
Here is my the databases section on settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
}
I created the eb environment through the eb create command and then proceeded to create an RDS database on the eb console on their website. Is there anything else I need to do to connect Django to MySQL? Something to do with security groups or something?
Thanks!
Thanks for your response #DrewPierce, but the problem was a simple one, and it's also extremely silly. Turns out the dollar sign in my RDS password was causing a problem. Changed it to a simpler password and I successfully logged in.
Hope this helps anyone else with a similar issue!
I'm following this tutorial:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html
I'm on Step 6. I already created my AWS Elastic Beanstalk Python Application and now I want to link it to my Django application.
This is my directory:
~/Documents/myapp
myapp manage.py requirements.txt .ebextensions .elasticbeanstalk .git .gitignore
Inside my .ebextensions is myapp.config:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- option_name: AWS_SECRET_KEY
value: myUsersSecretKey
- option_name: AWS_ACCESS_KEY_ID
value: myUsersAcessKey
Inside
~/Documents/myapp/myapp
is my settings.py, views.py etc. I made the database in my settings.py to:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.environ['RDS_DB_NAME'],
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
}
}
as it said in the tutorial. I then did
git add .
git commit -m "eb configuration"
git aws.push
it said that the environment update initiated successfully. After I waited for the status to be ready, I went to the URL and it is still the AWS Elastic Beanstalk Python success page and is not my django application. Any idea why?
Edit: I then tried following this tutorial:
https://www.youtube.com/watch?v=YJoOnKiSYws (pause at 7:44)
and changed my config file to:
container_commands:
01_syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: myapp/wsgi.py
- option_name: DJANGO_SETTINGS_MODULE
value: myapp.settings
- namespace: aws:autoscaling:launchconfiguration
option_name: EC2KeyName
value: myKeyPairName
and now when I refreshed the page after the status was ready, it gave an internal server error saying:
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the sever administrator, root#localhost and inform them of the time and error occured, and anything you might have done that may have acused the error.
More information about this error may be available in the server error log.
EDIT 2:
Is my wsgi.py file supposed to look a certain way when dealing with AWS elastic beanstalk? I left the wsgi.py file how it was when it was first created because the tutorial did as well. This is my wsgi.py:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "ayflare.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I think there are a couple of things here,
You might want to try using "python manage.py syncdb --noinput" in
the config instead of using "django-admin.py". I seem to remember
encountering some issues until I switched to "python manage.py".
Another thing that might be worthwhile doing is enabling DEBUG in
your django settings file to see if you can get a more detailed error
message about the issue.
Sometimes, if you mess up the deployment is EB, rebuilding the
environment can help smooth out issues once you sort them in your
code/config files. You can do this in the Elastic Beanstalk page in
the AWS Management Console.
Your first myapp.config file looks correct. The updated one that you got from the youtube video doesn't seem correct if you are following the official Amazon Docs.