We Keep Coding

An error occurs when adding a container command to Django aws eb

I am using Python 3.7 Django 3.0.9 postgresql 11
-ebextension/django.config
container_commands:
01_migrate:
command: "django-admin.py migrate"
leader_only: true
02_compilemessages:
command: "django-admin.py compilemessages"
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: config.wsgi:application
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: config.settings
Error message when running eb deploy
2020-08-04 14:48:51 INFO Environment update is starting.
2020-08-04 14:49:30 INFO Deploying new version to instance(s).
2020-08-04 14:49:45 ERROR [Instance: i-0719a2ed67b621907] Command failed on instance. An unexpected error has occurred [ErrorCode: 0000000001].
2020-08-04 14:49:45 INFO Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2020-08-04 14:49:45 ERROR Unsuccessful command execution on instance id(s) 'i-0719a2ed67b621907'. Aborting the operation.
2020-08-04 14:49:45 ERROR Failed to deploy application.
eb logs error content
2020/08/04 11:35:55.132182 ERROR Requirements file provided! Importing into Pipfile…
2020/08/04 11:35:56.227858 ERROR Error occurred during build: Command 01_migrate failed
2020/08/04 11:35:56.227884 ERROR An error occurred during execution of command app-deploy - PostBuildEbExtension. Stop running the command. Error: Container commands build failed. Please refer to /var/log/cfn-init.log for more details.
After checking the command 01_migrate failed message,
it seems that there is a problem with the continer commands, so I ran django-admin migrate.
django.core.exceptions.ImproperlyConfigured: Requested setting
DATABASES, but settings are not configured. You must either define the
environment variable DJANGO_SETTINGS_MODULE or call
settings.configure() before accessing settings.
I have defined environment variables in django.config, but there is an error, I cannot deploy my site
django-admin migrate is replaced by python manage.py migrate but still doesn't deploy
You can also find the detailed code on my GitHub
https://github.com/dopza86/air_bnb_clone
I don't know how to do it, I need your help,thank you

When your container_commands command run, you won't be in your python's environment. Also make sure that env variables are set before your container_commands run.
For this you can modify your .ebextentions:
01-env_var.config
option_settings:
aws:elasticbeanstalk:container:python:
WSGIPath: config.wsgi:application
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: config.settings
05-packages.config
packages:
yum:
postgresql-devel: []
10-django.config
container_commands:
01_migrate:
command: |
env # just to print out the env variables for checking in logs
source $PYTHONPATH/activate
django-admin.py migrate
leader_only: true
02_compilemessages:
command: |
env # just to print out the env variables for checking in logs
source $PYTHONPATH/activate
django-admin.py compilemessages

It seems like AWS has changed the way to customize the nginx reverse proxy and buried the documentation deep:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/platforms-linux-extend.html
In short, instead of using .ebextensions with a files: directive, one needs to create a nested path: .platform > .nginf > conf.d, and place there the nginx customization .conf files.

Heroku looking for Procfile in wrong directory

I'm trying to work with Django on Heroku and I'm following this tutorial with its Django template https://devcenter.heroku.com/articles/django-app-configuration
https://devcenter.heroku.com/articles/deploying-python
But when i run 'heroku local web' it always look in the wrong directory. I've tried to move the project
(env) D:\Study\Workbench\heroku-testing\testing\env\codeShareApp>heroku local web
[WARN] No ENV file found
[WARN] ENOENT: no such file or directory, open 'C:\Users\William\Procfile'
[FAIL] No Procfile and no package.json file found in Current Directory - See
run_foreman.js --help
The project directory is as follows
env/
codeShareApp/
.idea/
codeShareApp/
.env
manage.py
Procfile
requirements.txt
runtime
include/
Lib/
Scripts/
tcl/
EDIT:
I tried to make a new project with directory like below, but still got the same error
codeShareApp/
codeShareApp/
env/
.env
manage.py
Procfile
requirements.txt
runtime

I encounter this problem also in a project; I solved this by Making sure that the Procfile is using the correct syntax, and that there are no extra tabs or newlines in the file.
especially whitespcae.

I dont know why, but I tried it on another machine and it works just fine. It seems that it has something to do with heroku cli 6.13 because the error only occurs after it updates from version 5.12 to 6.12

The documentation says:
The file must be placed in the root directory of your application. It will not function if placed in a subdirectory.
Try to do this and it should work. See the docs for more information.

Running Django locally with heroku fails due to missing Procfile

I have a Django 1.11/Python 3.5 app that I built and want to run on Heroku locally. It's a simple SPA using the Heroku Django template provided on GitHub (https://github.com/heroku/heroku-django-template). I followed a Heroku tutorial (https://devcenter.heroku.com/articles/deploying-python#how-to-keep-build-artifacts-out-of-git), but I cannot seem to run the app locally using the following command:
heroku local web
Running this produces the following error:
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: EACCES: permission denied, open '.env'
at Object.fs.openSync (fs.js:584:18)
at Object.fs.readFileSync (fs.js:491:33)
at loadEnvsFile (/snap/heroku/414/lib/node_modules/heroku-cli/node_modules/foreman/lib/envs.js:133:15)
at Array.map (native)
at loadEnvs (/snap/heroku/414/lib/node_modules/heroku-cli/node_modules/foreman/lib/envs.js:148:30)
at Command.<anonymous> (/snap/heroku/414/lib/node_modules/heroku-cli/node_modules/foreman/nf.js:72:16)
at Command.listener (/snap/heroku/414/lib/node_modules/heroku-cli/node_modules/commander/index.js:301:8)
at emitTwo (events.js:106:13)
at Command.emit (events.js:194:7)
at Command.parseArgs (/snap/heroku/414/lib/node_modules/heroku-cli/node_modules/commander/index.js:615:12)
My .env file looks like this:
WEB_CONCURRENCY=3
SECRET_APP_KEY="xxxxxxxxxxxxxxxxxxxx"
I ran chmod 777 on .env but I get the same error.
When I run the following command:
heroku local
I get the following error:
[WARN] EACCES: permission denied, open 'Procfile'
[FAIL] No Procfile and no package.json file found in Current Directory - See run.js --help
▸ Cannot convert undefined or null to object
My Procfile looks like this:
web: gunicorn personal_website.wsgi
Now I cannot understand why when running "heroku local web" I get the previously mentioned error, especially after giving it the necessary permissions.
Also, others have had the same error when running "heroku local", but the answer is to "make sure the formatting of your Procfile is correct." Well mine is correct and have tried many variations to it.
I have seen others with this issue on SO and some have been resolved, unfortunately for me none of them worked.
What could I possibly be doing wrong here?

Running Django migrations when deploying to Elastic Beanstalk

I have my Django app set up on Elastic Beanstalk and recently made a change to the DB that I would like to have applied to the live DB now. I understand that I need to set this up as a container command, and after checking the DB I can see that the migration was run, but I can't figure out how to have more controls over the migration. For example, I only want a migration to run when necessary but from my understanding, the container will run the migration on every deploy assuming the command is still listed in the config file. Also, on occassion, I will be given options during a migration such as:
Any objects realted to these content types by a foreign key will also be deleted.
Are you sure you want to delete these content types?
If you're unsure, answer 'no'
How do I set up the container command to respond to this with a yes during the deployment phase?
This is my current config file
container_commands:
01_migrate:
command: 'source /opt/python/run/venv/bin/actiate && python app/manage.py makemigrations'
command: 'source /opt/python/run/venv/bin/activate && python app/manage.py migrate'
Is there a way to set these 2 commands to only run when necessary and to respond to the yes/no options I receive during a migration?

I'm not sure there is a specific way to answer yes or no. but you can append --noinput to your container command. Use the --noinput option to suppress all user prompting, such as “Are you sure?” confirmation messages.
try
command: 'source /opt/python/run/venv/bin/activate && python app/manage.py migrate --noinput'
OR..
You can ssh into your elasticbean instance and run your command manually.
Then you'll have more control over the migrations.
Install awsebcli with pip install awsebcli
Type eb ssh Your EnvironmentName
Navigate to your eb instance app directory with:
sudo -s
source /opt/python/run/venv/bin/activate
source /opt/python/current/env
cd /opt/python/current/app
then run your command.
./manage.py migrate
I hope this helps

Aside from the automatic migration that you can add to deploy script (which runs every time you update the environment, and may not be desirable if you have long running migration or other Django management commands), you can ssh into an EB instance to run migration manually.
Here is how to manually run migration (and any other Django management commands) while working with Amazon Linux 2 (Python 3.7, 3.8) created by Elastic Beanstalk:
First, from your EB cli: eb ssh to connect an instance.
The virtual environment can be activated by
source /var/app/venv/*/bin/activate
The manage.py can be ran by
python3 /var/app/current/manage.py
Now the only tricky bit is to get Elastic Beanstalk's environment variables. You can access them by /opt/elasticbeanstalk/bin/get-config, I'm not super familiar with bash script, but here is a little script that I use to get and set environment variables, maybe someone can improve it to make it less hard-coded:
#! /bin/bash
export DJANGO_SECRET_KEY=$(/opt/elasticbeanstalk/bin/get-config environment -k DJANGO_SECRET_KEY)
...
More info regarding Amazon Linux 2 splatform script tools: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/custom-platforms-scripts.html

Make sure that the same settings are used when migrating and running!
Thus I would recommend you change this kind of code in django.config
container_commands:
01_migrate:
command: "source /opt/python/run/venv/bin/activate && python manage.py migrate"
leader_only: true
to:
container_commands:
01_migrate:
command: "django-admin migrate"
leader_only: true
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: fund.productionSettings
as recommended here. This will help you avoid issues with wrong settings used.
More on manage.py v.s. django-admin.py.

django-admin method not working as it was not configured properly. You can also use python manage.py migrate in
.ebextentions/django.config
container_commands:
01_migrate:
command: "python manage.py migrate"
leader_only: true

In reference to Oscar Chen answer, you can set environmental variables using eb cli with
eb setenv key1=value1 key2=valu2 ...etc

The trick is that the full output of container_commands is in /var/log/cfn-init-cmd.log (Amazon Linux 2 Elastic Beanstalk released November 2020).
To view this you would run:
eb ssh [environment-name]
sudo tail -n 50 -f /var/log/cfn-init-cmd.log
This doesn't seem to be documented anywhere obvious and it's not displayed by eb logs; I found it by hunting around in /var/log.
The Django example management command django-admin.py migrate did not work for me. Instead I had to use something like:
01_migrate:
command: "$PYTHONPATH/python manage.py migrate"
leader_only: true
02_collectstatic:
command: "$PYTHONPATH/python manage.py collectstatic --noinput --verbosity=0 --clear"
To see the values of your environment variables at deploy time, you can create a debug command like:
03_debug:
command: "env"
You can see most of these environment variable with eb ssh; sudo cat /opt/elasticbeanstalk/deployment/env, but there seem to be some subtle differences at deploy time, hence using env above to be sure.
Here you'll see that $PYTHONPATH is being in a non-typical way, pointing to the virtualenv's bin directory, not the site-packages directory.

This answer looks like it will work for you if you just want to send "yes" to a few prompts.
You might also consider the --noinput flag so that your config looks like:
container_commands:
01_migrate:
command: 'source /opt/python/run/venv/bin/actiate && python app/manage.py makemigrations'
command: 'source /opt/python/run/venv/bin/activate && python app/manage.py migrate --noinput
This takes the default setting, which is "no".
It also appears that there's an open issue/fix to solve this problem a better way.

Create Django Superuser on AWS Elastic Beanstalk

I have a custom user class called MyUser. It works fine locally with registrations, logins and so on. I'm trying to deploy my application to AWS Elastic Beanstalk and I'm running into some problems with creating my superuser.
I tried making a script file and run it as the official AWS guide suggests. Didnt work well so I decided to try a secondary method suggested here and create a custom manage.py command to create my user.
When I deploy I get the following errors in the log.
[Instance: i-8a0a6d6e Module: AWSEBAutoScalingGroup ConfigSet: null] Command failed on instance. Return code: 1 Output: [CMD-AppDeploy/AppDeployStage0/EbExtensionPostBuild] command failed with error code 1: Error occurred during build: Command 02_createsu failed.
[2015-03-10T08:05:20.464Z] INFO [17937] : Command processor returning results:
{"status":"FAILURE","api_version":"1.0","truncated":"false","results":[{"status":"FAILURE","msg":"[CMD-AppDeploy/AppDeployStage0/EbExtensionPostBuild] command failed with error code 1: Error occurred during build: Command 02_createsu failed","returncode":1,"events":[]}]}
[2015-03-10T08:05:20.463Z] ERROR [17937] : Command execution failed: [CMD-AppDeploy/AppDeployStage0/EbExtensionPostBuild] command failed with error code 1: Error occurred during build: Command 02_createsu failed (ElasticBeanstalk::ActivityFatalError)
at /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.1/lib/elasticbeanstalk/activity.rb:189:in `rescue in exec'
...
caused by: command failed with error code 1: Error occurred during build: Command 02_createsu failed (Executor::NonZeroExitStatus)
The code looks like the following:
This is my mysite.config file in .ebextensions/
01_syncdb and 03_collectstatic works fine.
container_commands:
01_syncdb:
command: "django-admin.py migrate --noinput"
leader_only: true
02_createsu:
command: "manage.py createsu"
leader_only: true
03_collectstatic:
command: "django-admin.py collectstatic --noinput"
option_settings:
- namespace: aws:elasticbeanstalk:container:python
option_name: WSGIPath
value: treerating/wsgi.py
- option_name: DJANGO_SETTINGS_MODULE
value: treerating.settings
This is my /profiles/management/commands/createsu.py file:
from django.core.management.base import BaseCommand
from django.contrib.auth.models import User
from profiles.models import MyUser
class Command(BaseCommand):
def handle(self, *args, **options):
if MyUser.objects.count() == 0:
MyUser.objects.create_superuser("admin", "treerating", "password")
And I have __init__.py files in both /management/ and /commands/ folders.
I tried this command locally from command line and it works fine and creates the user without errors. So there shouldnt be any issue with the command itself or the MyUser.objects.create_superuser().
EDIT: I tried changing my def handle(): function to only set a variable to True and I still get the same errors. So it seems like the problem is not related to the create_superuser function or the handle, but more something with using manage.py.
Any ideas?
EDIT 2:
I tried executing the command by SSH and failed. I then followed the instructions in this post and set the Python Path's manually with:
source /opt/python/run/venv/bin/activate
and
source /opt/python/current/env
I was then able to successfully create my user.
The official AWS Django Deployment guide does not mention anything about this. But I guess you are suppose to set your Python Path's in the .config file somehow. I'm not sure exactly how to do this so if someone still want to answer that, I will test it and accept it as answer if that will solve the deployment errors.

Double-check the link to your secondary method. You can set the python path in the option settings (.ebextensions/02_python.config) that you've created:
option_settings:
"aws:elasticbeanstalk:application:environment":
DJANGO_SETTINGS_MODULE: "iotd.settings"
"PYTHONPATH": "/opt/python/current/app/iotd:$PYTHONPATH"
"ALLOWED_HOSTS": ".elasticbeanstalk.com"
However, I've done this and am still experiencing the issue you've described, so you'll have to see if it fixes it.
EDIT: It turns out my issue was a file structure issue. I had the management directory in the project directory, when it should have been placed one level deeper in the directory of one of my apps.
This placed it one level deeper beneath my manage.py and settings.py than is shown in the example, but it is working fine now.

I know this could be late but I just wanted to share that I solved this issue by adding the file /profiles/management/commands/createsu.py into the app folder you are using.
In my case was:
easy/easyapp/management/commands/createsu.py
where easy is my project and easyapp my app.

Another alternative that worked for me is to just go directly into the config.yml file and change the wsgi path there. You can get access with the eb config command and just go down 50 lines or so, make your changes, escape and save. This is only an environment-specific solution though.