When deploying a Django app using dokku I am getting a following error
Collectstatic configuration error. To debug, run:
$ heroku run python ./manage.py collectstatic --noinput
I found no way to run heroku run python ./manage.py collectstatic --noinput for a dokku container, but when I am trying dokku run my app python ./manage.py collectstatic --noinput, the static files are successfully copied to the STATIC_ROOT folder and no error message given.
I could solve the problem by placing collectstatic command into Procfile:
web: python manage.py collectstatic --noinput ; gunicorn myapp.wsgi
Still, I would love to know what was causing the problem and how can it be debugged. Any ideas?
You should have four settings in your settings.py file called MEDIA_ROOT, MEDIA_URL, STATIC_ROOT and STATIC_URL.
I set mine like so:
MEDIA_ROOT = 'media'
STATIC_ROOT = 'static'
MEDIA_URL = '/media'
STATIC_URL = '/static'
Inside the docker container that gets created, you will find your application under /app which makes the media path /app/media/ and the static path /app/static/.
Unfortunately if you don't have a media and static folder committed in git, it won't get created under /app automatically.
Since git doesn't allow you to commit an empty folder (it only commits files), I do the following in my projects:
mkdir media static
touch media/.dir
touch static/.dir
git add media/.dir static/.dir
git commit -m 'Make media and static directories'
The 'touch' command creates an empty file, then you 'git add' the two newly-created files and check them in.
Now when you push, the directories will be there to contain the media and static files. Just keep in mind that every time you 'git push', a new container is created, and the old one is destroyed. While this isn't a problem for your static files, your media will be lost unless you store it somewhere else.
Related
I have a django application thats is rendering the static files when run outside docker container. I added a Dockerfile so that I can run it in a container. However, the static files are not getting served when running in the container
The sample app that I have used can be found here
I have added the following Dockerfile at the root level
FROM python:3.8
#add project files to the usr/src/app folder
ADD . /usr/src/app
#set directoty where CMD will execute
WORKDIR /usr/src/app/wisdompets
COPY requirements.txt ./
# Get pip to download and install requirements:
RUN pip install --no-cache-dir -r requirements.txt
# Expose ports
EXPOSE 8000
# default command to execute
CMD exec gunicorn wisdompets.wsgi:application --bind 0.0.0.0:8000 --workers 3
And the requirements.txt file at root level as below
django==3.0.3
gunicorn==20.0.4
The static files settings in my settings.py is as follows
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static')
]
Please let me know if there's something wrong with my configuration
You are missing collectstatic command which will package static files alongside the app.
Check this link for reference.
Revisa keeps you configured.
STATIC_ROOT
You should have no problems when you run the command.
python manage.py \
collectstatic \
--verbosity 0 \
--noinput \
--clear
If it's fine but still doesn't render the statics.
Check the gunicorn documentation.
I recommend you to use uwsgi is much more elegant.
https://uwsgi-docs.readthedocs.io/en/latest/
I'm not able to get my django project to run with whitenoise and compressed staticfiles (including libsass). In links below, I read that it's only possible by offline compression of needed staticfiles. But when I started up the docker container, running compress command
docker-compose -f production.yml run --rm django python manage.py compress
gives me error:
ValueError: Missing staticfiles manifest entry for 'sass/app.scss'
While trying to request the site gives me following error (as expected?):
compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "..." is missing from offline manifest. You may need to run "python manage.py compress"
Settings are as follows (build with cookiecutter-django, see link for complete code base below):
STATIC_ROOT = str(ROOT_DIR("staticfiles"))
STATIC_URL = "/static/"
STATICFILES_DIRS = [str(APPS_DIR.path("static"))]
STATICFILES_FINDERS = [
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
STATICFILES_FINDERS += ["compressor.finders.CompressorFinder"]
COMPRESS_PRECOMPILERS = [("text/x-scss", "django_libsass.SassCompiler")]
COMPRESS_CACHEABLE_PRECOMPILERS = (("text/x-scss", "django_libsass.SassCompiler"),)
COMPRESS_ENABLED = env.bool("COMPRESS_ENABLED", default=True)
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage"
COMPRESS_URL = STATIC_URL
So after searching the internet for 1 day; I'm stuck... Thx for any help or suggestion!
Code base: https://github.com/rl-institut/E_Metrobus/tree/compress
which is build with cookiecutter-django-foundation
including the following changes to config/setttings/production.py:
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage" # Instead of pre-set "storages.backends.s3boto3.S3Boto3Storage"
COMPRESS_ROOT = STATIC_ROOT # Just in case
COMPRESS_OFFLINE = True # Needed to run compress offline
Possible related links:
Whitenoise and django-compressor cause 404 for compressed files
Possible to use WhiteNoise with Django-Compressor?
Django staticfiles not found on Heroku (with whitenoise)
https://github.com/django-compressor/django-compressor/issues/486
EDIT
Solved it using Justins answer (see below, with additonal changes).
My mistake was trying to compress files with already running container, giving me the error above. After changing Dockerfile with following lines (Notice the duplicate collectstatic cmd!):
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
python /app/manage.py collectstatic --noinput
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
and rebuilding image everything worked like a charm :)
Additionally, diverging from settings above, I had to set COMPRESS_ENABLED=True in my settings/env file.
I just had the same problem.
Add this to project/compose/production/django/start
python /app/manage.py compress --force
i.e.
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
this is weird but it work very well.
collect and compress static files by whitenoise
python manage.py collectstatic --clear
set COMPRESS_STORAGE = 'compressor.storage.BrotliCompressorFileStorage'
to make .br files in CACHE directory
python manage.py compress --force
set COMPRESS_STORAGE = 'compressor.storage.GzipCompressorFileStorage'
to make .gzfiles in CACHE directory
python manage.py compress --force
to add new compressed files to whitenoise: manifest.json, manifest.json.gz,
manifest.json.br
--no-post-process option is to tell whitenoise not to compress static files again.
python manage.py collectstatic --no-post-process
make sure to run the commands in order.
to test if whitenoise is working
python manage.py runserver --nostatic
I am trying to set up a development version of a Django Rest (1.11) application on a mac.
I have run the python manage.py collectstatic command and this resulted in the files being copied into a 'static' folder.
However when I run the application (python manage.py runserver) none of the static files load in the browser (no css). The Network tab shows the status of the static files as 404.
My settings.py file has the following:
STATIC_URL = '/static/'
STATIC_ROOT = BASE_DIR + '/abba_rest/static/'
Appreciate any advice on how to debug this issue.
I got the static files to load by using the --insecure flag when starting the server;
python manage.py runserver --insecure
Using Django 1.11.6, python 3.6.2
Short summary: static files are collected correctly on server but only one app's sub directory is served, the admin's static files return as 404.
Running the local development server everything works as expected. After deploying to my EB instance none of the static files for the admin app load. If i run eb ssh and look around on the server I can see the collected static folder, along with all the different app's subfolders. The files are all there where I'd expect. The links produced on the django front-end map in a sensible way to these paths, it's just that anything in static/admin returns a 404, while the files in static/entities work fine.
Testing paths on the front end:
/static/admin/css/base.css (404)
/static/entities/style.css (works)
On server, all collected files in static look fine:
/opt/python/current/app/static/admin/css
/opt/python/current/app/static/entities
No idea why one sub directory is server fine but the others return 404s.
From my django.config
option_settings:
"aws:elasticbeanstalk:application:environment":
DJANGO_SETTINGS_MODULE: "spf.settings"
"PYTHONPATH": "/opt/python/current/app:$PYTHONPATH"
"ALLOWED_HOSTS": ".elasticbeanstalk.com"
"aws:elasticbeanstalk:container:python":
WSGIPath: spf/wsgi.py
NumProcesses: 3
NumThreads: 20
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "static/"
...
03_collectstatic:
command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
Running the collectstatic command locally produces the correct static folder with the same subdirectories for each app too.
Sheepish self-answer here:
In the AWS configuration for my eb instance I had set some static file aliases to serve directly from one of my sub apps. (Look under configuration->Software Configuration->Static Files). These aliases override any wsgi settings you may have in your django application. :)
I have a basic django app (Newsdiffs)that runs just fine at localhost:8000 with python website/manage.py runserver but I'd like to migrate it to Heroku and I can't figure out what my next step is.
I thought getting it running locally would translate to running it on Heroku, but I'm realizing that python website/manage.py runserver is launching the dev settings and I'm not sure how to tell it to use the main settings.
All that is in my Procfile is this:
web: python website/manage.py runserver
Locally, that works fine, though it launches it at http://127.0.0.1:8000/ which is probably not what I want on Heroku. So how do I figure out where to set the hostname and port? I don't see either in the app anyplace.
I have just drawn this list for myself two days ago.
It was put together after having followed the steps described in Heroku's help pages for python.
It's by no means definitive nor perfect, and it will change, but it's a valid trace, since I was able to put the site online.
Some issues remain, to be checked thoroughly, e.g. the location of the media/ directory where files are uploaded should/could live outside your project for security reasons (now it works, but I have noticed if the dyno sleeps then the files are not reached/displayed by the template later).
The same goes for the staticfiles/ directory (although this one seems to work fine).
Also, you might want to set django's debug mode to false.
So here it is:
My first steps to deploy an EXISTING django application to Heroku
ASSUMPTIONS:
a) your django project is in a virtual environment already
b) you have already collected all your project's required packages with
pip freeze > requirements.txt
and committed it to git
git add requirements.txt
git commit -m 'my prj requirements'
0) Activate your project's virtual environment
workon xyz #using virtualenvwrapper
then go to your django project's directory (DPD for short) if not already taken there
cd ~/prj/xyz (or cdproject with virtualenvwrapper if setup properly)
and create a new git branch for heroku twiddling to prevent messing things up
git checkout -b he
1) Create the app on heroku
heroku create xyz
that also adds heroku as a remote of your repo
2) Add the needed packages to requirements.txt
vi requirements.txt
add
dj-database-url==0.3.0
django-postgrespool==0.3.0
gunicorn==19.3.0
psycopg2==2.6
django-toolbelt==0.0.1
static3==0.5.1
whitenoise==2.0.3
3) Install all dependencies in the local venv
pip install -r requirements.txt --allow-all-external
4) Setup the heroku django settings
cd xyz
create a copy
cp setting.py settings_heroku.py
and edit it
vi settings_heroku.py
import os
import dj_database_url
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'), )
MEDIA_ROOT = os.path.join(BASE_DIR, "media")
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
SECRET_KEY = os.environ["DJANGO_SECRET_KEY"]
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
replace django's std db cfg with
DATABASES['default'] = dj_database_url.config()
DATABASES['default']['ENGINE'] = 'django_postgrespool'
and
WSGI_APPLICATION = 'xyz.wsgi_heroku.application'
5) Configure the necessary environment variables (heroku configs)
edit the .env file
vi .env
e.g.
DJANGO_SECRET_KEY=whatever
EMAIL_HOST_USER=youruser#gmail.com
EMAIL_HOST_PASSWORD=whateveritis
and/or set them manually if needed (in my case .env had no effect, wasn't loaded apparently, and had to set the vars manually for now)
heroku config:set DJANGO_SECRET_KEY=whatever
heroku config:set EMAIL_HOST_USER=youruser#gmail.com
heroku config:set EMAIL_HOST_PASSWORD=whateveritis
6) Create a separate wsgi file for heroku
cd xyx
cp wsgi.py wsgi_heroku.py
and edit it to make it point to the right settings
vi wsgi_heroku.py
from whitenoise.django import DjangoWhiteNoise
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "xyz.settings_heroku")
application = get_wsgi_application()
application = DjangoWhiteNoise(application)
7) Make sure all the templates use
{% load staticfiles %}
8) Define the Procfile file so that it points to the right wsgi
e.g.
cd ~/prj/xyz (DPD)
vi Procfile
add
web: gunicorn xyz.wsgi_heroku --log-file -
9) Collect all static content into DPD/staticfiles/
locally, make sure django points to the right wsgi settings
export WSGI_APPLICATION=blogger.wsgi_heroku.application
python manage.py collectstatic
10) add the changes to the local git repo (he branch)
git add --all .
git commit -m 'first 4 heroku'
11) check the whole thing works locally
heroku local # in heroku's help they also add `web`, not needed?!
12) push your code to heroku
git push heroku he:master
13) make sure a instance of the app is running
heroku ps:scale web=1
14) create the tables on the heroku DB
heroku run python manage.py migrate
Note: if you see a message that says, “You just installed Django’s auth system, which means you don’t have any superusers defined. Would you like to create one now?”, type no.
15) add the superuser to the heroku DB
heroku run bash
python manage.py createsuperuser
and fill in the details, as usual
16) Populate the DB with the necessary fixtures
heroku run python manage.py loaddata yourfile.json
17) Visit the website page on heroku's webserver
heroku open
or go to
https://xyz.herokuapp.com/
and the admin
https://xyz.herokuapp.com/admin/
and the DB
https://xyz.herokuapp.com/db
Useful commands:
View the app's logs
heroku logs [--tail]
List add-ons deployed
heroku addons
and use one:
heroku addons:open <add-on-name>
Run a command on heroku (the remote env, where you are deploying)
heroku run python manage.py shell
heroku run bash
Set a config var on Heroku
heroku config:set VARNAME=whatever
View the config vars that are set (including the DB's)
heroku config
View postgres DB details
heroku pg
If you know some python and have a lot of experience building web apps in other languages but don't totally understand where Heroku fits, I highly recommend Discover Flask, which patched a lot of the holes in my understanding of how these pieces all fit together.
Some of the things that I worked out:
you really do need an isolated virtual environment if you're going to deploy to Heroku, because Heroku installs Python modules from the requirements.txt file.
Gunicorn is a web server, and you definitely need to run your app under Gunicorn or it won't run on Heroku.
The "Procfile" doesn't just give the command you use to run the app locally. And Heroku requires it. So if you've got an app that was built to run on Heroku and it doesn't include a Procfile, they left something out.
You don't tell Heroku what your hostname is. When you run heroku create it should tell you what your domain name is going to be. And every time you run git push heroku master (or whatever branch you're pushing, maybe it isn't master), Heroku will (try to) restart your app.
Heroku doesn't support sqlite. You have to run your Production DB in Postgres.
This doesn't directly answer my question, but it does fill in some of the missing pieces that were making it hard for me to even ask the right question. RTFM notwithstanding. :)