Django Whitenoise with compressed staticfiles - python

I'm not able to get my django project to run with whitenoise and compressed staticfiles (including libsass). In links below, I read that it's only possible by offline compression of needed staticfiles. But when I started up the docker container, running compress command
docker-compose -f production.yml run --rm django python manage.py compress
gives me error:
ValueError: Missing staticfiles manifest entry for 'sass/app.scss'
While trying to request the site gives me following error (as expected?):
compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "..." is missing from offline manifest. You may need to run "python manage.py compress"
Settings are as follows (build with cookiecutter-django, see link for complete code base below):
STATIC_ROOT = str(ROOT_DIR("staticfiles"))
STATIC_URL = "/static/"
STATICFILES_DIRS = [str(APPS_DIR.path("static"))]
STATICFILES_FINDERS = [
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
STATICFILES_FINDERS += ["compressor.finders.CompressorFinder"]
COMPRESS_PRECOMPILERS = [("text/x-scss", "django_libsass.SassCompiler")]
COMPRESS_CACHEABLE_PRECOMPILERS = (("text/x-scss", "django_libsass.SassCompiler"),)
COMPRESS_ENABLED = env.bool("COMPRESS_ENABLED", default=True)
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage"
COMPRESS_URL = STATIC_URL
So after searching the internet for 1 day; I'm stuck... Thx for any help or suggestion!
Code base: https://github.com/rl-institut/E_Metrobus/tree/compress
which is build with cookiecutter-django-foundation
including the following changes to config/setttings/production.py:
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage" # Instead of pre-set "storages.backends.s3boto3.S3Boto3Storage"
COMPRESS_ROOT = STATIC_ROOT # Just in case
COMPRESS_OFFLINE = True # Needed to run compress offline
Possible related links:
Whitenoise and django-compressor cause 404 for compressed files
Possible to use WhiteNoise with Django-Compressor?
Django staticfiles not found on Heroku (with whitenoise)
https://github.com/django-compressor/django-compressor/issues/486
EDIT
Solved it using Justins answer (see below, with additonal changes).
My mistake was trying to compress files with already running container, giving me the error above. After changing Dockerfile with following lines (Notice the duplicate collectstatic cmd!):
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
python /app/manage.py collectstatic --noinput
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
and rebuilding image everything worked like a charm :)
Additionally, diverging from settings above, I had to set COMPRESS_ENABLED=True in my settings/env file.

I just had the same problem.
Add this to project/compose/production/django/start
python /app/manage.py compress --force
i.e.
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app

this is weird but it work very well.
collect and compress static files by whitenoise
python manage.py collectstatic --clear
set COMPRESS_STORAGE = 'compressor.storage.BrotliCompressorFileStorage'
to make .br files in CACHE directory
python manage.py compress --force
set COMPRESS_STORAGE = 'compressor.storage.GzipCompressorFileStorage'
to make .gzfiles in CACHE directory
python manage.py compress --force
to add new compressed files to whitenoise: manifest.json, manifest.json.gz,
manifest.json.br
--no-post-process option is to tell whitenoise not to compress static files again.
python manage.py collectstatic --no-post-process
make sure to run the commands in order.
to test if whitenoise is working
python manage.py runserver --nostatic

Related

Use npm library for django on runserver environment

I made static folder in my root project.
in settins.py
STATIC_URL = '/static/'
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'static'),
)
Install npm library under static directory.
myproject/static$ npm -i js-cookie
then There comes the files like this
myproject/static/node_modules/js.cookie/dist/js.cookie.min.js
then run server
myproject$ python manage.py runserver
then use in template.html
However this shows the error
GET http://localhost:8000/static/node_modules/js.cookie/dist/js.cookie.min.js net::ERR_ABORTED 404 (Not Found)
I have two quesitons.
Is it good practice to use npm library under django project??
Maybe runserver doesn't use mypoject/static?

Django static files not serving when running inside Docker container

I have a django application thats is rendering the static files when run outside docker container. I added a Dockerfile so that I can run it in a container. However, the static files are not getting served when running in the container
The sample app that I have used can be found here
I have added the following Dockerfile at the root level
FROM python:3.8
#add project files to the usr/src/app folder
ADD . /usr/src/app
#set directoty where CMD will execute
WORKDIR /usr/src/app/wisdompets
COPY requirements.txt ./
# Get pip to download and install requirements:
RUN pip install --no-cache-dir -r requirements.txt
# Expose ports
EXPOSE 8000
# default command to execute
CMD exec gunicorn wisdompets.wsgi:application --bind 0.0.0.0:8000 --workers 3
And the requirements.txt file at root level as below
django==3.0.3
gunicorn==20.0.4
The static files settings in my settings.py is as follows
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static')
]
Please let me know if there's something wrong with my configuration
You are missing collectstatic command which will package static files alongside the app.
Check this link for reference.
Revisa keeps you configured.
STATIC_ROOT
You should have no problems when you run the command.
python manage.py \
collectstatic \
--verbosity 0 \
--noinput \
--clear
If it's fine but still doesn't render the statics.
Check the gunicorn documentation.
I recommend you to use uwsgi is much more elegant.
https://uwsgi-docs.readthedocs.io/en/latest/

Static files not loading (Django 1.11)

I am trying to set up a development version of a Django Rest (1.11) application on a mac.
I have run the python manage.py collectstatic command and this resulted in the files being copied into a 'static' folder.
However when I run the application (python manage.py runserver) none of the static files load in the browser (no css). The Network tab shows the status of the static files as 404.
My settings.py file has the following:
STATIC_URL = '/static/'
STATIC_ROOT = BASE_DIR + '/abba_rest/static/'
Appreciate any advice on how to debug this issue.
I got the static files to load by using the --insecure flag when starting the server;
python manage.py runserver --insecure

How to deploy / migrate an existing django app / project to a production server on Heroku?

I have a basic django app (Newsdiffs)that runs just fine at localhost:8000 with python website/manage.py runserver but I'd like to migrate it to Heroku and I can't figure out what my next step is.
I thought getting it running locally would translate to running it on Heroku, but I'm realizing that python website/manage.py runserver is launching the dev settings and I'm not sure how to tell it to use the main settings.
All that is in my Procfile is this:
web: python website/manage.py runserver
Locally, that works fine, though it launches it at http://127.0.0.1:8000/ which is probably not what I want on Heroku. So how do I figure out where to set the hostname and port? I don't see either in the app anyplace.
I have just drawn this list for myself two days ago.
It was put together after having followed the steps described in Heroku's help pages for python.
It's by no means definitive nor perfect, and it will change, but it's a valid trace, since I was able to put the site online.
Some issues remain, to be checked thoroughly, e.g. the location of the media/ directory where files are uploaded should/could live outside your project for security reasons (now it works, but I have noticed if the dyno sleeps then the files are not reached/displayed by the template later).
The same goes for the staticfiles/ directory (although this one seems to work fine).
Also, you might want to set django's debug mode to false.
So here it is:
My first steps to deploy an EXISTING django application to Heroku
ASSUMPTIONS:
a) your django project is in a virtual environment already
b) you have already collected all your project's required packages with
pip freeze > requirements.txt
and committed it to git
git add requirements.txt
git commit -m 'my prj requirements'
0) Activate your project's virtual environment
workon xyz #using virtualenvwrapper
then go to your django project's directory (DPD for short) if not already taken there
cd ~/prj/xyz (or cdproject with virtualenvwrapper if setup properly)
and create a new git branch for heroku twiddling to prevent messing things up
git checkout -b he
1) Create the app on heroku
heroku create xyz
that also adds heroku as a remote of your repo
2) Add the needed packages to requirements.txt
vi requirements.txt
add
dj-database-url==0.3.0
django-postgrespool==0.3.0
gunicorn==19.3.0
psycopg2==2.6
django-toolbelt==0.0.1
static3==0.5.1
whitenoise==2.0.3
3) Install all dependencies in the local venv
pip install -r requirements.txt --allow-all-external
4) Setup the heroku django settings
cd xyz
create a copy
cp setting.py settings_heroku.py
and edit it
vi settings_heroku.py
import os
import dj_database_url
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'), )
MEDIA_ROOT = os.path.join(BASE_DIR, "media")
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
SECRET_KEY = os.environ["DJANGO_SECRET_KEY"]
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
replace django's std db cfg with
DATABASES['default'] = dj_database_url.config()
DATABASES['default']['ENGINE'] = 'django_postgrespool'
and
WSGI_APPLICATION = 'xyz.wsgi_heroku.application'
5) Configure the necessary environment variables (heroku configs)
edit the .env file
vi .env
e.g.
DJANGO_SECRET_KEY=whatever
EMAIL_HOST_USER=youruser#gmail.com
EMAIL_HOST_PASSWORD=whateveritis
and/or set them manually if needed (in my case .env had no effect, wasn't loaded apparently, and had to set the vars manually for now)
heroku config:set DJANGO_SECRET_KEY=whatever
heroku config:set EMAIL_HOST_USER=youruser#gmail.com
heroku config:set EMAIL_HOST_PASSWORD=whateveritis
6) Create a separate wsgi file for heroku
cd xyx
cp wsgi.py wsgi_heroku.py
and edit it to make it point to the right settings
vi wsgi_heroku.py
from whitenoise.django import DjangoWhiteNoise
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "xyz.settings_heroku")
application = get_wsgi_application()
application = DjangoWhiteNoise(application)
7) Make sure all the templates use
{% load staticfiles %}
8) Define the Procfile file so that it points to the right wsgi
e.g.
cd ~/prj/xyz (DPD)
vi Procfile
add
web: gunicorn xyz.wsgi_heroku --log-file -
9) Collect all static content into DPD/staticfiles/
locally, make sure django points to the right wsgi settings
export WSGI_APPLICATION=blogger.wsgi_heroku.application
python manage.py collectstatic
10) add the changes to the local git repo (he branch)
git add --all .
git commit -m 'first 4 heroku'
11) check the whole thing works locally
heroku local # in heroku's help they also add `web`, not needed?!
12) push your code to heroku
git push heroku he:master
13) make sure a instance of the app is running
heroku ps:scale web=1
14) create the tables on the heroku DB
heroku run python manage.py migrate
Note: if you see a message that says, “You just installed Django’s auth system, which means you don’t have any superusers defined. Would you like to create one now?”, type no.
15) add the superuser to the heroku DB
heroku run bash
python manage.py createsuperuser
and fill in the details, as usual
16) Populate the DB with the necessary fixtures
heroku run python manage.py loaddata yourfile.json
17) Visit the website page on heroku's webserver
heroku open
or go to
https://xyz.herokuapp.com/
and the admin
https://xyz.herokuapp.com/admin/
and the DB
https://xyz.herokuapp.com/db
Useful commands:
View the app's logs
heroku logs [--tail]
List add-ons deployed
heroku addons
and use one:
heroku addons:open <add-on-name>
Run a command on heroku (the remote env, where you are deploying)
heroku run python manage.py shell
heroku run bash
Set a config var on Heroku
heroku config:set VARNAME=whatever
View the config vars that are set (including the DB's)
heroku config
View postgres DB details
heroku pg
If you know some python and have a lot of experience building web apps in other languages but don't totally understand where Heroku fits, I highly recommend Discover Flask, which patched a lot of the holes in my understanding of how these pieces all fit together.
Some of the things that I worked out:
you really do need an isolated virtual environment if you're going to deploy to Heroku, because Heroku installs Python modules from the requirements.txt file.
Gunicorn is a web server, and you definitely need to run your app under Gunicorn or it won't run on Heroku.
The "Procfile" doesn't just give the command you use to run the app locally. And Heroku requires it. So if you've got an app that was built to run on Heroku and it doesn't include a Procfile, they left something out.
You don't tell Heroku what your hostname is. When you run heroku create it should tell you what your domain name is going to be. And every time you run git push heroku master (or whatever branch you're pushing, maybe it isn't master), Heroku will (try to) restart your app.
Heroku doesn't support sqlite. You have to run your Production DB in Postgres.
This doesn't directly answer my question, but it does fill in some of the missing pieces that were making it hard for me to even ask the right question. RTFM notwithstanding. :)

Collectstatic configuration error when deploying Django app with dokku

When deploying a Django app using dokku I am getting a following error
Collectstatic configuration error. To debug, run:
$ heroku run python ./manage.py collectstatic --noinput
I found no way to run heroku run python ./manage.py collectstatic --noinput for a dokku container, but when I am trying dokku run my app python ./manage.py collectstatic --noinput, the static files are successfully copied to the STATIC_ROOT folder and no error message given.
I could solve the problem by placing collectstatic command into Procfile:
web: python manage.py collectstatic --noinput ; gunicorn myapp.wsgi
Still, I would love to know what was causing the problem and how can it be debugged. Any ideas?
You should have four settings in your settings.py file called MEDIA_ROOT, MEDIA_URL, STATIC_ROOT and STATIC_URL.
I set mine like so:
MEDIA_ROOT = 'media'
STATIC_ROOT = 'static'
MEDIA_URL = '/media'
STATIC_URL = '/static'
Inside the docker container that gets created, you will find your application under /app which makes the media path /app/media/ and the static path /app/static/.
Unfortunately if you don't have a media and static folder committed in git, it won't get created under /app automatically.
Since git doesn't allow you to commit an empty folder (it only commits files), I do the following in my projects:
mkdir media static
touch media/.dir
touch static/.dir
git add media/.dir static/.dir
git commit -m 'Make media and static directories'
The 'touch' command creates an empty file, then you 'git add' the two newly-created files and check them in.
Now when you push, the directories will be there to contain the media and static files. Just keep in mind that every time you 'git push', a new container is created, and the old one is destroyed. While this isn't a problem for your static files, your media will be lost unless you store it somewhere else.

Categories