I'm not able to get my django project to run with whitenoise and compressed staticfiles (including libsass). In links below, I read that it's only possible by offline compression of needed staticfiles. But when I started up the docker container, running compress command
docker-compose -f production.yml run --rm django python manage.py compress
gives me error:
ValueError: Missing staticfiles manifest entry for 'sass/app.scss'
While trying to request the site gives me following error (as expected?):
compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "..." is missing from offline manifest. You may need to run "python manage.py compress"
Settings are as follows (build with cookiecutter-django, see link for complete code base below):
STATIC_ROOT = str(ROOT_DIR("staticfiles"))
STATIC_URL = "/static/"
STATICFILES_DIRS = [str(APPS_DIR.path("static"))]
STATICFILES_FINDERS = [
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
STATICFILES_FINDERS += ["compressor.finders.CompressorFinder"]
COMPRESS_PRECOMPILERS = [("text/x-scss", "django_libsass.SassCompiler")]
COMPRESS_CACHEABLE_PRECOMPILERS = (("text/x-scss", "django_libsass.SassCompiler"),)
COMPRESS_ENABLED = env.bool("COMPRESS_ENABLED", default=True)
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage"
COMPRESS_URL = STATIC_URL
So after searching the internet for 1 day; I'm stuck... Thx for any help or suggestion!
Code base: https://github.com/rl-institut/E_Metrobus/tree/compress
which is build with cookiecutter-django-foundation
including the following changes to config/setttings/production.py:
COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage" # Instead of pre-set "storages.backends.s3boto3.S3Boto3Storage"
COMPRESS_ROOT = STATIC_ROOT # Just in case
COMPRESS_OFFLINE = True # Needed to run compress offline
Possible related links:
Whitenoise and django-compressor cause 404 for compressed files
Possible to use WhiteNoise with Django-Compressor?
Django staticfiles not found on Heroku (with whitenoise)
https://github.com/django-compressor/django-compressor/issues/486
EDIT
Solved it using Justins answer (see below, with additonal changes).
My mistake was trying to compress files with already running container, giving me the error above. After changing Dockerfile with following lines (Notice the duplicate collectstatic cmd!):
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
python /app/manage.py collectstatic --noinput
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
and rebuilding image everything worked like a charm :)
Additionally, diverging from settings above, I had to set COMPRESS_ENABLED=True in my settings/env file.
I just had the same problem.
Add this to project/compose/production/django/start
python /app/manage.py compress --force
i.e.
python /app/manage.py collectstatic --noinput
python /app/manage.py compress --force
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
this is weird but it work very well.
collect and compress static files by whitenoise
python manage.py collectstatic --clear
set COMPRESS_STORAGE = 'compressor.storage.BrotliCompressorFileStorage'
to make .br files in CACHE directory
python manage.py compress --force
set COMPRESS_STORAGE = 'compressor.storage.GzipCompressorFileStorage'
to make .gzfiles in CACHE directory
python manage.py compress --force
to add new compressed files to whitenoise: manifest.json, manifest.json.gz,
manifest.json.br
--no-post-process option is to tell whitenoise not to compress static files again.
python manage.py collectstatic --no-post-process
make sure to run the commands in order.
to test if whitenoise is working
python manage.py runserver --nostatic
I'm trying to deploy a Django app to Heroku, it starts to build, download and installs everything, but that's what I get when it comes to collecting static files
$ python manage.py collectstatic --noinput
remote: Traceback (most recent call last):
remote: File "manage.py", line 10, in <module>
remote: execute_from_command_line(sys.argv)
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
remote: utility.execute()
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
remote: self.fetch_command(subcommand).run_from_argv(self.argv)
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
remote: self.execute(*args, **cmd_options)
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
remote: output = self.handle(*args, **options)
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 168, in handle
remote: collected = self.collect()
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 98, in collect
remote: for path, storage in finder.list(self.ignore_patterns):
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/finders.py", line 112, in list
remote: for path in utils.get_files(storage, ignore_patterns):
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/staticfiles/utils.py", line 28, in get_files
remote: directories, files = storage.listdir(location)
remote: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/files/storage.py", line 300, in listdir
remote: for entry in os.listdir(path):
remote: OSError: [Errno 2] No such file or directory: '/app/blogproject/static'
remote:
remote: ! Error while running '$ python manage.py collectstatic --noinput'.
remote: See traceback above for details.
remote:
remote: You may need to update application code to resolve this error.
remote: Or, you can disable collectstatic for this application:
remote:
remote: $ heroku config:set DISABLE_COLLECTSTATIC=1
remote:
remote: https://devcenter.heroku.com/articles/django-assets
remote:
remote: ! Push rejected, failed to compile Python app
remote:
remote: Verifying deploy...
remote:
remote: ! Push rejected to pin-a-voyage.
This is the whole settings.py file
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
import os
import dj_database_url
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.8/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '*********************'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
# Application definition
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'blog',
'custom_user',
'django_markdown',
'parsley',
)
#### AUTH ###
AUTH_USER_MODEL = 'custom_user.CustomUser'
AUTHENTICATION_BACKENDS = (
'custom_user.backends.CustomUserAuth',
'django.contrib.auth.backends.ModelBackend',
# 'django.contrib.auth.backends.RemoteUserBackend',
)
#############
#### EMAIL ###
EMAIL_USE_TLS = True
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_PASSWORD = '***' #my gmail password
EMAIL_HOST_USER = 'voyage.pin#gmail.com' #my gmail username
DEFAULT_FROM_EMAIL = 'voyage.pin#gmail.com'
SERVER_EMAIL = 'voyage.pin#gmail.com'
EMAIL_PORT = 587
DEFAULT_FROM_EMAIL = EMAIL_HOST_USER
##############
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
)
ROOT_URLCONF = 'blogproject.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'blogproject.wsgi.application'
# Database
# https://docs.djangoproject.com/en/1.8/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'blogproject',
'USER': '***',
'PASSWORD': '***',
'HOST': 'localhost',
'PORT': '',
}
}
# Internationalization
# https://docs.djangoproject.com/en/1.8/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Update database configuration with $DATABASE_URL.
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)
# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# Allow all host headers
ALLOWED_HOSTS = ['*']
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.8/howto/static-files/
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles')
STATIC_URL = '/static/'
# Extra places for collectstatic to find static files.
STATICFILES_DIRS = (
os.path.join(PROJECT_ROOT, 'static'),
)
# Simplified static file serving.
# https://warehouse.python.org/project/whitenoise/
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
This is the structure of the project
blog-project -- blog -- migrations
-- static
-- templates
-- blogproject
-- blogprojectenv
-- custom_user
-- media
-- .git
Any thoughts?
I just updated to Django 1.10 today and had the exact same problem.
Your static settings are identical to mine as well.
This worked for me, run the following commands:
disable the collectstatic during a deploy
heroku config:set DISABLE_COLLECTSTATIC=1
deploy
git push heroku master
run migrations (django 1.10 added at least one)
heroku run python manage.py migrate
run collectstatic using bower
heroku run 'bower install --config.interactive=false;grunt prep;python manage.py collectstatic --noinput'
enable collecstatic for future deploys
heroku config:unset DISABLE_COLLECTSTATIC
try it on your own (optional)
heroku run python manage.py collectstatic
future deploys should work as normal from now on
You have STATICFILES_DIRS configured to expect a static directory in the same directory as your settings.py file, so make sure it's there not somewhere else.
Also, do you have any files in that static directory? If you don't then git won't track it and so although it exists locally it won't exist in git. The usual solution to this is to create an empty file called .keep in the directory which will ensure that git tracks it. But once you have some static files in this directory then it won't be a problem anymore.
DO NOT disable collectstatic on heroku with heroku config:set DISABLE_COLLECTSTATIC=1. This will just hide the error and not make your app healthy.
Instead, it's better to understand why the collectstatic command fails because it means something is not right with your settings.
Step 1
Run locally both commands:
python manage.py collectstatic
python manage.py test
You should see one or more error messages. Most of the time, it's a missing variable (for ex: STATIC_ROOT) you must add to your project settings.py file.
It's necessary to add the test command because some collectstatic related issues will only surface with test, such as this one
Step 2
Once you've fixed all the error messages locally, push again to heroku.
Troubleshooting
Remember you can also run commands directly in your heroku VM.
If you cannot reproduce locally, run the collecstatic command in heroku and check what's going on directly in your production environment:
python manage.py collectstatic --dry-run --noinput
(Same goes for heroku console obviously)
Run python manage.py collectstatic locally and fix any errors. In my case there were reference errors that prevented that command from running successfully.
if you use django-heroku library
maybe you forget for put this setting in the bottom of line text settings.py for can possible read all config parameters
import django_heroku
django_heroku.settings(locals())
as like as the documentation:
Usage of Django-Heroku
In settings.py, at the very bottom::
…
# Configure Django App for Heroku.
import django_heroku
django_heroku.settings(locals())
This will automatically configure DATABASE_URL, ALLOWED_HOSTS, WhiteNoise (for static assets), Logging, and Heroku CI for your application.
p.s: sorry for my bad english
This error has occurred because you do not have staticfiles in
your Project's Root Directory.
Don't worry. The solution is SIMPLE.
You only need TWO STEPS.
Step 1: Open your settings.py file and write
import os
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent.parent
STATIC_ROOT = BASE_DIR / 'staticfiles'
Step 2: Run below given command in terminal in your Project's root directory.
(If you are using any Virtual environment for Django Project then go inside your Virtual environment and then go into your Project's root directory and then run below given command.)
python manage.py collectstatic
Congrats : Your Problem is Solved.
Now, you can COMMIT and PUSH your "changes" so that it is reflected in your Repository and then you are good to go.
This worked for me:
step 1 - heroku config:set DISABLE_COLLECTSTATIC=1
step 2 - git push heroku master
I face same problem..
Follow this step
heroku config:set DISABLE_COLLECTSTATIC=1
git push heroku master
python manage.py collectstatic
python manage.py test
If any error occurred after running test..check your
STATIC_ROOT is correct like this ==> STATIC_ROOT = os.path.join(BASE_DIR, 'static').
After run collectstatic command check all static files are
store in static directory for your root dir. level(manage.py
dir. level)...
heroku run python manage.py collectstatic.
heroku run python manage.py migrate
heroku config:unset DISABLE_COLLECTSTATIC (for future use).
Heroku had made a document with suggestions on how to handle this https://devcenter.heroku.com/articles/django-assets
add to settings.py
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'static'),
)
make a directory in the root of your project called staticfiles, put a favicon or something in there, just make sure git tracks it. Then the collectstatic command should finish on heroku.
Ran to that issue after trying to deploy an app again. The problem got cured after I specified these commands:
$ heroku config:set SECRET_KEY="*secret_key*"
$ heroku config:set DEBUG_VALUE="True"
$ heroku config:set EMAIL_USER="*user-email*"
$ heroku config:set EMAIL_PASS="*pass*"
These variables in settings.py were invoked with local environment variables,
which heroku didn't have on its environment, hence the error.
It seems to me that it's having problems creating that blogproject/static folder. I see you have a static folder inside your blog app, but it should be up one level in your blogproject folder.
Try creating a static folder inside your blogproject folder and that error should go away.
Today, not all of the requirements came in properly with $ pipenv install django from the heroku-django-template and $ pip install -r requirements.txt.
The latest version of the template includes a /static folder with a humans.txt, so the previous solution is likely not the proplem
Try running $ pipenv install whitenoise and then $ pip freeze > requirements.txt.
If that works, I would recommend $ pip install psycopg2 --ignore-installed and $ pip freeze > requirements.txt as well, otherwise you will similarly have problems migrating.
I faced the same issue while deploying my app. I realized I had updated my pip version, installed few plugins but forgot to create a fresh requirements.txt file.
Run pip freeze > requirements.txt in your terminal
Run python manage.py collectstatic
Now push the code to github and deploy to heroku server
Hope this helps if that is the case
insert this line of code to your setting.py file.
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
In my case was an error almost like described above, after push-ing that resulted in errors, I set the SECRET_KEY "heroku config:set SECRET_KEY='*************************'",
git push heroku main (again)
,
heroku run python manage.py migrate
,
heroku run python manage.py createsuperuser .. and everything
,
heroku open
and it worked :)
removing STATICFILES_DIRS worked in my case
heroku config:set DISABLE_COLLECTSTATIC=1 --app #yourappname
Just run the command
This problem occurs because Heroku tries to run manage.py.
While executing manage.py
we have to write like
python manage.py 'some_command'
But Heroku tries it as
python manage.py --noinput
So in this case we can make changes to our manage.py file:
Initialy it looks like this:
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault('DJANGO_SETTINGS_MODULE',
'your_project.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv) # just put this in try block
if __name__ == '__main__':
main()
So we change our main.py to:
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault('DJANGO_SETTINGS_MODULE',
'your_project.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
try:
execute_from_command_line(sys.argv) # just put this in try block
except:
pass
if __name__ == '__main__':
main()
If you used .env files and python-decouple you'll have to define the environmental variables in Heroku app settings > Config Vars. Otherwise collectstatic won't work.
After testing everything that was posted on this thread, here's what worked for me:
Keep the Heroku environment variable DISABLE_COLLECTSTATIC set to 0, as it won't really solve the issue, but just mask it and mess with your site's assets
When using django_heroku lib on the settings file, include the argument "staticfiles=False", like this: django_heroku.settings(locals(), staticfiles=False)
The STATIC_ROOT variable on settings.py should be set as STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles'), being BASE_DIR set as BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))). This is described on Heroku's django-assets doc
After doing all that, run a python manage.py migrateon Heroku CLI and you should see a message about assets downloaded.
Before making it to actually work, the collectstatic command python manage.py collectstatic on Heroku CLI was giving a correct output, so be aware that you may get no errors with this command, but still have something wrong going on.
When deploying a Django app using dokku I am getting a following error
Collectstatic configuration error. To debug, run:
$ heroku run python ./manage.py collectstatic --noinput
I found no way to run heroku run python ./manage.py collectstatic --noinput for a dokku container, but when I am trying dokku run my app python ./manage.py collectstatic --noinput, the static files are successfully copied to the STATIC_ROOT folder and no error message given.
I could solve the problem by placing collectstatic command into Procfile:
web: python manage.py collectstatic --noinput ; gunicorn myapp.wsgi
Still, I would love to know what was causing the problem and how can it be debugged. Any ideas?
You should have four settings in your settings.py file called MEDIA_ROOT, MEDIA_URL, STATIC_ROOT and STATIC_URL.
I set mine like so:
MEDIA_ROOT = 'media'
STATIC_ROOT = 'static'
MEDIA_URL = '/media'
STATIC_URL = '/static'
Inside the docker container that gets created, you will find your application under /app which makes the media path /app/media/ and the static path /app/static/.
Unfortunately if you don't have a media and static folder committed in git, it won't get created under /app automatically.
Since git doesn't allow you to commit an empty folder (it only commits files), I do the following in my projects:
mkdir media static
touch media/.dir
touch static/.dir
git add media/.dir static/.dir
git commit -m 'Make media and static directories'
The 'touch' command creates an empty file, then you 'git add' the two newly-created files and check them in.
Now when you push, the directories will be there to contain the media and static files. Just keep in mind that every time you 'git push', a new container is created, and the old one is destroyed. While this isn't a problem for your static files, your media will be lost unless you store it somewhere else.