We Keep Coding

django admin static files are uploaded to s3 via `collectstatic` but not served when running locally

I'm following this tutorial for storing static and media files on s3. This is what my static files configuration in settings.py looks like:
USE_S3 = os.getenv('USE_S3') == 'TRUE'
## AWS Configuration
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = None # differs from the tutorial because the bucket is private
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MEDIA_URL = '/mediafiles/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'mediafiles'
After setting all the env variables I run python manage.py collectstatic and I can see that the files have been successfully uploaded to my s3 bucket. I.e. I see static/admin/ directory on s3 with fonts, css, etc.
However when I run the server locally the admin panel is missing all the css. I'm not sure why django cannot find the admin static files given that collectstatic worked. I found several other tutorials here, here and here but I can't find what I'm missing. For all of them it seems like it's supposed to "just work" after running collectstatic... What did I forget?
Note The admin panel is fine when USE_S3 == False

So the objects in the s3 bucket weren't public. I made the assets public and now it's working.

heroku not serving css

I have the following code in my settings.py in django.
DEFAULT_FILE_STORAGE = 'hhhh.utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'hhhh.utils.StaticRootS3BotoStorage'
S3DIRECT_REGION = 'us-west-2'
S3_URL = '//%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
MEDIA_URL = '//%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
MEDIA_ROOT = MEDIA_URL
STATIC_URL = S3_URL + 'static/'
STATIC_ROOT = STATIC_URL + 'static_root/'
Heroku is not serving the ststic files. any ideas. I have the allowed hosts set to my site and heroku.

If I am reading your variables correctly, your static URL is built as so:
S3_URL + static + static_root
So, if your s3 bucket is named hhhh, then the final URL is
//hhhh.s3.amazonaws.com/static/static_root
Do files exist in that location?
For more info, Heroku offers a sample settings.py file regarding Django static files here: serving static assets with Django: https://devcenter.heroku.com/articles/django-assets
This page speaks specifically to hosting s3 files on s3 with heroku:
http://www.jorgechang.com/blog/howto-deploy-a-fault-tolerant-django-app-on-aws-part-2-moving-static-media-files-to-s3/
The author's STATIC_ROOT variable is blank, because static locations of files are set- and then the code later refers to the files on an Amazons3 location- it seems his code collects static files from a specific place and puts them in S3, then references them from there. You seem to be attempting to refer directly to an amazon s3 URL on your static_root var, so this laws gives you an alternate way to go it.

Serving static files in development

I've got some static files that are currently serving from Amazon S3. This obviously isn't effective because I can't collectstatic every time I make a minor change. Whenever I'm developing I want to serve from my project where the static files are collected from. The project structure looks partially like this:
myproject
-app
--templates
-myproject
--static
---app
----css
----js
----img
My settings.py looks partially like this:
STATIC_ROOT = 'staticfiles'
MEDIA_ROOT = os.path.join(BASE_DIR, 'mediafiles')
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'static/app'),
)
AWS_HEADERS = {
'Expires': 'Thu, 31 Dec 2099 20:00:00 GMT',
'Cache-Control': 'max-age=94608000',
}
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
if DEBUG:
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_URL = '/static/'
else:
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATIC_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
STATICFILES_LOCATION = 'static'
MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
What I'm trying to achieve is that if I'm running development (when DEBUG would be true) serve static files from the project, rather than fetching from S3.
At the end of my urlpatterns in urls.py I have + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT).
Finally, in my base.html I have:
<!doctype html>
...
<link rel="stylesheet" href="{% static '/css/normalize.css'%}">
If I'm running with debug=True and I view the source of the page I see <link rel="stylesheet" href="/css/normalize.css"> which isn't where the file is. If I run with debug=False and view the source I get<link rel="stylesheet" href="<AMAZON BUCKET>/css/normalize.css">.
What am I missing to serve locally?

You have to pass a relative path to the static template tag, without the heading slash:
{% static 'css/normalize.css' %}

How to tell Django collectstatic to use Amazon S3?

I have followed this howto to setup django + S3. Specifically:
import os
# AWS credentials
AWS_STORAGE_BUCKET_NAME = os.environ['AWS_STORAGE_BUCKET_NAME']
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
# boto config
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_HEADERS = { # see http://developer.yahoo.com/performance/rules.html#expires
'Expires': 'Thu, 31 Dec 2099 20:00:00 GMT',
'Cache-Control': 'max-age=94608000',
}
# For the static files
STATICFILES_LOCATION = 'static'
STATICFILES_STORAGE = 'myapp.custom_storages.StaticStorage'
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, STATICFILES_LOCATION)
# For the media files
MEDIAFILES_LOCATION = 'media'
DEFAULT_FILE_STORAGE = 'myapp.custom_storages.MediaStorage'
MEDIA_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
My custom storages are simple S3BotoStorages:
from django.conf import settings
from storages.backends.s3boto import S3BotoStorage
class StaticStorage(S3BotoStorage):
location = settings.STATICFILES_LOCATION
class MediaStorage(S3BotoStorage):
location = settings.MEDIAFILES_LOCATION
I would expect that collectstatic would honor this configuration (as explained in the howto), and use the myapp.custom_storages.StaticStorage to upload collected static files to S3. Instead, it just uses the local filesystem. Since I have:
STATIC_ROOT = os.path.join(BASE_DIR, 'mycollectstatic')
(just because having 'static' there seems too confusing to me), I can clearly see that:
» python manage.py collectstatic
You have requested to collect static files at the destination
location as specified in your settings:
/absolute-path/mycollectstatic
This will overwrite existing files!
Are you sure you want to do this?
Type 'yes' to continue, or 'no' to cancel:
So it seems that collectstatic command is using STATIC_ROOT even when STATICFILES_STORAGE = 'myapp.custom_storages.StaticStorage'. Is this expected?
Should I configure STATIC_ROOT differently when using another STATICFILES_STORAGE? Where is this documented?

Django 1.4 serving media files works, but not static

I am using Django 1.4 and for some reason i am able to serve media files, but not the static ones...
Here is my code:
settings:
MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
MEDIA_URL = '/media/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static/')
STATIC_URL = '/static/'
ADMIN_MEDIA_PREFIX = '/static/admin/'
urls.py:
(r'^media/(?P<path>.*)$', 'django.views.static.serve',{'document_root': settings.MEDIA_ROOT}),
(r'^static/(?P<path>.*)$', 'django.views.static.serve',{'document_root': settings.STATIC_ROOT}),
base.html:
<link href="{{ STATIC_URL }}css/bootstrap.css" rel="stylesheet">
<link href="{{ MEDIA_URL }}css/bootstrap-colorpicker.css" rel="stylesheet">
i get a 404 http not found... what am i doing wrong? An i did create the static folder in my project right next to media
http://mysite.com:8000/static/css/bootstrap.css

Your static folder should be under one app that you use it for.
For example, I have a project named my_project and an application named my_app, I have some static files used in my_app so I put them under ~/project_path/my_project/my_app/static
NB: my_app must be in INSTALLED_APPS. See STATICFILES_FINDERS documentation.
Edit:
As a best practice, you should have a global static folder in one app (the main one), for example a static folder how contains your html template basic resources as jquery, bootstrap, your global style.
And for the static files how's required only for one app, for example app foo, these files should be under foo/static folder

I suggest removing the explicit media and static views and allowing the staticfiles app to create them (when DEBUG is True under development).
Check the default finders are present in your settings.py
https://docs.djangoproject.com/en/1.4/ref/contrib/staticfiles/#std:setting-STATICFILES_FINDERS
Either add your project static directory to STATICFILES_DIRS (https://docs.djangoproject.com/en/1.4/ref/contrib/staticfiles/#std:setting-STATICFILES_DIRS) or place app specific static folders under each app. The app needs to be listed in the INSTALLED_APPS for the finders to locate the static content.
Do not place static files into STATIC_ROOT yourself. This directory is managed by the collectstatic command. See https://docs.djangoproject.com/en/dev/howto/static-files/#deployment

Here is how i define my media url in settings.py:
import os
ABSOLUTE_PATH = os.path.dirname(__file__)
MEDIA_ROOT = os.path.join(ABSOLUTE_PATH, '../media')
MEDIA_URL = '/media/'
STATIC_ROOT = os.path.join(ABSOLUTE_PATH, '../static')
STATIC_URL = "/static/"
So like you see the difference is ../media and ../static
Is mysite in your installed apps ? Django look inside your installed apps and check if there's a static folder there.