I have configured an S3 bucket to store and serve static and media files for a Django website, currently just trying to get the static files needed for the admin pages and all that.
Here is all the static and AWS config info in my settings file:
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DEFAULT_FILE_STORAGE = 'config.storage_backends.MediaStorage'
#used to authenticate with S3
AWS_ACCESS_KEY_ID = 'AKIAWWJOJKZGFSJO2UPW' #not real one
AWS_SECRET_ACCESS_KEY = 'KNg1z5wXWiDRAIh4zLiHgbD2N3wtWZTK' #not real one
#for endpoints to send or retrieve files
AWS_STORAGE_BUCKET_NAME = 'my-static' #not real bucket name
AWS_DEFAULT_ACL = None
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400',}
AWS_LOCATION = 'static'
STATIC_ROOT = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'config/static'),
]
Of course I replaced any sensitive variables with fake ones for the purpose of this post. I have gone through many tutorials and other posts and I seem to have my STATIC_URL configured correctly but whenever I runserver and go to the admin pages, none of the css is applied. I do not think it is properly retrieving the static files (they are all uploaded to the S3 bucket) from the bucket, but I am stuck on what to do.
I am sure you might have missed some configurations like public and static file hosting on your s3 bucket.
Please go through this article Using Amazon S3 to Store your Django Site's Static and Media Files
I have personally tried this and it worked for me. The article explains every step.
Related
I am using Django 4.0.1 in my project, and right prior to deploying my site, I am faced with the issue of handling my static files. Due to the limit of my server, I have decided to instead serve these static files via CDN.
I have already configured my STATIC_URL option in settings.py:
STATIC_URL = 'assets/'
I am aware that in the Django documentation, they say that this url refers to the static files located in STATIC_ROOT. Of course, normally the latter is an absolute path on your server where the collectstatic command collects the static files and put them there, but I am wondering if I can configure this STATIC_ROOT to point a path which is not on my server.
To be precise, I want to know whether I can point STATIC_ROOT to my CDN storage. In that way I can still use STATIC_URL to refer to my static assets, while being able to serve them via CDN.
Yes, it's actually a recommended way to serve static files for years. Not sure why Django didn't add it to its core.
You can use django-storages, it supports almost every cloud provider. You can use it for media and/or static.
Yes definitely you can use it for any other server. For example while we use AWS S3 as a server to serve static files it should look like this :
USE_S3 = os.getenv('USE_S3') == 'TRUE'
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
I'm testing my django project on s3 locally. But when running the server it doesn't seem to be serving static files
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
But it doesn' t seem working
when I check my browser's console it shows http error code of 403 request forbidden when accessing to my static files. why this happens, should I change my s3 settings.
this error can be of the s3 side. check if you have permission to access objects and buckets. as i see your settings, you are using django-storages. not django-s3-storages. their are some changes in between.
for me personally, in case of django-storages; by adding region name to the setting helped.so make sure that you also included
AWS_S3_REGION_NAME = 'the region here'
hope it works for you too.
I'm following this tutorial for storing static and media files on s3. This is what my static files configuration in settings.py looks like:
USE_S3 = os.getenv('USE_S3') == 'TRUE'
## AWS Configuration
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = None # differs from the tutorial because the bucket is private
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MEDIA_URL = '/mediafiles/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'mediafiles'
After setting all the env variables I run python manage.py collectstatic and I can see that the files have been successfully uploaded to my s3 bucket. I.e. I see static/admin/ directory on s3 with fonts, css, etc.
However when I run the server locally the admin panel is missing all the css. I'm not sure why django cannot find the admin static files given that collectstatic worked. I found several other tutorials here, here and here but I can't find what I'm missing. For all of them it seems like it's supposed to "just work" after running collectstatic... What did I forget?
Note The admin panel is fine when USE_S3 == False
So the objects in the s3 bucket weren't public. I made the assets public and now it's working.
I have the following code in my settings.py in django.
DEFAULT_FILE_STORAGE = 'hhhh.utils.MediaRootS3BotoStorage'
STATICFILES_STORAGE = 'hhhh.utils.StaticRootS3BotoStorage'
S3DIRECT_REGION = 'us-west-2'
S3_URL = '//%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
MEDIA_URL = '//%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
MEDIA_ROOT = MEDIA_URL
STATIC_URL = S3_URL + 'static/'
STATIC_ROOT = STATIC_URL + 'static_root/'
Heroku is not serving the ststic files. any ideas. I have the allowed hosts set to my site and heroku.
If I am reading your variables correctly, your static URL is built as so:
S3_URL + static + static_root
So, if your s3 bucket is named hhhh, then the final URL is
//hhhh.s3.amazonaws.com/static/static_root
Do files exist in that location?
For more info, Heroku offers a sample settings.py file regarding Django static files here: serving static assets with Django: https://devcenter.heroku.com/articles/django-assets
This page speaks specifically to hosting s3 files on s3 with heroku:
http://www.jorgechang.com/blog/howto-deploy-a-fault-tolerant-django-app-on-aws-part-2-moving-static-media-files-to-s3/
The author's STATIC_ROOT variable is blank, because static locations of files are set- and then the code later refers to the files on an Amazons3 location- it seems his code collects static files from a specific place and puts them in S3, then references them from there. You seem to be attempting to refer directly to an amazon s3 URL on your static_root var, so this laws gives you an alternate way to go it.
I have setup an AWS S3 bucket in order to transfer my static files in a remote CDN using the application django-storages,
everything worked fine until I tried to compress my static files before uploading to S3 using django_compressor.
I have setup all the variables according django_compressor documentation for django-storages (https://django_compressor.readthedocs.org/en/latest/remote-storages/index.html)
I uploaded all the files in S3 using 'manage.py collectstatic' then:
When I do 'manage.py compress' I get this error:
CommandError: An error occured during rendering ../templates/base.html: 'https://my_bucket.s3.amazonaws.com/css/bootstrap.2.3.1.css' isn't accessible via COMPRESS_URL ('https://my_bucket.s3-external-3.amazonaws.com/') and can't be compressed
What's wrong with my setup?
Here is my settings.py configuration for django-storages and django_compressor:
COMPRESS_URL = 'https://mybucket_name.s3-external-3.amazonaws.com/'
STATIC_URL = COMPRESS_URL
DEFAULT_FILE_STORAGE = 'my_project.boto_custom.CachedS3BotoStorage'
AWS_ACCESS_KEY_ID = 'XXX'
AWS_SECRET_ACCESS_KEY = 'XXX'
AWS_STORAGE_BUCKET_NAME = 'mybucket_name'
COMPRESS_ROOT = STATIC_ROOT
COMPRESS_STORAGE = 'my_project.boto_custom.CachedS3BotoStorage'
STATICFILES_STORAGE = 'my_project.boto_custom.CachedS3BotoStorage'
COMPRESS_OFFLINE = True
Thanks for your help
I fixed it by adding one variable and it worked:
AWS_S3_CUSTOM_DOMAIN = 'my_bucket.s3-external-3.amazonaws.com'
If you have separate S3 buckets for static and media you can also put it in your subclass of S3BotoStorage like so:
class CachedS3BotoStorage(S3BotoStorage):
custom_domain = 'my_bucket.s3-external-3.amazonaws.com'
(or better yet set it to settings.AWS_S3_CUSTOM_STATIC_DOMAIN or something)