GAE: Read static file - python

I'm using app engine and I don't know how to read a static file from my project. I have a file structure that globally looks like this:
- html (static folder)
- staticfile1.html
- staticfile2.html
- script
- main.py
app.yaml
In my app.yaml I already set the application_readably attribute to true for the html dir:
- url: /html
static_dir: html
application_readable: true
I tried several methods to access the staticfile1.html but either of them returns this error:
[Errno 13] file not accessible: u'/html/staticfile1.html'
At the moment my code looks like this:
INDEX_HTML = open('/html'+self.request.path).read()
self.response.out.write(INDEX_HTML)
I hope someone knows how I can read the static file.
Thanks in advance.

You'll probably need to use relative paths based on the current module's __file__ attribute. e.g. from main.py, you'd do something like:
import os
_HERE = os.path.basename(__file__)
_HTML_DIR = os.path.join(_HERE, os.pardir, 'html')

Related

How to access Environment Variables in Django with Django Environ?

I'm a Node developer but I need to create a Django app (i'm totally beginner in Django).
I need to read some data from an API but ofc, I shouldn't hardcode the API url.
So having API_BASE_URL=api.domain.com in my .env file, in Node I would access the variables in my functions this way:
import ('dotenv/config');
import axios from 'axios';
baseUrl = process.env.API_BASE_URL;
function getApiData() {
return axios.get(baseUrl);
}
So how would be the Python/Django version of it?
Saying I have the function bellow:
import ???
def get_api_data():
url = ????
import environ
# reading .env file
environ.Env.read_env()
def get_api_data():
url = env('API_BASE_URL')
Let's say you have a .env file saved in the same directory as your manage.py file.
You can then go to settings.py and do:
from decouple import config
API_BASE_URL = config('API_BASE_URL')
Assuming your .env file looks like:
API_BASE_URL='some.url'

Change static folder from config in Flask

Has anyone tried this snippet flask config based static folder code cnippet?
The code:
import flask
class MyFlask(flask.Flask):
#property
def static_folder(self):
if self.config.get('STATIC_FOLDER') is not None:
return os.path.join(self.root_path,
self.config.get('STATIC_FOLDER'))
#static_folder.setter
def static_folder(self, value):
self.config.get('STATIC_FOLDER') = value
# Now these are equivalent:
app = Flask(__name__, static_folder='foo')
app = MyFlask(__name__)
app.config['STATIC_FOLDER'] = 'foo'
In my case in complains about this line:
self.config.get('STATIC_FOLDER') = value
The error message: Can't assign to function call
Does anyone how to set the static_folder from the config.py file in Flask?
Okay, I assume you want to use a custom path to the static folder for whatever reason. I wanted to do the same for the sake of better app modularity.
Here's my app folder structure:
instance/
core/
|_templates/
|_static/
|_views.py
run.py
config.py
As you can see, my static folder is inside the core folder.
In run.py, you can do the following:
app = Flask(__name__, static_url_path=None)
if __name__ == '__main__':
app.config.from_object('config')
# config file has STATIC_FOLDER='/core/static'
app.static_url_path=app.config.get('STATIC_FOLDER')
# set the absolute path to the static folder
app.static_folder=app.root_path + app.static_url_path
print(app.static_url_path)
print(app.static_folder)
app.run(
host=app.config.get('HOST'),
port=app.config.get('PORT'),
threaded=True
)
This is what I did, and it works perfectly fine. I'm using flask 0.12.
I don't know anything about that snippet, but
some_function(...) = some_value
is never valid Python (Python doesn't have l-values). It looks like config has a dict-like interface, so the offending line should probably just be
self.config['STATIC_FOLDER'] = value
Probably a copy-and-paste error from the getter definition above the setter.
app = Flask(__name__, static_url_path="/STATIC_FOLDER", static_folder='STATIC_FOLDER')
Yes, In one of my projects I am using/setting a custom path for STATIC_FOLDER. You can set the path to STATIC_FOLDER in config.py like below:
STATIC_PATH = '<project-name>/<path-to-static-folder>/'
ex:
STATIC_PATH = 'myApp/static/'
If you can write your project structure then I can answer it as per your requirements.
FYI if you want your directory to be outside of the server directory the best solutions I found so far are either to make a copy of your directory into the server directory before startup in your main(), or to create a symlink.

Serve stylesheet with webapp2 outside of Google App Engine

So I have successfully deployed an app using webapp2/jinja2 and a Paste server, but am having trouble serving static stylesheets.
I have had luck accessing static files via this method, as well as implementing a StaticFileHandler I found with some google-fu:
import os
import mimetypes
import webapp2
import logging
class StaticFileHandler(webapp2.RequestHandler):
def get(self, path):
abs_path = os.path.abspath(os.path.join(self.app.config.get('webapp2_static.static_file_path', 'static'), path))
if os.path.isdir(abs_path) or abs_path.find(os.getcwd()) != 0:
self.response.set_status(403)
return
try:
f = open(abs_path, 'r')
self.response.headers.add_header('Content-Type', mimetypes.guess_type(abs_path)[0])
self.response.out.write(f.read())
f.close()
except:
self.response.set_status(404)
where my main app routing looks like:
app = webapp2.WSGIApplication([('/', HelloWorld),
(r'/display', DisplayHandler),
(r'/static/(.+)', StaticFileHandler)
], debug=True)
My css files are in a folder under the app root: /static/css/main.css
I can access the file via direct url, and even link it as a stylesheet, but the styles won't apply. Any ideas? Is there another way to serve stylesheets? Some way to implement an app.yaml similar to GAE?
you dont need a static file handler.
upload the app with the static file folder by adding this to your app.yaml
- url: /static/
static_dir: static
docs are here: https://developers.google.com/appengine/docs/python/config/appconfig#Static_Directory_Handlers
edit:
see answer below in comments
#Mnemon, hats off to you for solving my problem. I would upvote you but I'm not allowed to do that. You convinced me that if it's not the only webapp2 way without GAE, it's at least a way that will work.
But also I can contribute that your solution is now installable as "pip install webapp2_static", from pipi--- by an author who seems to be using his real name... you I'm sure. Other webapp2 docs that I found helpful are available here.
I'm implementing your code on a Linux desktop development server, using paste, which you also used:
def main():
from paste import httpserver
httpserver.serve(app, host='127.0.0.1', port='8080')
But with the code as you have it above (which appears to be utterly identical to that of webapp2_static.py file), I don't find that putting my css files in a folder named static in the app root works as you said.
For example, I have /home/user/proj/public_html/app/app.py, where the py file contains your code plus other "views" for my ultra-simple site. (I don't know how paste really works, so maybe for now the public_html is just in there for reference so that I don't become confused when I'm uploading stuff onto the production server.)
So if I put the css stylesheets into a folder named /static, then, if I put /static in as either a subdirectory of /app or of /public_html I find that neither location works; I must instead make it a subdirectory of /proj.
I wasn't expecting that, but the cure for me is to change the default 'static' in your app.configure.get(..., 'static') call, to 'public_html/app/static'. Then it works, with the /static folder inside /app.
Similarly using the pipi code with './app/static/ in place of the default 'static' doesn't work; I found that I need ./public_html/app/static instead (or maybe it was just /public_html/app/static or even public_html/app/static... I forgot... one of those worked).
I tested how your computation of abs_path works and have reworked it in the code below, in which I have junked your approach in favor of something more Djangoesque. To wit, in my one app py file I put at the top the following:
STATIC_DIR = os.sep + 'tostatic' + os.path.abspath(os.path.dirname(__file__)) + os.sep + 'static'
Then in the page to which I want to add css, my Home page in my case, I put a very readable:
<link href="{{STATIC_DIR}}/dist/css/bootstrap.min.css" rel="stylesheet" type="text/css">
For the "view" that generates my Home page I have (env is a jinja2 Environment object that takes a template loader as an argument):
class Home(webapp2.RequestHandler):
def get(self):
template = env.get_template('index.html')
template_values = {'STATIC_DIR': STATIC_DIR }
self.response.write(template.render(template_values))
And finally the URL routing is as in:
app = webapp2.WSGIApplication(
[
(r'/', Home),
(r'/tostatic/(.+)', StaticView),
], debug=True)
The view for the static file serving is now:
class StaticView(webapp2.RequestHandler):
def get(self, path):
path = os.sep + path
try:
f = open(path, 'r')
self.response.headers.add_header('Content-Type', mimetypes.guess_type(path)[0])
self.response.out.write(f.read())
f.close()
except Exception, e:
print 'Problem in StaticView:', e
self.response.set_status(404)
To finally close, the problem that I had with your approach is the one that I and other near noobs have with the departure of URLs from the legacy association with the file system. In your approach "static" is both a sub-directory and a string between slashes at the front of the URL that tells the interpreter which view (which webapp2.RequestHandler subclass) to run. You take the /static from the rest of the URL and then later hard-code it back on. And when it comes time to decide what to put in for href in the tag the HTML page coder has to remember that duplicity. With the {{STATIC_DIR}} template variable approach it's clear what to do. And it's easy to redefine the location of the static files--- only the STATIC_DIR declaration has to be changed.
I found that self.response.set_status(404) shows up in Firebug, but not Firefox. Evidently with webapp2 you must provide and serve your own HTTP status code pages.
self.response.headers.add_header('Content-Type', mimetypes.guess_type(abs_path)[0])
self.response.headers['Content-Type'] = mimetypes.guess_type(abs_path)[0]

How do I set up gaeunit 2.0a with my Django app?

I am trying to set up Google App Engine unit testing for my web application. I downloaded the file from here.
I followed the instructions in the readmen by copying the directory gaeunit into the directory with the rest of my apps and registering 'gaeunit' in settings.py. This didn't seem sufficient to actually get things going. I also stuck url('^test(.*)', include('gaeunit.urls')) into my urls.py file.
When I go to the url http://localhost:8000/test, I get the following error:
[Errno 2] No such file or directory: '../../gaeunit/test'
Any suggestions? I'm not sure what I've done wrong. Thanks!
You can copy url but in your urls.py include
(r'^test', include('gaeunit.urls')),
Also you will have to change test path in gaeunit.py
_LOCAL_DJANGO_TEST_DIR = 'test'
And in your main web test files under test folder include following line for django test setup
self.application = django.core.handlers.wsgi.WSGIHandler()
According to the instructions on the main page you should add the URL to app.yaml, not urls.py.
- url: /test.*
script: gaeunit.py

Read a file on App Engine with Python?

Is it possible to open a file on GAE just to read its contents and get the last modified tag?
I get a IOError: [Errno 13] file not accessible:
I know that i cannot delete or update but i believe reading should be possible
Has anyone faced a similar problem?
os.stat(f,'r').st_mtim
You've probably declared the file as static in app.yaml. Static files are not available to your application; if you need to serve them both as static files and read them as application files, you'll need to include 2 copies in your project (ideally using symlinks, so you don't actually have to maintain an actual copy.)
Update Nov 2014:
As suggested in the comments, you can now do this with the application_readable flag:
application_readable
Optional. By default, files declared in static file handlers are
uploaded as static data and are only served to end users, they cannot
be read by an application. If this field is set to true, the files are
also uploaded as code data so your application can read them. Both
uploads are charged against your code and static data storage resource
quotas.
See https://cloud.google.com/appengine/docs/python/config/appconfig#Static_Directory_Handlers
You can read files, but they're on Goooogle's wacky GAE filesystem so you have to use a relative path. I just whipped up a quick app with a main.py file and test.txt in the same folder. Don't forget the 'e' on st_mtime.
import os
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
path = os.path.join(os.path.split(__file__)[0], 'test.txt')
self.response.out.write(os.stat(path).st_mtime)
def main():
application = webapp.WSGIApplication([('/', MainHandler)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
+1 for the new "application_readable: true" feature. Before using this new feature I did run into an issue with GAEs' "wacky" file system while getting the NLP Montylingua to import.
Issue: Monty uses the open(filename,'rb') and a file pointer to file_ptr.read() in bytes from the static files. My implementation worked on my local windows system but failed upon deployment!
The fix: Specify the expected bytes to read file_ptr.read(4) #4 binary bytes
Appears to be something related to the 64 bit GAE server wanting to read in more (8 by default) bytes. Anyways, took a while to find that issue. Montylingua loads now.
I came up strange but working solution :) Jinja :)
Serving static files directly sometimes become a headache with GAE. Possible trade-off from performance let you move straigh forward with Jinja
- url: /posts/(.*\.(md|mdown|markdown))
mime_type: text/plain
static_files: static/posts/\1
upload: posts/(.*\.(md|mdown|markdown))
from jinja2 import Environment
from jinja2.loaders import FileSystemLoader
posts = Environment(loader=FileSystemLoader('static/posts/')) # Note that we use static_files folder defined in app.yaml
post = posts.get_template('2013-11-13.markdown')
import markdown2 # Does not need of course
class Main(webapp2.RequestHandler):
def get ( self ):
self.response.headers[ 'Content-Type' ] = 'text/html'
self.response.write ( markdown2.markdown( post.render()) ) # Jinja + Markdown Render function
Did you get it ;) I tested and It worked.
With webapp2, supposing you have pages/index.html at the same path as main.py:
#!/usr/bin/env python
import webapp2, os
class MainHandler(webapp2.RequestHandler):
def get(self):
path = os.path.join(os.path.split(__file__)[0], 'pages/index.html')
with open(path, 'r') as f:
page_content = f.read()
self.response.write(page_content)
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
I can't see an answer for when the file hasn't been marked as static, and you're trying to read it in mode 'rt'; apparently that doesn't work. You can however open files just fine in mode 'rb', or just plain 'r'. (I wasted about 10 minutes on that 't'.)

Categories