I have recently been trying to learning about WSGI and moreover, how the web works in regards to Python. So I've been reading through Werkzeug and PEP333 to learn.
However I've run up against a small question, that I think I understand but probably don't, so I would appreciate your steering in the right direction.
PEP333 states:
The application object is simply a callable object that accepts two
arguments. The term "object" should not be misconstrued as requiring
an actual object instance: a function, method, class, or instance with
a call method are all acceptable for use as an application object.
Application objects must be able to be invoked more than once, as
virtually all servers/gateways (other than CGI) will make such
repeated requests.
The implementation:
class AppClass:
"""Produce the same output, but using a class
(Note: 'AppClass' is the "application" here, so calling it
returns an instance of 'AppClass', which is then the iterable
return value of the "application callable" as required by
the spec.
If we wanted to use *instances* of 'AppClass' as application
objects instead, we would have to implement a '__call__'
method, which would be invoked to execute the application,
and we would need to create an instance for use by the
server or gateway.
"""
def __init__(self, environ, start_response):
self.environ = environ
self.start = start_response
def __iter__(self):
status = '200 OK'
response_headers = [('Content-type', 'text/plain')]
self.start(status, response_headers)
yield "Hello world!\n"
My question here is just to clarify if I have understood it correctly.
It states that AppClass is the application, and when we call it, it returns an instance of AppClass. But then further down states that 'if we wanted to use instances of AppClass ass application objects instead', is this saying that when the server side of WSGI calls the AppClass object, there is only one instance running?
For example. The server can issue multiple requests (200 OK's) to the Application for more responses, hence why iter is placed into the class. But each request runs through the same singular AppClass instance, each request to the server basically doesn't instantiate more than one instance of the AppClass?
Sorry if this is long winded, and apologies again if I haven't made much sense. I'm trying to improve atm.
Appreciate your inputs as always.
Thanks.
The server technology will call your app (in this case the class AppClass, causing an object construction) for each request. This is because each request will have a potentially unique environ.
The neat thing about this is it doesn't mean your app has to be a class, I often find it useful to define my wsgi app (or middleware) as a function returning a function:
# I'd strongly suggest using a web framework instead to define your application
def my_application(environ, start_response):
start_response(str('200 OK'), [(str('Content-Type'), str('text/plain'))])
return [b'hello world!\n']
def my_middleware(app):
def middleware_func(environ, start_response):
# do something or call the inner app
return app(environ, start_response)
return middleware_func
# expose `app` for whatever server tech you're using (such as uwsgi)
app = my_application
app = my_middleware(app)
Another common pattern involves defining an object to store some application state which is constructed once:
class MyApplication(object):
def __init__(self):
# potentially some expensive initialization
self.routes = ...
def __call__(self, environ, start_response):
# Called once per request, must call `start_response` and then
# return something iterable -- could even be `return self` if
# this class were to define `__iter__`
...
return [...]
app = MyApplication(...)
As for PEP333, I'd suggest reading PEP3333 instead -- it contains largely the same information, but clarifies the datatypes used throughout.
For background on various ways that WSGI application objects can be implemented read this blog post on the topic.
http://blog.dscpl.com.au/2011/01/implementing-wsgi-application-objects.html
I would also suggest reading the following, which talks about how Python web servers in general work.
https://ruslanspivak.com/lsbaws-part1/
Unless you really have a need you probably just want to use a framework. Avoid trying to write anything from scratch with WSGI.
Related
I'm writing a CLI to interact with elasticsearch using the elasticsearch-py library. I'm trying to mock elasticsearch-py functions in order to test my functions without calling my real cluster.
I read this question and this one but I still don't understand.
main.py
Escli inherits from cliff's App class
class Escli(App):
_es = elasticsearch5.Elasticsearch()
settings.py
from escli.main import Escli
class Settings:
def get(self, sections):
raise NotImplementedError()
class ClusterSettings(Settings):
def get(self, setting, persistency='transient'):
settings = Escli._es.cluster\
.get_settings(include_defaults=True, flat_settings=True)\
.get(persistency)\
.get(setting)
return settings
settings_test.py
import escli.settings
class TestClusterSettings(TestCase):
def setUp(self):
self.patcher = patch('elasticsearch5.Elasticsearch')
self.MockClass = self.patcher.start()
def test_get(self):
# Note this is an empty dict to show my point
# it will contain childs dict to allow my .get(persistency).get(setting)
self.MockClass.return_value.cluster.get_settings.return_value = {}
cluster_settings = escli.settings.ClusterSettings()
ret = cluster_settings.get('cluster.routing.allocation.node_concurrent_recoveries', persistency='transient')
# ret should contain a subset of my dict defined above
I want to have Escli._es.cluster.get_settings() to return what I want (a dict object) in order to not make the real HTTP call, but it keeps doing it.
What I know:
In order to mock an instance method I have to do something like
MagicMockObject.return_value.InstanceMethodName.return_value = ...
I cannot patch Escli._es.cluster.get_settings because Python tries to import Escli as module, which cannot work. So I'm patching the whole lib.
I desperately tried to put some return_value everywhere but I cannot understand why I can't mock that thing properly.
You should be mocking with respect to where you are testing. Based on the example provided, this means that the Escli class you are using in the settings.py module needs to be mocked with respect to settings.py. So, more practically, your patch call would look like this inside setUp instead:
self.patcher = patch('escli.settings.Escli')
With this, you are now mocking what you want in the right place based on how your tests are running.
Furthermore, to add more robustness to your testing, you might want to consider speccing for the Elasticsearch instance you are creating in order to validate that you are in fact calling valid methods that correlate to Elasticsearch. With that in mind, you can do something like this, instead:
self.patcher = patch('escli.settings.Escli', Mock(Elasticsearch))
To read a bit more about what exactly is meant by spec, check the patch section in the documentation.
As a final note, if you are interested in exploring the great world of pytest, there is a pytest-elasticsearch plugin created to assist with this.
I am using cherrypy as a web server, and I want to check a user's logged-in status before returning the page. This works on methods in the main Application class (in site.py) but gives an error when I call the same decorated function on method in a class that is one layer deeper in the webpage tree (in a separate file).
validate_user() is the function used as a decorator. It either passes a user to the page or sends them to a 401 restricted page, as a cherrypy.Tool, like this:
from user import validate_user
cherrypy.tools.validate_user = cherrypy.Tool('before_handler', validate_user)
I attach different sections of the site to the main site.py file's Application class by assigning instances of the sub-classes as variables accordingly:
from user import UserAuthentication
class Root:
user = UserAuthentication() # maps user/login, user/register, user/logout, etc
admin = Admin()
api = Api()
#cherrypy.expose
#cherrypy.tools.validate_user()
def how_to(self, **kw):
from other_stuff import how_to_page
return how_to_page(kw)
This, however, does not work when I try to use the validate_user() inside the Admin or Api or Analysis sections. These are in separate files.
import cherrypy
class Analyze:
#cherrypy.expose
#cherrypy.tools.validate_user() #### THIS LINE GIVES ERROR ####
def explore(self, *args, **kw): # #addkw(fetch=['uid'])
import explore
kw['uid'] = cherrypy.session.get('uid',-1)
return explore.explorer(args, kw)
The error is that cherrypy.tools doesn't have a validate_user function or method. But other things I assign in site.py do appear in cherrypy here. What's the reason why I can't use this tool in a separate file that is part of my overall site map?
If this is relevant, the validate_user() function simply looks at the cherrypy.request.cookie, finds the 'session_token' value, and compares it to our database and passes it along if the ID matches.
Sorry I don't know if the Analyze() and Api() and User() pages are subclasses, or nested classes, or extended methods, or what. So I can't give this a precise title. Do I need to pass in the parent class to them somehow?
The issue here is that Python processes everything except the function/method bodies during import. So in site.py, when you import user (or from user import <anything>), that causes all of the user module to be processed before the Python interpreter has gotten to the definition of the validate_user tool, including the decorator, which is attempting to access that tool by value (rather than by a reference).
CherryPy has another mechanism for decorating functions with config that will enable tools on those handlers. Instead of #cherrypy.tools.validate_user, use:
#cherrypy.config(**{"tools.validate_user.on": True})
This decorator works because instead of needing to access validate_user from cherrypy.tools to install itself on the handler, it instead configures CherryPy to install that tool on the handler later, when the handler is invoked.
If that tool is needed for all methods on that class, you can use that config decorator on the class itself.
You could alternatively, enable that tool for given endpoints in the server config, as mentioned in the other question.
I am using python flask framework. I write a decorator which will be need a parameter, and this parameter will be dynamic.
my decorator like below, will be get a key ,and using the key fetch data from redis.
def redis_hash_shop_style(key):
def fn_wrapper(f):
#wraps(f)
def decorated_function(*args, **kwargs):
data = redis_hash(key)
return data
return decorated_function
return fn_wrapper
and I have a class to using this decorater, code like this
class ShopAreaAndStyleListAPI(Resource):
#redis_hash_shop_style(key='shop_{}_style'.format(g.city.id))
def get(self):
# if not found from redis, query from mysql
pass
As you see, my decorator need a parameter named key, and I pass the key like this
#redis_hash_shop_style(key='shop_{}_style'.format(g.city.id))
g.city.id will be get the city's id, if everything is ok, the key will be like this
shop_100_style
but I got the error:
class ShopAreaAndStyleListAPI(Resource):
File "xx.py", line 659, in ShopAreaAndStyleListAPI
#redis_hash_shop_style(key='shop_{}_style'.format(g.city.id))
File "/Users/xx/.virtualenvs/yy/lib/python2.7/site-packages/werkzeug/local.py", line 347, in __getattr__
return getattr(self._get_current_object(), name)
File "/Users/xx/.virtualenvs/yy/lib/python2.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
return self.__local()
File "/Users/xx/.virtualenvs/yy/lib/python2.7/site-packages/flask/globals.py", line 44, in _lookup_app_object
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that
needed to interface with the current application object in a way.
To solve this set up an application context with app.app_context().
See the documentation for more information.
I am quite confused , in flask, how to pass a dynamic parameter to a decorator?
Thanks.
If we check the docs for flask application global, flask.g, it says:
To share data that is valid for one request only from one function to another, a global variable is not good enough because it would break in threaded environments. Flask provides you with a special object that ensures it is only valid for the active request and that will return different values for each request.
This is achieved by using a thread-local proxy (in flask/globals.py):
g = LocalProxy(partial(_lookup_app_object, 'g'))
The other thing we should keep in mind is that Python is executing the first pass of our decorator during the "compile" phase, outside of any request, or flask application. That means key argument get assigned a value of 'shop_{}_style'.format(g.city.id) when your application starts (when your class is being parsed/decorated), outside of flask request context.
But we can easily delay accessing to flask.g by using a lazy proxy, which fetches the value only when used, via callback function. Let's use the one already bundled with flask, the werkzeug.local.LocalProxy:
from werkzeug.local import LocalProxy
class ShopAreaAndStyleListAPI(Resource):
#redis_hash_shop_style(key=LocalProxy(lambda: 'shop_{}_style'.format(g.city.id)))
def get(self):
# if not found from redis, query from mysql
pass
In general (for non-flask or non-werkzeug apps), we can use a similar LazyProxy from the ProxyTypes package.
Unrelated to this, you'll also want to fix your redis_hash_shop_style decorator to not only fetch from redis, but to also update (or create) the value if stale (or non-existing), by calling the wrapped f() when appropriate.
This is my first outing with CherryPy so forgive any stupidity.
I'm trying to write a RESTful API that in part deals with adding/removing people. I want to be able to GET/PUT/DELETE example.com/people/.
The dispatcher seems to be behaving totally differently for the index method vs a defined function:
class people:
"""This is the class for CherryPy that deals with CRUD on people"""
#cherrypy.expose
def index(self, name):
return name
#cherrypy.expose
def who(self, name):
return name
root = webroot()
root.people = people()
cherrypy.quickstart(root)
If I call example.com/people/tom, I get a 404, if I call example.com/people/who/tom I get 'tom' returned.
Can anyone see what I'm doing wrong? Is there a way I can pass /xxx to index?
Indexes are a bit different when it comes to URL arguments.
The index method has a special role in CherryPy: it handles intermediate URI’s that end in a slash; for example, the URI /orders/items/ might map to root.orders.items.index. The index method can take additional keyword arguments if the request includes querystring or POST params; see Keyword Arguments, next. However, unlike all other page handlers, it cannot take positional arguments
source
However, the url of example.com/people?name=tom should work as you expect.
I learning Python (coming from a dotnet background) and developing an app which interacts with a webservice.
The web service is flat, in that it has numerous calls some of which are related to sessions e.g. logging on etc, whereas other calls are related to retrieving/setting business data.
To accompany the webservice, there are a couple of python classes which wrap all the calls. I am looking to develop a client on top of that class but give the client more OO structure.
The design of my own app was to have a Session-type class which would be responsible for logging on/maintaining the connection etc , but itself would be injected into a Business-type class which is responsible for making all the business calls.
So the stack is something like
WebService (Soap)
WebServiceWrapper (Python)
Session (Python)
Business (Python)
Here's a sample of my code (I've renamed some methods to try and make stuff more explicit)
from webServiceWrapper import webServiceAPI
class Session():
def __init__(self, user, password):
self._api = webServiceAPI()
self.login = self._api.login(user, password)
def webServiceCalls(self):
return self._api()
class Business():
def __init__(self, service):
self._service=service
def getBusinessData(self):
return self._service.get_business_data()
and my unit test
class exchange(unittest.TestCase):
def setUp(self):
self.service = Session("username","password")
self._business = Business(self.service.webServiceCalls())
def testBusinessReturnsData(self):
self.assertFalse(self._business.getBusinessData()==None)
The unit test fails fails on
return self._api()
saying that the underlying class is not callable
TypeError: 'webServiceAPI' is not callable
My first q is, is that the python way? Is the OO thinking which underpins app development with static languages good for dynamic as well? (That's probably quite a big q!)
My second q is that, if this kind of architecture is ok, what am I doing wrong (I guess in terms of passing references to objects in this way)?
Many thx
S
If WebserviceAPI is an object just remove the parentheses like that:
return self._api
You already created an instance of the object in the constructor.
Maybe add the definition of WebserviceAPI to the question, I can only guess at the moment.
I don't see anything that is wrong or un-Pythonic here. Pythonistas often point out the differences to static languages like Java or C#, but many real-world application mostly use a simple static class design in Python, too.
I guess that webServiceAPI is not a class, thus it can't be called.
If you are using Python 2.x, always inherit from the object type, otherwise you'll get a “classic class” (a relict from ancient times that is kept for background compatibility).