I love CherryPy's API for sessions, except for one detail. Instead of saying cherrypy.session["spam"] I'd like to be able to just say session["spam"].
Unfortunately, I can't simply have a global from cherrypy import session in one of my modules, because the cherrypy.session object isn't created until the first time a page request is made. Is there some way to get CherryPy to initialize its session object immediately instead of on the first page request?
I have two ugly alternatives if the answer is no:
First, I can do something like this
def import_session():
global session
while not hasattr(cherrypy, "session"):
sleep(0.1)
session = cherrypy.session
Thread(target=import_session).start()
This feels like a big kludge, but I really hate writing cherrypy.session["spam"] every time, so to me it's worth it.
My second solution is to do something like
class SessionKludge:
def __getitem__(self, name):
return cherrypy.session[name]
def __setitem__(self, name, val):
cherrypy.session[name] = val
session = SessionKludge()
but this feels like an even bigger kludge and I'd need to do more work to implement the other dictionary functions such as .get
So I'd definitely prefer a simple way to initialize the object myself. Does anyone know how to do this?
For CherryPy 3.1, you would need to find the right subclass of Session, run its 'setup' classmethod, and then set cherrypy.session to a ThreadLocalProxy. That all happens in cherrypy.lib.sessions.init, in the following chunks:
# Find the storage class and call setup (first time only).
storage_class = storage_type.title() + 'Session'
storage_class = globals()[storage_class]
if not hasattr(cherrypy, "session"):
if hasattr(storage_class, "setup"):
storage_class.setup(**kwargs)
# Create cherrypy.session which will proxy to cherrypy.serving.session
if not hasattr(cherrypy, "session"):
cherrypy.session = cherrypy._ThreadLocalProxy('session')
Reducing (replace FileSession with the subclass you want):
FileSession.setup(**kwargs)
cherrypy.session = cherrypy._ThreadLocalProxy('session')
The "kwargs" consist of "timeout", "clean_freq", and any subclass-specific entries from tools.sessions.* config.
Related
I have seen different "patterns" in handling this case so I am wondering if one has any drawbacks comapred to the other.
So lets assume that we wish to create a new object of class MyClass and add it to the database. We can do the following:
class MyClass:
pass
def builder_method_for_myclass():
# A lot of code here..
return MyClass()
my_object=builder_method_for_myclass()
with db.managed_session() as s:
s.add(my_object)
which seems that only keeps the session open for adding the new object but I have also seen cases where the entire builder method is called and executed within the managed session like so:
class MyClass:
pass
def builder_method_for_myclass():
# A lot of code here..
return MyClass()
with db.managed_session() as s:
my_object=builder_method_for_myclass()
are there any downsides in either of these methods and if yes what are they? Cant find something specific about this in the documentation.
When you build objects depending on objects fetched from a session you have to be in a session. So a factory function can only execute outside a session for the simplest cases. Usually you have to pass the session around or make it available on a thread local.
For example in this case to build a product I need to fetch the product category from the database into the session. So my product factory function depends on the session instance. The new product is created and added to the same session that the category is also in. An implicit commit should also occur when the session ends, ie the context manager completes.
def build_product(session, category_name):
category = session.query(ProductCategory).where(
ProductCategory.name == category_name).first()
return Product(category=category)
with db.managed_session() as s:
my_product = build_product(s, "clothing")
s.add(my_product)
I'm writing a CLI to interact with elasticsearch using the elasticsearch-py library. I'm trying to mock elasticsearch-py functions in order to test my functions without calling my real cluster.
I read this question and this one but I still don't understand.
main.py
Escli inherits from cliff's App class
class Escli(App):
_es = elasticsearch5.Elasticsearch()
settings.py
from escli.main import Escli
class Settings:
def get(self, sections):
raise NotImplementedError()
class ClusterSettings(Settings):
def get(self, setting, persistency='transient'):
settings = Escli._es.cluster\
.get_settings(include_defaults=True, flat_settings=True)\
.get(persistency)\
.get(setting)
return settings
settings_test.py
import escli.settings
class TestClusterSettings(TestCase):
def setUp(self):
self.patcher = patch('elasticsearch5.Elasticsearch')
self.MockClass = self.patcher.start()
def test_get(self):
# Note this is an empty dict to show my point
# it will contain childs dict to allow my .get(persistency).get(setting)
self.MockClass.return_value.cluster.get_settings.return_value = {}
cluster_settings = escli.settings.ClusterSettings()
ret = cluster_settings.get('cluster.routing.allocation.node_concurrent_recoveries', persistency='transient')
# ret should contain a subset of my dict defined above
I want to have Escli._es.cluster.get_settings() to return what I want (a dict object) in order to not make the real HTTP call, but it keeps doing it.
What I know:
In order to mock an instance method I have to do something like
MagicMockObject.return_value.InstanceMethodName.return_value = ...
I cannot patch Escli._es.cluster.get_settings because Python tries to import Escli as module, which cannot work. So I'm patching the whole lib.
I desperately tried to put some return_value everywhere but I cannot understand why I can't mock that thing properly.
You should be mocking with respect to where you are testing. Based on the example provided, this means that the Escli class you are using in the settings.py module needs to be mocked with respect to settings.py. So, more practically, your patch call would look like this inside setUp instead:
self.patcher = patch('escli.settings.Escli')
With this, you are now mocking what you want in the right place based on how your tests are running.
Furthermore, to add more robustness to your testing, you might want to consider speccing for the Elasticsearch instance you are creating in order to validate that you are in fact calling valid methods that correlate to Elasticsearch. With that in mind, you can do something like this, instead:
self.patcher = patch('escli.settings.Escli', Mock(Elasticsearch))
To read a bit more about what exactly is meant by spec, check the patch section in the documentation.
As a final note, if you are interested in exploring the great world of pytest, there is a pytest-elasticsearch plugin created to assist with this.
Let's say I have a class which stores important, dynamic data which I need to render my sites. This class should be created individually per user, and the data from the class needs to get updated according to some user input, so I have at least two views:
#app.route('/')
def index():
myclass = MyClass()
return render_template('index.html')
#app.route('/_update', methods=['GET'])
def update():
ret_data = {"value": request.args.get('c', type=float)}
a = myclass.calculate(ret_data['value'])
return jsonify(result=a)
Ofcourse it can't work this way, because myclass wouldn't exist in update() - so a working solution would be to make myclass global on creation. But this doesn't seem clean and it ruins the possibility for individual classes per session.
So, is there any clean way to access a class instance in different views, or how should I handle this, it doesn't feel like an uncommon scenario to me.
Secondly, I would like to have the class instance created for every user, but also closed when every a user closes his browser window etc. - I don't really get how to do this, I have a __del__() function in my class, but it won't be used if I set the instance to global.
Thank you very much!
You have a fundamental misunderstanding about how web applications work. They are stateless: nothing is shared between requests for any particular user. On the contrary, any global variable will be accessible to whichever user happens to hit the application next.
The way to store state for a user is to put it in the database. You can use a session cookie to associate an individual user with their particular data.
As Daniel Rosemann pointed out, it's probably not how one should design a web application. There is however a way to reach that functionality using global variables plus multiple instances. I don't know enough about python to estimate how wrong (or even dangerous) the use of global variables is, but it seems working for me - I'm happy about every comment on this solution:
Setting two global dicts, one to store the class instances, one for keep track if the instance is still relevant:
global session_instances, session_alive
session_instances = {}
session_alive = {}
In my root view I create a uuid and save the class instance with it in the dict and start a thread which should close my class after some time:
#app.route('/')
def index():
if not session.get('uid'):
session['uid'] = uuid.uuid4()
session_instances.update({session['uid'] : pyd2d.d2d()})
session_alive.update({session['uid'] : 0})
t = Thread(target=close_session, args = (session['uid'], ))
t.start()
return render_template('index.html')
The thread responsible for closing (e.g. 15 seconds after the last request):
def close_session(uid):
global session_alive
while True:
time.sleep(15)
if session_alive[uid] == 0:
session_instances[uid].stop()
break
session_alive[uid] = 0
And finally, to update the timer anytime a request is send:
#app.before_request
def before_request():
global session_alive
if session.get('uid'):
session_alive[session['uid']] = 1
This seems to work just fine. Should I feel bad about using global variables, or is it ok in cases like this? I appreciate any input!
I'm writing a test for a caching mechanism. The mechanism has two cache layers, the request cache and redis. The request cache uses Flask.g, an object that stores values for the duration of the request. It does this by creating a dictionary, on the Flask.g._cache attribute.
However, I think that the exact attribute is an implementation detail that my unit test shouldn't care about. I want to make sure it stores its values on Flask.g, but I don't care about how it does that. What would be a good way to test this?
I'm using the Python mock module, so I know I can mock out `Flask.g, but I'm not sure if there's a way to test whether there has been any property access on it, without caring about which property it is.
Is this even the right approach for tests like this?
Personally you shouldn't be mocking Flask.g if you are testing endpoints. since you would be creating a self.app (I may be unsure about this portion)
Secondly you will need to mock the redis client with something like this being the returned structure.
class RedisTestCase(object):
saved_dictionary = {}
def keys(self, pattern):
found_keys = []
for key in RedisTestCase.saved_dictionary: #loop keys for pattern
if pattern:
found_keys.append(key)
return found_keys
def delete(self, keys):
for key in keys:
if key in RedisTestCase.saved_dictionary:
del RedisTestCase.saved_dictionary[key]
def mset(self, items):
RedisTestCase.saved_dictionary.update(items)
def pipeline(self):
return RedisTestCase()
I have a module which wrap an json api for querying song cover/remix data with limits for number of requests per hour/minute. I'd like to keep an optional cache of json responses without forcing users to adjust a cache/context parameter every time. What is a good way of initializing a library/module in python? Or would you recommend I just do the explicit thing and use a cache named parameter in every call that eventually request json data?
I was thinking of doing
_cache = None
class LFU(object):
...
NO_CACHE, LFU = "NO_CACHE", "LFU"
def set_cache_strategy(strategy):
if _cache == NO_CACHE:
_cache = None
else:
_cache = LFU()
import second_hand_songs_wrapper as s
s.set_cache_strategy(s.LFU)
l1 = s.ShsLabel.get_from_resource_id(123)
l2 = s.ShsLabel.get_from_resource_id(123,use_cache=Fale)
edit:
I'm probably only planning on having two strategies one with/ one without a cache.
Other possible alternative initialization schemes I can think of of the top of my head include using enviromental variables, initializing _cache by hand in the user code to None/LFU(), and using explicit cache everywhere(possibly defaulting to having a cache).
Note the reason I don't set cache on an instance of the class is that I currently use a never instantiated class(use class functions + class state as a singleton) to abstract downloading the json data along with some convenience/methods to download certain urls. I could instantiate the downloader class but then I'd have to pass the instance explicitly to each function, or use another global variable for a convenience version of the class. The downloader class also keeps track of # of requests(website has limit per minute/hour) so having multiple downloader objects would cause more trouble.
There's nothing wrong in setting a default, even if that default is None. I would note though that having the pseudo-constants as well as a conditional (provided that's all you use the values for) is redundant. Try:
caching_strategies = {'NO_CACHE' : lambda: None,
'LFU' : LFU}
_cache = caching_strategies['NO_CACHE']
def set_cache_strategy(strategy):
_cache = caching_methods[strategy]()
If you want to provide a convenience method for the available strategies, just wrap caching_strategies.keys(). Really though, as far as your strategies go, you should probably have all your strategies inherit from some base strategy, and just create a no_cache strategy class that inherits from that and stubs all the methods for your standardized caching inteface.