Here is my problem. I have created a pretty heavy readonly class making many database calls with a static "factory" method. The goal of this method is to avoid killing the database by looking in a pool of already-created objects if an identical instance of the same object (same type, same init parameters) already exists.
If something was found, the method will just return it. No problem. But if not, how may I create an instance of the object, in a way that works with inheritance?
>>> class A(Object):
>>> #classmethod
>>> def get_cached_obj(self, some_identifier):
>>> # Should do something like `return A(idenfier)`, but in a way that works
>>> class B(A):
>>> pass
>>> A.get_cached_obj('foo') # Should do the same as A('foo')
>>> A().get_cached_obj('foo') # Should do the same as A('foo')
>>> B.get_cached_obj('bar') # Should do the same as B('bar')
>>> B().get_cached_obj('bar') # Should do the same as B('bar')
Thanks.
import weakref
class A(object):
_get_obj_cache = weakref.WeakValueDictionary()
#classmethod
def get_obj(cls, identifier):
cache = cls._get_obj_cache
obj = cache.get((cls, identifier))
if obj is None:
obj = cache[(cls, identifier)] = cls(identifier)
return obj
class B(A):
pass
Because a WeakValueDictionary is used, the objects will remain cached as long as you have any other reference to them, and you can call SomeClass.get_obj(identifier) as many times as you like to get that same object. If I've understood you correctly, it's the cls(identifier) which will hit the database and what you want to call less frequently, since you know the objects are immutable.
If you want to keep objects in the cache even if they are no longer referenced elsewhere, then change the WeakValueDictionary into a normal dict.
This requires that identifier is suitable for a dict key, and if it's a string as you have in your example code, then it is.
One usual approach is this.
class SomeClass( object ):
# Something that is precious and needs to be pooled.
class SomeClassPool( object ):
def __init__( self ):
self.pool= [ SomeClass() ]
def getInstance( self ):
if len(self.pool) == 0:
self.pool.append( SomeClass() )
# maybe allocate several, depends on the costs
return self.pool.pop()
def release( self, anInstance ):
self.pool.append( anInstance )
We separate the pool from the objects being pooled. They have nothing to do with each other.
You can subclass the objects being pooled all you want.
You can -- independently -- change the pooling strategies without breaking or retesting the objects being pooled.
Expanding on S.Lott's comment:
"I want to return the correct instance
each time, without removing it from
the pool". You mean you want a
dictionary of objects? -S.Lott
the_cache = {}
def get_obj(cls, identifier):
key = (cls, identifier)
if key not in the_cache:
the_cache[key] = cls(identifier)
return the_cache[key]
or
def get_obj(cls, identifier):
key = (cls, identifier)
try:
return the_cache[key]
except KeyError:
the_cache[key] = cls(identifier)
return the_cache[key]
Related
I have a class that I need:
First instance MUST receive a parameter.
All the following instances have this parameter be optional.
If it is not passed then I will use the parameter of the previous object init.
For that, I need to share a variable between the objects (all objects belong to classes with the same parent).
For example:
class MyClass:
shared_variable = None
def __init__(self, paremeter_optional=None):
if paremeter_optional is None: # Parameter optional not given
if self.shared_variable is None:
print("Error! First intance must have the parameter")
sys.exit(-1)
else:
paremeter_optional = self.shared_variable # Use last parameter
self.shared_variable = paremeter_optional # Save it for next object
objA = MyClass(3)
objB = MyClass()
Because the shared_variable is not consistent/shared across inits, when running the above code I get the error:
Error! First intance must have the parameter
(After the second init of objB)
Of course, I could use a global variable but I want to avoid it if possible and use some best practices for this.
Update: Having misunderstood the original problem, I would still recommend being explicit, rather than having the class track information better tracked outside the class.
class MyClass:
def __init__(self, parameter):
...
objA = MyClass(3)
objB = MyClass(4)
objC = MyClass(5)
objD = MyClass(5) # Be explicit; don't "remember" what was used for objC
If objC and objD are "related" enough that objD can rely on the initialization of objC, and you want to be DRY, use something like
objC, objD = [MyClass(5) for _ in range(2)]
Original answer:
I wouldn't make this something you set from an instance at all; it's a class attribute, and so should be set at the class level only.
class MyClass:
shared_variable = None
def __init__(self):
if self.shared_variable is None:
raise RuntimeError("shared_variable must be set before instantiating")
...
MyClass.shared_variable = 3
objA = MyClass()
objB = MyClass()
Assigning a value to self.shared_variable makes self.shared_variable an instance attribute so that the value is not shared among instances.
You can instead assign the value explicitly to the class attribute by referencing the attribute of the instance's class object instead.
Change:
self.shared_variable = paremeter_optional
to:
self.__class__.shared_variable = paremeter_optional
I have a class whose attributes are not known beforehand:
class Event():
def __init__(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
and another one which is basically a list of objects Event:
class Collection(list):
def __init__(self):
self.members = []
def add(self,new):
try:
self.members.extend(new)
except TypeError:
self.members.append(new)
Let's say now that I define 3 objects Event:
a = Event(name="a",value=1)
b = Event(name="b",value=2)
c = Event(name="c",other=True)
And I create a Collection from them:
col = Collection()
col.add([a,b,c])
What I want is to be able to print out all the values of the objects in the list for a given attribute (if the attribute does not exist for an object, it should return None or any other pre-defined value). For example:
print col.name #should return ["a","b","c"]
print col.value #should return [1,2,None]
I have read the following answer: Extract list of attributes from list of objects in python
But that doesn't work here since the name of my attribute is not known by advance, and some might not even be defined. How should I define my class Collection(), or maybe even re-think everything to achieve my goal ?
This is a variation of "I want to create dynamic variable names". The solution here is the same: use a dictionary.
class Event(object):
def __init__(self, **kwargs):
self.attributes = dict(kwargs)
Your Collection class will need a custom __getattr__ method, so that it can look up values in its Event list instead.
class Collection(object):
# assume self.events is a list of Event objects
def __getattr__(self, name):
return [event.attributes.get(name) for event in self.events]
You could stick with your current implementation of Event, and have Collection look at event.__dict__ instead of event.attributes. I don't recall, though, if __dict__ might contain anything else besides the attributes you explicitly set. I'd err on the side of caution.
You can just override the __getattr__ method of the Collection class, which is called when an attribute is accessed. In order to access to unknown set of attributes you can use event.__dict__. So, a possible solution is like this:
def __getattr__(self, name):
return [m.__dict__.get(name) for m in self.members]
I have a dict of different types for which I want to add a simple getter based on the name of the actual parameter.
For example, for three storage parameters, let's say:
self.storage = {'total':100,'used':88,'free':1}
I am looking now for a way (if possible?) to generate a function on the fly with some meta-programming magic.
Instead of
class spaceObj(object):
def getSize(what='total'):
return storage[what]
or hard coding
#property
def getSizeTotal():
return storage['total']
but
class spaceObj(object):
# manipulting the object's index and magic
#property
def getSize:
return ???
so that calling mySpaceObj.getSizeFree would be derived - with getSize only defined once in the object and related functions derived from it by manipulating the objects function list.
Is something like that possible?
While certainly possible to get an unknown attribute from a class as a property, this is not a pythonic approach (__getattr__ magic methods are rather rubyist)
class spaceObj(object):
storage = None
def __init__(self): # this is for testing only
self.storage = {'total':100,'used':88,'free':1}
def __getattr__(self, item):
if item[:7] == 'getSize': # check if an undefined attribute starts with this
return self.getSize(item[7:])
def getSize(self, what='total'):
return self.storage[what.lower()]
print (spaceObj().getSizeTotal) # 100
You can put the values into the object as properties:
class SpaceObj(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
storage = {'total':100,'used':88,'free':1}
o = SpaceObj(**storage)
print o.total
or
o = SpaceObj(total=100, used=88, free=1)
print o.total
or using __getattr__:
class SpaceObj(object):
def __init__(self, **kwargs):
self.storage = kwargs
def __getattr__(self,name):
return self.storage[name]
o = SpaceObj(total=100, used=88, free=1)
print o.total
The latter approach takes a bit more code but it's more safe; if you have a method foo and someone create the instance with SpaceObj(foo=1), then the method will be overwritten with the first approach.
>>> import new
>>> funcstr = "def wat(): print \"wat\";return;"
>>> funcbin = compile(funcstr,'','exec')
>>> ns = {}
>>> exec funcbin in ns
>>> watfunction = new.function(ns["wat"].func_code,globals(),"wat")
>>> globals()["wat"]=watfunction
>>> wat()
wat
I find it very interesting the way how SQLAlchemy constructing query strings, eg:
(Session.query(model.User)
.filter(model.User.age > 18)
.order_by(model.User.age)
.all())
As far as I can see, there applied some kind of Proxy Pattern. In my small project I need to make similar string construction using OOP approach. So, I tried to reconstitute this behavior.
Firstly, some kind of object, one of plenty similar objects:
class SomeObject(object):
items = None
def __init__(self):
self.items = []
def __call__(self):
return ' '.join(self.items) if self.items is not None else ''
def a(self):
self.items.append('a')
return self
def b(self):
self.items.append('b')
return self
All methods of this object return self, so I can call them in any order and unlimited number of times.
Secondly, proxy object, that will call subject's methods if it's not a perform method, which calls object to see the resulting string.
import operator
class Proxy(object):
def __init__(self, some_object):
self.some_object = some_object
def __getattr__(self, name):
self.method = operator.methodcaller(name)
return self
def __call__(self, *args, **kw):
self.some_object = self.method(self.some_object, *args, **kw)
return self
def perform(self):
return self.some_object()
And finally:
>>> obj = SomeObject()
>>> p = Proxy(obj)
>>> print p.a().a().b().perform()
a a b
What can you say about this implementation? Is there better ways to make the desirable amount of classes that would make such a string cunstructing with the same syntax?
PS: Sorry for my english, it's not my primary language.
Actually what you are looking at is not a proxy pattern but the builder pattern, and yes your implementation is IMHO is the classic one (using the Fluent interface pattern).
I don't know what SQLAlchemy does, but I would implement the interface by having the Session.query() method return a Query object with methods like filter(), order_by(), all() etc. Each of these methods simply returns a new Query object taking into account the applied changes. This allows for method chaining as in your first example.
Your own code example has numerous problems. One example
obj = SomeObject()
p = Proxy(obj)
a = p.a
b = p.b
print a().perform() # prints b
I am programming a simulations for single neurons. Therefore I have to handle a lot of Parameters. Now the Idea is that I have two classes, one for a SingleParameter and a Collection of parameters. I use property() to access the parameter value easy and to make the code more readable. This works perfect for a sinlge parameter but I don't know how to implement it for the collection as I want to name the property in Collection after the SingleParameter. Here an example:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(self):
return self._v
def set(self, value):
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
# par1 and par2 I can access perfectly via 'p1.v = ...'
# or get its value with 'p1.v'
class Collection(object):
def __init__(self):
self.dict = {}
def __getitem__(self, name):
return self.dict[name] # get the whole object
# to get the value instead:
# return self.dict[name].v
def add(self, parameter):
self.dict[parameter.name] = parameter
# now comes the part that I don't know how to implement with property():
# It shoule be something like
# self.__dict__[parameter.name] = property(...) ?
col = Collection()
col.add(par1)
col.add(par2)
col['par1'] # gives the whole object
# Now here is what I would like to get:
# col.par1 -> should result like col['par1'].v
# col.par1 = 5 -> should result like col['par1'].v = 5
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?
Look at built-in functions getattr and setattr. You'll probably be a lot happier.
Using the same get/set functions for both classes forces you into an ugly hack with the argument list. Very sketchy, this is how I would do it:
In class SingleParameter, define get and set as usual:
def get(self):
return self._s
def set(self, value):
self._s = value
In class Collection, you cannot know the information until you create the property, so you define the metaset/metaget function and particularize them only later with a lambda function:
def metaget(self, par):
return par.s
def metaset(self, value, par):
par.s = value
def add(self, par):
self[par.name] = par
setattr(Collection, par.name,
property(
fget=lambda x : Collection.metaget(x, par),
fset=lambda x, y : Collection.metaset(x,y, par))
Properties are meant to dynamically evaluate attributes or to make them read-only. What you need is customizing attribute access. __getattr__ and __setattr__ do that really fine, and there's also __getattribute__ if __getattr__ is not enough.
See Python docs on customizing attribute access for details.
Have you looked at the traits package? It seems that you are reinventing the wheel here with your parameter classes. Traits also have additional features that might be useful for your type of application (incidently I know a person that happily uses traits in neural simulations).
Now I implemented a solution with set-/getattr:
class Collection(object):
...
def __setattr__(self, name, value):
if 'dict' in self.__dict__:
if name in self.dict:
self[name].v = value
else:
self.__dict__[name] = value
def __getattr__(self, name):
return self[name].v
There is one thing I quite don't like that much: The attributes are not in the __dict__. And if I have them there as well I would have a copy of the value - which can be dangerous...
Finally I succeded to implement the classes with property(). Thanks a lot for the advice. It took me quite a bit to work it out - but I can promise you that this exercise helps you to understand better pythons OOP.
I implemented it also with __getattr__ and __setattr__ but still don't know the advantages and disadvantages to the property-solution. But this seems to be worth another question. The property-solutions seems to be quit clean.
So here is the code:
class SingleParameter(object):
def __init__(self, name, default_value=0, unit='not specified'):
self.name = name
self.default_value = default_value
self.unit = unit
self.set(default_value)
def get(*args):
self = args[0]
print "get(): "
print args
return self._v
def set(*args):
print "set(): "
print args
self = args[0]
value = args[-1]
self._v = value
v = property(fget=get, fset=set, doc='value of parameter')
class Collection(dict):
# inheriting from dict saves the methods: __getitem__ and __init__
def add(self, par):
self[par.name] = par
# Now here comes the tricky part.
# (Note: this property call the get() and set() methods with one
# more argument than the property of SingleParameter)
setattr(Collection, par.name,
property(fget=par.get, fset=par.set))
# Applying the classes:
par1 = SingleParameter(name='par1', default_value=10, unit='mV')
par2 = SingleParameter(name='par2', default_value=20, unit='mA')
col = Collection()
col.add(par1)
col.add(par2)
# Setting parameter values:
par1.v = 13
col.par1 = 14
# Getting parameter values:
par1.v
col.par1
# checking identity:
par1.v is col.par1
# to access the whole object:
col['par1']
As I am new I am not sure how to move on:
how to treat follow up questions (like this itself):
get() is seems to be called twice - why?
oop-design: property vs. "__getattr__ & __setattr__" - when should I use what?
is it rude to check the own answer to the own question as accepted?
is it recommended to rename the title in order to put correlated questions or questions elaborated with the same example into the same context?
Other questions that I put to understand property():
Why do managed attributes just work for class attributes and not for instance attributes in python?
How can I assign a new class attribute via __dict__ in python?
I have a class that does something similar, but I did the following in the collection object:
setattr(self, par.name, par.v)