I want to do something like "calling a method of a property":
class my_property(property):
def __init__(self, ...):
...
def some_method(self):
# do something fancy like a plausibility check, publishing
# etc. in context of the property.
This improved property would now be used like a normal property except I can call an extra method:
setattr(self.__class__, 'some_value', property(getter, setter))
self.some_value = 5
self.some_value.some_method()
Of course this won't work because some_method() would be called on what the property returned - which is 5 in my case.
Same for this approach:
some_function(self.some_value)
What I could do now is s.th. like this:
some_function(self, "some_value")
But this is bad because this way tools like pylint cannot help me any more which is important for me (e.g. check whether self has an attribute some_value).
Is there a way to use the expression self.some_value without writing it in parenthesis and without just evaluating the property?
My approaches:
Don't use properties. Use getters and setters instead.
self.some_value.set(5)
x = self.some_value.get()
self.some_value.publish()
Use unintuitive black magic
self.some_value = 5 # will assign a value using `__set__()`
x = self.some_value # will read a value using `__get__()`
self.some_value = publish # calls `__set__()` with a well known magic value
publish would be some object known to __set__ which would make __set__ accomplish some task.
Use even darker magic
self.some_value = 5 # will assign a value using `__set__()`
x = self.some_value # will read a value using `__get__()`
some_method(self.some_value) # checks whether `__set__()` has been called from some_method()
This can by done by examining the callstack inside __get()__ and check whether some_method() is the next element. Instead of returning a value it would produce the desired behavior.
Background:
I have implemented some sort of property tree which makes use of properties to implement complex get/get semantics. It can be used like this:
root.some.subtree.element = 5
element is a property-attribute of root.some.subtree. This assignment would call it's __set__ method which e.g. might store the value on disc or send it through TCP.
In the same way
print(root.some.subtree.element)
might read and return some value from any provider.
Propertyies have a set and a get semantic but now I need a new semantic, e.g. publish. So what I would best like to have would be a property implementation with a new __publish__() method.
Of course I can easily read the property value and call a function like this:
publish_value('root.some.subtree.element', root.some.subtree.element)
But in this case pylint can't warn me if I misspelled the string expression.
Edit:
Originally I wrote
self.some_value = property(...)
to apply a property to self. This doesn't work this way so I corrected it. My intention is to let any object (in my case self) have a property which I can define extra operations on.
Related
I am trying to set the attribute values of a certain class AuxiliaryClass than is instantiated in a method from MainClass class in the most efficient way possible.
AuxiliaryClass is instantiated within a method of MainClass - see below. However, AuxiliaryClass has many different attributes and I need to set the value of those attributes once the class has been instantiated - see the last 3 lines of my code.
Note: due to design constraints I cannot explain here, my classes only contain methods, meaning that I need to declare attributes as methods (see below).
class AuxiliaryClass(object):
def FirstMethod(self):
return None
...
def NthMethod(self):
return None
class MainClass(object):
def Auxiliary(self):
return AuxiliaryClass()
def main():
obj = MainClass()
obj.Auxiliary().FirstMethod = #some_value
...
obj.Auxiliary().NthMethod = #some_other_value
# ~~> further code
Basically I want to replace these last 3 lines of code with something neater, more elegant and more efficient. I know I could use a dictionary if I was instantiating AuxiliaryClass directly:
d = {'FirstMethod' : some_value,
...
'NthMethod' : some_other_value}
obj = AuxiliaryClass(**d)
But this does not seem to work for the structure of my problem. Finally, I need to set the values of AuxiliaryClass's attributes once MainClass has been instantiated (so I can't set the attribute's values within method Auxiliary).
Is there a better way to do this than obj.Auxiliary().IthMethod = some_value?
EDIT
A couple of people have said that the following lines:
obj.Auxiliary().FirstMethod = #some_value
...
obj.Auxiliary().NthMethod = #some_other_value
will have no effect because they will immediately get garbage collected. I do not really understand what this means, but if I execute the following lines (after the lines above):
print(obj.Auxiliary().FirstMethod())
...
print(obj.Auxiliary().NthMethod())
I am getting the values I entered previously.
To speed things up, and make the customization somewhat cleaner, you can cache the results of the AuxilliaryClass constructor/singleton/accessor, and loop over a dict calling setattr().
Try something like this:
init_values = {
'FirstMethod' : some_value,
:
'NthMethod' : some_other_value,
}
def main():
obj = MainClass()
aux = obj.Auxiliary() # cache the call, only make it once
for attr,value in init_values.items(): # python3 here, iteritems() in P2
setattr(aux, attr, value)
# other stuff below this point
I understand what is happening here: my code has a series of decorators before all methods which allow memoization. I do not know exactly how they work but when used the problem described above - namely, that lines of type obj.Auxiliary().IthMethod = some_value get immediately garbage collected - does not occur.
Unfortunately I cannot give further details regarding these decorators as 1) I do not understand them very well and 2) I cannot transmit this information outside my company. I think under this circumstances it is difficult to answer my question because I cannot fully disclose all the necessary details.
First, if you guys think the way I'm trying to do things is not Pythonic, feel free to offer alternative suggestions.
I have an object whose functionality needs to change based on outside events. What I've been doing originally is create a new object that inherits from original (let's call it OrigObject()) and overwrites the methods that change (let's call the new object NewObject()). Then I modified both constructors such that they can take in a complete object of the other type to fill in its own values based on the passed in object. Then when I'd need to change functionality, I'd just execute myObject = NewObject(myObject).
I'm starting to see several problems with that approach now. First of all, other places that reference the object need to be updated to reference the new type as well (the above statement, for example, would only update the local myObject variable). But that's not hard to update, only annoying part is remembering to update it in other places each time I change the object in order to prevent weird program behavior.
Second, I'm noticing scenarios where I need a single method from NewObject(), but the other methods from OrigObject(), and I need to be able to switch the functionality on the fly. It doesn't seem like the best solution anymore to be using inheritance, where I'd need to make M*N different classes (where M is the number of methods the class has that can change, and N is the number of variations for each method) that inherit from OrigObject().
I was thinking of using attribute remapping instead, but I seem to be running into issues with it. For example, say I have something like this:
def hybrid_type2(someobj, a):
#do something else
...
class OrigObject(object):
...
def hybrid_fun(self, a):
#do something
...
def switch(type):
if type == 1:
self.hybrid_fun = OrigObject.hybrid_fun
else:
self.fybrid_fun = hybrid_type2
Problem is, after doing this and trying to call the new hybrid_fun after switching it, I get an error saying that hybrid_type2() takes exactly 2 arguments, but I'm passing it one. The object doesn't seem to be passing itself as an argument to the new function anymore like it does with its own methods, anything I can do to remedy that?
I tried including hybrid_type2 inside the class as well and then using self.hybrid_fun = self.hybrid_type2 works, but using self.hybrid_fun = OrigObject.hybrid_fun causes a similar error (complaining that the first argument should be of type OrigObject). I know I can instead define OrigObject.hybrid_fun() logic inside OrigObject.hybrid_type1() so I can revert it back the same way I'm setting it (relative to the instance, rather than relative to the class to avoid having object not be the first argument). But I wanted to ask here if there is a cleaner approach I'm not seeing here? Thanks
EDIT:
Thanks guys, I've given points for several of the solutions that worked well. I essentially ended up using a Strategy pattern using types.MethodType(), I've accepted the answer that explained how to do the Strategy pattern in python (the Wikipedia article was more general, and the use of interfaces is not needed in Python).
Use the types module to create an instance method for a particular instance.
eg.
import types
def strategyA(possible_self):
pass
instance = OrigObject()
instance.strategy = types.MethodType(strategyA, instance)
instance.strategy()
Note that this only effects this specific instance, no other instances will be effected.
You want the Strategy Pattern.
Read about descriptors in Python. The next code should work:
else:
self.fybrid_fun = hybrid_type2.__get__(self, OrigObject)
What about defining it like so:
def hybrid_type2(someobj, a):
#do something else
...
def hybrid_type1(someobj, a):
#do something
...
class OrigObject(object):
def __init__(self):
...
self.run_the_fun = hybrid_type1
...
def hybrid_fun(self, a):
self.run_the_fun(self, a)
def type_switch(self, type):
if type == 1:
self.run_the_fun = hybrid_type1
else:
self.run_the_fun = hybrid_type2
You can change class at runtime:
class OrigObject(object):
...
def hybrid_fun(self, a):
#do something
...
def switch(self):
self.__class__ = DerivedObject
class DerivedObject(OrigObject):
def hybrid_fun(self, a):
#do the other thing
...
def switch(self):
self.__class__ = OrigObject
I have a bunch of variables that are equal to values pulled from a database. Sometimes, the database doesn't have a value and returns "NoneType". I'm taking these variables and using them to build an XML file. When the variable is NoneType, it causes the XML value to read "None" rather than blank as I'd prefer.
My question is: Is there an efficient way to go through all the variables at once and search for a NoneType and, if found, turn it to a blank string?
ex.
from types import *
[Connection to database omitted]
color = database.color
size = database.size
shape = database.shape
name = database.name
... etc
I could obviously do something like this:
if type(color) is NoneType:
color = ""
but that would become tedious for the 15+ variables I have. Is there a more efficient way to go through and check each variable for it's type and then correct it, if necessary? Something like creating a function to do the check/correction and having an automated way of passing each variable through that function?
All the solutions given here will make your code shorter and less tedious, but if you really have a lot of variables I think you will appreciate this, since it won't make you add even a single extra character of code for each variable:
class NoneWrapper(object):
def __init__(self, wrapped):
self.wrapped = wrapped
def __getattr__(self, name):
value = getattr(self.wrapped, name)
if value is None:
return ''
else:
return value
mydb = NoneWrapper(database)
color = mydb.color
size = mydb.size
shape = mydb.shape
name = mydb.name
# All of these will be set to an empty string if their
# original value in the database is none
Edit
I thought it was obvious, but I keep forgetting it takes time until all the fun Python magickery becomes a second nature. :) So how NoneWrapper does its magic? It's very simple, really. Each python class can define some "special" methods names that are easy to identify, because they are always surrounded by two underscores from each side. The most common and well-known of these methods is __init__(), which initializes each instance of the class, but there are many other useful special methods, and one of them is __getattr__(). This method is called whenever someone tries to access an attribute. of an instance of your class, and you can customize it to customize attribute access.
What NoneWrapper does is to override getattr, so whenever someone tries to read an attribute of mydb (which is a NoneWrapper instance), it reads the attribute with the specified name from the wrapped object (in this case, database) and return it - unless it's value is None, in which case it returns an empty string.
I should add here that both object variables and methods are attributes, and, in fact, for Python they are essentially the same thing: all attributes are variables that could be changed, and methods just happen to be variables that have their value set to a function of special type (bound method). So you can also use getattr() to control access to functions, which could lead to many interesting uses.
The way I would do it, although I don't know if it is the best, would be to put the variables you want to check and then use a for statement to iterate through the list.
check_vars = [color,size,shape,name]
for var in check_vars:
if type(var) is NoneType:
var = ""
To add variables all you have to do is add them to the list.
If you're already getting them one at a time, it's not that much longer to write:
def none_to_blank(value):
if value is None:
return ""
return value
color = none_to_blank(database.color)
size = none_to_blank(database.size)
shape = none_to_blank(database.shape)
name = none_to_blank(database.name)
Incidentally, use of "import *" is generally discouraged. Import only what you're using.
you can simply use:
color = database.color or ""
another way is to use a function:
def filter_None(var):
"" if (a is None) else a
color = filter_None(database.color)
I don't know how the database object is structured but another solution is to modify the database object like:
def myget(self, varname):
value = self.__dict__[varname]
return "" if (value is None) else value
DataBase.myget = myget
database = DataBase(...)
[...]
color = database.myget("color")
you can do better using descriptors or properties
I create a class whose objects are initialized with
a bunch of XML code. The class has the ability to extract various parameters out of that XML and to cache them inside the object state variables. The potential amount of these parameters is large and most probably, the user will not need most of them. That is why I have decided to perform a "lazy" initialization.
In the following test case such a parameter is title. When the user tries to access it for the first time, the getter function parses the XML, properly initializes the state variable and return its value:
class MyClass(object):
def __init__(self, xml=None):
self.xml = xml
self.title = None
def get_title(self):
if self.__title is None:
self.__title = self.__title_from_xml()
return self.__title
def set_title(self, value):
self.__title = value
title = property(get_title, set_title, None, "Citation title")
def __title_from_xml(self):
#parse the XML and return the title
return title
This looks nice and works fine for me. However, I am disturbed a little bit by the fact that the getter function is actually a "setter" one in the sense that it has a very significant side effect on the object. Is this a legitimate concern? If so, how should I address it?
This design pattern is called Lazy initialization and it has legitimate use.
While the getter certainly performs a side-effect, that's not traditionally what one would consider a bad side-effect. Since the getter always returns the same thing (barring any intervening changes in state), it has no user-visible side-effects. This is a typical use for properties, so there's nothing to be concerned about.
Quite some years later but well: while lazy initialization is fine in itself, I would definitly not postpone xml parsing etc until someone accesses the object's title. Computed attributes are supposed to behave like plain attributes, and a plain attribute access will never raise (assuming the attribute exists of course).
FWIW I had a very similar case in some project I took over, with xml parsing errors happening at the most unexpected places, due to the previous developper using properties the very same way as in the OP example, and had to fix it by putting the parsing and validation part at instanciation time.
So, use properties for lazy initialization only if and when you know the first access will never ever raise. Actually, never use a property for anything that might raise (at least when getting - setting is a different situation). Else, dont use a property, make the getter an explicit method and clearly document it might raise this or that.
NB : using a property to cache something is not the problem here, this by itself is fine.
I'm writing a program that uses genetic techniques to evolve equations.
I want to be able to submit the function 'mainfunc' to the Parallel Python 'submit' function.
The function 'mainfunc' calls two or three methods defined in the Utility class.
They instantiate other classes and call various methods.
I think what I want is all of it in one NAMESPACE.
So I've instantiated some (maybe it should be all) of the classes inside the function 'mainfunc'.
I call the Utility method 'generate()'. If we were to follow it's chain of execution
it would involve all of the classes and methods in the code.
Now, the equations are stored in a tree. Each time a tree is generated, mutated or cross
bred, the nodes need to be given a new key so they can be accessed from a dictionary attribute of the tree. The class 'KeySeq' generates these keys.
In Parallel Python, I'm going to send multiple instances of 'mainfunc' to the 'submit' function of PP. Each has to be able to access 'KeySeq'. It would be nice if they all accessed the same instance of KeySeq so that none of the nodes on the returned trees had the same key, but I could get around that if necessary.
So: my question is about stuffing EVERYTHING into mainfunc.
Thanks
(Edit) If I don't include everything in mainfunc, I have to try to tell PP about dependent functions, etc by passing various arguements in various places. I'm trying to avoid that.
(late Edit) if ks.next() is called inside the 'generate() function, it returns the error 'NameError: global name 'ks' is not defined'
class KeySeq:
"Iterator to produce sequential \
integers for keys in dict"
def __init__(self, data = 0):
self.data = data
def __iter__(self):
return self
def next(self):
self.data = self.data + 1
return self.data
class One:
'some code'
class Two:
'some code'
class Three:
'some code'
class Utilities:
def generate(x):
'___________'
def obfiscate(y):
'___________'
def ruminate(z):
'__________'
def mainfunc(z):
ks = KeySeq()
one = One()
two = Two()
three = Three()
utilities = Utilities()
list_of_interest = utilities.generate(5)
return list_of_interest
result = mainfunc(params)
It's fine to structure your program that way. A lot of command line utilities follow the same pattern:
#imports, utilities, other functions
def main(arg):
#...
if __name__ == '__main__':
import sys
main(sys.argv[1])
That way you can call the main function from another module by importing it, or you can run it from the command line.
If you want all of the instances of mainfunc to use the same KeySeq object, you can use the default parameter value trick:
def mainfunc(ks=KeySeq()):
key = ks.next()
As long as you don't actually pass in a value of ks, all calls to mainfunc will use the instance of KeySeq that was created when the function was defined.
Here's why, in case you don't know: A function is an object. It has attributes. One of its attributes is named func_defaults; it's a tuple containing the default values of all of the arguments in its signature that have defaults. When you call a function and don't provide a value for an argument that has a default, the function retrieves the value from func_defaults. So when you call mainfunc without providing a value for ks, it gets the KeySeq() instance out of the func_defaults tuple. Which, for that instance of mainfunc, is always the same KeySeq instance.
Now, you say that you're going to send "multiple instances of mainfunc to the submit function of PP." Do you really mean multiple instances? If so, the mechanism I'm describing won't work.
But it's tricky to create multiple instances of a function (and the code you've posted doesn't). For example, this function does return a new instance of g every time it's called:
>>> def f():
def g(x=[]):
return x
return g
>>> g1 = f()
>>> g2 = f()
>>> g1().append('a')
>>> g2().append('b')
>>> g1()
['a']
>>> g2()
['b']
If I call g() with no argument, it returns the default value (initially an empty list) from its func_defaults tuple. Since g1 and g2 are different instances of the g function, their default value for the x argument is also a different instance, which the above demonstrates.
If you'd like to make this more explicit than using a tricky side-effect of default values, here's another way to do it:
def mainfunc():
if not hasattr(mainfunc, "ks"):
setattr(mainfunc, "ks", KeySeq())
key = mainfunc.ks.next()
Finally, a super important point that the code you've posted overlooks: If you're going to be doing parallel processing on shared data, the code that touches that data needs to implement locking. Look at the callback.py example in the Parallel Python documentation and see how locking is used in the Sum class, and why.
Your concept of classes in Python is not sound I think. Perhaps, it would be a good idea to review the basics. This link will help.
Python Basics - Classes