I find myself with a layout problem. I have an instance method that returns an instance of another class, but to do so requires the first instance. For example:
class bar:
def __init__(self, barvalue, foo_instance, flag, x):
# munge barvalue based on foo_instance, flag and x
self._data = barvalue
class foo(basefoo):
def __init__(self, value, flag, x):
self._data=value # same as basefoo
if flag:
self._extra=x #absent in basefoo
innervalue=value
self.munged = bar(innervalue, self, flag, x)
The self.munged = bar(value, self, x, y) seems a bit hackish, so I would like to check if this the cleanest way.
Nested class tangent
If need be, bar could be made a sub-class [sic. nested class] of foo — the idea of bar(basefoo(value),barvalue, flag, x) is nice, but I could write a class method to covert basefoo to a foo.
But I am not sure what I would gain. In a previous edit I misunderstood how the nested class would work. It turns out the inner class (bar) does not receive the self form self.bar. I thought it was because __new__ gets called before __init__, but this is not the case as __new__ does not receive the self form self.bar. but this is a topic for another question.
class foo(basefoo):
class bar:
def __init__(self, barvalue, foo_instance, flag, x):
# munge barvalue based on foo_instance, flag and x
self._data = barvalue
def __init__(self, value, flag, x):
self._data=value # same as basefoo
if flag:
self._extra=x #absent in basefoo
innervalue=value
self.munged = self.bar(innervalue, self)
#NOT self.munged = self.bar(innervalue)
and force the user to covert a basefoo object to a foo object to get bar if need be? Is this the right line of thought?
Concern
This is the second time I have found myself writing this layout —the other turned messy see GitHub repo of messy code. So I don't want to repeat the messiness.
So, is this the best way or is there a clever trick?
Also, does this situation have a name? My google-fu failed me.
Related
I have seen super().__init__(*args) used to call the super constructor safely (in a way that does not fail to diamond inheritence). However I cannot find a way to call different super constructors with different arguments in this way.
Here is an example illustraiting the problem.
from typing import TypeVar, Generic
X = TypeVar("X")
Y = TypeVar("Y")
class Base:
def __init__(self):
pass
class Left(Base, Generic[X]):
def __init__(self, x:X):
super().__init__()
self.lft = x
class TopRight(Base, Generic[Y]):
def __init__(self, y:Y):
super().__init__()
self.rgh = y
class BottomRight(TopRight[Y], Generic[Y]):
def __init__(self, y:Y):
super().__init__(y + y)
class Root(Left[X], BottomRight[Y], Generic[X, Y]):
def __init__(self, x:X, y:Y):
pass #issue here
#does not work
#super().__init__(x)
#super().__init__(y)
#calls base twice
#Left[X].__init__(x)
#BottomRight[Y].__init__(y)
How do I call Left.__init__(x) and BottomRight.__init__(y) seperately and safely?
The thing is that to be use in cooperative form, the intermediate classes have to accept the arguments that are not "aimed" at them, and pass those on on their own super call, in a way that becomes transparent.
You them do not place multiple calls to your ancestor classes: you let the language runtime do that for you.
Your code should be written:
from typing import Generic, TypeVar
X = TypeVar("X")
Y = TypeVar("Y")
class Base:
def __init__(self):
pass
class Left(Base, Generic[X]):
def __init__(self, x:X, **kwargs):
super().__init__(**kwargs)
self.lft = x
class TopRight(Base, Generic[Y]):
def __init__(self, y:Y, **kwargs):
super().__init__(**kwargs)
self.rgh = y
class BottomRight(TopRight[Y], Generic[Y]):
def __init__(self, y:Y, **kwargs): # <- when this is executed, "y" is extracted from kwargs
super().__init__(y=y + y, **kwargs) # <- "x" remains in kwargs, but this class does not have to care about it.
class Root(Left[X], BottomRight[Y], Generic[X, Y]):
def __init__(self, x:X, y:Y):
super().__init__(x=x, y=y) # <- will traverse all superclasses, "Generic" being last
Also, note that depending on your project's ends, and final complexity, these type annotations may gain you nothing, and instead, add complexity to a code otherwise trivial. They are not always a gain in Python projects, although due to circunstances the tooling (i.e. IDEs), might recommend them.
Also, check this similar answer from a few days ago, were I detail a bit more of Python method resolution order mechanisms, and point to the official documentation on them: In multiple inheritance in Python, init of parent class A and B is done at the same time?
I have the following problem and I need advice on how to solve it the best technically in Python. As I am new to programming I would like to have some advice.
So I will have the following object and they should store something. Here is an example:
object 1: cash dividends (they will have the following properties)
exdate (will store a list of dates)
recorddate (will store a list of dates)
paydate (will store a list of dates)
ISIN (will store a list of text)
object 2: stocksplits (they will have the following prpoerties)
stockplitratio (will be some ration)
exdate(list of dates)
...
I have tried to solve it like this:
class cashDividends(object):
def __init__(self, _gross,_net,_ISIN, _paydate, _exdate, _recorddate, _frequency, _type, _announceddate, _currency):
self.gross = _gross
self.net = _net
self.ISIN = _ISIN
self.paydate = _paydate
self.exdate = _exdate
self.recorddate = _recorddate
self.frequency = _frequency
self.type = _type
self.announceddate = _announceddate
self.currency = _currency
So if I have this I would have to create another class named stockplits and then define an __init__ function again.
However is there a way where I can have one class like "Corporate Actions" and then have stock splits and cashdividends in there ?
Sure you can! In python you can pass classes to other classes.
Here a simple example:
class A():
def __init__(self):
self.x = 0
class B():
def __init__(self):
self.x = 1
class Container():
def __init__(self, objects):
self.x = [obj.x for obj in objects]
a = A()
b = B()
c = Container([a,b])
c.x
[0,1]
If I understood correctly what you want is an object that has other objects from a class you created as property?
class CorporateActions(object):
def __init__(self, aCashDividend, aStockSplit):
self.cashDividend = aCashDividend
self.stockSplit = aStockSplit
myCashDividends = CashDividends(...) #corresponding parameters here
myStockSplit = StockSplit(...)
myCorporateActions = CorporateActions(myCashDividends, myStockSplit)
Strictly speaking this answer isn't an answer for the final question. However, it is a way to make your life slightly easier.
Consider creating a sort-of template class (I'm using this term loosely; there's no such thing in Python) that does the __init__ work for you. Like this:
class KwargAttrs():
def __init__(self, **kwargs):
for k,v in kwargs.items():
setattr(self, k, v)
def _update(self, **kwargs):
args_dict = {k:(kwargs[k] if k in kwargs else self.__dict__[k]) for k in self.__dict__}
self.__dict__.update(args_dict)
This class uses every supplied keyword argument as an object attribute. Use it this way:
class CashDividends(KwargAttrs):
def __init__(self, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
# save the namespace before it gets polluted
super().__init__(**locals())
# work that might pollute local namespace goes here
# OPTIONAL: update the argument values in case they were modified:
super()._update(**locals())
Using a method like this, you don't have to go through the argument list and assign every single object attribute; it happens automatically.
We bookend everything you need to accomplish in the __init__ method with method calls to the parent-class via super(). We do this because locals() returns a dict every variable in the function's current namespace, so you need to 1.) capture that namespace before any other work pollutes it and 2.) update the namespace in case any work changes the argument values.
The call to update is optional, but the values of the supplied arguments will not be updated if something is done to them after the call to super().__init__() (that is, unless you change the values using setattr(self, 'argname, value)`, which is not a bad idea).
You can continue using this class like so:
class StockSplits(KwargAttrs):
def __init__(self, stocksplitratio, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
super().__init__(**locals())
As mentioned in the other answers you can create a container for our other classes, but you can even do that using this same template class:
class CorporateActions(KwargAttrs):
def __init__(self, stock_splits , cash_dividends):
super().__init__(**locals())
ca = CorporateActions(stock_splits = StockSplits(<arguments>), cash_dividends = CashDividends(<arguments>) )
Question
How can you extend a python property?
A subclass can extend a super class's function by calling it in the overloaded version, and then operating on the result. Here's an example of what I mean when I say "extending a function":
# Extending a function (a tongue-in-cheek example)
class NormalMath(object):
def __init__(self, number):
self.number = number
def add_pi(self):
n = self.number
return n + 3.1415
class NewMath(object):
def add_pi(self):
# NewMath doesn't know how NormalMath added pi (and shouldn't need to).
# It just uses the result.
n = NormalMath.add_pi(self)
# In NewMath, fractions are considered too hard for our users.
# We therefore silently convert them to integers.
return int(n)
Is there an analogous operation to extending functions, but for functions that use the property decorator?
I want to do some additional calculations immediately after getting an expensive-to-compute attribute. I need to keep the attribute's access lazy. I don't want the user to have to invoke a special routine to make the calculations. basically, I don't want the user to ever know the calculations were made in the first place. However, the attribute must remain a property, since i've got legacy code I need to support.
Maybe this is a job for decorators? If I'm not mistaken, decorator is a function that wraps another function, and I'm looking to wrap a property with some more calculations, and then present it as a property again, which seems like a similar idea... but I can't quite figure it out.
My Specific Problem
I've got a base class LogFile with an expensive-to-construct attribute .dataframe. I've implemented it as a property (with the property decorator), so it won't actually parse the log file until I ask for the dataframe. So far, it works great. I can construct a bunch (100+) LogFile objects, and use cheaper methods to filter and select only the important ones to parse. And whenever I'm using the same LogFile over and over, i only have to parse it the first time I access the dataframe.
Now I need to write a LogFile subclass, SensorLog, that adds some extra columns to the base class's dataframe attribute, but I can't quite figure out the syntax to call the super class's dataframe construction routines (without knowing anything about their internal workings), then operate on the resulting dataframe, and then cache/return it.
# Base Class - rules for parsing/interacting with data.
class LogFile(object):
def __init__(self, file_name):
# file name to find the log file
self.file_name = file_name
# non-public variable to cache results of parse()
self._dataframe = None
def parse(self):
with open(self.file_name) as infile:
...
...
# Complex rules to interpret the file
...
...
self._dataframe = pandas.DataFrame(stuff)
#property
def dataframe(self):
"""
Returns the dataframe; parses file if necessary. This works great!
"""
if self._dataframe is None:
self.parse()
return self._dataframe
#dataframe.setter
def dataframe(self,value):
self._dataframe = value
# Sub class - adds more information to data, but does't parse
# must preserve established .dataframe interface
class SensorLog(LogFile):
def __init__(self, file_name):
# Call the super's constructor
LogFile.__init__(self, file_name)
# SensorLog doesn't actually know about (and doesn't rely on) the ._dataframe cache, so it overrides it just in case.
self._dataframe = None
# THIS IS THE PART I CAN'T FIGURE OUT
# Here's my best guess, but it doesn't quite work:
#property
def dataframe(self):
# use parent class's getter, invoking the hidden parse function and any other operations LogFile might do.
self._dataframe = LogFile.dataframe.getter()
# Add additional calculated columns
self._dataframe['extra_stuff'] = 'hello world!'
return self._dataframe
#dataframe.setter
def dataframe(self, value):
self._dataframe = value
Now, when these classes are used in an interactive session, the user should be able to interact with either in the same way.
>>> log = LogFile('data.csv')
>>> print log.dataframe
#### DataFrame with 10 columns goes here ####
>>> sensor = SensorLog('data.csv')
>>> print sensor.dataframe
#### DataFrame with 11 columns goes here ####
I have lots of existing code that takes a LogFile instance which provides a .dataframe attribute and dos something interesting (mostly plotting). I would LOVE to have SensorLog instances present the same interface so they can use the same code. Is it possible to extend the super-class's dataframe getter to take advantage of existing routines? How? Or am I better off doing this a different way?
Thanks for reading that huge wall of text. You are an internet super hero, dear reader. Got any ideas?
You should be calling the superclass properties, not bypassing them via self._dataframe. Here's a generic example:
class A(object):
def __init__(self):
self.__prop = None
#property
def prop(self):
return self.__prop
#prop.setter
def prop(self, value):
self.__prop = value
class B(A):
def __init__(self):
super(B, self).__init__()
#property
def prop(self):
value = A.prop.fget(self)
value['extra'] = 'stuff'
return value
#prop.setter
def prop(self, value):
A.prop.fset(self, value)
And using it:
b = B()
b.prop = dict((('a', 1), ('b', 2)))
print(b.prop)
Outputs:
{'a': 1, 'b': 2, 'extra': 'stuff'}
I would generally recommend placing side-effects in setters instead of getters, like this:
class A(object):
def __init__(self):
self.__prop = None
#property
def prop(self):
return self.__prop
#prop.setter
def prop(self, value):
self.__prop = value
class B(A):
def __init__(self):
super(B, self).__init__()
#property
def prop(self):
return A.prop.fget(self)
#prop.setter
def prop(self, value):
value['extra'] = 'stuff'
A.prop.fset(self, value)
Having costly operations within a getter is also generally to be avoided (such as your parse method).
If I understand correctly what you want to do is call the parent's method from the child instance. The usual way to do that is by using the super built-in.
I've taken your tongue-in-cheek example and modified it to use super in order to show you:
class NormalMath(object):
def __init__(self, number):
self.number = number
def add_pi(self):
n = self.number
return n + 3.1415
class NewMath(NormalMath):
def add_pi(self):
# this will call NormalMath's add_pi with
normal_maths_pi_plus_num = super(NewMath, self).add_pi()
return int(normal_maths_pi_plus_num)
In your Log example, instead of calling:
self._dataframe = LogFile.dataframe.getter()
you should call:
self._dataframe = super(SensorLog, self).dataframe
You can read more about super here
Edit: Even thought the example I gave you deals with methods, to do the same with #properties shouldn't be a problem.
You have some possibilities to consider:
1/ Inherit from logfile and override parse in your derived sensor class. It should be possible to modify your methods that work on dataframe to work regardless of the number of members that dataframe has - as you are using pandas a lot of it is done for you.
2/ Make sensor an instance of logfile then provide its own parse method.
3/ Generalise parse, and possibly some of your other methods, to use a list of data descriptors and possibly a dictionary of methods/rules either set in your class initialiser or set by a methods.
4/ Look at either making more use of the methods already in pandas, or possibly, extending pandas to provide the missing methods if you and others think that they would be accepted into pandas as useful extensions.
Personally I think that you would find the benefits of options 3 or 4 to be the most powerful.
The problem is that you're missing a self going into the parent class. If your parent is a singleton then a #staticmethod should work.
class X():
x=1
#staticmethod
def getx():
return X.x
class Y(X):
y=2
def getyx(self):
return X.getx()+self.y
wx = Y()
wx.getyx()
3
I have recently stated trying to use the newer style of classes in Python (those derived from object). As an excersise to familiarise myself with them I am trying to define a class which has a number of class instances as attributes, with each of these class instances describing a different type of data, e.g. 1d lists, 2d arrays, scalars etc. Essentially I wish to be able to write
some_class.data_type.some_variable
where data_type is a class instance describing a collection of variables. Below is my first attempt at implementing this, using just a profiles_1d instance and rather generic names:
class profiles_1d(object):
def __init__(self, x, y1=None, y2=None, y3=None):
self.x = x
self.y1 = y1
self.y2 = y2
self.y3 = y3
class collection(object):
def __init__(self):
self._profiles_1d = None
def get_profiles(self):
return self._profiles_1d
def set_profiles(self, x, *args, **kwargs):
self._profiles_1d = profiles_1d(x, *args, **kwargs)
def del_profiles(self):
self._profiles_1d = None
profiles1d = property(fget=get_profiles, fset=set_profiles, fdel=del_profiles,
doc="One dimensional profiles")
Is the above code roughly an appropriate way of tackling this problem. The examples I have seen of using property just set the value of some variable. Here I require my set method to initialise an instance of some class. If not, any other suggestions of better ways to implement this would be greatly appreciated.
In addition, is the way I am defining my set method ok? Generally the set method, as far as I understand, defines what to do when the user types, in this example,
collection.profiles1d = ...
The only way I can correctly set the attributes of the profiles_1d instance with the above code is to type collection.set_profiles([...], y1=[...], ...), but I think that I shouldn't be directly calling this method. Ideally I would want to type collection.profiles = ([...], y1=[...], ...): is this correct/possible?
Finally, I have seen a decorators mentioned alot with repect to the new style of classes, but this is something I know very little about. Is the use of decorators appropriate here? Is this something I should know more about for this problem?
First, it's good you're learning new-style classes. They've got lots of advantages.
The modern way to make properties in Python is:
class Collection(object):
def __init__(self):
self._profiles_1d = None
#property
def profiles(self):
"""One dimensional profiles"""
return self._profiles_1d
#profiles.setter
def profiles(self, argtuple):
args, kwargs = argtuple
self._profiles_1d = profiles_1d(*args, **kwargs)
#profiles.deleter
def profiles(self):
self._profiles_1d = None
then set profiles by doing
collection = Collection()
collection.profiles = (arg1, arg2, arg3), {'kwarg1':val1, 'kwarg2':val2}
Notice all three methods having the same name.
This is not normally done; either have them pass the attributes to collections constructor or have them create the profiles_1d themselves and then do collections.profiles = myprofiles1d or pass it to the constructor.
When you want the attribute to manage access to itself instead of the class managing access to the attribute, make the attribute a class with a descriptor. Do this if, unlike in the property example above, you actually want the data stored inside the attribute (instead of another, faux-private instance variable). Also, it's good for if you're going to use the same property over and over again -- make it a descriptor and you don't need to write the code multiple times or use a base class.
I actually like the page by #S.Lott -- Building Skills in Python's Attributes, Properties and Descriptors.
When creating propertys (or other descriptors) that need to call other instance methods the naming convention is to prepend an _ to those methods; so your names above would be _get_profiles, _set_profiles, and _del_profiles.
In Python 2.6+ each property is also a decorator, so you don't have to create the (otherwise useless) _name methods:
#property
def test(self):
return self._test
#test.setter
def test(self, newvalue):
# validate newvalue if necessary
self._test = newvalue
#test.deleter
def test(self):
del self._test
It looks like your code is trying to set profiles on the class instead of instances -- if this is so, properties on the class won't work as collections.profiles would be overridden with a profiles_1d object, clobbering the property... if this is really what you want, you'll have to make a metaclass and put the property there instead.
Hopefully you are talking about instances, so the class would look like:
class Collection(object): # notice the capital C in Collection
def __init__(self):
self._profiles_1d = None
#property
def profiles1d(self):
"One dimensional profiles"
return self._profiles_1d
#profiles1d.setter
def profiles1d(self, value):
self._profiles_1d = profiles_1d(*value)
#profiles1d.deleter
def profiles1d(self):
del self._profiles_1d
and then you would do something like:
collection = Collection()
collection.profiles1d = x, y1, y2, y3
A couple things to note: the setter method gets called with only two items: self, and the new value (which is why you were having to call set_profiles1d manually); when doing an assignment, keyword naming is not an option (that only works in function calls, which an assignment is not). If it makes sense for you, you can get fancy and do something like:
collection.profiles1d = (x, dict(y1=y1, y2=y2, y3=y3))
and then change the setter to:
#profiles1d.setter
def profiles1d(self, value):
x, y = value
self._profiles_1d = profiles_1d(x, **y)
which is still fairly readable (although I prefer the x, y1, y2, y3 version myself).
Can a python class contain an instance of itself as a data container may look like this?
class A:
def __init__(self, val):
self.a = A(val)
self.val = val
aa = A(2)
#this will cause RuntimeError: maximum recursion depth exceeded
My purpose is using this class as a data container contain a copy inside if it when it be inited to reduce the deepcopy action.
It may used as an "undo" chain give a chance to get the init val's value when it's necessary.
Is it possible for such an action?
This won't work, for the reason already given:
Python sees A(2) and calls A.__init__.
A.__init__ calls A(val).
A(val) calls A.__init__.
GOTO 2
I assume you're doing this so that you have a log of what val has been; that is, if sometime later you decide that you want val to be 3 instead, you don't throw away the original value 2. How about:
Code
class A( object ):
#property
def val( self ):
return self.history[ -1 ]
#val.setter
def val( self, value ):
self.history.append( value )
def __init__( self, val ):
self.history = [ ]
self.val = val
Explanation
A( object ): classes should now inherit from object. Just because, basically.
#property: this tells python that every time we ask for A.val, it should call A.val() and return the result. This is a decorator; look up the property builtin function for more information.
#val.setter: this is similar to the above, but tells Python that every time we try to assign to A.val it should call the following function instead. Instead of setting A.val, it appends the value to the history list.
Yes, a class can contain an instance of itself, you just can't create it on initiation for the reasons described by others.
For example this class will do it,
class A:
def __init__(self,value):
self.value=value
def setProperty(self,subvalue):
self.innerInstance=A(subvalue)
You can then instantiate it and set its inner copy of itself like this:
>>>OuterInstance=A(123)
>>>OuterInstance.setProperty(456)
And verify it worked with:
>>>OuterInstance.innerInstance.value
456
Well, the class you have in self.a also has self.a, and so furher without an end. I'm not sure I get the purpose of what you are doing, but you might try something like this:
class A:
def __init__(self, val, copy = True):
if copy:
self.a = A(val, False)
self.val = val