We Keep Coding

completion/intellisense on (*args ,**kwargs) functions

Is there a way to have completion/intellisense on (*args ,**kwargs) functions?
For instance:
class GetVar(GetVarInterface):
#classmethod
def fromcustom(cls,locorvar,offset=0,varType="int", name=None,deref=False,member=None):
return GetVarCustom(locorvar,offset,varType, name,deref,member)
class GetVarCustom(GetVar):
def __init__(self,locorvar,offset=0,varType="int", name=None,deref=False,member=None):
I wanted to implement this without specifying every argument of the constructor (For example using *vars, **kwargs) but didn't want to lose completion/intellisense abilities. Is there a way?
The disadvantage in the current implementation is that you would have to replicate the signature twice for every change...

The only option is to add a comment under the function to hint the arguments, otherwise you can't; if the ide is reading that a function has undefined arguments, it will show you that it's undefined.
A "solution" is to just use the common arguments and pass the rest as kwargs, or you can keep the original init.
class Single_Init:
def __init__(self, val_a, val_b, name=None):
self.val_a = val_a
self.val_b = val_b
self.name = name
class Single_Init_B(Single_Init):
# The previous contructor is calld
def get_result(self):
return self.val_a + self.val_b
class Split_Const:
def op_offset(self, offset):
self.offset = offset
def __init__(self, name, member=None, **kwargs):
""" You olso can hint in a func coment """
self.name = name
self.member = member
if 'offset' in kwargs:
self.offset = kwargs['offset']
else:
self.offset = None
if __name__ == '__main__':
single = Single_Init_B(2, 3)
print('Single:', single.get_result())
split = Split_Const('Name')
split.op_offset(0.5)
print('Split:', split.offset)

Got the solution outside this site..
#functools.wraps(functools.partial(GetVarCustom.__init__,1))
def f(*args,**kwargs):
return GetVarCustom(*args,**kwargs)
Of course, it would have been easier in case of a standard function. However, you need to update the assigned attribute of wraps. Otherwise it will change the function name.
#functools.wraps(GetVarCustom.value,assigned=['__doc__'])
def getvalue(*args,**kwargs):
return self_custom.value(*args,**kwargs)

Class heritage in Python with dynamically generated method

I've created a module that includes 3 main things:
A main class with a __new__ method and child classes that inherit this one;
A make_method decorator that can associate a method to an existing class:
def make_method(obj):
'''Decorator to make the function a method of *obj*.
In the current context::
#make_method(Axes)
def toto(ax, ...):
...
makes *toto* a method of `Axes`, so that one can directly use:
ax.toto()
COPYRIGHT: from Yannick Copin
'''
def decorate(f):
setattr(obj, f.__name__, f)
return f
return decorate
A generic class.
The main class has a __new__ method that uses the make_method decorator to create a plotting function that will work for any child class. Here is the example:
class MainClass():
FREEPARAMETERS = ['mu_1', 'sigma_1']
PLTALL = ["self.param['" + k + "']" for k in FREEPARAMETERS]
def __new__(cls, *arg, **kwargs):
'''Returns the class with dynamically generated methods'''
obj = super(MainClass, cls).__new__(cls)
exec("#make_method(MainClass)\n" +
"def plot_a(self, x_lin):\n" +
" return self.likelihood_tot(x_lin, %s)"
% (", ".join(MainClass.PLTALL)))
return(obj)
Then, if I create a ChildClass that way:
class ChildClass(MainClass):
FREEPARAMETERS = ['mu_1', 'sigma_1', 'mu_2', 'sigma_2']
PLTALL = ["self.param['" + k + "']" for k in FREEPARAMETERS]
it will have a plot_a method that will only require to have an x_lin without having to enter the parameters by hand (provided a likelihood_tot method that takes the correct parameters).
All of that works very well (it may be a bit of an overkill for what I actually need but it's sweet). Then comes the generic class:
class generic():
'''Usage:
gen = MyModule.generic()
gen.set_model('model')
fitted_model_1 = gen.fit(dataset1)
fitted_model_2 = gen.fit(dataset2)
fitted_model_1.plot_a(x_lin)
fitted_model_2.plot_a(x_lin)'''
def set_model(self, classname):
'''Associates an uninstantiated class to self.model from its string'''
self.model = getattr(sys.modules[__name__], classname)
def fit(self, dataset, **kwargs):
'''Instantiates the class with the pandas,
apply the `minimize` method,
and gives that back as an output'''
model = self.model(dataset)
model.minimize(**kwargs)
return(model)
As stated in its documentation, the idea is to be able to call for a generic model to which I can for example pass different datasets without having to manually instance the class every time. Works fine.
The issue arises when doing the following, where ChildClass1 and ChildClass2 have different FREEPARAMETERS:
gen.set_model('ChildClass1')
fitted_childclass1 = gen.fit(dataset)
gen.set_model('ChildClass2')
fitted_childclass2 = gen.fit(dataset)
fitted_childclass2.plot_a(x_lin)
fitted_childclass1.plot_a(x_lin)
The first plot_a(x_lin), associated with the last call of the generic class, works fine. The second, however, gives me a KeyError: 'mu_2' or a TypeError: likelihood_tot() missing n required positional arguments depending on the relative number of FREEPARAMETERS between the latest instanced class and the one before, which means that somehow, the dynamically generated plot_a of fitted_childclass1 now has the parameters of fitted_childclass2.
Yet, when calling for childclass1.PLTALL or childclass2.PLTALL, I do get the expected result. It would seem that they are not interchanged after instantiation. All methods that are not dynamically generated are correct.
I can of course call the plot_a functions in between each, but that's obviously not the point. I would like both childclass1 and childclass2 to behave as if I instantiated them like:
childclass1 = MyModule.ChildClass1(dataset)
childclass1.minimize()
childclass2 = MyModule.ChildClass2(dataset)
childclass2.minimize()

More efficient way of setting default method argument to instance attribute [duplicate]

I want to pass a default argument to an instance method using the value of an attribute of the instance:
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=self.format):
print(formatting)
When trying that, I get the following error message:
NameError: name 'self' is not defined
I want the method to behave like this:
C("abc").process() # prints "abc"
C("abc").process("xyz") # prints "xyz"
What is the problem here, why does this not work? And how could I make this work?

You can't really define this as the default value, since the default value is evaluated when the method is defined which is before any instances exist. The usual pattern is to do something like this instead:
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=None):
if formatting is None:
formatting = self.format
print(formatting)
self.format will only be used if formatting is None.
To demonstrate the point of how default values work, see this example:
def mk_default():
print("mk_default has been called!")
def myfun(foo=mk_default()):
print("myfun has been called.")
print("about to test functions")
myfun("testing")
myfun("testing again")
And the output here:
mk_default has been called!
about to test functions
myfun has been called.
myfun has been called.
Notice how mk_default was called only once, and that happened before the function was ever called!

In Python, the name self is not special. It's just a convention for the parameter name, which is why there is a self parameter in __init__. (Actually, __init__ is not very special either, and in particular it does not actually create the object... that's a longer story)
C("abc").process() creates a C instance, looks up the process method in the C class, and calls that method with the C instance as the first parameter. So it will end up in the self parameter if you provided it.
Even if you had that parameter, though, you would not be allowed to write something like def process(self, formatting = self.formatting), because self is not in scope yet at the point where you set the default value. In Python, the default value for a parameter is calculated when the function is compiled, and "stuck" to the function. (This is the same reason why, if you use a default like [], that list will remember changes between calls to the function.)
How could I make this work?
The traditional way is to use None as a default, and check for that value and replace it inside the function. You may find it is a little safer to make a special value for the purpose (an object instance is all you need, as long as you hide it so that the calling code does not use the same instance) instead of None. Either way, you should check for this value with is, not ==.

Since you want to use self.format as a default argument this implies that the method needs to be instance specific (i.e. there is no way to define this at class level). Instead you can define the specific method during the class' __init__ for example. This is where you have access to instance specific attributes.
One approach is to use functools.partial in order to obtain an updated (specific) version of the method:
from functools import partial
class C:
def __init__(self, format):
self.format = format
self.process = partial(self.process, formatting=self.format)
def process(self, formatting):
print(formatting)
c = C('default')
c.process()
# c.process('custom') # Doesn't work!
c.process(formatting='custom')
Note that with this approach you can only pass the corresponding argument by keyword, since if you provided it by position, this would create a conflict in partial.
Another approach is to define and set the method in __init__:
from types import MethodType
class C:
def __init__(self, format):
self.format = format
def process(self, formatting=self.format):
print(formatting)
self.process = MethodType(process, self)
c = C('test')
c.process()
c.process('custom')
c.process(formatting='custom')
This allows also passing the argument by position, however the method resolution order becomes less apparent (which can affect the IDE inspection for example, but I suppose there are IDE specific workarounds for that).
Another approach would be to create a custom type for these kind of "instance attribute defaults" together with a special decorator that performs the corresponding getattr argument filling:
import inspect
class Attribute:
def __init__(self, name):
self.name = name
def decorator(method):
signature = inspect.signature(method)
def wrapper(self, *args, **kwargs):
bound = signature.bind(*((self,) + args), **kwargs)
bound.apply_defaults()
bound.arguments.update({k: getattr(self, v.name) for k, v in bound.arguments.items()
if isinstance(v, Attribute)})
return method(*bound.args, **bound.kwargs)
return wrapper
class C:
def __init__(self, format):
self.format = format
#decorator
def process(self, formatting=Attribute('format')):
print(formatting)
c = C('test')
c.process()
c.process('custom')
c.process(formatting='custom')

You can't access self in the method definition. My workaround is this -
class Test:
def __init__(self):
self.default_v = 20
def test(self, v=None):
v = v or self.default_v
print(v)
Test().test()
> 20
Test().test(10)
> 10

"self" need to be pass as the first argument to any class functions if you want them to behave as non-static methods.
it refers to the object itself. You could not pass "self" as default argument as it's position is fix as first argument.
In your case instead of "formatting=self.format" use "formatting=None" and then assign value from code as below:
[EDIT]
class c:
def __init__(self, cformat):
self.cformat = cformat
def process(self, formatting=None):
print "Formating---",formatting
if formatting == None:
formatting = self.cformat
print formatting
return formatting
else:
print formatting
return formatting
c("abc").process() # prints "abc"
c("abc").process("xyz") # prints "xyz"
Note : do not use "format" as variable name, 'cause it is built-in function in python

Instead of creating a list of if-thens that span your default arguements, one can make use of a 'defaults' dictionary and create new instances of a class by using eval():
class foo():
def __init__(self,arg):
self.arg = arg
class bar():
def __init__(self,*args,**kwargs):
#default values are given in a dictionary
defaults = {'foo1':'foo()','foo2':'foo()'}
for key in defaults.keys():
#if key is passed through kwargs, use that value of that key
if key in kwargs: setattr(self,key,kwargs[key])
#if no key is not passed through kwargs
#create a new instance of the default value
else: setattr(self,key, eval(defaults[key]))
I throw this at the beginning of every class that instantiates another class as a default argument. It avoids python evaluating the default at compile... I would love a cleaner pythonic approach, but lo'.

Create multiple classes or multiple objects in Python?

I have the following problem and I need advice on how to solve it the best technically in Python. As I am new to programming I would like to have some advice.
So I will have the following object and they should store something. Here is an example:
object 1: cash dividends (they will have the following properties)
exdate (will store a list of dates)
recorddate (will store a list of dates)
paydate (will store a list of dates)
ISIN (will store a list of text)
object 2: stocksplits (they will have the following prpoerties)
stockplitratio (will be some ration)
exdate(list of dates)
...
I have tried to solve it like this:
class cashDividends(object):
def __init__(self, _gross,_net,_ISIN, _paydate, _exdate, _recorddate, _frequency, _type, _announceddate, _currency):
self.gross = _gross
self.net = _net
self.ISIN = _ISIN
self.paydate = _paydate
self.exdate = _exdate
self.recorddate = _recorddate
self.frequency = _frequency
self.type = _type
self.announceddate = _announceddate
self.currency = _currency
So if I have this I would have to create another class named stockplits and then define an __init__ function again.
However is there a way where I can have one class like "Corporate Actions" and then have stock splits and cashdividends in there ?

Sure you can! In python you can pass classes to other classes.
Here a simple example:
class A():
def __init__(self):
self.x = 0
class B():
def __init__(self):
self.x = 1
class Container():
def __init__(self, objects):
self.x = [obj.x for obj in objects]
a = A()
b = B()
c = Container([a,b])
c.x
[0,1]

If I understood correctly what you want is an object that has other objects from a class you created as property?
class CorporateActions(object):
def __init__(self, aCashDividend, aStockSplit):
self.cashDividend = aCashDividend
self.stockSplit = aStockSplit
myCashDividends = CashDividends(...) #corresponding parameters here
myStockSplit = StockSplit(...)
myCorporateActions = CorporateActions(myCashDividends, myStockSplit)

Strictly speaking this answer isn't an answer for the final question. However, it is a way to make your life slightly easier.
Consider creating a sort-of template class (I'm using this term loosely; there's no such thing in Python) that does the __init__ work for you. Like this:
class KwargAttrs():
def __init__(self, **kwargs):
for k,v in kwargs.items():
setattr(self, k, v)
def _update(self, **kwargs):
args_dict = {k:(kwargs[k] if k in kwargs else self.__dict__[k]) for k in self.__dict__}
self.__dict__.update(args_dict)
This class uses every supplied keyword argument as an object attribute. Use it this way:
class CashDividends(KwargAttrs):
def __init__(self, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
# save the namespace before it gets polluted
super().__init__(**locals())
# work that might pollute local namespace goes here
# OPTIONAL: update the argument values in case they were modified:
super()._update(**locals())
Using a method like this, you don't have to go through the argument list and assign every single object attribute; it happens automatically.
We bookend everything you need to accomplish in the __init__ method with method calls to the parent-class via super(). We do this because locals() returns a dict every variable in the function's current namespace, so you need to 1.) capture that namespace before any other work pollutes it and 2.) update the namespace in case any work changes the argument values.
The call to update is optional, but the values of the supplied arguments will not be updated if something is done to them after the call to super().__init__() (that is, unless you change the values using setattr(self, 'argname, value)`, which is not a bad idea).
You can continue using this class like so:
class StockSplits(KwargAttrs):
def __init__(self, stocksplitratio, gross, net, ISIN, paydate, exdate, recorddate, frequency, type, announceddate, currency):
super().__init__(**locals())
As mentioned in the other answers you can create a container for our other classes, but you can even do that using this same template class:
class CorporateActions(KwargAttrs):
def __init__(self, stock_splits , cash_dividends):
super().__init__(**locals())
ca = CorporateActions(stock_splits = StockSplits(<arguments>), cash_dividends = CashDividends(<arguments>) )

Python - extending properties like you'd extend a function

Question
How can you extend a python property?
A subclass can extend a super class's function by calling it in the overloaded version, and then operating on the result. Here's an example of what I mean when I say "extending a function":
# Extending a function (a tongue-in-cheek example)
class NormalMath(object):
def __init__(self, number):
self.number = number
def add_pi(self):
n = self.number
return n + 3.1415
class NewMath(object):
def add_pi(self):
# NewMath doesn't know how NormalMath added pi (and shouldn't need to).
# It just uses the result.
n = NormalMath.add_pi(self)
# In NewMath, fractions are considered too hard for our users.
# We therefore silently convert them to integers.
return int(n)
Is there an analogous operation to extending functions, but for functions that use the property decorator?
I want to do some additional calculations immediately after getting an expensive-to-compute attribute. I need to keep the attribute's access lazy. I don't want the user to have to invoke a special routine to make the calculations. basically, I don't want the user to ever know the calculations were made in the first place. However, the attribute must remain a property, since i've got legacy code I need to support.
Maybe this is a job for decorators? If I'm not mistaken, decorator is a function that wraps another function, and I'm looking to wrap a property with some more calculations, and then present it as a property again, which seems like a similar idea... but I can't quite figure it out.
My Specific Problem
I've got a base class LogFile with an expensive-to-construct attribute .dataframe. I've implemented it as a property (with the property decorator), so it won't actually parse the log file until I ask for the dataframe. So far, it works great. I can construct a bunch (100+) LogFile objects, and use cheaper methods to filter and select only the important ones to parse. And whenever I'm using the same LogFile over and over, i only have to parse it the first time I access the dataframe.
Now I need to write a LogFile subclass, SensorLog, that adds some extra columns to the base class's dataframe attribute, but I can't quite figure out the syntax to call the super class's dataframe construction routines (without knowing anything about their internal workings), then operate on the resulting dataframe, and then cache/return it.
# Base Class - rules for parsing/interacting with data.
class LogFile(object):
def __init__(self, file_name):
# file name to find the log file
self.file_name = file_name
# non-public variable to cache results of parse()
self._dataframe = None
def parse(self):
with open(self.file_name) as infile:
...
...
# Complex rules to interpret the file
...
...
self._dataframe = pandas.DataFrame(stuff)
#property
def dataframe(self):
"""
Returns the dataframe; parses file if necessary. This works great!
"""
if self._dataframe is None:
self.parse()
return self._dataframe
#dataframe.setter
def dataframe(self,value):
self._dataframe = value
# Sub class - adds more information to data, but does't parse
# must preserve established .dataframe interface
class SensorLog(LogFile):
def __init__(self, file_name):
# Call the super's constructor
LogFile.__init__(self, file_name)
# SensorLog doesn't actually know about (and doesn't rely on) the ._dataframe cache, so it overrides it just in case.
self._dataframe = None
# THIS IS THE PART I CAN'T FIGURE OUT
# Here's my best guess, but it doesn't quite work:
#property
def dataframe(self):
# use parent class's getter, invoking the hidden parse function and any other operations LogFile might do.
self._dataframe = LogFile.dataframe.getter()
# Add additional calculated columns
self._dataframe['extra_stuff'] = 'hello world!'
return self._dataframe
#dataframe.setter
def dataframe(self, value):
self._dataframe = value
Now, when these classes are used in an interactive session, the user should be able to interact with either in the same way.
>>> log = LogFile('data.csv')
>>> print log.dataframe
#### DataFrame with 10 columns goes here ####
>>> sensor = SensorLog('data.csv')
>>> print sensor.dataframe
#### DataFrame with 11 columns goes here ####
I have lots of existing code that takes a LogFile instance which provides a .dataframe attribute and dos something interesting (mostly plotting). I would LOVE to have SensorLog instances present the same interface so they can use the same code. Is it possible to extend the super-class's dataframe getter to take advantage of existing routines? How? Or am I better off doing this a different way?
Thanks for reading that huge wall of text. You are an internet super hero, dear reader. Got any ideas?

You should be calling the superclass properties, not bypassing them via self._dataframe. Here's a generic example:
class A(object):
def __init__(self):
self.__prop = None
#property
def prop(self):
return self.__prop
#prop.setter
def prop(self, value):
self.__prop = value
class B(A):
def __init__(self):
super(B, self).__init__()
#property
def prop(self):
value = A.prop.fget(self)
value['extra'] = 'stuff'
return value
#prop.setter
def prop(self, value):
A.prop.fset(self, value)
And using it:
b = B()
b.prop = dict((('a', 1), ('b', 2)))
print(b.prop)
Outputs:
{'a': 1, 'b': 2, 'extra': 'stuff'}
I would generally recommend placing side-effects in setters instead of getters, like this:
class A(object):
def __init__(self):
self.__prop = None
#property
def prop(self):
return self.__prop
#prop.setter
def prop(self, value):
self.__prop = value
class B(A):
def __init__(self):
super(B, self).__init__()
#property
def prop(self):
return A.prop.fget(self)
#prop.setter
def prop(self, value):
value['extra'] = 'stuff'
A.prop.fset(self, value)
Having costly operations within a getter is also generally to be avoided (such as your parse method).

If I understand correctly what you want to do is call the parent's method from the child instance. The usual way to do that is by using the super built-in.
I've taken your tongue-in-cheek example and modified it to use super in order to show you:
class NormalMath(object):
def __init__(self, number):
self.number = number
def add_pi(self):
n = self.number
return n + 3.1415
class NewMath(NormalMath):
def add_pi(self):
# this will call NormalMath's add_pi with
normal_maths_pi_plus_num = super(NewMath, self).add_pi()
return int(normal_maths_pi_plus_num)
In your Log example, instead of calling:
self._dataframe = LogFile.dataframe.getter()
you should call:
self._dataframe = super(SensorLog, self).dataframe
You can read more about super here
Edit: Even thought the example I gave you deals with methods, to do the same with #properties shouldn't be a problem.

You have some possibilities to consider:
1/ Inherit from logfile and override parse in your derived sensor class. It should be possible to modify your methods that work on dataframe to work regardless of the number of members that dataframe has - as you are using pandas a lot of it is done for you.
2/ Make sensor an instance of logfile then provide its own parse method.
3/ Generalise parse, and possibly some of your other methods, to use a list of data descriptors and possibly a dictionary of methods/rules either set in your class initialiser or set by a methods.
4/ Look at either making more use of the methods already in pandas, or possibly, extending pandas to provide the missing methods if you and others think that they would be accepted into pandas as useful extensions.
Personally I think that you would find the benefits of options 3 or 4 to be the most powerful.

The problem is that you're missing a self going into the parent class. If your parent is a singleton then a #staticmethod should work.
class X():
x=1
#staticmethod
def getx():
return X.x
class Y(X):
y=2
def getyx(self):
return X.getx()+self.y
wx = Y()
wx.getyx()
3