Could anyone explain to me why A()+A() does give an error, but B()+B() works as expected? I came across this when I was writing a larger piece of code, but this seems to be the smallest code necessary to reproduce it.
from types import MethodType
D = {'__add__': lambda x, y: "".join((repr(x), repr(y)))}
class A(object):
def __getattr__(self, item):
if item == '__coerce__':
raise AttributeError()
return MethodType(D[item], self)
def __repr__(self):
return "A"
class B():
def __getattr__(self, item):
if item == '__coerce__':
raise AttributeError()
return MethodType(D[item], self)
def __repr__(self):
return "B"
try:
A()+A()
except Exception as e:
print e
B()+B()
Does anyone have an explanation?
That's because new style classes never invoke __coerce__ with binary operators. Also for special methods __getattr__ is never invoked in new style classes: From Data model coercion rules:
New-style classes (those derived from object) never invoke the
__coerce__() method in response to a binary operator; the only time __coerce__() is invoked is when the built-in function coerce() is called.
Related
It is known that in Python, due to optimization concerns, we cannot add/modify member functions of a built-in class, e.g., adding an sed function to the built-in str class to perform re.sub(). Thus, the only way to achieve so is to inherit the class (or subclassing). i.e.,
class String(str):
def __init__(self, value='', **kwargs):
super().__init__()
def sed(self, src, tgt):
return String(re.sub(src, tgt, self))
The problem with this is that after sub-classing, member functions return base-class instance instead of the inherited class instance. For example, I would like to chain String edits String(' A b C d E [!] ').sed(...).lower().sed(...).strip().sed('\[.*\]', '').split() and so on. However, functions such as .lower() and .strip() returns an str instead of String, so cannot perform .sed(...) afterwards. And I do not want to keep casting to String after every function call.
So I did a manual over-ride of every base-class methods as follows:
class String(str):
for func in dir(str):
if not func.startswith('_'):
exec(f'{func}=lambda *args, **kwargs: [(String(i) if type(i)==str else i) for i in [str.{func}(*args, **kwargs)]][0]')
def __init__(self, value='', **kwargs):
super().__init__()
def sed(self, src, tgt):
return String(re.sub(src, tgt, self))
However, not every member function returns a simple str object, e.g., for functions such as .split(), they return a list of str; other functions like .isalpha() or .find() return boolean or integer. In general, I want to add more string-morphing functions and do not want to manually over-ride member functions of each return type in order to return inherited-class objects rather than base-class objects. So is there a more elegant way of doing this? Thanks!
Python's built-in classes are not designed to support that style of inheritance
easily. Also, the whole idea seems flawed to my eye. Even if you do figure out
a way to solve the problem as you've framed it, what's the advantage over good
old functions?
# Special String objects with new methods.
s = String('foo bar')
result = s.sed('...', '...')
# Regular str instances passed to ordinary functions.
s = 'foo bar'
result = sed(s, '...', '...')
That said, here's one way to try. I have not tested it
extensively, it might have a flaw, and I would never use it in real code.
The basic idea is to capture objects returned during low-level
attribute access, and if the object is callable return
a wrapped version of it that will perform the needed
data conversions.
import re
from functools import wraps
class String(str):
def __getattribute__(self, attr):
obj = object.__getattribute__(self, attr)
return wrapped(obj) if callable(obj) else obj
def __init__(self, value='', **kwargs):
super().__init__()
def sed(self, src, tgt):
return re.sub(src, tgt, self)
def wrapped(func):
#wraps(func)
def wrapper(*xs, **kws):
obj = func(*xs, **kws)
return convert(obj)
return wrapper
def convert(obj):
if isinstance(obj, str):
return String(obj)
elif isinstance(obj, list):
return [convert(x) for x in obj]
elif isinstance(obj, tuple):
return tuple(convert(x) for x in obj)
else:
return obj
Demo:
s = String('foo bar')
got = s.sed('foo', 'bzz').upper().split()
print(got)
print(type(got))
print(type(got[0]))
Output:
['BZZ', 'BAR']
<class 'list'>
<class '__main__.String'>
Im trying to create a "function with inheritance" so that I dont have to have a bunch of similar functions with copy pasted code. And this is effectively what I ended up with:
class TypeConverter:
__metaclass__ = ABCMeta
def __convert__(self, thing):
if type(thing) == list:
return [self.convert_one(self,t) for t in thing]
else:
return self.convert_one(self,thing)
def __new__(self, thing):
return self.__convert__(self,thing)
#abstractmethod
def convert_one(self, thing):
pass
class HexToInt(TypeConverter):
def convert_one(self, _hex):
return int(_hex, 16)
class IntToHex(TypeConverter):
def convert_one(self, _int):
return hex(_int)
In py3 it runs fine and the objects work like functions which is the intent:
>>> HexToInt(['ff' , 'fe'])
[255, 254]
>>> IntToHex(255)
'0xff'
In py2 it does not work (but thats where I need it to work):
unbound method __convert__() must be called with HexToInt instance as first argument (got ABCMeta instance instead)
Ideally TypeConverter can hold all the some logic about whether or not to return a sequence and what type of sequence it should be etc, and the child classes implement implement some other specific logic. I dont want to have have regular function objects since id have to instantiate and then call on the object I want it to work exactly like a function; i.e does the thing when the parans close,like in the example above.
How bad of an idea is this? What are the problems here? Can I make it work even if I shouldn't?
Any and all thoughts highly appreciated.
Update, it works:
class TypeConverter(object):
def __convert(this, thing):
if type(thing) == list:
return [this.convert_one(t) for t in thing]
else:
return this.convert_one( thing)
def __new__(cls, thing):
c = super(TypeConverter, cls).__new__(cls)
return cls.__convert(c, thing)
#abstractmethod
def convert_one(self, thing):
pass
class HexToInt(TypeConverter):
def convert_one(self, _hex):
return int(_hex, 16)
class IntToHex(TypeConverter):
def convert_one(self, _int):
return hex(_int)
This would be a lot simpler as a simple decorator:
def one_or_many(f):
def wrapper(arg):
if isinstance(arg, list):
return list(map(f, arg))
else:
return f(arg)
return wrapper
#one_or_many
def hex_to_int(_hex):
return int(_hex, 16)
int_to_hex = one_or_many(hex)
No need for classes if you don't actually need classes.
I started learning Selenium and I am curious how to realize behavior from the PythonOrgSearch class which inherits from unittest.TestCase. Namely, each method which starts with test_ will be called automatically after initialization. I know this behavior is implemented in TestCase but I am interested in how to make something similar. Is there a design pattern that will take care of this?
And one bonus question, what is the point of assert True, since the condition is always True
import unittest
from selenium import webdriver
class PythonOrgSearch(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome("C:\chorme\chromedriver.exe")
self.driver.get("http://www.python.org")
def test_example(self):
print("Test")
assert True
def not_test(self):
print("Not a test")
def tearDown(self):
self.driver.close()
if __name__ == "__main__":
unittest.main()
You can do what you want with a metaclass which can customize the construction of your own classes. This a very powerful and general technique and arguably a Python design pattern.
Below is an example of it being applied to what you want to do. The metaclass' __new__() method looks through the contents of the class being defined—which is when it gets called—and looks for callable attributes whose names start with test_. After doing that, it defines __init__() and post_init() methods and makes them part of the class. The former calls the latter method which then iteratively calls all the methods defined that had matching names.
class MyMetaClass(type):
""" Create class that calls an added post_init() method which in turn calls
all method's whose names start with "test_".
"""
def __new__(meta, classname, bases, classdict):
# Get any class __init__() method defined.
class_init = classdict.get('__init__', lambda *_, **__: None)
test_funcs = [value for key, value in classdict.items()
if key.startswith('test_') and callable(value)]
def __init__(self, *args, **kwargs):
print('In metaclass generated __init__()')
class_init(self, *args, **kwargs) # Call class' __init__() method.
self.post_init()
def post_init(self):
print('In metaclass generated post_init()')
for method in test_funcs:
print(f'calling {classname}.{method.__name__}()')
method(self)
classdict.update({'__init__': __init__, # Attach methods to class.
'post_init': post_init})
return type.__new__(meta, classname, bases, classdict)
class Example(metaclass=MyMetaClass):
def __init__(self, arg, macnab=None):
print(f'in Example.__init__({arg!r}, macnab={macnab!r})')
def setUp(self):
pass
def test_example1(self):
print("Test1")
def test_example2(self):
print("Test2")
def not_test(self):
print("Not a test")
def tearDown(self):
print("Also not a test")
pass
print('Creating instance of Example')
Example = Example(42, macnab='keyword')
Output:
Creating instance of Example
In metaclass generated __init__()
in Example.__init__(42, macnab='keyword')
In metaclass generated post_init()
calling Example.test_example1()
Test1
calling Example.test_example2()
Test2
For the first question, you can use dir() on self to get a list of its member (Ufficial Documentation for dir).
After that, you can test the name pattern in some simple way, and if it is callable you can call it:
for name in dir(self):
if name[:5] == 'test_' and callable(getattr(self, name)):
res = getattr(self, name)()
print(res)
Concerning you bonus question, it is a common practice to force the function to be overloaded.
I would imagine they're simply finding callable methods that begin with "test_" using the dir() function. Something you could achieve pretty easily like:
class CustomTestCaseRunner:
def run(self):
methods = [
m for m in dir(self)
if callable(getattr(self, m))
and m.startswith("test_")
]
for m in methods:
print(f"Running {self.__class__.__name__}.{m}")
getattr(self, m)()
class MyTest(CustomTestCaseRunner):
def test_foo(self):
assert True
def test_bar(self):
assert 1
MyTest().run()
# Running MyTest.test_bar
# Running MyTest.test_foo
As for your second question about assert True, it is unlikely you'd ever actually assert True in live code. That function appears to just be an example. assert is typically used on the response from a function. Here are a few examples:
assert isinstance(1, int)
assert isinstance("foo", str)
When the condition evaluates to False, it will raise an AssertionError which will fail your test case.
I understand what I am asking here is probably not the best code design, but the reason for me asking is strictly academic. I am trying to understand how to make this concept work.
Typically, I will return self from a class method so that the following methods can be chained together. My understanding is by returning self, I am simply returning an instance of the class, for the following methods to work on.
But in this case, I am trying to figure out how to return both self and another value from the method. The idea is if I do not want to chain, or I do not call any class attributes, I want to retrieve the data from the method being called.
Consider this example:
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return self, 'lol'
def newmethod(self):
self.hold = self.hold * 2
return self, 2
t = Test()
t.methoda().newmethod()
print(t.hold)
In this case, I will get an AttributeError: 'tuple' object has no attribute 'newmethod' which is to be expected because the methoda method is returning a tuple which does not have any methods or attributes called newmethod.
My question is not about unpacking multiple returns, but more about how can I continue to chain methods when the preceding methods are returning multiple values. I also understand that I can control the methods return with an argument to it, but that is not what I am trying to do.
As mentioned previously, I do realize this is probably a bad question, and I am happy to delete the post if the question doesnt make any sense.
Following the suggestion by #JohnColeman, you can return a special tuple with attribute lookup delegated to your object if it is not a normal tuple attribute. That way it acts like a normal tuple except when you are chaining methods.
You can implement this as follows:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
return getattr(super().__getitem__(0), name)
class Test(object):
def __init__(self):
self.hold = None
def methoda(self):
self.hold = 'lol'
return ChainResult(self, 'lol')
def newmethod(self):
self.hold = self.hold * 2
return ChainResult(self, 2)
Testing:
>>> t = Test()
>>> t.methoda().newmethod()
>>> print(t.hold)
lollol
The returned result does indeed act as a tuple:
>>> t, res = t.methoda().newmethod()
>>> print(res)
2
>>> print(isinstance(t.methoda().newmethod(), tuple))
True
You could imagine all sorts of semantics with this, such as forwarding the returned values to the next method in the chain using closure:
class ChainResult(tuple):
def __new__(cls, *args):
return super(ChainResult, cls).__new__(cls, args)
def __getattribute__(self, name):
try:
return getattr(super(), name)
except AttributeError:
attr = getattr(super().__getitem__(0), name)
if callable(attr):
chain_results = super().__getitem__(slice(1, None))
return lambda *args, **kw: attr(*(chain_results+args), **kw)
else:
return attr
For example,
class Test:
...
def methodb(self, *args):
print(*args)
would produce
>>> t = Test()
>>> t.methoda().methodb('catz')
lol catz
It would be nice if you could make ChainResults invisible. You can almost do it by initializing the tuple base class with the normal results and saving your object in a separate attribute used only for chaining. Then use a class decorator that wraps every method with ChainResults(self, self.method(*args, **kw)). It will work okay for methods that return a tuple but a single value return will act like a length 1 tuple, so you will need something like obj.method()[0] or result, = obj.method() to work with it. I played a bit with delegating to tuple for a multiple return or to the value itself for a single return; maybe it could be made to work but it introduces so many ambiguities that I doubt it could work well.
I don't know if this will make sense, but...
I'm trying to dynamically assign methods to an object.
#translate this
object.key(value)
#into this
object.method({key:value})
To be more specific in my example, I have an object (which I didn't write), lets call it motor, which has some generic methods set, status and a few others. Some take a dictionary as an argument and some take a list. To change the motor's speed, and see the result, I use:
motor.set({'move_at':10})
print motor.status('velocity')
The motor object, then formats this request into a JSON-RPC string, and sends it to an IO daemon. The python motor object doesn't care what the arguments are, it just handles JSON formatting and sockets. The strings move_at and velocity are just two of what might be hundreds of valid arguments.
What I'd like to do is the following instead:
motor.move_at(10)
print motor.velocity()
I'd like to do it in a generic way since I have so many different arguments I can pass. What I don't want to do is this:
# create a new function for every possible argument
def move_at(self,x)
return self.set({'move_at':x})
def velocity(self)
return self.status('velocity')
#and a hundred more...
I did some searching on this which suggested the solution lies with lambdas and meta programming, two subjects I haven't been able to get my head around.
UPDATE:
Based on the code from user470379 I've come up with the following...
# This is what I have now....
class Motor(object):
def set(self,a_dict):
print "Setting a value", a_dict
def status(self,a_list):
print "requesting the status of", a_list
return 10
# Now to extend it....
class MyMotor(Motor):
def __getattr__(self,name):
def special_fn(*value):
# What we return depends on how many arguments there are.
if len(value) == 0: return self.status((name))
if len(value) == 1: return self.set({name:value[0]})
return special_fn
def __setattr__(self,attr,value): # This is based on some other answers
self.set({attr:value})
x = MyMotor()
x.move_at = 20 # Uses __setattr__
x.move_at(10) # May remove this style from __getattr__ to simplify code.
print x.velocity()
output:
Setting a value {'move_at': 20}
Setting a value {'move_at': 10}
10
Thank you to everyone who helped!
What about creating your own __getattr__ for the class that returns a function created on the fly? IIRC, there's some tricky cases to watch out for between __getattr__ and __getattribute__ that I don't recall off the top of my head, I'm sure someone will post a comment to remind me:
def __getattr__(self, name):
def set_fn(self, value):
return self.set({name:value})
return set_fn
Then what should happen is that calling an attribute that doesn't exist (ie: move_at) will call the __getattr__ function and create a new function that will be returned (set_fn above). The name variable of that function will be bound to the name parameter passed into __getattr__ ("move_at" in this case). Then that new function will be called with the arguments you passed (10 in this case).
Edit
A more concise version using lambdas (untested):
def __getattr__(self, name):
return lambda value: self.set({name:value})
There are a lot of different potential answers to this, but many of them will probably involve subclassing the object and/or writing or overriding the __getattr__ function.
Essentially, the __getattr__ function is called whenever python can't find an attribute in the usual way.
Assuming you can subclass your object, here's a simple example of what you might do (it's a bit clumsy but it's a start):
class foo(object):
def __init__(self):
print "initting " + repr(self)
self.a = 5
def meth(self):
print self.a
class newfoo(foo):
def __init__(self):
super(newfoo, self).__init__()
def meth2(): # Or, use a lambda: ...
print "meth2: " + str(self.a) # but you don't have to
self.methdict = { "meth2":meth2 }
def __getattr__(self, name):
return self.methdict[name]
f = foo()
g = newfoo()
f.meth()
g.meth()
g.meth2()
Output:
initting <__main__.foo object at 0xb7701e4c>
initting <__main__.newfoo object at 0xb7701e8c>
5
5
meth2: 5
You seem to have certain "properties" of your object that can be set by
obj.set({"name": value})
and queried by
obj.status("name")
A common way to go in Python is to map this behaviour to what looks like simple attribute access. So we write
obj.name = value
to set the property, and we simply use
obj.name
to query it. This can easily be implemented using the __getattr__() and __setattr__() special methods:
class MyMotor(Motor):
def __init__(self, *args, **kw):
self._init_flag = True
Motor.__init__(self, *args, **kw)
self._init_flag = False
def __getattr__(self, name):
return self.status(name)
def __setattr__(self, name, value):
if self._init_flag or hasattr(self, name):
return Motor.__setattr__(self, name, value)
return self.set({name: value})
Note that this code disallows the dynamic creation of new "real" attributes of Motor instances after the initialisation. If this is needed, corresponding exceptions could be added to the __setattr__() implementation.
Instead of setting with function-call syntax, consider using assignment (with =). Similarly, just use attribute syntax to get a value, instead of function-call syntax. Then you can use __getattr__ and __setattr__:
class OtherType(object): # this is the one you didn't write
# dummy implementations for the example:
def set(self, D):
print "setting", D
def status(self, key):
return "<value of %s>" % key
class Blah(object):
def __init__(self, parent):
object.__setattr__(self, "_parent", parent)
def __getattr__(self, attr):
return self._parent.status(attr)
def __setattr__(self, attr, value):
self._parent.set({attr: value})
obj = Blah(OtherType())
obj.velocity = 42 # prints setting {'velocity': 42}
print obj.velocity # prints <value of velocity>