A simplified version of my problem: I want to write a method in python that takes in one parameter, either a list of strings or my custom object which holds a list of strings. Then return the size of the list. The method is specifically for a user to call so I want it to be simple for the user (essentially I don't want two methods doing the same exact thing except for a single line of code and I don't want to import non python standard libraries)
I realize overloading is not possible in python like it is in Java.
What is a good way to go about this/what is the standard way? The solutions I have thought of are:
Write two different methods.
Write one method with two parameters and defaults, check for defaults move accordingly.
Write one method with one parameter, check what kind of object is passed in, move accordingly (not entirely sure if this type checking is possible)
From a design perspective if statements for each type of object I want to handle does not seem great in the long run, but I don't see any other solutions (besides separate methods)
Thank you for suggestions!
In python, you use a single dispatch function to establish a single method with different implementations based on the argument type (specifically, on the type of the first argument).
from functools import singledispatch
#singledispatch
def process_str_list(str_list):
raise NotImplementedError
#process_str_list.register(list)
def _(str_list):
# process raw list of strings
#process_str_list.register(MyStrListClass)
def _(str_list)
# process my object
To invoke the function, simply call process_str_list with your raw list or object. The type determination and implementation multiplexing takes place internally.
EDIT: Just wanted to add that the PEP that introduced single dispatch says:
It is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects.
Single dispatch is the pythonic way to approach that behavior.
As alfasin suggested, you can implement a __len__ function for your object;
class A(object):
def __init__(self, mylist):
self.mylist = mylist
def __len__(self):
return len(self.mylist)
For more complex distinctions, you can use isinstance at the function level:
def myfunction(obj):
if isinstance(obj, list):
# when the object is a list
elif isinstance(obj, MyClass):
# when the object is something else
else:
raise ValueError('wrong type!')
Related
I have some code I want to improve. I got a suggestion to use a complex solution using functools which I could not understand.
Code Explanation: I am trying to create converter for Strings.
What I want to do here is to run some fixed code before a convert function is executed. However that execution depends on variable argument like country code and validation lengths for that string.
This is what I implemented taking inspiration from: https://www.scaler.com/topics/python/python-decorators/
I don't understand why we need 3 levels of functions nesting here just to implement decorator that requires arguments country_code and valid_lengths.
import functools
from collections.abc import Callable
class Number:
def prevalidate(country_code: str, valid_lengths: list[int]): # type: ignore
def decorator(func: Callable):
#functools.wraps(func)
def wrapper(num: str, validate=False):
if num.startswith(country_code):
num = num[2:]
if validate and len(num) not in valid_lengths:
raise ValueError(f"{num} is not valid {country_code} number")
return func(num, validate)
return wrapper
return decorator
#staticmethod
#prevalidate(country_code="DZ", valid_lengths=[13])
def convert_dz(num: str, validate=False) -> str:
return num[4:6] + num[-4:]
... # other similar methods
num = Number.convert_dz("W/2011/012346") # => 1012346
Let me explain each level first.
The outer level is the decorator factory, which produces your decorator based on some input values.
The second level is the decorator which is a function taking a function as argument and returns a new function which wraps the original function.
The inner level is the wrapper, which is the function which will replace the original function.
Now, you wonder why level 1. and 2. are not merged. Indeed they can be merged, but the three layers are motivated by the shortcut given by the # symbol. The #deco on a function func is equivalent to overwriting the name of the function with func = deco(func), and #deco_factory(args) is equivalent to deco = deco_factory(args); func=deco(func). So it is the # symbol which will only pass the function as single argument. Still, you can manually decorate functions, but you may confuse other python developers which are already used to the three layer design.
Edit:
I did not yet comment to your code example, but just explained the title question. Note, that every time you call the decorator factory with the same arguments, you are creating a new decorator. It would be better if you just reuse a single instance of the decorator. Moreover, if the input values of the decorator factory change the way your class Number behaves, you should better add those values to the class constructor, I mean the __init__ method, and work with instances of Number.
Now, the implementation may not require a decorator, because adding self.prevalidate(num) at the beginning of each function is just a one-liner and is more explicit than the decorator, but there might be more ways to achieve it.
I am trying to make a class that acts like a list. However, I would like to be able to use all of the built-in list repr() functions and such, Is there any way to do something like this:
class myList:
...
def __getitem__(self, index):
#custom getitem function
def __setitem(self,index,value):
#custom setitem function
def toList(self):
#returns a list
#and a catch-all case:
def __*__(*args):
return self.toList().__*__(*args)
So, My question is, is it possible to make a catch-all case for something like that in python, to catch things like __repr__, __str__, __iter__, etc.
Thanks!
It’s not possible to trap such calls with a single method, in CPython at least. While you can define special methods like __getattr__, even defining the low-level __getattribute__ on a metaclass doesn’t work because the interpreter scans for special method names to build a fast lookup table of the sort used by types defined in C. Anything that doesn’t actually possess such individual methods will not respond to the interpreter’s internal calls (e.g., from repr).
What you can do is dynamically generate (or augment, as below) a class that has whatever special methods you want. The readability of the result might suffer too much to make this approach worthwhile:
def mkfwd(n):
def fwd(self,*a): return getattr(self.toList(),n)(*a)
return fwd
for m in "repr","str":
m="__%s__"%m
setattr(myList,m,mkfwd(m))
A Python class's __call__ method lets us specify how a class member should be behave as a function. Can we do the "opposite", i.e. specify how a class member should behave as an argument to an arbitrary other function?
As a simple example, suppose I have a ListWrapper class that wraps lists, and when I call a function f on a member of this class, I want f to be mapped over the wrapped list. For instance:
x = WrappedList([1, 2, 3])
print(x + 1) # should be WrappedList([2, 3, 4])
d = {1: "a", 2: "b", 3:"c"}
print(d[x]) # should be WrappedList(["a", "b", "c"])
Calling the hypothetical __call__ analogue I'm looking for __arg__, we could imagine something like this:
class WrappedList(object):
def __init__(self, to_wrap):
self.wrapped = to_wrap
def __arg__(self, func):
return WrappedList(map(func, self.wrapped))
Now, I know that (1) __arg__ doesn't exist in this form, and (2) it's easy to get the behavior in this simple example without any tricks. But is there a way to approximate the behavior I'm looking for in the general case?
You can't do this in general.*
You can do something equivalent for most of the builtin operators (like your + example), and a handful of builtin functions (like abs). They're implemented by calling special methods on the operands, as described in the reference docs.
Of course that means writing a whole bunch of special methods for each of your types—but it wouldn't be too hard to write a base class (or decorator or metaclass, if that doesn't fit your design) that implements all those special methods in one place, by calling the subclass's __arg__ and then doing the default thing:
class ArgyBase:
def __add__(self, other):
return self.__arg__() + other
def __radd__(self, other):
return other + self.__arg__()
# ... and so on
And if you want to extend that to a whole suite of functions that you create yourself, you can give them all similar special-method protocols similar to the builtin ones, and expand your base class to cover them. Or you can just short-circuit that and use the __arg__ protocol directly in those functions. To avoid lots of repetition, I'd use a decorator for that.
def argify(func):
def _arg(arg):
try:
return arg.__arg__()
except AttributeError:
return arg
#functools.wraps(func)
def wrapper(*args, **kwargs):
args = map(_arg, args)
kwargs = {kw: _arg(arg) for arg in args}
return func(*args, **kwargs)
return wrapper
#argify
def spam(a, b):
return a + 2 * b
And if you really want to, you can go around wrapping other people's functions:
sin = argify(math.sin)
… or even monkeypatching their modules:
requests.get = argify(requests.get)
… or monkeypatching a whole module dynamically a la early versions of gevent, but I'm not going to even show that, because at this point we're getting into don't-do-this-for-multiple-reasons territory.
You mentioned in a comment that you'd like to do this to a bunch of someone else's functions without having to specify them in advance, if possible. Does that mean every function that ever gets constructed in any module you import? Well, you can even do that if you're willing to create an import hook, but that seems like an even worse idea. Explaining how to write an import hook and either AST-patch each function creation node or insert wrappers around the bytecode or the like is way too much to get into here, but if your research abilities exceed your common sense, you can figure it out. :)
As a side note, if I were doing this, I wouldn't call the method __arg__, I'd call it either arg or _arg. Besides being reserved for future use by the language, the dunder-method style implies things that aren't true here (special-method lookup instead of a normal call, you can search for it in the docs, etc.).
* There are languages where you can, such as C++, where a combination of implicit casting and typed variables instead of typed values means you can get a method called on your objects just by giving them an odd type with an implicit conversion operator to the expected type.
Pardon incompetence of style from Python novice here.
I have a class that takes one parameter for establishing the initial data. There are two ways how the initial data can come in: either a list of strings, or a dictionary with string keys and integer values.
Right now I implement only one version of the constructor, the one that takes the dictionary for parameter, with {} as default value. The list parameter init is implemented as a method, ie
myClass = MyClass()
myClass.initList(listVar)
I can surely live with this, but this certainly is not perfect. So, I decided to turn here for some Pythonic wisdom: how such polymorphic constructors should be implemented? Should I try and fail to read initData.keys() in order to sniff if this is dictionary? Or maybe sniffing parameter types to implement lousy polymorphism where it's not welcome by design is considered non-pythonic?
In an ideal world you'd write one constructor that could take either a list or dict without knowing the difference (i.e. duck typed). Of course, this isn't very realistic since these are pretty different ducks.
Understandably, too, you have a little heartburn about the idea of checking the actual instance types, because it breaks with the idea of duck typing. But, in python 2.6 an interesting module called abc was introduced which allows the definition of "abstract base classes". To be considered an instance of an abstract base class, one doesn't actually have to inherit from it, but rather only has to implement all its abstract methods.
The collections module includes some abstract base classes that would be of interest here, namely collections.Sequence and collections.Mapping. Thus you could write your __init__ functions like:
def __init__(self, somedata):
if isinstance(somedata, collections.Sequence):
# somedata is a list or some other Sequence
elif isinstance(somedata, collections.Mapping):
# somedata is a dict or some other Mapping
http://docs.python.org/2/library/collections.html#collections-abstract-base-classes contains the specifics of which methods are provided by each ABC. If you stick to these, then your code can now accept any object which fits one of these abstract base classes. And, as far as taking the builtin dict and list types, you can see that:
>>> isinstance([], collections.Sequence)
True
>>> isinstance([], collections.Mapping)
False
>>> isinstance({}, collections.Sequence)
False
>>> isinstance({}, collections.Mapping)
True
And, almost by accident, you just made it work for tuple too. You probably didn't care if it was really a list, just that you can read the elements out of it. But, if you had checked isinstance(somedata, list) you would have ruled out tuple. This is what using an ABC buys you.
As #Jan-PhilipGehrcke notes, pythonic can be hard to quantify. To me it means:
easy to read
easy to maintain
simple is better than complex is better than complicated
etcetera, etcetera, and so forth (see the Zen of Python for the complete list, which you get by typing import this in the interpreter)
So, the most pythonic solution depends on what you have to do for each supported initializer, and how many of them you have. I would say if you have only a handful, and each one can be handled by only a few lines of code, then use isinstance and __init__:
class MyClass(object):
def __init__(self, initializer):
"""
initialize internal data structures with 'initializer'
"""
if isinstance(initializer, dict):
for k, v in itit_dict.items():
# do something with k & v
setattr(self, k, v)
elif isinstance(initializer, (list, tuple)):
for item in initializer:
setattr(self, item, None)
On the other hand, if you have many possible initializers, or if any one of them requires a lot of code to handle, then you'll want to have one classmethod constructor for each possible init type, with the most common usage being in __init__:
class MyClass(object):
def __init__(self, init_dict={}):
"""
initialize internal data structures with 'init_dict'
"""
for k, v in itit_dict.items():
# do something with k & v
setattr(self, k, v)
#classmethod
def from_sequence(cls, init_list):
"""
initialize internal data structures with 'init_list'
"""
result = cls()
for item in init_list:
setattr(result, item, None)
return result
This keeps each possible constructor simple, clean, and easy to understand.
As a side note: using mutable objects as defaults (like I do in the above __init__) needs to be done with care; the reason is that defaults are only evaluated once, and then whatever the result is will be used for every subsequent invocation. This is only a problem when you modify that object in your function, because those modifications will then be seen by every subsequent invocation -- and unless you wanted to create a cache that's probably not the behavior you were looking for. This is not a problem with my example because I am not modifying init_dict, just iterating over it (which is a no-op if the caller hasn't replaced it as it's empty).
There is no function overloading in Python. Using if isinstance(...) in your __init__() method would be very simple to read and understand:
class Foo(object):
def __init__(self, arg):
if isinstance(arg, dict):
...
elif isinstance(arg, list):
...
else:
raise Exception("big bang")
You can use *args and **kwargs to do it.
But if you want to know what type of parametr is you should use type() or isinstance().
I'd like to serialize Python objects to and from the plist format (this can be done with plistlib). My idea was to write a class PlistObject which wraps other objects:
def __init__(self, anObject):
self.theObject = anObject
and provides a "write" method:
def write(self, pathOrFile):
plistlib.writeToPlist(self.theObject.__dict__, pathOrFile)
Now it would be nice if the PlistObject behaved just like wrapped object itself, meaning that all attributes and methods are somehow "forwarded" to the wrapped object. I realize that the methods __getattr__ and __setattr__ can be used for complex attribute operations:
def __getattr__(self, name):
return self.theObject.__getattr__(name)
But then of course I run into the problem that the constructor now produces an infinite recursion, since also self.theObject = anObject tries to access the wrapped object.
How can I avoid this? If the whole idea seems like a bad one, tell me too.
Unless I'm missing something, this will work just fine:
def __getattr__(self, name):
return getattr(self.theObject, name)
Edit: for those thinking that the lookup of self.theObject will result in an infinite recursive call to __getattr__, let me show you:
>>> class Test:
... a = "a"
... def __init__(self):
... self.b = "b"
... def __getattr__(self, name):
... return 'Custom: %s' % name
...
>>> Test.a
'a'
>>> Test().a
'a'
>>> Test().b
'b'
>>> Test().c
'Custom: c'
__getattr__ is only called as a last resort. Since theObject can be found in __dict__, no issues arise.
But then of course I run into the problem that the constructor now produces an infinite recursion, since also self.theObject = anObject tries to access the wrapped object.
That's why the manual suggests that you do this for all "real" attribute accesses.
theobj = object.__getattribute__(self, "theObject")
I'm glad to see others have been able to help you with the recursive call to __getattr__. Since you've asked for comments on the general approach of serializing to plist, I just wanted to chime in with a few thoughts.
Python's plist implementation handles basic types only, and provides no extension mechanism for you to instruct it on serializing/deserializing complex types. If you define a custom class, for example, writePlist won't be able to help, as you've discovered since you're passing the instance's __dict__ for serialization.
This has a couple implications:
You won't be able to use this to serialize any objects that contain other objects of non-basic type without converting them to a __dict__, and so-on recursively for the entire network graph.
If you roll your own network graph walker to serialize all non-basic objects that can be reached, you'll have to worry about circles in the graph where one object has another in a property, which in turn holds a reference back to the first, etc etc.
Given then, you may wish to look at pickle instead as it can handle all of these and more. If you need the plist format for other reasons, and you're sure you can stick to "simple" object dicts, then you may wish to just use a simple function... trying to have the PlistObject mock every possible function in the contained object is an onion with potentially many layers as you need to handle all the possibilities of the wrapped instance.
Something as simple as this may be more pythonic, and keep the usability of the wrapped object simpler by not wrapping it in the first place:
def to_plist(obj, f_handle):
writePlist(obj.__dict__, f_handle)
I know that doesn't seem very sexy, but it is a lot more maintainable in my opinion than a wrapper given the severe limits of the plist format, and certainly better than artificially forcing all objects in your application to inherit from a common base class when there's nothing in your business domain that actually indicates those disparate objects are related.