Suppose I have following methods:
from torch import device
def run_cuda(device:device, count:int):
...
def gen_noise(device:device, width:int, height:int):
...
If I were to call these, I have to:
device = DEVICE
run_cuda(device, count=8)
gen_noise(device, width=128, height=128)
What I'm trying to achieve is to somehow remove multiple device argument in some way for better readabiliy. So I'd rather:
device = DEVICE
def use_device(device:device):
...
# multiline
device_user = use_device(device)
device_user.run_cuda(count=8)
device_user.gen_noise(width=128, height=128)
# inline
use_device(device).run_cuda(count=8)
use_device(device).gen_noise(width=128, height=128)
obviously I could simply wrap device class and manually define methods in it. But It feels like it would be quite better to define simple use_device(device) function that takes device argument, and pass into following function by 'calling' it.
Maybe something like so:
def use_device(device:device):
# call arbitrary methods that has 'device' argument.
return ExtensionMethodLikeCallable[[device, *args, **kwargs],any]()
Is this possible in Python?
Thank you.
Edit:
Justification for this implementation could be the fact that I could then:
use_device(device).use_dimension(512,512).use_iteration(8).gen_noise()
instead of:
gen_noise(device=device, width=512, height=512, iteration=8)
which one has 'better' readability is admittedly arguable but let's say it's just my personal preference.
Related
I have written a library. Some of its functions and methods operate on sets of Hashables, e.g.:
def some_function(my_set: set[Hashable]) -> None:
...
my_set.update(...)
...
How can I define something like an UpdatableSet (and use it instead of "set" in the signature of some_function), that demands existence of an update method, but allows for using some other class (from an external library) than set, that provides all necessary methods, in function calls?
def some_function(my_set: UpdatableSet[Hashable]) -> None:
...
my_set.update(...)
...
from intbitset import intbitset # see PyPI
some_set = intbitset(rhs=100)
some_function(some_set)
MutableSet[Hashable] is not enough, since it does not guarantee that there is an update method.
I use MyPy for type checking.
I thought of something like the following, but the register method is not found. And I do not know, if this is the right approach. Maybe defining some generic protocol would be the right way.
class UpdatableSet(MutableSet[_T], Generic[_T], ABC):
def update(self, other) -> None:
pass
UpdatableSet.register(set)
UpdatableSet.register(intbitset)
The comment of #SUTerliakov answers the question, and I was able to solve the problem this way:
The proper and type-safe solution would be generic Protocol[T] that defines all
methods of set you need. If there's only 5-6 methods, it's also convenient enough. –
SUTerliakov
Apr 12 at 19:34
I am writing a python script that provides a more user friendly API to a command line tool. Some of the necessary command calls take a lot of parameters (up to around 10 sometimes), but that is not good practice in Python. They can't just be defaults; it has to be possible to set all the parameters for a given call.
My current structure is an API class that has functions such as expose_image(), and then an interface class to handle the construction of the subprocess command and the call. I don't see that adding more classes will help, as the API class still has to generate and pass the parameters in some way.
One solution I have come up with is to fill a dictionary or namedtuple with the parameters and pass it as **kwargs, which makes things look a little nicer, but less explicit.
Is there a better way of handling this?
Thanks!
It is commendable that you want to build a Pythonic API rather than just an API for this command.
I'm not sure why you disregard default parameters though? If the default is None, you could treat that as a guide to not add things to the command line.
For example, suppose you want to call the tree command. You could have something like:
def my_tree(dirs_only=False, full_prefix=False, max_level=None, pattern=None):
cmd_line = ['tree']
if dirs_only:
cmd_line.append('-d')
if full_prefix:
cmd_line.append('-f')
if max_level is not None:
cmd_line.append('-L')
cmd_line.append(str(max_level))
if pattern is not None:
cmd_line.append('-P')
cmd_line.append(pattern)
subprocess.do_something_with(cmd_line)
Callers of my_tree could then interact with it like in the shell:
my_tree()
my_tree(dirs_only=True)
my_tree(pattern='Foo*')
my_tree(pattern='Foo*', max_level=2, full_prefix=True)
In languages such as Java, C# or Dart, you often see "fluent" APIs, and perhaps those might help. It would result in code such as:
my_tree().call()
my_tree().dirs_only().call()
my_tree().with_pattern('Foo*').call()
my_tree() \
.with_pattern('Foo*') \
.with_max_level(2) \
.full_prefix() \
.call()
Though the invocation looks nicer, there is a lot of boilerplate you need to write in order to obtain said niceity, which definitely feels a little bit un-Pythonic.
Like you said, ** of kvargs are convenient way to pass several arguments to your function, however it always better to declare arguments explicitly in the function definition:
def store(data, database,
user, password,
host=DEFAULT_HOST,
port=PG_DEFAULT_PORT,
chunk_size=64,
flags=None):
pass
# call
params = {"data": generate_data(),
"database": "mydb",
"user": "guest",
"password": "guest",
"chunk_size": 128
}
store(**params)
Another way is to use "Parameters" class, like this (an example from pika library):
class ConnectionParameters(Parameters):
def __init__(self,
host=None,
port=None,
virtual_host=None,
credentials=None,
channel_max=None,
frame_max=None,
heartbeat_interval=None,
ssl=None,
ssl_options=None,
connection_attempts=None,
retry_delay=None,
socket_timeout=None,
locale=None,
backpressure_detection=None):
super(ConnectionParameters, self).__init__()
# Create the default credentials object
if not credentials:
credentials = self._credentials(self.DEFAULT_USERNAME,
self.DEFAULT_PASSWORD)
...
# call
conn_params = pika.ConnectionParameters(host=self._host,
port=self._port,
credentials=cred)
conn = pika.BlockingConnection(parameters=conn_params)
Many languages support ad-hoc polymorphism (a.k.a. function overloading) out of the box. However, it seems that Python opted out of it. Still, I can imagine there might be a trick or a library that is able to pull it off in Python. Does anyone know of such a tool?
For example, in Haskell one might use this to generate test data for different types:
-- In some testing library:
class Randomizable a where
genRandom :: a
-- Overload for different types
instance Randomizable String where genRandom = ...
instance Randomizable Int where genRandom = ...
instance Randomizable Bool where genRandom = ...
-- In some client project, we might have a custom type:
instance Randomizable VeryCustomType where genRandom = ...
The beauty of this is that I can extend genRandom for my own custom types without touching the testing library.
How would you achieve something like this in Python?
Python is not a strongly typed language, so it really doesn't matter if yo have an instance of Randomizable or an instance of some other class which has the same methods.
One way to get the appearance of what you want could be this:
types_ = {}
def registerType ( dtype , cls ) :
types_[dtype] = cls
def RandomizableT ( dtype ) :
return types_[dtype]
Firstly, yes, I did define a function with a capital letter, but it's meant to act more like a class. For example:
registerType ( int , TheLibrary.Randomizable )
registerType ( str , MyLibrary.MyStringRandomizable )
Then, later:
type = ... # get whatever type you want to randomize
randomizer = RandomizableT(type) ()
print randomizer.getRandom()
A Python function cannot be automatically specialised based on static compile-time typing. Therefore its result can only depend on its arguments received at run-time and on the global (or local) environment, unless the function itself is modifiable in-place and can carry some state.
Your generic function genRandom takes no arguments besides the typing information. Thus in Python it should at least receive the type as an argument. Since built-in classes cannot be modified, the generic function (instance) implementation for such classes should be somehow supplied through the global environment or included into the function itself.
I've found out that since Python 3.4, there is #functools.singledispatch decorator. However, it works only for functions which receive a type instance (object) as the first argument, so it is not clear how it could be applied in your example. I am also a bit confused by its rationale:
In addition, it is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects.
I understand that anti-pattern is a jargon term for a pattern which is considered undesirable (and does not at all mean the absence of a pattern). The rationale thus claims that inspecting types of arguments is undesirable, and this claim is used to justify introducing a tool that will simplify ... dispatching on the type of an argument. (Incidentally, note that according to PEP 20, "Explicit is better than implicit.")
The "Alternative approaches" section of PEP 443 "Single-dispatch generic functions" however seems worth reading. There are several references to possible solutions, including one to "Five-minute Multimethods in Python" article by Guido van Rossum from 2005.
Does this count for ad hock polymorphism?
class A:
def __init__(self):
pass
def aFunc(self):
print "In A"
class B:
def __init__(self):
pass
def aFunc(self):
print "In B"
f = A()
f.aFunc()
f = B()
f.aFunc()
output
In A
In B
Another version of polymorphism
from module import aName
If two modules use the same interface, you could import either one and use it in your code.
One example of this is from xml.etree.ElementTree import XMLParser
I have a third-party Python library that allows me to register a callback function that it called later.
While the code works okay with functions, when I tried to pass a method it fails, the callback is never called.
I have no control over the third party library (source code not available).
def old_callbackFunction(param, data):
print data
class MyClass():
def callbackFunction(self, param, data):
print data
myObj = MyClass()
# old_setCallback(myObj.callbackFunction, param = "x") # this would work
setCallback(myObj.callbackFunction, param = "x") # this is never called
Sorin actually figured this out himself, with help from my comment, but he indicated that he wanted me to post the original comment as an answer. I was reluctant to post this originally because I'm unsure of the precise behavior of the setCallback and callbackFunction code; use at your own risk and modify as reason dictates.
The best way to wrap a function is to use functools.partial:
from functools import partial
setCallback(partial(myObj.callbackFunction), param="x")
You may also use a lambda (but you'll lose style points with the pythonistas):
setCallback(lambda param, data: myObj.callbackFunction(param, data), param="x")
I manage a fairly large python-based quantum chemistry suite, PyQuante. I'm currently struggling with how to set various defaults so that users can choose among different options at runtime.
For example, I have three different methods for computing electron repulsion integrals. Let's call them a,b,c. I used to simply pick the one I liked best (say, c), and have that hard-wired into the module that computes these integrals.
I have now modified this to use a module, Defaults.py, that contains all such hard-wires. But this is set at compile/install time. I would now like users to be able to override these options at runtime, say, using a .pyquanterc.py file.
In my integral routines, I currently have something like
from Defaults import integral_method
I know about dictionaries, and the .update() method. But I don't know how I would use this in real life. My defaults module looks like
integral_method = c
should I modify the end of Defaults.py to look for a .pythonrc.py file and override these values? E.g.
if os.path.exists('$HOME/.pythonrc.py'): do_something
If so, what should do_something look like?
With your current setup, the user can change the default functions in his scripts quite easily:
import Defaults
Defaults.integral_method = somefunc
If the user adds this to his script, all your modules that use integral_method from Defaults will use somefunc to calculate integrals.
I might do this via a factory class.
class IntegralSolver:
"""
Factory class containing methods for solving integrals.
>>> solver = IntegralSolver("method1")
>>> solver(x)
# solution via method1
Can also be used directly:
>>> IntegralSolver.method2(x)
# solution via method2
"""
def __init__(self, method):
self.__call__ = getattr(self, method)
#staticmethod
def method1(x):
return method1_solution
#staticmethod
def method2(x):
return method2_solution
It really depends on how your user runs the toolset. If they twiddle the python code each time, just setting a block at the top labeled OPTIONS should be good. If they run it off the command line, use the argparse library to allow them to switch options on the command line. Perhaps have it read the options out of a file with configParser to read a default file with your options, and if the user sets it, an additional file with their options.