Is there a way to do something like that in python?
def foo(str(arg).lower()):
print arg
foo('LOL')
I wanted to create decorator in the first place but whatever i try i could not get it to work as I wanted, i either get generator returned or list.
Edit:
This is what I wanted to accomplish.
def to_lowercase(function):
def wrapper(*args):
return function(*[x.lower() if isinstance(x, str) else x for x in args])
return wrapper
its possible to write a decorator to do what you seek:
decorators are functions that accept a function as the first argument and arguments as next arguments. Then a different function is returned, or the arguments are modified in some way before being applied to the input-function.
Here we can modify the arguments to lower case if they are instances of str and leave unaltered otherwise using comprehensions:
def tolower(f, *args, **kwargs):
def newfun(*args, **kwargs):
as_ = [a.lower() if isinstance(a, str) else a for a in args]
kws_ = {k: (v.lower() if isinstance(v, str) else v) for k,v in kwargs.items()}
return f(*as_, **kws_)
return newfun
#tolower
def print_lower(x, y, z):
print(x)
print(y)
print(z)
print_lower('A', 'B', 1)
# outputs
a
b
1
Related
I'm looking for a nice functional way to do the following:
def add(x, y):
return x + y
def neg(x):
return -x
def c(x, y):
# Apply neg to inputs for add
_x = neg(x)
_y = neg(y)
return add(_x, _y)
neg_sum = c(2, 2) # -4
It seems related to currying, but all of the examples I can find use functions that only have one input variable. I would like something that looks like this:
def add(x, y):
return x + y
def neg(x):
return -x
c = apply(neg, add)
neg_sum = c(2, 2) # -4
This is a fairly direct way to do it:
def add(x, y):
return x + y
def neg(x):
return -x
def apply(g, f):
# h is a function that returns
# f(g(arg1), g(arg2), ...)
def h(*args):
return f(*map(g, args))
return h
# or this:
# def apply(g, f):
# return lambda *args: f(*map(g, args))
c = apply(neg, add)
neg_sum = c(2, 2) # -4
Note that when you use *myvar as an argument in a function definition, myvar becomes a list of all non-keyword arguments that are received. And if you call a function with *expression as an argument, then all the items in expression are unpacked and sent as separate arguments to the function. I use these two behaviors to make h accept an unknown list of arguments, then apply function g to each one (with map), then pass all of them as arguments to f.
A different approach, depending on how extensible you need this to be, is to create an object which implements your operator methods, which each return the same object, allowing you to chain operators together in arbitrary orders.
If you can cope with it always returning a list, you might be able to make it work.
class mathifier:
def __init__(self,values):
self.values = values
def neg(self):
self.values = [-value for value in self.values]
return self
def add(self):
self.values = [sum(self.values)]
return self
print (mathifier([2,3]).neg().add().values)
And you can still get your named function for any set of chained functions:
neg_add = lambda x : mathifier(x).neg().add()
print(neg_add([2,3]).values)
From Matthias Fripp's answer, I asked myself : I'd like to compose add and neg both ways : add_neg(*args) and neg_add(*args). This requires hacking Matthias suggestion a bit. The idea is to get some hint on the arity (number of args) of the functions to compose. This information is obtained with a bit of introspection, thanks to inspect module. With this in mind, we adapt the way args are passed through the chain of funcs. The main assumption here is that we deal with real functions, in the mathematical sense, i.e. functions returning ONE float, and taking at least one argument.
from functools import reduce
from inspect import getfullargspec
def arity_one(func):
spec = getfullargspec(func)
return len(spec[0])==1 and spec[1] is None
def add(*args):
return reduce(lambda x,y:x+y, args, 0)
def neg(x):
return -x
def compose(fun1,fun2):
def comp(*args):
if arity_one(fun2): return fun1(*(map( fun2, args)))
else: return fun1(fun2(*args))
return comp
neg_add = compose(neg, add)
add_neg = compose(add, neg)
print(f"-2+(-3) = {add_neg(2, 3)}")
print(f"-(2+3) = {neg_add(2, 3)}")
The solution is still very adhoc...
As a Mathematica user, I like functions that automatically "threads over lists" (as the Mathematica people call it - see http://reference.wolfram.com/mathematica/ref/Listable.html). That means that if a function is given a list instead of a single value, it automatically uses each list entry as an argument and returns a list of the results - e.g.
myfunc([1,2,3,4]) -> [myfunc(1),myfunc(2),myfunc(3),myfunc(4)]
I implemented this principle in Python like this:
def myfunc(x):
if isinstance(x,list):
return [myfunc(thisx) for thisx in x]
#rest of the function
Is this a good way to do it? Can you think of any downsides of this implementation or the strategy overall?
If this is something you're going to do in a lot of functions, you could use a Python decorator. Here's a simple but useful one.
def threads_over_lists(fn):
def wrapped(x, *args, **kwargs):
if isinstance(x, list):
return [fn(e, *args, **kwargs) for e in x]
return fn(x, *args, **kwargs)
return wrapped
This way, just adding the line #threads_over_lists before your function would make it behave this way. For example:
#threads_over_lists
def add_1(val):
return val + 1
print add_1(10)
print add_1([10, 15, 20])
# if there are multiple arguments, threads only over the first element,
# keeping others the same
#threads_over_lists
def add_2_numbers(x, y):
return x + y
print add_2_numbers(1, 10)
print add_2_numbers([1, 2, 3], 10)
You should also consider whether you want this to vectorize only over lists, or also over other iterable objects like tuples and generators. This is a useful StackOverflow question for determining that. Be careful, though- a string is iterable, but you probably won't want your function operating on each character within it.
That's a good way to do it.
However, you would have to do it for each function you write.
To avoid that, you could use a decorator like this one :
def threads(fun):
def wrapper(element_or_list):
if isinstance(element_or_list, list):
return [fun(element) for element in element_or_list]
else:
return fun(element_or_list)
return wrapper
#threads
def plusone(e):
return e + 1
print(plusone(1))
print(plusone([1, 2, 3]))
A number of times I've thought that generator style can be more direct that returning a list, for example,
def foo(input_array):
for x in input_array:
yield processed(x)
vs.
def bar(input_array):
accumulator = []
for x in input_array:
accumulator.append(processed(x))
return accumulator
(okay, if it were really that simple, I'd write map, but you get the point: the generator version is cleaner). However, a return type of a generator is not always desired. Is there a built-in decorator that I can use to change foo into a function returning a list or tuple? The way I'd write it myself is,
import functools
def transform_return_value(transformer):
def inner(f):
#functools.wraps(f)
def new_f(*argv, **kwargs):
return transformer(f(*argv, **kwargs))
return new_f
return inner
#transform_return_value(list)
def foo(input_array):
for x in input_array:
yield processed(x)
To the best of my knowledge (and I've looked, because I've wondered exactly the same thing), no: there is no direct way of doing this with the standard library.
There is a thoroughly tested listify wrapper in unstdlib.py library, though: https://github.com/shazow/unstdlib.py/blob/master/unstdlib/standard/list_.py#L149
def listify(fn=None, wrapper=list):
"""
A decorator which wraps a function's return value in ``list(...)``.
Useful when an algorithm can be expressed more cleanly as a generator but
the function should return an list.
Example::
>>> #listify
... def get_lengths(iterable):
... for i in iterable:
... yield len(i)
>>> get_lengths(["spam", "eggs"])
[4, 4]
>>>
>>> #listify(wrapper=tuple)
... def get_lengths_tuple(iterable):
... for i in iterable:
... yield len(i)
>>> get_lengths_tuple(["foo", "bar"])
(3, 3)
"""
def listify_return(fn):
#wraps(fn)
def listify_helper(*args, **kw):
return wrapper(fn(*args, **kw))
return listify_helper
if fn is None:
return listify_return
return listify_return(fn)
Although #David Wolever's answer is suerly the cleanest way, one thing I often find myself doing (as it doesn't require to define an external decorator) is writing the generator as a local function, like this:
def foo(input_array):
def gen():
for x in input_array:
yield processed(x)
return list(gen())
Here's an alternative, simple decorator without any bells and whistles:
from functools import wraps
from types import GeneratorType
def listify(func):
"""decorator for making generator functions return a list instead"""
#wraps(func)
def new_func(*args, **kwargs):
r = func(*args, **kwargs)
if isinstance(r, GeneratorType):
return list(r)
else:
return r
return new_func
For efficient and concise list definitions try using list comprehension:
def foo(input_array):
return [processed(x) for x in input_array]
If you want a function to return a list, have it return a list. This is much cleaner, easier to understand, read and debug than using decorator.
You may prefer to write this inline, rather than call a function.
Background:
I mostly run python scripts from the command line in pipelines and so my arguments are always strings that need to be type casted to the appropriate type. I make a lot of little scripts each day and type casting each parameter for every script takes more time than it should.
Question:
Is there a canonical way to automatically type cast parameters for a function?
My Way:
I've developed a decorator to do what I want if there isn't a better way. The decorator is the autocast fxn below. The decorated fxn is fxn2 in the example. Note that at the end of the code block I passed 1 and 2 as strings and if you run the script it will automatically add them. Is this a good way to do this?
def estimateType(var):
#first test bools
if var == 'True':
return True
elif var == 'False':
return False
else:
#int
try:
return int(var)
except ValueError:
pass
#float
try:
return float(var)
except ValueError:
pass
#string
try:
return str(var)
except ValueError:
raise NameError('Something Messed Up Autocasting var %s (%s)'
% (var, type(var)))
def autocast(dFxn):
'''Still need to figure out if you pass a variable with kw args!!!
I guess I can just pass the dictionary to the fxn **args?'''
def wrapped(*c, **d):
print c, d
t = [estimateType(x) for x in c]
return dFxn(*t)
return wrapped
#autocast
def fxn2(one, two):
print one + two
fxn2('1', '2')
EDIT: For anyone that comes here and wants the updated and concise working version go here:
https://github.com/sequenceGeek/cgAutoCast
And here is also quick working version based on above:
def boolify(s):
if s == 'True' or s == 'true':
return True
if s == 'False' or s == 'false':
return False
raise ValueError('Not Boolean Value!')
def estimateType(var):
'''guesses the str representation of the variables type'''
var = str(var) #important if the parameters aren't strings...
for caster in (boolify, int, float):
try:
return caster(var)
except ValueError:
pass
return var
def autocast(dFxn):
def wrapped(*c, **d):
cp = [estimateType(x) for x in c]
dp = dict( (i, estimateType(j)) for (i,j) in d.items())
return dFxn(*cp, **dp)
return wrapped
######usage######
#autocast
def randomFunction(firstVar, secondVar):
print firstVar + secondVar
randomFunction('1', '2')
If you want to auto-convert values:
def boolify(s):
if s == 'True':
return True
if s == 'False':
return False
raise ValueError("huh?")
def autoconvert(s):
for fn in (boolify, int, float):
try:
return fn(s)
except ValueError:
pass
return s
You can adjust boolify to accept other boolean values if you like.
You could just use plain eval to input string if you trust the source:
>>> eval("3.2", {}, {})
3.2
>>> eval("True", {}, {})
True
But if you don't trust the source, you could use literal_eval from ast module.
>>> ast.literal_eval("'hi'")
'hi'
>>> ast.literal_eval("(5, 3, ['a', 'b'])")
(5, 3, ['a', 'b'])
Edit:
As Ned Batchelder's comment, it won't accept non-quoted strings, so I added a workaround, also an example about autocaste decorator with keyword arguments.
import ast
def my_eval(s):
try:
return ast.literal_eval(s)
except ValueError: #maybe it's a string, eval failed, return anyway
return s #thanks gnibbler
def autocaste(func):
def wrapped(*c, **d):
cp = [my_eval(x) for x in c]
dp = {i: my_eval(j) for i,j in d.items()} #for Python 2.6+
#you can use dict((i, my_eval(j)) for i,j in d.items()) for older versions
return func(*cp, **dp)
return wrapped
#autocaste
def f(a, b):
return a + b
print(f("3.4", "1")) # 4.4
print(f("s", "sd")) # ssd
print(my_eval("True")) # True
print(my_eval("None")) # None
print(my_eval("[1, 2, (3, 4)]")) # [1, 2, (3, 4)]
I'd imagine you can make a type signature system with a function decorator, much like you have, only one that takes arguments. For example:
#signature(int, str, int)
func(x, y, z):
...
Such a decorator can be built rather easily. Something like this (EDIT -- works!):
def signature(*args, **kwargs):
def decorator(fn):
def wrapped(*fn_args, **fn_kwargs):
new_args = [t(raw) for t, raw in zip(args, fn_args)]
new_kwargs = dict([(k, kwargs[k](v)) for k, v in fn_kwargs.items()])
return fn(*new_args, **new_kwargs)
return wrapped
return decorator
And just like that, you can now imbue functions with type signatures!
#signature(int, int)
def foo(x, y):
print type(x)
print type(y)
print x+y
>>> foo('3','4')
<type: 'int'>
<type: 'int'>
7
Basically, this is an type-explicit version of #utdemir's method.
If you're parsing arguments from the command line, you should use the argparse module (if you're using Python 2.7).
Each argument can have an expected type so knowing what to do with it should be relatively straightforward. You can even define your own types.
...quite often the command-line string should instead be interpreted as another type, like a float or int. The type keyword argument of add_argument() allows any necessary type-checking and type conversions to be performed. Common built-in types and functions can be used directly as the value of the type argument:
parser = argparse.ArgumentParser()
parser.add_argument('foo', type=int)
parser.add_argument('bar', type=file)
parser.parse_args('2 temp.txt'.split())
>>> Namespace(bar=<open file 'temp.txt', mode 'r' at 0x...>, foo=2)
There are couple of problems in your snippet.
#first test bools
if var == 'True':
return True
elif var == 'False':
return False
This would always check for True because you are testing against the strings 'True' and 'False'.
There is not an automatic coercion of types in python. Your arguments when you receive via *args and **kwargs can be anything. First will look for list of values (each of which can be any datatype, primitive and complex) and second will look for a mapping (with any valid mapping possible). So if you write a decorator, you will end up with a good list of error checks.
Normally, if you wish to send in str, just when the function is invoked, typecast it to string via (str) and send it.
I know I arrived late at this game, but how about eval?
def my_cast(a):
try:
return eval(a)
except:
return a
or alternatively (and more safely):
from ast import literal_eval
def mycast(a):
try:
return literal_eval(a)
except:
return a
For a decorator I am writing I would like to manipulate a specific named parameter of a function. Consider the following decorator:
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
Applied on the next function:
#square_param('dividend')
def quotient(divisor=1,dividend=0):
return dividend/divisor
This will work if dividend is called as a keyword argument e.g.:
>>> quotient(dividend=2)
4
However, when given as a positional argument this will fail.
>>> quotient(3,4)
TypeError: quotient() got multiple values for keyword argument 'dividend'
With Python 3 I could solve this by forcing the parameter to be always given as a keyword:
#square_param('dividend')
def quotient(divisor=1,*,dividend=0):
return dividend/divisor
but I would like to support Python 2 and also I would like to put as little restrictions on the function.
Is there a way that I can fix this behaviour in my decorator?
Firstly, your square_param decorator doesn't work because it doesn't return the functions. It needs to be:
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
Now I took #Dirk's advice and looked into the inspect module. You can do it by checking first if the parameter is one of the function's positional arguments, and second if that positional argument has been specified, and then modifying that argument position. You also need to make sure you only modify kwargs if the parameter was supplied as a keyword argument.
import inspect
def square_param(param):
def func_decorator(func):
def func_caller(*args,**kwargs):
funparams = inspect.getargspec(func).args
if param in funparams:
i = funparams.index(param)
if len(args) > i:
args = list(args) # Make mutable
args[i] = args[i] * args[i]
if param in kwargs:
kwargs[param] = kwargs[param] * kwargs[param]
return func(*args,**kwargs)
return func_caller
return func_decorator
even without using Inspect we can get function params
>>> func = lambda x, y, args: (x, y, {})
>>> func.func_code.co_argcount
3
>>> func.func_code.co_varnames
('x', 'y', 'args')
This may only be tangentially related, but I found it useful to solve a similar problem. I wanted to meld *args and **kwargs into a single dictionary so that my following code could process without regard to how the args came in, and I didn't want to mutate the existing kwargs variable, otherwise I just would have use kwargs.update().
all_args = {**kwargs, **{k: v for k, v in zip(list(inspect.signature(func).parameters), args)}}
# optionally delete `self`
del (all_args['self'])
Update: While this works, this answer has a better technique. In part:
bound_args = inspect.signature(f).bind(*args, **kwargs)
bound_args.apply_defaults()