I'm creating a small python app that formats a file's name to a set of rules. I'm having problems finding a way of applying a list of general formatting functions to the same string. I want to apply one function, then another, then another.
I've managed to find a way that works, but I feel that it's very clumsy.
Here I have a list of lists that includes one function and a dictionary of the kwargs. (All of these functions have a "text" parameter that is not included in the dictionary).
functions = [
[SRF.change, {'old': '.', 'new': ' '}],
[SRF.surround, {'value': SU.get_year}],
[SRF.remove, {'chars': '[],'}],
[SRF.capitalize_words, {}],
[SRF.remove_including, {'value': 'mp4'}]]
I then pass it into the custom_rename function. It loops over the list of functions and applies it to the "text" variable. As you can see, the variable changes every time func(text, **kwargs) is called.
def custom_rename(text, functions_list):
# Apply a list of functions to a string
for func_list in functions_list:
func = func_list[0] # Function
kwargs = func_list[1] # Dictionary
try:
text = func(text, **kwargs)
except AttributeError:
pass
return text
Is there a more elegant way of doing this? I, for example, do not like that I have to know that the function is in position [0] and the dictionary is in [1].
Instead of storing [function, arguments] lists, you can use functools.partial to create callables with the arguments already baked in:
from functools import partial
functions = [
partial(SRF.change, old='.', new=' '),
partial(SRF.surround, value=SU.get_year),
partial(SRF.remove, chars='[],'),
SRF.capitalize_words,
partial(SRF.remove_including, value='mp4')
]
Now your custom_rename function can be simplified to this:
def custom_rename(text, functions_list):
# Apply a list of functions to a string
for func in functions_list:
try:
text = func(text)
except AttributeError:
pass
return text
Related
I'm creating a function that takes in a callback function as an argument. I want to be able to use it like this:
def callback1(result, found_index):
# do stuffs
def callback2(result):
# do same stuffs even though it's missing the found_index parameter
somefunct(callback1)
somefunct(callback2)
# somefunct calls the callback function like this:
def somefunct(callback):
# do stuffs, and assign result and found_index
callback(result, found_index) # should not throw error
For context, I am somewhat trying to replicate how javascript's callback functions work for the .forEach function on arrays. You can make a function that takes in only the array item on that specific iteration, or the array item and index, or even the array item, index, and original array:
let some_array = ["apple", "orange", "banana"];
function callback1(value, index) {
console.log(`Item at index ${index}: ${value}`);
}
function callback2(value) {
console.log(`Value: ${value}`);
}
some_array.forEach(callback1); // runs with no errors
some_array.forEach(callback2); // runs with no errors
Furthermore, I don't want the callback function to force the * operator, but also allow them to use it if needed. Thank you, wonderful people of python.
(Posting this separately since it's fundamentally different to my other answer.)
If you need to pass a lot of values to some callbacks, without requiring other callbacks to declare a lot of unused parameters, a neat solution is to encapsulate all of those values in a single object. You can use collections.namedtuple to define a value type with named attributes, and then the callback can take one parameter and decide which attributes to use.
from collections import namedtuple
SomeFunctionResult = namedtuple('SomeFunctionResult', 'foo bar baz qux quz')
def some_function(callback):
result = SomeFunctionResult('foo', 'bar', 'baz', 'qux', 'quz')
callback(result)
Example:
>>> some_function(lambda r: print(r.foo, r.bar))
foo bar
>>> some_function(lambda r: print(r.baz, r.qux, r.quz))
baz qux quz
The downside is that this makes some_function less usable with existing functions which might expect to receive foo directly, rather than an object with a foo attribute. In that case, you have to write some_function(lambda r: blah(r.foo)) which is not as neat as some_function(blah).
The simplest approach would be to unify the signatures of your callbacks. Let's say you defined your forEach function as follows
def forEach(iterable, callback):
for index, elem in enumerate(iterable):
callback(elem, index)
You could then define Python analogs of the callack1 and callback2 Javascript functions as
def callback1(value, index):
print(f"Item at index {index}: {value}")
def callback2(value, _index):
print(f"Value: {value})
Rather than performing any complicated parameter-count-reasoning, exception handling, or dynamic dispatch within forEach, we delegate the decision of how to handle the value and index arguments to the callbacks themselves. If you need to adapt a single-parameter callback to work with forEach, you could simply use a wrapper lambda that discards the second argument:
forEach(some_iterable, lambda value, _index: callback(value))
However, at this point, you just have an obfuscated for loop, which would be much more cleanly expressed as
for elem in some_iterable:
callback(elem)
In this case, it is easier to ask for forgiveness than permission.
def some_function(callback):
result = 'foo'
found_index = 5
try:
callback(result, found_index)
except TypeError:
callback(result)
Example:
>>> some_function(print)
foo 5
>>> some_function(lambda x: print(x))
foo
this is the modified python code snippet you have provided that produces error , this works with no problem , you just have to unify the callback arguments number and type for each callback function called within the main function and define somefunc before calling it .
def callback1(result, found_index):
# do stuffs
result="overridden result in callback 1"
found_index ="overridden found_index in callback 1"
print(result,found_index)
def callback2(result,found_index):
# do same stuffs even though it's missing the found_index parameter
result="overridden result in callback 2"
print(result,found_index)
# somefunct calls the callback function like this:
def somefunct(callback):
# do stuffs, and assign result and found_index
result = "overridden result in somefunct"
found_index = "overridden index in somefunct"
callback(result, found_index) # NOW it should not throw error as the callback is fed with the 2 arguments used in callback1 and ignored in callback2
somefunct(callback1)
somefunct(callback2)
use optional arguments and check how much elemnts returned, sort of switch case:
https://linux.die.net/diveintopython/html/power_of_introspection/optional_arguments.html
Let's say I have a function that can take various kinds of parameter values, but I don't want to (as a constraint) pass arguments explicitly. Instead, I want to pass them as a string.:
def func(param)
return param+param
a = 'param=4'
func(<do something to a>(a))
>>8
Is this possible in python?
I want to use this idea in Django to create Query filters based on GET parameters in a dictionary and then just chain them using their keys.
lookup_dic = {'user': 'author=user',
'draft': 'Q(publish_date_lte=timezone.now())|
Q(publish_date_isnull=True)'}
Based on whether the user and draft keywords are passed in the GET parameters, this would be read out like:
queryset.objects.filter(author=user).filter(Q(publish_date_lte=timezone.now())|
Q(publish_date_isnull=True))
I understand that I can do this by replacing the author=user by Q(author__name=user), but I wanted to know if this string comprehension feature is implemented in python in general?
Use eval
def func(param=0):
return param+param
a = 'param=4'
eval('func(' + a +')')
Are you looking for this?
def func(param):
return param + param
a = 'param=4'
parameter, value = a.split("=")
print(func(**{parameter: int(value)}))
# >> 8
all.
I was wondering if it was possible to set multiple keywords at once (via list?) in a function call.
For example, if you do:
foo, bar = 1, 2
print(foo, bar)
The output is (1,2).
For the function
def printer(foo, bar)
print(foo,bar)
Is it possible to do something like:
printer([foo, bar] = [1,2])
where both keywords are being set with a list?
In particular, the reason why I ask is because I have a function that returns two variables, scale and offset:
def scaleOffset(...):
'stuff happens here
return [scale, offset]
I would like to pass both of these variables to a different function that accepts them as keywords, perhaps as a nested call.
def secondFunction(scale=None, offset=None):
'more stuff
So far I haven't found a way of doing a call like this:
secondFunction([scale,offset] = scaleOffset())
To pass args as a list
arg_list = ["foo", "bar"]
my_func(*arg_list)
To pass kwargs, use a dictionary
kwarg_dict = {"keyword": "value"}
my_func(**kwarg_dict)
I do scientific programming, and often want to show users prompts and variable pairs, let them edit the variables, and then do the calulations with the new variables. I do this so often, that I wrote a wxPython class to move this code out of the main program. You set up a list for each variable with the type of the variable (string, float, int), the prompt, and the variable's current value. You then place all of these lists in one big list, and my utility creates a neatly formated wxPython panel with prompts and the current values which can be edited.
When I started, I only had a few variables, so I would write out each variable.
s='this is a string'; i=1; f=3.14
my_list=[ ['s','your string here',s], ['i','your int here',i], ['f','your float here'],]
input_panel = Input(my_list)
# the rest of the window is created, the input_panel is added to the window, the user is
# allowed to make choices, and control returns when the user hits the calculate button
s,i,f = input_panel.results() # the .results() function returns the values in a list
Now I want to use this routine for a lot of variables (10-30), and this approach is breaking down. I can create the input list to the function over multiple lines using the list.append() statements. When the code returns from the function, though, I get this huge list that needs to be unpacked into the right variables. This is difficult to manage, and it looks like it will be easy to get the input list and output list out of sync. And worse than that, it looks kludgy.
What is the best way to pass lots of variables to a function in Python with extra information so that they can be edited, and then get the variables back so that I can use them in the rest of the program?
If I could pass the variables by reference into the function, then users could change them or not, and I would use the values once the program returned from the function. I would only need to build the input list over multiple lines, and there wouldn't be any possiblity of the input list getting out of sync with the output list. But Python doesn't allow this.
Should I break the big lists into smaller lists that then get combined into big lists for passing into and out of the functions? Or does this just add more places to make errors?
The simplest thing to do would be to create a class. Instead of dealing with a list of variables, the class will have attributes. Then you just use a single instance of the class.
There are two decent options that come to mind.
The first is to use a dictionary to gather all the variables in one place:
d = {}
d['var1'] = [1,2,3]
d['var2'] = 'asdf'
foo(d)
The second is to use a class to bundle all the arguments. This could be something as simple as:
class Foo(object):
pass
f = Foo()
f.var1 = [1,2,3]
f.var2 = 'asdf'
foo(f)
In this case I would prefer the class over the dictionary, simply because you could eventually provide a definition for the class to make its use clearer or to provide methods that handle some of the packing and unpacking work.
To me, the ideal solution is to use a class like this:
>>> class Vars(object):
... def __init__(self, **argd):
... self.__dict__.update(argd)
...
>>> x = Vars(x=1, y=2)
>>> x.x
1
>>> x.y
2
You can also build a dictionary and pass it like this:
>>> some_dict = {'x' : 1, 'y' : 2}
>>> #the two stars below mean to pass the dict as keyword arguments
>>> x = Vars(**some_dict)
>>> x.x
1
>>> x.y
2
You may then get data or alter it as need be when passing it to a function:
>>> def foo(some_vars):
... some_vars.z = 3 #note that we're creating the member z
...
>>> foo(x)
>>> x.z
3
If I could pass the variables by reference into the function, then users could change them or not, and I would use the values once the program returned from the function.
You can obtain much the same effect as "pass by reference" by passing a dict (or for syntactic convenience a Bunch, see http://code.activestate.com/recipes/52308/).
if you have a finite set of these cases, you could write specific wrapper functions for each one. Each wrapper would do the work of building and unpacking lists taht are passed to the internal function.
I would recommend using a dictionary
or a class to accumulate all details
about your variables
value
prompt text
A list to store the order in which you want them to be displayed
Then use good old iteration to prepare input and collect output
This way you will only be modifying a small manageable section of the code time and again.
Of course you should encapsulate all this into a class if your comfortable working with classes.
"""Store all variables
"""
vars = {}
"""Store the order of display
"""
order = []
"""Define a function that will store details and order of the variable definitions
"""
def makeVar(parent, order, name, value, prompt):
parent[name] = dict(zip(('value', 'prompt'), (value, prompt)))
order.append(name)
"""Create your variable definitions in order
"""
makeVar(vars, order, 's', 'this is a string', 'your string here')
makeVar(vars, order, 'i', 1, 'your int here')
makeVar(vars, order, 'f', 3.14, 'your float here')
"""Use a list comprehension to prepare your input
"""
my_list = [[name, vars[name]['prompt'], vars[name]['value']] for name in order]
input_panel = Input(my_list)
out_list = input_panel.results();
"""Collect your output
"""
for i in range(0, len(order)):
vars[order[i]]['value'] = out_list[i];
I have a small python script which i use everyday......it basically reads a file and for each line i basically apply different string functions like strip(), replace() etc....im constanstly editing the file and commenting to change the functions. Depending on the file I'm dealing with, I use different functions. For example I got a file where for each line, i need to use line.replace(' ','') and line.strip()...
What's the best way to make all of these as part of my script? So I can just say assign numbers to each functions and just say apply function 1 and 4 for each line.
First of all, many string functions – including strip and replace – are deprecated. The following answer uses string methods instead. (Instead of string.strip(" Hello "), I use the equivalent of " Hello ".strip().)
Here's some code that will simplify the job for you. The following code assumes that whatever methods you call on your string, that method will return another string.
class O(object):
c = str.capitalize
r = str.replace
s = str.strip
def process_line(line, *ops):
i = iter(ops)
while True:
try:
op = i.next()
args = i.next()
except StopIteration:
break
line = op(line, *args)
return line
The O class exists so that your highly abbreviated method names don't pollute your namespace. When you want to add more string methods, you add them to O in the same format as those given.
The process_line function is where all the interesting things happen. First, here is a description of the argument format:
The first argument is the string to be processed.
The remaining arguments must be given in pairs.
The first argument of the pair is a string method. Use the shortened method names here.
The second argument of the pair is a list representing the arguments to that particular string method.
The process_line function returns the string that emerges after all these operations have performed.
Here is some example code showing how you would use the above code in your own scripts. I've separated the arguments of process_line across multiple lines to show the grouping of the arguments. Of course, if you're just hacking away and using this code in day-to-day scripts, you can compress all the arguments onto one line; this actually makes it a little easier to read.
f = open("parrot_sketch.txt")
for line in f:
p = process_line(
line,
O.r, ["He's resting...", "This is an ex-parrot!"],
O.c, [],
O.s, []
)
print p
Of course, if you very specifically wanted to use numerals, you could name your functions O.f1, O.f2, O.f3… but I'm assuming that wasn't the spirit of your question.
If you insist on numbers, you can't do much better than a dict (as gimel suggests) or list of functions (with indices zero and up). With names, though, you don't necessarily need an auxiliary data structure (such as gimel's suggested dict), since you can simply use getattr to retrieve the method to call from the object itself or its type. E.g.:
def all_lines(somefile, methods):
"""Apply a sequence of methods to all lines of some file and yield the results.
Args:
somefile: an open file or other iterable yielding lines
methods: a string that's a whitespace-separated sequence of method names.
(note that the methods must be callable without arguments beyond the
str to which they're being applied)
"""
tobecalled = [getattr(str, name) for name in methods.split()]
for line in somefile:
for tocall in tobecalled: line = tocall(line)
yield line
It is possible to map string operations to numbers:
>>> import string
>>> ops = {1:string.split, 2:string.replace}
>>> my = "a,b,c"
>>> ops[1](",", my)
[',']
>>> ops[1](my, ",")
['a', 'b', 'c']
>>> ops[2](my, ",", "-")
'a-b-c'
>>>
But maybe string descriptions of the operations will be more readable.
>>> ops2={"split":string.split, "replace":string.replace}
>>> ops2["split"](my, ",")
['a', 'b', 'c']
>>>
Note:
Instead of using the string module, you can use the str type for the same effect.
>>> ops={1:str.split, 2:str.replace}
To map names (or numbers) to different string operations, I'd do something like
OPERATIONS = dict(
strip = str.strip,
lower = str.lower,
removespaces = lambda s: s.replace(' ', ''),
maketitle = lamdba s: s.title().center(80, '-'),
# etc
)
def process(myfile, ops):
for line in myfile:
for op in ops:
line = OPERATIONS[op](line)
yield line
which you use like this
for line in process(afile, ['strip', 'removespaces']):
...