We Keep Coding

Good design patterns for multiple function outputs

If you are incrementally designing a function that could have variable number of outputs, what is the best way to design that function? E.g.
def function(input):
return output1, output2, ...
or
def function(input):
return dict(output1=...)
In both cases, you need a bunch of if statements to sort through and utilize the outputs; the difference is where the if statements are used (within function or outside of the function over the dictionary). I am not sure what principle to use to decide on what to do.

If you need to return multiple things, it means the function is either complex and you should break it down, or you need an object with attributes that is "processed" in the function. A dict is a standard object, but you can also create your own, depends if you want to go more the OOP way or the functional/procedural way

What's the pythonic way to pass arguments between functions?

I have some arguments taken from the user and passed along function to function (each function in a different class), until it eventually gets to a function that does some processing and then the solution is returned up the chain. Up the chain, the functions become more and more abstract merging results from multiple runs of the lower functions.
Where should I use *args and **kwargs?
I think *args and *kwargs can be used for every function where the function doesn't use the arguments explicitly. But, the actual arguments need to be defined at the top_level so that the user knows what the function expects.
Where should I define what the inputs mean?
I think they should be defined at the top_level because that's the one the end-user might want to see the documentation for.
Where should I define what default values?
Again, I think they should be defined at the top_level because that's the one the end-user interacts with.
This is a simple example to demonstrate the passing of the arguments, where I haven't shown how the functions become more and more abstract or how they interact with different classes, as I felt it was unnecessary detail.
def top_level(a=1, b=1, c=1, d=1, e=1):
""" Compute sum of five numbers.
:param a: int, a
:param b: int, b
:param c: int, c
:param d: int, d
:param e: int, e
:return: int, sum
"""
return mid_level(a, b, c, d, e)
def mid_level(*args, **kwargs):
return bottom_level(*args, **kwargs)
def bottom_level(a, b, c, d, e):
return a + b + c + d + e
print top_level(1, 2, 3)
8
Is there a Python convention for passing arguments like this?

I'm not going to answer your question because it would be like answering the question "what's the best way to use a screwdriver to tighten a nut?". I.e. I do not believe that the tools you are asking for guidance with (*args and **kwargs) are designed to solve the problem you want to solve.
Instead I'll answer this question: "how do I associate some data with a set of functions?", and the answer to that is clearly Use Classes.
Welcome to object-oriented programming. I think you're going to enjoy it!
This is a very basic example of what I mean, but it was hard to know exactly what you wanted from your example since it was simple, but the basic principle is encapsulate your data in a class, and then operate on it using the class's methods.
You can then call between methods in the class without needing to pass loads of arguments around all the time (such as the .calculate() method below), which you don't know whether the top layer will need or a bottom layer.
You can just document the parameters in one place, the __init__ method.
You can customize through subclassing transparently to the code (because if you override a method in a subclass, it can still be used by the more generic superclass), as I've done for the .reduce(x, y) method below.
Example:
class ReductionCalculator:
def __init__(self, *args):
self.args = args
def calculate(self):
start = self.args[0]
for arg in self.args[1:]:
start = self.reduce(start, arg)
return start
class Summer(ReductionCalculator):
def reduce(self, x, y):
return x + y
class Multiplier(ReductionCalculator):
def reduce(self, x, y):
return x * y
summer = Summer(1, 2, 4)
print('sum: %d' % (summer.calculate(),))
multiplier = Multiplier(1, 2, 4)
print('sum: %d' % (multiplier.calculate(),))

How about this approach: create a class, call it AllInputs, that represents the collection of all the "arguments taken from the user." The only purpose of this class is to serve as a container for a set of values. One instance of this class gets initialized, of course, at the top level of the program.
class AllInputs:
def __init__(self,a=1, b=1, c=1, d=1, e=1):
""" Compute sum of five numbers.
:param a: int, a
:param b: int, b
:param c: int, c
:param d: int, d
:param e: int, e
"""
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
This object, call it all_inputs, is now passed as the single argument to all of the functions in your example. If a function doesn't use any of the fields in the object, that's fine; it just passes it along to the lower-level function where the real work gets done. To refactor your example, you would now have:
def top_level(all_inputs):
""" Compute sum of all inputs
:return: int, sum
"""
return mid_level(all_inputs)
def mid_level(all_inputs):
return bottom_level(all_inputs)
def bottom_level(all_inputs):
return (all_inputs.a + all_inputs.b + all_inputs.c +
all_inputs.d + all_inputs.e)
all_inputs = AllInputs(1, 2, 3)
print top_level(all_inputs)
8
I don't know if this is "Pythonic" or "non-Pythonic" and I don't care. I think it's a good programming idea to group together the data that the program will use. The initialization process, which combines default values with others taken from the user, is centralized in one place where it's easy to understand. It's reasonably self-documenting. You say the function calls are distributed across several classes, and that's no problem. The function calls are clean and the program flow is easy to follow. There is potential for optimization by placing some of the calculation inside AllInputs so you can avoid duplicating code.
What I don't like in your example (and I think you don't like it either, or you probably wouldn't have asked the question in the first place) is how it uses the *args syntax. When I see that syntax, I take it as a hint that all the arguments have the same semantic meaning, like in the standard library function os.path.join. In your application, if I understand the question, the low-level functions require the argument list to be in a specific order and have specific meanings (your example doesn't reflect that but the text suggests it). It's confusing to see arguments that get passed into a function as *args and then, at a lower level, their specific names and meanings appear once again. Grouping them into a single object makes it clear what's going on.

This isn't the most common pattern, but I've seen it for command line programs that have levels of nested commands: sub-commands, sub-sub-commands and so on. That's a model where "upper" level functions may be more or less dispatchers and not have information about what parameters are needed by the sub-functions within a given route. The purest scenario for this model is when the sub-commands are plugins and the "upper" layers have literally no information about the sub-functions, other than a calling convention the plug-ins are expected to adhere to.
In these cases, I'd argue the pythonic way is to pass parameters from higher-level to lower-level functions, and let the worker level decide which are useful. The range of possible parameters would be defined in the calling convention. This is pythonic on the basis of DRY -- don't repeat yourself. If the low-level / worker function defines what inputs are required or optional, it would often make sense to not repeat this information at the higher levels.
The same could be said for any inversion-of-control flow design, not just CLI applications w/ plug-ins. There are many application designs where I wouldn't use this approach, but it works here.
An input's meaning must be set at the topmost level it can arise in -- as an interface spec to lower levels (a convention, not programmatic). Otherwise the inputs would have no semantic meaning.
If an input can be used by multiple sub-functions, i.e. there's a chaining or pipeline concept in the control flow, then an input's default will also need to be defined at the topmost level for the input.

I would argue that passing arguments down several levels of functions is not pythonic in itself.
From the Zen of Python:
Simple is better than complex
Flat is better than nested
Edit:
If there are a lot of arguments and the functions inbetween just pass them down, I would probably wrap them up in a tuple and unwrap them at the lowest level.

Preventing duplication of code; parametrize a single function, or create two separate functions?

I'm in the process of combing through and simplifying a codebase for a python project. The nature of the selenium-wrapping library that I'm using called Splinter is that I find myself writing a lot of code with minor differences, maybe in the element I'm searching for, or a tiny change in selection logic, different parameters, etc.
Currently, I might have two separate functions with 90% of the code copy and pasted between them.
My two ideas for condensing this and being consistent are as follows:
1) Create three functions: A, B, and C. Functions A and B would be called directly and have single parameters. These functions then call function C with both the parameter they were given, and then the correct 'identifying' parameter, to change the way that function C works. Function C would never be called directly.
def A(x):
return C(x, 0)
def B(y):
return C(y, 1)
def C(a, b):
if b:
# Logic for B
else:
# Logic for A
2) Create one function, and have it take two parameters. First parameter is what you would otherwise pass into A or B, and the second parameter is the 'identifier' portion.
def D(x,i):
if i == 'case 1':
elif i == 'case 2':
else:
The 'identifier' part is what I don't like. Is it a smell to have a function depend on the caller using specific keywords, or values, in a parameter?
Option 1 produces two more functions to take care of, but option 2 causes a user to know these special 'ID' values in order for the function call to work correctly.
Which of these two patterns would be a better solution in this case?

Out of the two options presented, the first sounds like a better approach.
This is because the second option unnecessarily reveals an implementation detail which all callers must know about and depend on.
When implementing the first option, you can make the shared function, C(), private in the class or module. In python, this is often done by naming convention: prepend the function name with a single underscore for module privates and prepend the function name with double underscores for class privates.

This sounds like a good reason to use a decorator function. The wrapper function does the duplicate code while the innermost function is defined as the separate code. For example see the answers for How to make a chain of function decorators?
Thus, you would make the common code A and two functions B and C
def A():
#A
def B():
#A
def C():
For example Understanding Python Decorators in 12 Easy Steps!

Assign a numeric value that is returned from a function by using exec command

I need to assign a numeric value, which is returned from a function, to a variable name by using exec() command:
def func1(x,y):
return x+y
def main(x,y,n):
x=3; y=5; n=1
t = exec("func%s(%s,%s)" % (n,x,y))
return t**2
main(3,5,1)
I have many functions like "func1", "func2" and so on... I try to return t = func1(x,y) with theexec("func%s(%s,%s)" % (n,x,y)) statement. However, I cannot assign a value, returned fromexec().
There is a partly similar question on SE, but it is written for Python3 and is also not applicable to my case.
How can we resolve this problem or is there a more elegant way to perform such an operation, maybe without the use of'exec()'?
By the way, as "func%s(%s,%s)" % (n,x,y) is a statement, I used exec. Or should I better use eval?

It is almost always a really bad idea to get at functions and variables using their names as strings (or bits of their names as integers, as in the example code in the question). One reason why is that eval or exec can do literally anything and you generally want to avoid using code constructs whose behaviour is so hard to predict and reason about.
So here are two ways to get similar results with less pain.
First: Instead of passing around magic index numbers, like the 1 in the code above, pass the actual function which is, itself, a perfectly reasonable value for a variable in Python.
def main(x,y,f):
return f(x,y)**2
main(3, 5, func1)
(Incidentally, the definition of main in the question throws away the values of x,y,n that are passed in to it. That's probably a mistake.)
Second: Instead of making these mere functions, make them methods on classes, and pass around not the functions but either the classes themselves or instances of the classes.
class Solver:
def solve(self, x,y): return "eeeek, not implemented"
class Solver1:
def solve(self, x,y): return x+y
def main(x, y, obj):
return obj.solve(x,y)**2
main(3, 5, Solver1())
Which of these is the better approach depends on the details of your application. For instance, if you actually have multiple "parallel" sets of functions -- as well as your func1, func2, etc., there are also otherfunc1, otherfunc2 etc. -- then this is crying out for an object-oriented solution (the second approach above).
In general I would argue for the second approach; it tends to lead to cleaner code as your requirements grow.

Semantic Type Safety in Python

In my recent project I have the problem, that some values are often misinterpreted. For instance I calculate a wave as a sum of two waves (for which I need two amplitudes and two phase shifts), and then sample it at 4 points. I pass these tuples of four values to different functions, but sometimes I made the mistake to pass wave parameters instead of sample points.
These errors are hard to find, because all the calculations work without any error, but the values are totally meaningless in this context and so the results are just wrong.
What I want now is some kind of semantic type. I want to state that the one function returns sample points and the other function awaits sample points, and that I can do nothing that would conflict this declarations without immediately getting an error.
Is there any way to do this in python?

I would recommend implementing specific data types to be able to distinguish between different kind of information with the same structure.
You can simply subclass list for example and then do some type checking at runtime within your functions:
class WaveParameter(list):
pass
class Point(list):
pass
# you can use them just like lists
point = Point([1, 2, 3, 4])
wp = WaveParameter([5, 6])
# of course all methods from list are inherited
wp.append(7)
wp.append(8)
# let's check them
print(point)
print(wp)
# type checking examples
print isinstance(point, Point)
print isinstance(wp, Point)
print isinstance(point, WaveParameter)
print isinstance(wp, WaveParameter)
So you can include this kind of type checking in your functions, to make sure the correct kind of data was passed to it:
def example_function_with_waveparameter(data):
if not isinstance(data, WaveParameter):
log.error("received wrong parameter type (%s instead WaveParameter)" %
type(data))
# and then do the stuff
or simply assert:
def example_function_with_waveparameter(data):
assert(isinstance(data, WaveParameter))

Pyhon's notion of a "semantic type" is called a class, but as mentioned, Python is dynamically typed so even using custom classes instead of tuples you won't get any compile-time error - at best you'll get runtime errors if your classes are designed in such a way that trying to use one instead of the other will fail.
Now classes are not just about data, they are about behaviour too, so if you have functions that do waveform-specific computations these functions would probably become methods of the Waveform class, and idem for the Point part, and this might be enough to avoid logical errors like passing a "waveform" tuple to a function expecting a "point" tuple.
To make a long story short: if you want a statically typed functional language, Python is not the right tool (Haskell might be a better choice). If you really want / have to use Python, try using classes and methods instead of tuples and functions, it still won't detect type errors at compile-time but chances are you'll have less type errors AND that these type errors will be detected at runtime instead of producing wrong results.