Related
I have a function that calls subprocess.check_call() twice. I want to test all their possible outputs. I want to be able to set the first check_call() to return 1 and the second to return 0 and to do so for all possible combinations. The below is what I have so far. I am not sure how to adjust the expected return value
#patch('subprocess.check_call')
def test_hdfs_dir_func(mock_check_call):
for p, d in list(itertools.product([1, 0], repeat=2)):
if p or d:
You can assign the side_effect of your mock to an iterable and that will return the next value in the iterable each time it's called. In this case, you could do something like this:
import copy
import itertools
import subprocess
from unittest.mock import patch
#patch('subprocess.check_call')
def test_hdfs_dir_func(mock_check_call):
return_values = itertools.product([0, 1], repeat=2)
# Flatten the list; only one return value per call
mock_check_call.side_effect = itertools.chain.from_iterable(copy.copy(return_values))
for p, d in return_values:
assert p == subprocess.check_call()
assert d == subprocess.check_call()
Note a few things:
I don't have your original functions so I put my own calls to check_call in the loop.
I'm using copy on the original itertools.product return value because if I don't, it uses the original iterator. This exhausts that original iterator when what we want is 2 separate lists: one for the mock's side_effect and one for you to loop through in your test.
You can do other neat stuff with side_effect, not just raise. As shown above, you can change the return value for multiple calls: https://docs.python.org/3/library/unittest.mock-examples.html#side-effect-functions-and-iterables
Not only that, but you can see from the link above that you can also give it a function pointer. That allows you to do even more complex logic when keeping track of multiple mock calls.
def testfunction():
for i in range(10):
return('a')
print(testfunction())
I want 'a' outputed 10 times in one line. If I use print instead of return, it gives me 10 'a's but each on a new line. Can you help?
return terminates the current function, while print is a call to another function(atleast in python 3)
Any code after a return statement will not be run.
Python's way of printing 10 a's would be:
print('a' * 10)
In your case it would look like the following:
def testfunction ():
return 'a' * 10
print(testfunction ())
The reason its only printing once is because the return statment finishes the function (the return function stops the loop).
In order to print 'a' 10 times you want to do the following:
def testfunction():
for i in range(10):
print('a')
testfunction()
If you want "a" printed 10 times in one single line then you can simply go for:
def TestCode():
print("a"*10)
There's no need to use the for loop. For loop will just "a" for 10 times but every time it'll be a new line.
You can also take in a function argument and get "a" printed as many times as desired.
Such as:
def TestCode(times):
t = "a"*times
print(t)
Test:
TestCode(5)
>>> aaaaa
TestCode(7)
>>> aaaaaaa
print and return get mixed up when starting Python.
A function can return anything but it doesn't mean that the value will be printed for you to see. A function can even return another function (it's called functional programming).
The function below is adapted from your question and it returns a string object. When you call the function, it returns the string object into the variable called x. That contains all of the info you wanted and you can print that to the console.
You could have also used yield or print in your for loop but that may be outside of the scope.
def test_function(item:str="a", n:int=10):
line = item*n # this will be a string object
return line
ten_a_letters = test_function()
print(ten_a_letters)
"aaaaaaaaaa"
two_b_letters = test_function("b",2)
print(two_b_letters)
"bb"
I want 'a' outputed 10 times in one line. If I use print instead of
return, it gives me 10 'a's but each on a new line.
If you want to use print, the you need to pass a 2nd parameter as follows:
def testfunction():
for i in range(10):
print('a', end='')
However, I think the pythonic way would be to do the following:
def testfunction():
print('a' * 10)
When you use return you end the execution of the function immediately and only one value is returned.
Other answers here provide an easier way to solve your problem (which is great), but I would like to suggest a different approach using yield (instead of return) and create a generator (which might be an overkill but a valid alternative nonetheless):
def testfunction():
for i in range(10):
yield('a')
print(''.join(x for x in testfunction()))
1. What does "yield" keyword do?
def test ():
print('a' * 10)
test()
Output will be 'aaaaaaaaaa'.
def cons(a, b):
def pair(f):
return f(a, b)
return pair
def car(f):
def left(a, b):
return a
return f(left)
def cdr(f):
def right(a, b):
return b
return f(right)
Found this python code on git.
Just want to know what is f(a,b) in cons definition is, and how does it work?
(Not a function I guess)
cons is a function, that takes two arguments, and returns a function that takes another function, which will consume these two arguments.
For example, consider the following function:
def add(a, b):
return a + b
This is just a function that adds the two inputs, so, for instance, add(2, 5) == 7
As this function takes two arguments, we can use cons to call this function:
func_caller = cons(2, 5) # cons receives two arguments and returns a function, which we call func_caller
result = func_caller(add) # func_caller receives a function, that will process these two arguments
print(result) # result is the actual result of doing add(2, 5), i.e. 7
This technique is useful for wrapping functions and executing stuff, before and after calling the appropriate functions.
For example, we can modify our cons function to actually print the values before and after calling add:
def add(a, b):
print('Adding {} and {}'.format(a, b))
return a + b
def cons(a, b):
print('Received arguments {} and {}'.format(a, b))
def pair(f):
print('Calling {} with {} and {}'.format(f, a, b))
result = f(a, b)
print('Got {}'.format(result))
return result
return pair
With this update, we get the following outputs:
func_caller = cons(2, 5)
# prints "Received arguments 2 and 5" from inside cons
result = func_caller(add)
# prints "Calling add with 2 and 5" from inside pair
# prints "Adding 2 and 5" from inside add
# prints "Got 7" from inside pair
This isn't going to make any sense to you until you know what cons, car, and cdr mean.
In Lisp, lists are stored as a very simple form of linked list. A list is either nil (like None) for an empty list, or it's a pair of a value and another list. The cons function takes a value and a list and returns you another list just by making a pair:
def cons(head, rest):
return (head, rest)
And the car and cdr functions (they stand for "Contents of Address|Data Register", because those are the assembly language instructions used to implement them on a particular 1950s computer, but that isn't very helpful) return the first or second value from a pair:
def car(lst):
return lst[0]
def cdr(lst):
return lst[1]
So, you can make a list:
lst = cons(1, cons(2, cons(3, None)))
… and you can get the second value from it:
print(car(cdr(lst))
… and you can even write functions to get the nth value:
def nth(lst, n):
if n == 0:
return car(lst)
return nth(cdr(lst), n-1)
… or print out the whole list:
def printlist(lst):
if lst:
print(car(lst), end=' ')
printlist(cdr(lst))
If you understand how these work, the next step is to try them on those weird definitions you found.
They still do the same thing. So, the question is: How? And the bigger question is: What's the point?
Well, there's no practical point to using these weird functions; the real point is to show you that everything in computer science can be written with just functions, no built-in data structures like tuples (or even integers; that just takes a different trick).
The key is higher-order functions: functions that take functions as values and/or return other functions. You actually use these all the time: map, sort with a key, decorators, partial… they’re only confusing when they’re really simple:
def car(f):
def left(a, b):
return a
return f(left)
This takes a function, and calls it on a function that returns the first of its two arguments.
And cdr is similar.
It's hard to see how you'd use either of these, until you see cons:
def cons(a, b):
def pair(f):
return f(a, b)
return pair
This takes two things and returns a function that takes another function and applies it to those two things.
So, what do we get from cons(3, None)? We get a function that takes a function, and applies it to the arguments 3 and None:
def pair3(f):
return f(3, None)
And if we call cons(2, cons(3, None))?
def pair23(f):
return f(2, pair3)
And what happens if you call car on that function? Trace through it:
def left(a, b):
return a
return pair23(left)
That pair23(left) does this:
return left(2, pair3)
And left is dead simple:
return 2
So, we got the first element of (2, cons(3, None)).
What if you call cdr?
def right(a, b):
return a
return pair23(right)
That pair23(right) does this:
return right(2, pair3)
… and right is dead simple, so it just returns pair3.
You can work out that if we call car(cdr(pair23)), we're going to get the 3 out of it.
And now you can write lst = cons(1, cons(2, cons(3, None))), write the recursive nth and printlist functions above, and trace through how they work on lst.
I mentioned above that you can even get rid of integers. How do you do that? Read about Church numerals. You define zero and successor functions. Then you can define one as successor(zero) and two as successor(one). You can even recursively define add so that add(x, zero) is x but add(x, successor(y)) is successor(add(x, y)), and go on to define mul, etc.
You also need a special function you can use as a value for nil.
Anyway, once you've done that, using all of the other definitions above, you can do lst = cons(zero(cons(one, cons(two, cons(three, nil)))), and nth(lst, two) will give you back one. (Of course writing printlist will be a bit trickier…)
Obviously, this is all going to be a lot slower than just using tuples and integers and so on. But theoretically, it’s interesting.
Consider this: we could write a tiny dialect of Python that has only three kinds of statements—def, return, and expression statements—and only three kinds of expressions—literals, identifiers, and function calls—and it could do everything normal Python does. (In fact, you could get rid of statements altogether just by having a function-defining expression, which Python already has.) That tiny language would be a pain to use, but it would a lot easier to write a program to reason about programs in that tiny language. And we even know how to translate code using tuples, loops, etc. into code in this tiny subset language, which means we can write a program that reasons about that real Python code.
In fact, with a couple more tricks (curried functions and/or static function types, and lazy evaluation), the compiler/interpreter could do that kind of reasoning on the fly and optimize our code for us. It’s easy to tell programmatically that car(cdr(cons(2, cons(3, None)) is going to return 3 without having to actually evaluate most of those function calls, so we can just skip evaluating them and substitute 3 for the whole expression.
Of course this breaks down if any function can have side effects. You obviously can’t just substitute None for print(3) and get the same results. So instead, you need some clever trick where IO is handled by some magic object that evaluates functions to figure out what it should read and write, and then the whole rest of the program, the part that users write, becomes pure and can be optimized however you want. With a couple more abstractions, we can even make IO something that doesn’t have to be magical to do that.
And then you can build a standard library that gives you back all those things we gave up, written in terms of defining and calling functions, so it’s actually usable—but under the covers it’s all just reducing pure function calls, which is simple enough for a computer to optimize. And then you’ve basically written Haskell.
Sorry if this is a dumb question, but I've looked for a while and not really found the answer.
If I'm writing a python function, for example:
def function(in1, in2):
in1=in1+1
in2=in2+1
How do I make these changes stick?
I know why they dont, this has been addressed in many answers, but I couldn't find an answer to the question of how to actually make them do so. Without returning values or making some sort of class, is there really no way for a function to operate on its arguments in a global sense?
I also want these variables to not be global themselves, as in I want to be able to do this:
a=1
b=2
c=3
d=4
function(a,b)
function(c,d)
Is this just wishful thinking?
It can be done but I'm warning you - it won't be pretty! What you can do is to capture the caller frame in your function, then pick up the call line, parse it and extract the arguments passed, then compare them with your function signature and create an argument map, then call your function and once your function finishes compare the changes in the local stack and update the caller frame with the mapped changes. If you want to see how silly it can get, here's a demonstration:
# HERE BE DRAGONS
# No, really, here be dragons, this is strictly for demonstration purposes!!!
# Whenever you use this in code a sweet little pixie is brutally killed!
import ast
import inspect
import sys
def here_be_dragons(funct): # create a decorator so we can, hm, enhance 'any' function
def wrapper(*args, **kwargs):
caller = inspect.getouterframes(inspect.currentframe())[1] # pick up the caller
parsed = ast.parse(caller[4][0], mode="single") # parse the calling line
arg_map = {} # a map for our tracked args to establish global <=> local link
for node in ast.walk(parsed): # traverse the parsed code...
# and look for a call to our wrapped function
if isinstance(node, ast.Call) and node.func.id == funct.__name__:
# loop through all positional arguments of the wrapped function
for pos, var in enumerate(funct.func_code.co_varnames):
try: # and try to find them in the captured call
if isinstance(node.args[pos], ast.Name): # named argument!
arg_map[var] = node.args[pos].id # add to our map
except IndexError:
break # no more passed arguments
break # no need for further walking through the ast tree
def trace(frame, evt, arg): # a function to capture the wrapped locals
if evt == "return": # we're only interested in our function return
for arg in arg_map: # time to update our caller frame
caller[0].f_locals[arg_map[arg]] = frame.f_locals.get(arg, None)
profile = sys.getprofile() # in case something else is doing profiling
sys.setprofile(trace) # turn on profiling of the wrapped function
try:
return funct(*args, **kwargs)
finally:
sys.setprofile(profile) # reset our profiling
return wrapper
And now you can easily decorate your function to enable it to perform this ungodly travesty:
# Zap, there goes a pixie... Poor, poor, pixie. It will be missed.
#here_be_dragons
def your_function(in1, in2):
in1 = in1 + 1
in2 = in2 + 1
And now, demonstration:
a = 1
b = 2
c = 3
d = 4
# Now is the time to play and sing along: Queen - A Kind Of Magic...
your_function(a, b) # bam, two pixies down... don't you have mercy?
your_function(c, d) # now you're turning into a serial pixie killer...
print(a, b, c, d) # Woooo! You made it! At the expense of only three pixie lives. Savage!
# prints: (2, 3, 4, 5)
This, obviously, works only for non-nested functions with positional arguments, and only if you pass simple local arguments, feel free to go down the rabbit hole of handling keyword arguments, different stacks, returned/wrapped/chained calls, and other shenanigans if that's what you fancy.
Or, you know, you can use structures invented for this, like globals, classes, or even enclosed mutable objects. And stop murdering pixies.
If you are looking to modify the value of the variables you could have your code be
def func(a,b):
int1 = a + 2
int2 = b + 3
return int1,int2
a = 2
b = 3
a,b = func(a,b)
This allows you to actually change the values of the a and b variables with the function.
you can do:
def function(in1, in2):
return in1 + 1 , in2 + 1
a, b = function(a,b)
c, d = function(c,d)
python functions are closed -> when function(a,b) s called, a and b get reassigned to a local (to the function) references/pointers in1 and in2, which are not accessible outside of the function. provide references to those new values w/o using globals, you will need to pass that back through return.
When you pass an array or non primitive object into a function, you can modify the object's attributes and have those modifications be visible to other references for that object outside, because the object itself contain the pointers to those values, making the visible to anything else holding a pointer to that object.
I'm working through Downey's How To Think Like a Computer Scientist and I have a question regarding his print_backward() function for Linked List.
First, here's Downey's implementation of a Linked List in Python:
class Node:
#initialize with cargo (stores the value of the node)
#and the link. These are set to None initially.
def __init__(self, cargo = None, next = None):
self.cargo = cargo
self.next = next
def __str__(self):
return str(self.cargo)
We give this class the following cargo and link values:
#cargo
node1 = Node('a')
node2 = Node('b')
node3 = Node('c')
#link them
node1.next = node2
node2.next = node3
To print the linked list, we use another of Downey's functions.
def printList(node):
while node:
print node,
node = node.next
>>>printList(node1)
>>>a b c
All very straightforward. But I don't understand how the recursive call in the following function allows one to print the linked list backwards.
def print_backward(list):
if list == None : return
print_backward(list.next)
print list,
>>>print_backward(node1)
>>>c b a
Wouldn't calling "list.next" as the value of print_backward simply give you "b c"?
Note: Several people below have pointed out that this function is badly designed since, given any list, we cannot show that it will always reach the base case. Downey also points out this problem later in the same chapter.
In the forward-printing version, it prints each node before doing the recursive call. In the backward-printing version, it prints each node after doing the recursive call.
This is not coincidental.
Both of the functions recurse until the end of the list is reached. The difference is whether printing happens during this process or afterward.
Function calls use a stack, a last-in first-out data structure that remembers where the computer was executing code when the function call was made. What is put on in the stack in one order comes off in the opposite order. Thus, the recursion is "unwound" in the reverse order of the original calls. The printing occurs during the unwinding process, i.e., after each recursive call has completed.
def print_backward(list):
if list == None : return
print_backward(list.next)
print list,
Wouldn't calling "list.next" as the value of print_backward simply give you "b c"?
No; picture what happens when a->b->c gets passed to print_backward:
"[b c]" is passed to print_backward and then "a" is printed.
But "print_backward", before "a" is printed, calls itself. So:
[ a b c ] is not None, so b->c gets passed to print_backward
[ b c ] is passed to print_backward
[ c] is passed to print_backward
None is passed to print_backward
which returns
and then "c" is printed
and then "b" is printed
and then "a" is printed
quit.
If list isn't None, it calls print_backward, then prints the first member of the list. Expanded, this is exssentially what happens. You can see that when calls start returning, 'c' is printed, then 'b', then 'a'.
It looks like when actually printing a list, it prints the first node
print_backward(list='a','b','c')
print_backward(list='b','c')
print_backward(list='c')
print_backward(list=None)
list is None, so return
print 'c'
print 'b','c'
print 'a','b','c'
Sometimes I find it easier to think of recursion as merely constructing a list of calls to be made in a certain order. As the function continues, it builds up a bunch of calls until it finally gets to the base case. The base case is the situation where no further breaking down the of program is necessary; in this function, the base case is when there is nothing to print, in which case we just leave without doing anything with return.
The cool stuff usually happens on the way back as we unwind the recursive stack of function calls. currently, print_backward has been called on each element of the list, and it will now 'unwind', finishing the most recent calls first and the earlier calls last. This means that the 'instance' of print_backward created when you call it on the last element is the first one to finish, and thus the last element is the first one to be printed, followed by the second to last, third to last, etc., until the original function finally exits.
Take a look at this representation of what happened:
print_backward(node1) #first call to print_backward
print_backward(node2) #calls itself on next node
print_backward(node3) #calls itself on next node
print_backward(None) #calls itself on None. We can now start unwinding as this is the base case:
print Node3 #now the third invocation finishes...
print Node2 #and the second...
print Node1 #and the first.
While the function is called first on the earlier elements, the part that actually prints that element comes after the recursive call, so it won't actually execute until that recursive call finishes. In this case, that means that the print list part won't execute until all of the later elements have been printed first (in reverse order), thus giving you the list elements printed backwards. :D
It's using recursion. It "zips" all the way down until it gets to the end, then it prints every element as each call returns. Since the first one to get to print is the most recent called, it prints the list backwards.
No. There's two kinds of recursion:
Tail recursion: if there is nothing to do after the function returns except return its value. Function calls are not stacked.
Recursion that finds the base case first (in this case, null, then backwardly processes the list). Each function call is pushed into the stack, for later processing. In your exemple, the function is stacked as 'a'->'b'->'c'->null, then as the stack is popped, the author showed that by printing backwards: `if null return: print 'c' -> print 'b' -> print 'a'
In your case, the author only demonstrated a different concept of recursion, and used that to print the list backwards.
Your nodes look something like this:
node1 node2 node3
'a' => 'b' => 'c' => None
At the first call to print_backward, the variable list has the value 'a', subsequent calls to print_backward move one further down the line. Note that none of them print anything until you hit the guard (None) at which time, things get printed from the back to front as the print_backward that received node 'c' must return before the print_backward that received node 'b' can print (because the print statement is after the function call) and so on.
While I recognize that this is somebody else's code, there are a few things in here which are bad practice -- Best I tell you now while you're learning rather than later. First, don't use list as a variable name since it is the name of a builtin function/type in python. second the equality test if obj == None is better done by if obj is None, finally, it's always a good idea to have your classes inherit from object (class node(object):) as that makes it a new-style class.