I've a got a code that needs certain variable to be shared as like:
def example(arg):
req = urllib2.Request(r'{}'.format(arg))
...
def exampe2(arg):
# i need this func to access req
# i think i can't use req as a global var since the program might need to get imported and it would run from main() (which is again a function)
Would really like your help!
As said in the comments; you can do a pass-by-parameter methodology which'd be applied like this:
def example2(arg, req):
....
def example(arg):
req = urllib2.Request(r'{}'.format(arg))
...
return example2(..., req)
Or you could just as easily integrate the two functions, as you could combine the two arg parameters on example and example2.
this example might help i guess
def example1(arg):
example1.request = "from example1"
....
def example2(arg):
print(example1.request)
example1("arg1")
example2("arg2")
> from example one
otherwise you can make request as global and use that request varable inside your example2 function. But all you need to do is execute example1 before example2. Or you can return the request from example1 and assign example1 return value to another variable inside example2.
Just pass it as return values and parameters? This is a feature, since it allows you to keep things as local as possible. If your function needs lots of arguments or gives lots of output, it's often a sign that it can be broken up into multiple functions (a function should ideally do one clearly separated thing and be named as such).
In some cases of course, you want to pass around some data such as configuration options: you might create some new object for this but why not simply a dictionary?
def make_request(arg, config):
req = urllib2.Request(r'{}'.format(arg))
config['req'] = req
return config
Note that I returned the dict config, even though it is not necessary since dicts are mutable in Python. This just makes it clear in the code that I am modifying it. Now we can use the config:
def exampe2(arg, config):
arg = config['arg']
...do stuff..
Related
Given the following code,
def myfunc(a=None, b=None, c=None, **kw):
func(arga=a, argb=b, **kw)
#do something with c
def func(arga=None, argb=None, argc=None):
....
Can I replicate part of the signature of func, namely the missing args, without imitating every missing arg of func manually?
Put it more simply, I want to see argc in keywords of myfunc such that myfunc? would be different. It would contain argc. myfunc(a=None,b=None,c=None,argc=None)
#functools.wraps allows for wrapping a complete functions. Using partial can subtract args. But don't know to add.
yes, it is possible, though not trivial -
Python's introspection capabilities allow you to check all parameters the target function declares, and it is possible to build a new function programmatically that will include those attributes automatically.
I have written this for a project of mine, and had exposed the relevant code as my answer here: Signature-changing decorator: properly documenting additional argument
I will not mark this as duplicate, since the other question is more worried about documenting the new function.
If you want to give a try, with your code, maybe with something simpler, you can check the inspect.signature call from the standard library, which allows one to discover everything about parameters and default arguments of the target function.
Building a new function from this information is a bit more tricky, but possible - but one can always resort to a exec call which will can create a new function from a string template. The answer there follows this line.
I'm not sure what is being asked here either but I have here alternative code to functools.partial that might be adapted ???
(edit>)
The difference here from partial is that the mkcall argument is a string rather than a series of arguments. This string can then be formatted and analysed according to whatever appropriate requirements are needed before the target function is called.
(<edit)
def mkcall(fs, globals=None,locals=None):
class func:
def __init__(f,fcnm=None,params=None,globals=None,locals=None):
f.nm = fcnm
f.pm = params
f.globals = globals
f.locals = locals
def __call__(f):
s = f.nm + f.pm
eval(s,f.globals,f.locals)
if '(' in fs:
funcn,lbr,r = fs.partition('(')
tp = lbr + r
newf = func(funcn,tp,globals,locals)
callf = newf.__call__
else:
callf = eval(fs,globals,locals)
return callf
#call examples
# mkcall("func(arg)")
# mkcall("func")
I'm trying to mock a global variable which is list type.
Below is the illustration -
This is first_main.py -
url=[] -- The result off url.append(hello.replace(-test','')) is getting stored here which is in method_a() and this is global variable
def method_a():
url.append(hello.replace(-test',''))
return something
Now, url=[] is used in another method.
def method_b():
*Some codes and url=[] is used here in this code.*
print url
return True
Now, I'm testing method_b -
#mock.patch('first_main.url')
def test_method_b(self, mock_url_list):
mock_url_list.return_value.url.return_value = [['a','b'],['c','d']]
reponse = method_b()
print response
Now, if i put a print statement for url which is getting used in method_b it should return me the [['a','b'],['c','d']] instead of id.
------------Console------------
MagicMock name='url' id='090909090'
I want return_type in list ie. [['a','b'],['c','d']] instead of id.
Thank You
Let me know incase of clarification.
The default behaviour of mock.patch() is to substitute the target object with a MagicMock. Therefore print url will always print the MagicMock instance.
However, mock.patch() supports a new_callable argument which allows you to specify a different type of object. In your case you should be able to supply list.
#mock.patch('first_main.url', new_callable=list)
def test_method_b(self, mock_url_list):
mock_url_list.extend([['a','b'],['c','d']])
reponse = method_b()
print response
Note that in the above example you do not need to use return_value because you are not working with a mock. You can manipulate mock_url_list as a list.
The patch decorator ensures that the original global list is put back the way it was after the test completes.
Providing the answer-
No need to mock. Just import the module and alter the value of the global within setUp():
Refer the link given below-
Mocking a global variable
Thanks :)
In my python3 code, I have a series of functions into which I want to insert some common code. This code must set the same set of local variables within each function.
To better explain, consider the following function ...
def arbtrary_function(arg0, arg1):
### I want these local variables set ...
localvar0 = some_function_of(arg0)
localvar1 = some_other_function_of(arg0)
# ... 18 more similar settings ...
localvar20 = yet_another_function_of(arg0)
### End of local variable settings
# The rest of the function's code will use
# these local variables.
I want to begin a number of functions with this same code block.
I have tried doing this with a decorator, but I haven't been able to figure out how the decorator can set local variable values within the function that it is decorating.
If python3 had something like the C-preprocessor's macro facility, I could do this:
### Somehow define a "macro" with those 21 lines of common code ...
somehow_define_macro COMMON_CODE(ARG):
localvar0 = some_function_of(ARG)
localvar1 = some_other_function_of(ARG)
# ... 18 more similar settings ...
localvar20 = yet_another_function_of(ARG)
... and then, I could do this in each of the many functions into which I'd like to insert this common code:
def arbtrary_function(arg0, arg1):
COMMON_CODE(arg0)
But, of course, python doesn't have this C-preprocessor functionality.
I tried to do this with a function that accepts its caller's locals() dictionary as an argument. Even though the function makes changes to that supplied locals() dictionary, those changes don't make any new local variables available to the caller after the function returns. For example:
def modifylocals(loc, arg):
loc['localvar0'] = some_function_of(arg)
# ... etc. ...
def arbitrary_function(arg0, arg1):
modifylocals(locals(), arg0)
# The following properly prints the new value of localvar0
# that was set within locals() ...
print('value in locals() of localvar0: {}' \
.format(locals().get('localvar0', 'NOT SET')))
# However, the localvar0 local variable does not get created
# within my function. The following prints 'DOES NOT EXIST' ...
try:
print('value of localvar0: {}'.format(localvar0))
except:
print('DOES NOT EXIST')
I know I could also do the following, but then, I would have to change each function that calls the setmylocals() function if I ever want that function to set more or fewer local variables ...
def setmylocals(arg):
localvar0 = some_function_of(arg)
# ... etc. ...
localvar20 = yet_another_function_of(arg)
return (localvar0, ... etc. ..., localvar20)
def arbitrary_function(arg0, arg1):
localvar0, ..., localvar20 = setmylocals(arg0)
Does anyone know of a way I can implement this particular C-preprocessor-like functionality in python3?
PS: I know that if I refactor all the functions that use these local variables to use dictionary entries instead of the local variables in question, then this becomes a straightforward task. However, the functions I want to modify already exist in a large code base, and I don't want to refactor all of those functions. Being able to optionally pre-set local variable values in this "C-preprocessor-like" manner involves much less work and will have much less impact on the existing code base.
In other words, I could do this, but I don't want to ...
def setvariables(arg):
retval = {}
retval['localvar0'] = some_function_of(arg)
# ... etc. ...
return retval
def arbitrary_function(arg0, arg1):
vars = setvariables(arg0)
# ... and then change the rest of the code to use
# `vars['localvar0']` instead of `localvar0`, etc.
# However, I want to avoid all this refactoring
# of the existing code.
PPS: This discussion states that what I'm looking for is impossible:
Modifying locals in Python
But hope springs eternal. Does anyone know of any procedure for doing what I want?
Let's say I have a code like this:
def read_from_file(filename):
list = []
for i in filename:
value = i[0]
list.append(value)
return list
def other_function(other_filename):
"""
That's where my question comes in. How can I get the list
from the other function if I do not know the value "filename" will get?
I would like to use the "list" in this function
"""
read_from_file("apples.txt")
other_function("pears.txt")
I'm aware that this code might not work or might not be perfect. But the only thing I need is the answer to my question in the code.
You have two general options. You can make your list a global variable that all functions can access (usually this is not the right way), or you can pass it to other_function (the right way). So
def other_function(other_filename, anylist):
pass # your code here
somelist = read_from_file("apples.txt")
other_function("pears.txt.", somelist)
You need to "catch" the value return from the first function, and then pass that to the second function.
file_name = read_from_file('apples.txt')
other_function(file_name)
You need to store the returned value in a variable before you can pass it onto another function.
a = read_from_file("apples.txt")
There are at least three reasonable ways to achieve this and two which a beginner will probably never need:
Store the returned value of read_from_file and give it as a parameter to other_function (so adjust the signature to other_function(other_filename, whatever_list))
Make whatever_list a global variable.
Use an object and store whatever_list as a property of that object
(Use nested functions)
(Search for the value via garbage collector gc ;-)
)
Nested functions
def foo():
bla = "OK..."
def bar():
print(bla)
bar()
foo()
Global variables
What are the rules for local and global variables in Python? (official docs)
Global and Local Variables
Very short example
Misc
You should not use list as a variable name as you're overriding a built-in function.
You should use a descriptive name for your variables. What is the content of the list?
Using global variables can sometimes be avoided in a good way by creating objects. While I'm not always a fan of OOP, it sometimes is just what you need. Just have a look of one of the plenty tutorials (e.g. here), get familiar with it, figure out if it fits for your task. (And don't use it all the time just because you can. Python is not Java.)
There were several discussions on "returning multiple values in Python", e.g.
1,
2.
This is not the "multiple-value-return" pattern I'm trying to find here.
No matter what you use (tuple, list, dict, an object), it is still a single return value and you need to parse that return value (structure) somehow.
The real benefit of multiple return value is in the upgrade process. For example,
originally, you have
def func():
return 1
print func() + func()
Then you decided that func() can return some extra information but you don't want to break previous code (or modify them one by one). It looks like
def func():
return 1, "extra info"
value, extra = func()
print value # 1 (expected)
print extra # extra info (expected)
print func() + func() # (1, 'extra info', 1, 'extra info') (not expected, we want the previous behaviour, i.e. 2)
The previous codes (func() + func()) are broken. You have to fix it.
I don't know whether I made the question clear... You can see the CLISP example. Is there an equivalent way to implement this pattern in Python?
EDIT: I put the above clisp snippets online for your quick reference.
Let me put two use cases here for multiple return value pattern. Probably someone can have alternative solutions to the two cases:
Better support smooth upgrade. This is shown in the above example.
Have simpler client side codes. See following alternative solutions I have so far. Using exception can make the upgrade process smooth but it costs more codes.
Current alternatives: (they are not "multi-value-return" constructions, but they can be engineering solutions that satisfy some of the points listed above)
tuple, list, dict, an object. As is said, you need certain parsing from the client side. e.g. if ret.success == True: blabla. You need to ret = func() before that. It's much cleaner to write if func() == True: blabal.
Use Exception. As is discussed in this thread, when the "False" case is rare, it's a nice solution. Even in this case, the client side code is still too heavy.
Use an arg, e.g. def func(main_arg, detail=[]). The detail can be list or dict or even an object depending on your design. The func() returns only original simple value. Details go to the detail argument. Problem is that the client need to create a variable before invocation in order to hold the details.
Use a "verbose" indicator, e.g. def func(main_arg, verbose=False). When verbose == False (default; and the way client is using func()), return original simple value. When verbose == True, return an object which contains simple value and the details.
Use a "version" indicator. Same as "verbose" but we extend the idea there. In this way, you can upgrade the returned object for multiple times.
Use global detail_msg. This is like the old C-style error_msg. In this way, functions can always return simple values. The client side can refer to detail_msg when necessary. One can put detail_msg in global scope, class scope, or object scope depending on the use cases.
Use generator. yield simple_return and then yield detailed_return. This solution is nice in the callee's side. However, the caller has to do something like func().next() and func().next().next(). You can wrap it with an object and override the __call__ to simplify it a bit, e.g. func()(), but it looks unnatural from the caller's side.
Use a wrapper class for the return value. Override the class's methods to mimic the behaviour of original simple return value. Put detailed data in the class. We have adopted this alternative in our project in dealing with bool return type. see the relevant commit: https://github.com/fqj1994/snsapi/commit/589f0097912782ca670568fe027830f21ed1f6fc (I don't have enough reputation to put more links in the post... -_-//)
Here are some solutions:
Based on #yupbank 's answer, I formalized it into a decorator, see github.com/hupili/multiret
The 8th alternative above says we can wrap a class. This is the current engineering solution we adopted. In order to wrap more complex return values, we may use meta class to generate the required wrapper class on demand. Have not tried, but this sounds like a robust solution.
try inspect?
i did some try, and not very elegant, but at least is doable.. and works :)
import inspect
from functools import wraps
import re
def f1(*args):
return 2
def f2(*args):
return 3, 3
PATTERN = dict()
PATTERN[re.compile('(\w+) f()')] = f1
PATTERN[re.compile('(\w+), (\w+) = f()')] = f2
def execute_method_for(call_str):
for regex, f in PATTERN.iteritems():
if regex.findall(call_str):
return f()
def multi(f1, f2):
def liu(func):
#wraps(func)
def _(*args, **kwargs):
frame,filename,line_number,function_name,lines,index=\
inspect.getouterframes(inspect.currentframe())[1]
call_str = lines[0].strip()
return execute_method_for(call_str)
return _
return liu
#multi(f1, f2)
def f():
return 1
if __name__ == '__main__':
print f()
a, b = f()
print a, b
Your case does need code editing. However, if you need a hack, you can use function attributes to return extra values , without modifying return values.
def attr_store(varname, value):
def decorate(func):
setattr(func, varname, value)
return func
return decorate
#attr_store('extra',None)
def func(input_str):
func.extra = {'hello':input_str + " ,How r you?", 'num':2}
return 1
print(func("John")+func("Matt"))
print(func.extra)
Demo : http://codepad.org/0hJOVFcC
However, be aware that function attributes will behave like static variables, and you will need to assign values to them with care, appends and other modifiers will act on previous saved values.
the magic is you should use design pattern blablabla to not use actual operation when you process the result, but use a parameter as the operation method, for your case, you can use the following code:
def x():
#return 1
return 1, 'x'*1
def f(op, f1, f2):
print eval(str(f1) + op + str(f2))
f('+', x(), x())
if you want generic solution for more complicated situation, you can extend the f function, and specify the process operation via the op parameter