Is it possible to forward-declare a function in Python? I want to sort a list using my own cmp function before it is declared.
print "\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
I've put the definition of cmp_configs method after the invocation. It fails with this error:
NameError: name 'cmp_configs' is not defined
Is there any way to "declare" cmp_configs method before it's used?
Sometimes, it is difficult to reorganize code to avoid this problem. For instance, when implementing some forms of recursion:
def spam():
if end_condition():
return end_result()
else:
return eggs()
def eggs():
if end_condition():
return end_result()
else:
return spam()
Where end_condition and end_result have been previously defined.
Is the only solution to reorganize the code and always put definitions before invocations?
Wrap the invocation into a function of its own so that
foo()
def foo():
print "Hi!"
will break, but
def bar():
foo()
def foo():
print "Hi!"
bar()
will work properly.
The general rule in Python is that a function should be defined before its usage, which does not necessarily mean it needs to be higher in the code.
If you kick-start your script through the following:
if __name__=="__main__":
main()
then you probably do not have to worry about things like "forward declaration". You see, the interpreter would go loading up all your functions and then start your main() function. Of course, make sure you have all the imports correct too ;-)
Come to think of it, I've never heard such a thing as "forward declaration" in python... but then again, I might be wrong ;-)
If you don't want to define a function before it's used, and defining it afterwards is impossible, what about defining it in some other module?
Technically you still define it first, but it's clean.
You could create a recursion like the following:
def foo():
bar()
def bar():
foo()
Python's functions are anonymous just like values are anonymous, yet they can be bound to a name.
In the above code, foo() does not call a function with the name foo, it calls a function that happens to be bound to the name foo at the point the call is made. It is possible to redefine foo somewhere else, and bar would then call the new function.
Your problem cannot be solved because it's like asking to get a variable which has not been declared.
I apologize for reviving this thread, but there was a strategy not discussed here which may be applicable.
Using reflection it is possible to do something akin to forward declaration. For instance lets say you have a section of code that looks like this:
# We want to call a function called 'foo', but it hasn't been defined yet.
function_name = 'foo'
# Calling at this point would produce an error
# Here is the definition
def foo():
bar()
# Note that at this point the function is defined
# Time for some reflection...
globals()[function_name]()
So in this way we have determined what function we want to call before it is actually defined, effectively a forward declaration. In python the statement globals()[function_name]() is the same as foo() if function_name = 'foo' for the reasons discussed above, since python must lookup each function before calling it. If one were to use the timeit module to see how these two statements compare, they have the exact same computational cost.
Of course the example here is very useless, but if one were to have a complex structure which needed to execute a function, but must be declared before (or structurally it makes little sense to have it afterwards), one can just store a string and try to call the function later.
If the call to cmp_configs is inside its own function definition, you should be fine. I'll give an example.
def a():
b() # b() hasn't been defined yet, but that's fine because at this point, we're not
# actually calling it. We're just defining what should happen when a() is called.
a() # This call fails, because b() hasn't been defined yet,
# and thus trying to run a() fails.
def b():
print "hi"
a() # This call succeeds because everything has been defined.
In general, putting your code inside functions (such as main()) will resolve your problem; just call main() at the end of the file.
There is no such thing in python like forward declaration. You just have to make sure that your function is declared before it is needed.
Note that the body of a function isn't interpreted until the function is executed.
Consider the following example:
def a():
b() # won't be resolved until a is invoked.
def b():
print "hello"
a() # here b is already defined so this line won't fail.
You can think that a body of a function is just another script that will be interpreted once you call the function.
Sometimes an algorithm is easiest to understand top-down, starting with the overall structure and drilling down into the details.
You can do so without forward declarations:
def main():
make_omelet()
eat()
def make_omelet():
break_eggs()
whisk()
fry()
def break_eggs():
for egg in carton:
break(egg)
# ...
main()
# declare a fake function (prototype) with no body
def foo(): pass
def bar():
# use the prototype however you see fit
print(foo(), "world!")
# define the actual function (overwriting the prototype)
def foo():
return "Hello,"
bar()
Output:
Hello, world!
No, I don't believe there is any way to forward-declare a function in Python.
Imagine you are the Python interpreter. When you get to the line
print "\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
either you know what cmp_configs is or you don't. In order to proceed, you have to
know cmp_configs. It doesn't matter if there is recursion.
You can't forward-declare a function in Python. If you have logic executing before you've defined functions, you've probably got a problem anyways. Put your action in an if __name__ == '__main__' at the end of your script (by executing a function you name "main" if it's non-trivial) and your code will be more modular and you'll be able to use it as a module if you ever need to.
Also, replace that list comprehension with a generator express (i.e., print "\n".join(str(bla) for bla in sorted(mylist, cmp=cmp_configs)))
Also, don't use cmp, which is deprecated. Use key and provide a less-than function.
Import the file itself. Assuming the file is called test.py:
import test
if __name__=='__main__':
test.func()
else:
def func():
print('Func worked')
TL;DR: Python does not need forward declarations. Simply put your function calls inside function def definitions, and you'll be fine.
def foo(count):
print("foo "+str(count))
if(count>0):
bar(count-1)
def bar(count):
print("bar "+str(count))
if(count>0):
foo(count-1)
foo(3)
print("Finished.")
recursive function definitions, perfectly successfully gives:
foo 3
bar 2
foo 1
bar 0
Finished.
However,
bug(13)
def bug(count):
print("bug never runs "+str(count))
print("Does not print this.")
breaks at the top-level invocation of a function that hasn't been defined yet, and gives:
Traceback (most recent call last):
File "./test1.py", line 1, in <module>
bug(13)
NameError: name 'bug' is not defined
Python is an interpreted language, like Lisp. It has no type checking, only run-time function invocations, which succeed if the function name has been bound and fail if it's unbound.
Critically, a function def definition does not execute any of the funcalls inside its lines, it simply declares what the function body is going to consist of. Again, it doesn't even do type checking. So we can do this:
def uncalled():
wild_eyed_undefined_function()
print("I'm not invoked!")
print("Only run this one line.")
and it runs perfectly fine (!), with output
Only run this one line.
The key is the difference between definitions and invocations.
The interpreter executes everything that comes in at the top level, which means it tries to invoke it. If it's not inside a definition.
Your code is running into trouble because you attempted to invoke a function, at the top level in this case, before it was bound.
The solution is to put your non-top-level function invocations inside a function definition, then call that function sometime much later.
The business about "if __ main __" is an idiom based on this principle, but you have to understand why, instead of simply blindly following it.
There are certainly much more advanced topics concerning lambda functions and rebinding function names dynamically, but these are not what the OP was asking for. In addition, they can be solved using these same principles: (1) defs define a function, they do not invoke their lines; (2) you get in trouble when you invoke a function symbol that's unbound.
Python does not support forward declarations, but common workaround for this is use of the the following condition at the end of your script/code:
if __name__ == '__main__': main()
With this it will read entire file first and then evaluate condition and call main() function which will be able to call any forward declared function as it already read the entire file first. This condition leverages special variable __name__ which returns __main__ value whenever we run Python code from current file (when code was imported as a module, then __name__ returns module name).
"just reorganize my code so that I don't have this problem." Correct. Easy to do. Always works.
You can always provide the function prior to it's reference.
"However, there are cases when this is probably unavoidable, for instance when implementing some forms of recursion"
Can't see how that's even remotely possible. Please provide an example of a place where you cannot define the function prior to it's use.
Now wait a minute. When your module reaches the print statement in your example, before cmp_configs has been defined, what exactly is it that you expect it to do?
If your posting of a question using print is really trying to represent something like this:
fn = lambda mylist:"\n".join([str(bla)
for bla in sorted(mylist, cmp = cmp_configs)])
then there is no requirement to define cmp_configs before executing this statement, just define it later in the code and all will be well.
Now if you are trying to reference cmp_configs as a default value of an argument to the lambda, then this is a different story:
fn = lambda mylist,cmp_configs=cmp_configs : \
"\n".join([str(bla) for bla in sorted(mylist, cmp = cmp_configs)])
Now you need a cmp_configs variable defined before you reach this line.
[EDIT - this next part turns out not to be correct, since the default argument value will get assigned when the function is compiled, and that value will be used even if you change the value of cmp_configs later.]
Fortunately, Python being so type-accommodating as it is, does not care what you define as cmp_configs, so you could just preface with this statement:
cmp_configs = None
And the compiler will be happy. Just be sure to declare the real cmp_configs before you ever invoke fn.
Python technically has support for forward declaration.
if you define a function/class then set the body to pass, it will have an empty entry in the global table.
you can then "redefine" the function/class later on to implement the function/class.
unlike c/c++ forward declaration though, this does not work from outside the scope (i.e. another file) as they have their own "global" namespace
example:
def foo(): pass
foo()
def foo(): print("FOOOOO")
foo()
foo is declared both times
however the first time foo is called it does not do anything as the body is just pass
but the second time foo is called. it executes the new body of print("FOOOOO")
but again. note that this does not fix circular dependancies. this is because files have their own name and have their own definitions of functions
example 2:
class bar: pass
print(bar)
this prints <class '__main__.bar'> but if it was declared in another file it would be <class 'otherfile.foo'>
i know this post is old, but i though that this answer would be useful to anyone who keeps finding this post after the many years it has been posted for
One way is to create a handler function. Define the handler early on, and put the handler below all the methods you need to call.
Then when you invoke the handler method to call your functions, they will always be available.
The handler could take an argument nameOfMethodToCall. Then uses a bunch of if statements to call the right method.
This would solve your issue.
def foo():
print("foo")
#take input
nextAction=input('What would you like to do next?:')
return nextAction
def bar():
print("bar")
nextAction=input('What would you like to do next?:')
return nextAction
def handler(action):
if(action=="foo"):
nextAction = foo()
elif(action=="bar"):
nextAction = bar()
else:
print("You entered invalid input, defaulting to bar")
nextAction = "bar"
return nextAction
nextAction=input('What would you like to do next?:')
while 1:
nextAction = handler(nextAction)
I know the title is very confusing but essentially here is what I am trying to do:
def foo():
bar(name)
def evaluate_foo(name_table):
def bar(name, lookup_table=name_table):
print(name + "'s last name is " + lookup_table[name])
foo()
Basically, if I know that foo is going to make a call to bar, can I replace bar with my own function right before calling foo?
I am trying to provide a lookup table to bar, but the lookup table is generated inside of evaluate_foo. It's also important to me that someone using the function bar doesn't need to know about the lookup table that is being used by bar within it.
If I understand your question correctly, you can pass your function to foo to be executed. For instance you could do:
def foo(f):
f()
def evaluate_foo(some_param):
baz = some_param.get_special_number()
def bar(bar_param=baz):
print("Bar's result is " + str(baz))
foo(bar)
This won't work because foo() will lookup the function referred to by bar in the namespace it's defined in, not the namespace it's called in.
This sort of situation is what object inheritance is for, instead of having function foo() you should redesign it to be object whatever with method whatever.foo() and whatever.bar(). Then you can subclass whatever and write a new method bar() that foo() will find because it's defined in the class namespace.
Presumably you want to do this because you don't control foo() and so you can't convert it from a function into a method. In that case you can achieve this by monkey-patching bar() in the same namespace where foo() is defined, but that's some deep dark voodoo that is generally discouraged.
If I understand you correctly, you want to override a function definition globally. The following is not pretty, but it works (with some caveats below):
def foo():
print("foo")
def baz():
print("baz")
foo()
def bar():
def local_foo():
print("local foo")
global foo
foo = local_foo
baz()
if __name__ == "__main__":
bar()
The output is
baz
local foo
So, the caveats:
You can only do this for global functions, unless you want to jump through even more hoops.
The second point is that, unless you're just doing this for fun or are desperate, you should use other methods (as described by others) to achieve the same effect. E.g. some kind of design pattern.
I've recently made a move from C/C++ to Python. Having difficulties in understanding closures and decorators.
Declared the functions as such (an online blog had this piece code as an example)
def outer(some_func):
def inner():
print "Before Foo"
ret = some_func()
return ret + 1
return inner
def foo():
return 1
When I run the following code once
foo = outer(foo)
foo()
I get the expected output as
Before Foo
2
However if I run it more than once, I get strange output. I do not understand what happens.
# Writing it down twice instead of running the same cell
# twice on my IPython notebook to better explain things here
foo = outer(foo)
foo = outer(foo)
foo()
The output is as follows
Before Foo
Before Foo
3
Why do I get 3 as the output and why does "Before Foo" get printed twice?
Edit:- What happens when I decorate the function again? Running foo = outer(foo) is quite clear now. What happens when I run foo = outer(foo) again?
You're decorating the decorated function again. outer takes in a function and returns closure which of course is function by itself. There's nothing preventing you to pass the returned function as a parameter to the following call like you're doing in your example.
If you change your code to print the function being called and return value it's easier to understand what's going on:
def outer(some_func):
def inner():
print "Before inner, func: {0}".format(some_func)
ret = some_func()
print "After inner, return: {0}".format(ret + 1)
return ret + 1
return inner
def foo():
return 1
foo = outer(foo)
foo = outer(foo)
foo()
Output:
Before inner, func: <function inner at 0x7f1924b516e0>
Before inner, func: <function foo at 0x7f1924b51668>
After inner, return: 2
After inner, return: 3
Consider what foo refers to in each statement of foo=outer(foo). Initially you define foo to be a function that always returns 1. Once you call foo=outer(foo) though, it now has a new value. Try to imagine replacing the foo on the right side of the assignment with what foo would equal before you run that line. Remember, you're nesting functions here.
A couple ways that might help you wrap your head around what's happening are
Add a couple more print statements to different parts of your function to help understand what's being called and when.
Don't re-assign to the variable foo unless there is a reason to do so. Maybe choose foo1, foo2, etc. so you can look back to previous iterations of foo and see more clearly the composition that's taking place.
question hoebenkr, I don't understand how when you're assigning the variable ret to the parameter of the function, you call the function. Is that the de-facto standard for declaring a variable to a function? It promptly runs in python?
I've got a Python module which has several variables with hard-coded values which are used throughout the project. I'd like to bind the variables somehow to a function to redirect to a config file. Is this possible?
# hardcoded_values.py
foo = 'abc'
bar = 1234
# usage somewhere else in another module
from hardcoded_values import *
print foo
print bar
What I want to do is change only hardcoded_values.py, so that print foo transparently calls a function.
# hardcoded_values.py
import config
foo = SomeWrapper(config.get_value, 'foo') # or whatever you can think of to call config.get_value('foo')
...
config.get_value would be a function that is called with parameter 'foo' when using variable foo (as in print foo).
I'm pretty sure that you can't do what you want to do if you import like from hardcoded_values import *.
What you want to do is to set foo to some function, and then apply the property decorator (or equivalent) so that you can call foo as foo rather than foo(). You cannot apply the property decorator to modules for reasons detailed here: Why Is The property Decorator Only Defined For Classes?
Now, if you were to import hardcoded_values then I think there is a way to do what you want to hardcoded_values.foo. I have a pretty good feeling that what I am about to describe is a BAD IDEA that should never be used, but I think it is interesting.
BAD IDEA???
So say you wanted to replace a constant like os.EX_USAGE which on my system is 64 with some function, and then call it as os.EX_USAGE rather than os.EX_USAGE(). We need to be able to use the property decorator, but for that we need a type other than module.
So what can be done is to create a new type on the fly and dump in the __dict__ of a module with a type factory function that takes a module as an argument:
def module_class_factory(module):
ModuleClass = type('ModuleClass' + module.__name__,
(object,), module.__dict__)
return ModuleClass
Now I will import os:
>>> import os
>>> os.EX_USAGE
64
>>> os.getcwd()
'/Users/Eric'
Now I will make a class OsClass, and bind the name os to an instance of this class:
>>> OsClass = module_class_factory(os)
>>> os = OsClass()
>>> os.EX_USAGE
64
>>> os.getcwd()
'/Users/Eric'
Everything still seems to work. Now define a function to replace os.EX_USAGE, noting that it will need to take a dummy self argument:
>>> def foo(self):
... return 42
...
...and bind the class attribute OsClass.EX_USAGE to the function:
>>> OsClass.EX_USAGE = foo
>>> os.EX_USAGE()
42
It works when called by the os object! Now just apply the property decorator:
>>> OsClass.EX_USAGE = property(OsClass.EX_USAGE)
>>> os.EX_USAGE
42
now the constant defined in the module has been transparently replaced by a function call.
You definitely cannot do this if the client code of your module uses from hardcoded_variables import *. That makes references to the contents of hardcoded_variables in the other module's namespace, and you can't do anything with them after that.
If the client code can be changed to just import the module (with e.g. import hardcoded_variables) and then access its attributes (with hardcoded_variables.foo) you do have a chance, but it's a bit awkward.
Python caches modules that have been imported in sys.modules (which is a dictionary). You can replace a module in that dictionary with some other object, such as an instance of a custom class, and use property objects or other descriptors to implement special behavior when you access the object's attributes.
Try making your new hardcoded_variables.py look like this (and consider renaming it, too!):
import sys
class DummyModule(object):
def __init__(self):
self._bar = 1233
#property
def foo(self):
print("foo called!")
return "abc"
#property
def bar(self):
self._bar += 1
return self._bar
if __name__ != "__main__": # Note, this is the opposite of the usual boilerplate
sys.modules[__name__] = DummyModule()
If I understand correctly, you want your hardcoded_variables module evaluated every time you try to access a variable.
I would probably have hardcoded_variables in a document (e.g. json?) and a custom wrapper function like:
import json
def getSettings(var):
with open('path/to/variables.json') as infl:
data = json.load(infl)
return infl[var]