For the purpose of building a specialized debugger for my code I would like to trace some of the functions in my code, and to log the arguments they received in each call.
I would like to be able to do this without adding line of code to each function, or even a decorator around all functions, but to set a trace around the whole run.
This is somewhat similar to what you can do with sys.settrace in the sys module:
https://docs.python.org/2/library/sys.html
except that as far as I can tell the trace doesn't include the funcions' arguments.
So i would like to write a function that looks something like this:
def tracing_func(func_name, args):
if func_name in ['func', 'foo']:
log_func_args(func_name, args)
where log_func_args logs it in a file for later analysis.
Then set this function to be called whenever any function in my code is called, with the functions' name and args.
Can this be done?
Ok so sys.settrace does the trick pretty well:
https://docs.python.org/2/library/sys.html#sys.settrace
and an example:
https://pymotw.com/2/sys/tracing.html
note that the function you pass to settrace has to return a reference to itself (or to another function for further tracing in that scope).
Related
I have just been programming in python, and have been looking to just make simple programs to start getting a better understanding.
I am busy writing a rock paper scissors text game and to work out the games winner/looser etc I created a function. For a while I when I ran my program I kept getting an error when the function was called, it was that my function was not defined. My function was below my initial code.
However I for some reason moved my function to the top of my code under my global variable declarations, and now my function executes perfectly.
Why is this the case? is there a way I can have my functions below my main code but not get the error of my function being undefined. Would I just need to declare my function before and then call it later, if so, how would I declare an empty function?
I would love to understand so any help would be greatly appreciated.
Why is this the case?
Because that's the way Python works - all the code at the top-level of a module or script is executed sequentially, so functions are only defined after the def statement has been executed.
You have to understand that in Python everything is an object, including functions, classes etc, so the def statement is mostly a syntactic sugar that creates a function object (from the def block) and binds it to the function name, IOW a function is just another global variable in your module or script - and you wouldn't expect to be able to use a variable before you defined it, would you ?
is there a way I can have my functions below my main code but not get the error of my function being undefined.
Yes quite simply by putting your "main code" in a function and call this function at the end of the script:
import something
import another thing
def main():
bar = foo()
print("the answer is {}".format(bar))
def foo():
return 42
# this makes sure the main function will only be executed
# when using your .py file as a script, not when importing
# it as a module.
if __name__ == "__main__":
main()
Function has to be defined first before it is called.You have to define a function and afterwards function can be called anywhere in the program.
This is how Python works,in this all the things executes in a sequence.
I am new to Python so please don't flame me if the question is too basic :)
I have read that Python is executed from top - to - bottom.
If this is the case, why do programs go like this:
def func2():
pass
def func1():
func2()
def func():
func1()
if __name__ == '__main__':
func()
So from what I have seen, the main function goes at last and the other functions are stacked on top of it.
Am I wrong in saying this? If no, why isn't the main function or the function definitions written from top to bottom?
EDIT: I am asking why I can't do this:
if __name__ == '__main__':
func()
def func1():
func2()
Isn't this the natural order? You keep on adding stuff at the bottom, since it is executed from top to bottom.
The defs are just creating the functions. No code is executed, other than to parse the syntax and tie functions to those names.
The if is the first place code is actually executed. If you put it first, and call a function before it is defined, the result is a NameError. Therefore, you need to put it after the functions are defined.
Note that this is unlike PHP or JavaScript, where functions are 'hoisted' - any function definitions are processed and parsed before everything else. In PHP and JavaScript, it's perfectly legal to do what you are saying and define functions in the source lower down than where they are called. (One detail in JS is that functions defined like function(){} are hoisted, while functions defined like var func1=function(){}; are not. I don't know how it works with anonymous functions in PHP 5.3 yet).
See, in this, cat() will print correctly, and yip() gives you a NameError because the parser hasn't gotten to the definition of yip() at the time you call it.
def cat():
print 'meowin, yo'
cat()
yip()
def yip():
print 'barkin, yall'
meowin, yo
Traceback (most recent call last):
File "cat.py", line 5, in
yip()
NameError: name 'yip' is not defined
Python is executed from top to bottom, but executing a "def" block doesn't immediately execute the contained code. Instead it creates a function object with the given name in the current scope. Consider a Python file much like your example:
def func2():
print "func2"
def func1():
func2()
def func():
func1()
if __name__ == '__main__':
func()
What happens when this script is executed is as follows:
Firstly, a function object is created and bound to the name "func2" in the global scope. Then a function object is created and bound to the name "func1" in the global scope. Then one called "func". Then the "if" statement is executed, the condition is true and the "func()" statement is executed. At this point "func" is a function object found in the global scope, so it is invoked and its code runs. That code contains the "func1()" statement, which is resolved to a call to the "func1" function, and so on.
If you put the "if" statement at the top then when it executes there would not yet be anything defined with the name "func", so you would get an error. The important thing to recognise is that the "def" statement is itself a statement that is executed. It is not like in some other languages where definitions are a separate sort of declaration with no order of execution.
Also note that so long as the "if __name__..." bit is at the end of the file, it doesn't really matter what order the other declarations are in, since by the time any of them are called all of the "def"s will have already been executed.
Python does, in general, process commands from top to bottom. However, a function call will cause Python to execute that function, and continue downward only after that call has ended.
In your example, the Python interpreter executes the following steps:
Define func2.
Define func1.
Define func.
Process if statement if __name__ == '__main__':.
Call the func function (since the condition is true).
Call the func1 function (because that's what func does).
Call the func2 function (because that's what func1 does).
End, because after finishing the call to func2 it has also finished calling func1 and therefore has finished calling func, which was the final statement in the code.
It's also worth noting that you can have function calls written occur "before they're defined", as long as they aren't executed. Mentioning this here as it's a common new-to-python error. None of the other examples demonstrate this "it's ok, in one way" behavior. (though not really recommended)
def hi():
pie()
#hi() cannot be called here, doesn't work because hi depends on pie
def pie():
print('hi pie')
hi() # it's ok to call here!
You can even go a little further (slightly silly example, but I believe it clarifies the example)
runIt = False
def hi():
if runIt:
pie()
hi() #now ok, as the code won't call pie.
runIt = True
#hi() #wouldn't be ok as now the code will call pie.
def pie():
print('hi pie')
hi() # ok here too!
The if __name__ == "__main__" part goes at the end, because presumably whatever your main function does will need all the other definitions in the script. If there were any other function definitions below the main block, then it would not be possible for them to be used in there, since they haven't been seen by the interpreter at that point.
def statements simply define a function - they don't actually run it until the function is invoked. Hence why they generally come before whatever code uses them - they set the function up to be used in the future.
There's no hard requirement that def statements come before anything else, it's just fairly common to put the __main__ code at the bottom (among other things, it makes it clear what's in that section and what isn't, since anything below the if is part of that section). It also makes sure whatever you want to call from the section has already been def'd.
For the second part of your question (Style of coding in Python), I would take a look at PEP 8, which provides a style guide written by Guido van Rossum (the BFDL himself!) and provides the basics of writing Pythonic code.
It's true that the convention is to put the "main" function just above the "if __name__ == '__main__'" statement at the end of the file. I think that this is because we usually want functions to be near to their calling sites.
As for the called function coming above the calling function, I think that this is just a result of refactoring. At least in my case, I tend to write a function, and then take out bits of code to put into sub-functions, which I put on top so that I can see better.
This is just a primitive form of code organization. If there's a stronger cohesive unit, I'd be tempted to pull those functions into their own module.
There is two points that you need to know :
The main condition is to block the prevent code from being run when
the module is imported. In our case, the condition is true because
our file is not imported.
The code is working as follows :
Define func2
Define func1
Define func
Process if statement if __name__ == '__main__': (which in our case is True)
Call the func function
Call the func1
Call the func2 function
return the result(print the message func2)
End
Is there any way to call a function before its definition.
def Insert(value):
"""place value at an available leaf, then bubble up from there"""
heap.append(value)
BubbleUp(len(heap) - 1)
def BubbleUp(position):
print 'something'
This code shows "unresolved reference BubbleUp"
The code here doesn't show anything, least of all an error, because neither of the functions is called. What matters is the location of the call to Insert, and as long as it comes after BubbleUp (and why wouldn't it), there is no issue. Function definitions don't execute the function body, so you can define functions in whatever order you like, as long as you refrain from calling any of them until all necessary functions are defined.
I'm writing a python command line program which has some interdependent options, I would like for the user to be able to enter the options in whichever order they please.
Currently I am using the getopts library to parse the command line options, unfortunately that parses them in-order. I've thrown together a system of boolean flags to leave the processing of certain command line arguments until the one they're dependent on is processed, however I had the idea of using a Priority Queue of function calls which would execute after all the command line options are parsed.
I know that Python can store functions under variable names, but that seems to call the function at the same time.
For example:
help = obj.PrintHelp()
heapq.heappush(commandQ, (0, help))
Will print the help dialog immediately. How would I go about implementing my code such that it won't call PrintHelp() immediately upon assigning it a name.
EDIT:
Oh i just realized I was pushing into a queue called help, that's my mistake.
Thanks for the tip on removing the () after PrintHelp.
What if I want to now call a function that requires more than the self argument?
myFun = obj.parseFile(path)
heapq.heappush(commandQ, (1, myFun))
Would I just make the tuple bigger and take the command line argument?
If you heappush like this:
myFun = obj.parseFile
heapq.heappush(commandQ, (1, myFun, path))
then to later call the function, you could do this:
while commandQ:
x=heapq.heappop(commandQ)
func=x[1]
args=x[2:]
func(*args)
Use
help = obj.PrintHelp
without the parentheses. This makes help reference the function.
Later, you can call the function with help().
Note also (if I understand your situation correctly), you could just use the optparse or (if you have Python2.7 or better) argparse modules in the standard library to handle the command-line options in any order.
PS. help is a built-in function in Python. Naming a variable help overrides the built-in, making it difficult (though not impossible) to access the built-in. Generally, it's a good idea not to overwrite the names of built-ins.
Instead of using getopts, I would suggest using optparse (argparse, if you are using a newer python version): most probably, you will get everything you need, already implemented.
That said, in your example code, you are actually calling the function, while you should simply get its name:
help = obj.PrintHelp
heapq.heappush(help, (0, help))
If you want to store a complete function call in Python, you can do it one of two ways:
# option 1: hold the parameters separately
# I've also skipped saving the function in a 'help' variable'
heapq.heappush(commandQ, (0, obj.PrintHelp, param1, param2))
# later:
command = commandQ[0]
heapq.heappop(commandQ)
command[1](*command[2:]) # call the function (second item) with args (remainder of items)
Alternatively, you can use a helper to package the arguments up via lambda:
# option 2: build a no-argument anonymous function that knows what arguments
# to give the real one
# module scope
def makeCall(func, *args):
return lambda: func(*args)
# now you can:
help = makeCall(obj.PrintHelp, param1, param2)
heapq.heappush(commandQ, (0, help))
If you need keyword arguments, let me know and I'll edit to take care of those too.
I am new to Python so please don't flame me if the question is too basic :)
I have read that Python is executed from top - to - bottom.
If this is the case, why do programs go like this:
def func2():
pass
def func1():
func2()
def func():
func1()
if __name__ == '__main__':
func()
So from what I have seen, the main function goes at last and the other functions are stacked on top of it.
Am I wrong in saying this? If no, why isn't the main function or the function definitions written from top to bottom?
EDIT: I am asking why I can't do this:
if __name__ == '__main__':
func()
def func1():
func2()
Isn't this the natural order? You keep on adding stuff at the bottom, since it is executed from top to bottom.
The defs are just creating the functions. No code is executed, other than to parse the syntax and tie functions to those names.
The if is the first place code is actually executed. If you put it first, and call a function before it is defined, the result is a NameError. Therefore, you need to put it after the functions are defined.
Note that this is unlike PHP or JavaScript, where functions are 'hoisted' - any function definitions are processed and parsed before everything else. In PHP and JavaScript, it's perfectly legal to do what you are saying and define functions in the source lower down than where they are called. (One detail in JS is that functions defined like function(){} are hoisted, while functions defined like var func1=function(){}; are not. I don't know how it works with anonymous functions in PHP 5.3 yet).
See, in this, cat() will print correctly, and yip() gives you a NameError because the parser hasn't gotten to the definition of yip() at the time you call it.
def cat():
print 'meowin, yo'
cat()
yip()
def yip():
print 'barkin, yall'
meowin, yo
Traceback (most recent call last):
File "cat.py", line 5, in
yip()
NameError: name 'yip' is not defined
Python is executed from top to bottom, but executing a "def" block doesn't immediately execute the contained code. Instead it creates a function object with the given name in the current scope. Consider a Python file much like your example:
def func2():
print "func2"
def func1():
func2()
def func():
func1()
if __name__ == '__main__':
func()
What happens when this script is executed is as follows:
Firstly, a function object is created and bound to the name "func2" in the global scope. Then a function object is created and bound to the name "func1" in the global scope. Then one called "func". Then the "if" statement is executed, the condition is true and the "func()" statement is executed. At this point "func" is a function object found in the global scope, so it is invoked and its code runs. That code contains the "func1()" statement, which is resolved to a call to the "func1" function, and so on.
If you put the "if" statement at the top then when it executes there would not yet be anything defined with the name "func", so you would get an error. The important thing to recognise is that the "def" statement is itself a statement that is executed. It is not like in some other languages where definitions are a separate sort of declaration with no order of execution.
Also note that so long as the "if __name__..." bit is at the end of the file, it doesn't really matter what order the other declarations are in, since by the time any of them are called all of the "def"s will have already been executed.
Python does, in general, process commands from top to bottom. However, a function call will cause Python to execute that function, and continue downward only after that call has ended.
In your example, the Python interpreter executes the following steps:
Define func2.
Define func1.
Define func.
Process if statement if __name__ == '__main__':.
Call the func function (since the condition is true).
Call the func1 function (because that's what func does).
Call the func2 function (because that's what func1 does).
End, because after finishing the call to func2 it has also finished calling func1 and therefore has finished calling func, which was the final statement in the code.
It's also worth noting that you can have function calls written occur "before they're defined", as long as they aren't executed. Mentioning this here as it's a common new-to-python error. None of the other examples demonstrate this "it's ok, in one way" behavior. (though not really recommended)
def hi():
pie()
#hi() cannot be called here, doesn't work because hi depends on pie
def pie():
print('hi pie')
hi() # it's ok to call here!
You can even go a little further (slightly silly example, but I believe it clarifies the example)
runIt = False
def hi():
if runIt:
pie()
hi() #now ok, as the code won't call pie.
runIt = True
#hi() #wouldn't be ok as now the code will call pie.
def pie():
print('hi pie')
hi() # ok here too!
The if __name__ == "__main__" part goes at the end, because presumably whatever your main function does will need all the other definitions in the script. If there were any other function definitions below the main block, then it would not be possible for them to be used in there, since they haven't been seen by the interpreter at that point.
def statements simply define a function - they don't actually run it until the function is invoked. Hence why they generally come before whatever code uses them - they set the function up to be used in the future.
There's no hard requirement that def statements come before anything else, it's just fairly common to put the __main__ code at the bottom (among other things, it makes it clear what's in that section and what isn't, since anything below the if is part of that section). It also makes sure whatever you want to call from the section has already been def'd.
For the second part of your question (Style of coding in Python), I would take a look at PEP 8, which provides a style guide written by Guido van Rossum (the BFDL himself!) and provides the basics of writing Pythonic code.
It's true that the convention is to put the "main" function just above the "if __name__ == '__main__'" statement at the end of the file. I think that this is because we usually want functions to be near to their calling sites.
As for the called function coming above the calling function, I think that this is just a result of refactoring. At least in my case, I tend to write a function, and then take out bits of code to put into sub-functions, which I put on top so that I can see better.
This is just a primitive form of code organization. If there's a stronger cohesive unit, I'd be tempted to pull those functions into their own module.
There is two points that you need to know :
The main condition is to block the prevent code from being run when
the module is imported. In our case, the condition is true because
our file is not imported.
The code is working as follows :
Define func2
Define func1
Define func
Process if statement if __name__ == '__main__': (which in our case is True)
Call the func function
Call the func1
Call the func2 function
return the result(print the message func2)
End