We Keep Coding

How can attributes on functions survive wrapping?

Let's say I have the following function which has an attribute which marks it for special handling within a callback subsystem:
def my_func(msg):
print msg
my_func.my_marker = SPECIAL_CONSTANT
The problem is that if other bits of code wrap my_func with functools.partial or another decorator, then my_marker will be lost.
my_partial = partial(my_func, 'hello world')
print my_partial.my_marker
>>> AttributeError...
Is there a way to protect attributes on functions when wrapping? Is there a better way to store the metadata I'm currently storing in my_marker? It seems like storing a reference to the original function suffers from the same problem.

If you know that the partial you use is actually a partial, you can use it's func attribute.
E. g.
from functools import partial
SPECIAL_CONSTANT = 'bam'
def my_func(msg):
print msg
my_func.my_marker = SPECIAL_CONSTANT
my_partial = partial(my_func, 'hello world')
print my_partial.func.my_marker
If you really have to handle meta_data, maybe it is a better approach to write classes and override the __call__() method.

Here is another solution, using functools.update_wrapper.
from functools import partial, update_wrapper, WRAPPER_ASSIGNMENTS
SPECIAL_CONSTANT = 'bam'
def my_func(msg):
print msg
my_func.my_marker = SPECIAL_CONSTANT
my_partial = partial(my_func, 'hello world')
update_wrapper(my_partial, my_func, WRAPPER_ASSIGNMENTS + ('my_marker', ))
print my_partial.my_marker

Inspired by kindall's comment, cbo's answer and the source code:
from functools import partial as _partial, update_wrapper
def partial(func, *args, **keywords):
return update_wrapper(_partial(func, *args, **keywords), func)
An example showing it working, and caveats:
def my_func(msg):
print msg
my_func.my_marker = 'FOO'
my_partial = partial(my_func, 'hello world')
print my_func.my_marker
print my_partial.my_marker
# check other stuff is still there
print my_partial.func
print my_partial.args
print my_partial.keywords
# this works fine for READ ONLY stuff.
# so just remember:
my_partial.my_marker = 'BAR' # this _only_ updates partial, not the original
print my_func.my_marker
print my_partial.my_marker
my_func.my_marker = 'BAZ' # this _only_ updates original, not the partial
print my_func.my_marker
print my_partial.my_marker
You can do something like:
import functools
setattr(functools, 'partial', partial) # from above code
However, this is probably a bad idea as (1) it would need to be imported before any code which relied on it, (2) it could break imported code, (3) it might confuse future people and (4) the alternative, keeping it locally, is easy. Only do this if you want to force third party code to run your version.

Import only functions from a python file

I have many Python files (submission1.py, submission2.py, ... , submissionN.py) in the following format,
#submission1.py
def fun():
print('some fancy function')
fun()
I want to write a tester to test these submissions. (They are actually homeworks that I am grading.). I have a tester for the fun() which is able to test the function itself. However, my problem is, when I import submission.py, it runs the fun() since it calls it at the end of file.
I know that, using if __name__ == "__main__": is the correct way of handling this issue, however, our submissions does not have it since we did not teach it.
So, my question is, is there any way that I can import only fun() from the submission.py files without running the rest of the python file?

For simple scripts with just functions the following will work:
submission1.py:
def fun(x):
print(x)
fun("foo")
def fun2(x):
print(x)
fun2("bar")
print("debug print")
You can remove all bar the FunctionDef nodes then recompile:
import ast
import types
with open("submission1.py") as f:
p = ast.parse(f.read())
for node in p.body[:]:
if not isinstance(node, ast.FunctionDef):
p.body.remove(node)
module = types.ModuleType("mod")
code = compile(p, "mod.py", 'exec')
sys.modules["mod"] = module
exec(code, module.__dict__)
import mod
mod.fun("calling fun")
mod.fun2("calling fun2")
Output:
calling fun
calling fun2
The module body contains two Expr and one Print node which we remove in the loop keeping just the FunctionDef's.
[<_ast.FunctionDef object at 0x7fa33357f610>, <_ast.Expr object at 0x7fa330298a90>,
<_ast.FunctionDef object at 0x7fa330298b90>, <_ast.Expr object at 0x7fa330298cd0>,
<_ast.Print object at 0x7fa330298dd0>]
So after the loop out body only contains the functions:
[<_ast.FunctionDef object at 0x7f49a786a610>, <_ast.FunctionDef object at 0x7f49a4583b90>]
This will also catch where the functions are called with print which if the student was calling the function from an IDE where the functions have return statements is pretty likely, also to keep any imports of there are any you can keep ast.Import's and ast.ImportFrom's:
submission.py:
from math import *
import datetime
def fun(x):
print(x)
fun("foo")
def fun2(x):
return x
def get_date():
print(pi)
return datetime.datetime.now()
fun2("bar")
print("debug print")
print(fun2("hello world"))
print(get_date())
Compile then import:
for node in p.body[:]:
if not isinstance(node, (ast.FunctionDef,ast.Import, ast.ImportFrom)):
p.body.remove(node)
.....
import mod
mod.fun("calling fun")
print(mod.fun2("calling fun2"))
print(mod.get_date())
Output:
calling fun
calling fun2
3.14159265359
2015-05-09 12:29:02.472329
Lastly if you have some variables declared that you need to use you can keep them using ast.Assign:
submission.py:
from math import *
import datetime
AREA = 25
WIDTH = 35
def fun(x):
print(x)
fun("foo")
def fun2(x):
return x
def get_date():
print(pi)
return datetime.datetime.now()
fun2("bar")
print("debug print")
print(fun2("hello world"))
print(get_date()
Add ast.Assign:
for node in p.body[:]:
if not isinstance(node, (ast.FunctionDef,
ast.Import, ast.ImportFrom,ast.Assign)):
p.body.remove(node)
....
Output:
calling fun
calling fun2
3.14159265359
2015-05-09 12:34:18.015799
25
35
So it really all depends on how your modules are structured and what they should contain as to what you remove. If there are literally only functions then the first example will do what you want. If there are other parts that need to be kept it is just a matter of adding them to the isinstance check.
The listing of all the abstract grammar definitions is in the cpython source under Parser/Python.asdl.

You could use sys.settrace() to catch function definitions.
Whenever your fun() is defined, you save it somewhere, and you place a stub into the module you are importing, so that it won't get executed.
Assuming that fun() gets defined only once, this code should do the trick:
import sys
fun = None
def stub(*args, **kwargs):
pass
def wait_for_fun(frame, event, arg):
global fun
if frame.f_code.co_filename == '/path/to/module.py':
if 'fun' in frame.f_globals:
# The function has just been defined. Save it.
fun = frame.f_globals['fun']
# And replace it with our stub.
frame.f_globals['fun'] = stub
# Stop tracing the module execution.
return None
return wait_for_fun
sys.settrace(wait_for_fun)
import my_module
# Now fun() is available and we can test it.
fun(1, 2, 3)
# We can also put it again inside the module.
# This is important if other functions in the module need it.
my_module.fun = fun
This code can be improved in many ways, but it does its job.

maybe if you just want to import the fun () function from submission.py try
from submission import fun
To perform the function of fun, you must include the fun module
submission.fun()
or if you want to make it easier when calling the fun () function, give it a try
from submission import fun as FUN
FUN ()

Is there a way to print the class / function name everytime i print something in python?

In Python, is it possible to print the "class and function name" whenever the "Print" function is called?
For excample,
class SimpleClass:
def function1(self):
print "This is the printed msg"
c = SimpleClass()
c.function1()
"Output wanted"
SimpleClass:function1 - "This is the printed msg"

Use logging module instead of print statements.
The list of available attributes are located here, including: args, asctime, created, exc_info, filename, funcName, levelname, levelno, lineno, module, msecs, message, msg, name, pathname, process, processName, relativeCreated, thread, threadName.

You're using Python 2.x it looks like, so print is not a function and can't be overridden like you want. You could, however, make print a function as it is in Python 3.x using from __future__ import print_function assuming you're using Python 2.6 or later. Then you could pretty trivially override it with your own print that does as you like. (Well, the overriding is trivial; getting the class and method name is a little less so.)
from inspect import currentframe
_print = print
def print(*args, **kwargs):
frame = currentframe().f_back
fname = frame.f_code.co_name
self = frame.f_locals.get("self", None)
qname = (type(self).__name__ + "." + fname) if self else fname
_print(qname, end=": ")
_print(*args, **kwargs)
Digging self out of the function's local variables to see if it's a method is kind of a cheat, since that's only a convention and self could refer to any value, but it should work 99.44% of the time.

How do you find where a print statement is located?

I have a program depending on a large code base that prints a lot of irrelevant and annoying messages. I would like to clean them up a bit, but since their content is dynamically generated, I can't just grep for them.
Is there a way to place a hook on the print statement? (I use python 2.4, but I would be interested in results for any version). Is there another way to find from which "print" statement the output comes?

For CPython2.5 or older:
import sys
import inspect
import collections
_stdout = sys.stdout
Record = collections.namedtuple(
'Record',
'frame filename line_number function_name lines index')
class MyStream(object):
def __init__(self, target):
self.target = target
def write(self, text):
if text.strip():
record = Record(*inspect.getouterframes(inspect.currentframe())[1])
self.target.write(
'{f} {n}: '.format(f = record.filename, n = record.line_number))
self.target.write(text)
sys.stdout = MyStream(sys.stdout)
def foo():
print('Hi')
foo()
yields
/home/unutbu/pybin/test.py 20: Hi
For CPython2.6+ we can import the print function with
from __future__ import print_function
and then redirect it as we wish:
from __future__ import print_function
import sys
import inspect
import collections
Record = collections.namedtuple(
'Record',
'frame filename line_number function_name lines index')
def myprint(text):
if text.strip():
record = Record(*inspect.getouterframes(inspect.currentframe())[1])
sys.stdout.write('{f} {n}: '.format(f = record.filename, n = record.line_number))
sys.stdout.write(text + '\n')
def foo():
print('Hi')
print = myprint
foo()
Note that inspect.currentframe uses sys._getframe which is not part of all implementations of Python. So the solution above may only work for CPython.

Strictly speaking, code base that you depend on, as in libraries, shouldn't contain any print statements. So, you should really just remove all of them.
Other than that, you can monkey-patch stdout: Adding a datetime stamp to Python print

a very gross hack to make this work:
use your favorite text editor, use your search/find feature.
find all the print statements.
and input into each of them a number, or identifier manually. (or automatically if you do this what a script)
a script to do this would be simple, just have it look for for print with regex, and replace it with print ID, and then it will all be the same, but you will get numbers.
cheers.
edit
barring any strange formatting, the following code should do it for you.
note, this is just an example of a way you could do it. not really an answer.
import re
class inc():
def __init__(self):
self.x = 0
def get(self):
self.x += 1
return self.x
def replacer(filename_in, filename_out):
i = inc()
out = open(filename_out, 'w')
with open(filename_in) as f:
for line in f:
out.write("%s\n" % re.sub(r'print', 'print %d,' % i.get(), line))
i used an basic incrementer class in case you wanted to had some kind of more complex ID, instead of just having a counter.

In harsh circumstances (output done in some weird binary libraries) you could also use strace -e write (and more options). If you do not read strace's output, the straced program waits until you do, so you can send it a signal and see where it dies.

Here is a trick that Jeeeyul came up with for Java: Replace the output stream (i.e. sys.out) with something that notices when a line feed has been written.
If this flag is true, throw an exception when the next byte is being written. Catch the exception in the same place, walk up the stack trace until you find code that doesn't belong to your "debug stream writer".
Pseudocode:
class DebugPrintln:
def __init__(self):
self.wasLF = False
def write(self, x):
if self.wasLF:
self.wasLF = False
frames = traceback.extract_stack()
... find calling code and output it ...
if x == '\n':
self.wasLF = true
super.write(x)

How to determine file, function and line number?

In C++, I can print debug output like this:
printf(
"FILE: %s, FUNC: %s, LINE: %d, LOG: %s\n",
__FILE__,
__FUNCTION__,
__LINE__,
logmessage
);
How can I do something similar in Python?

There is a module named inspect which provides these information.
Example usage:
import inspect
def PrintFrame():
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename) # __FILE__ -> Test.py
print(info.function) # __FUNCTION__ -> Main
print(info.lineno) # __LINE__ -> 13
def Main():
PrintFrame() # for this line
Main()
However, please remember that there is an easier way to obtain the name of the currently executing file:
print(__file__)

For example
import inspect
frame = inspect.currentframe()
# __FILE__
fileName = frame.f_code.co_filename
# __LINE__
fileNo = frame.f_lineno
There's more here http://docs.python.org/library/inspect.html

Building on geowar's answer:
class __LINE__(object):
import sys
def __repr__(self):
try:
raise Exception
except:
return str(sys.exc_info()[2].tb_frame.f_back.f_lineno)
__LINE__ = __LINE__()
If you normally want to use __LINE__ in e.g. print (or any other time an implicit str() or repr() is taken), the above will allow you to omit the ()s.
(Obvious extension to add a __call__ left as an exercise to the reader.)

You can refer my answer:
https://stackoverflow.com/a/45973480/1591700
import sys
print sys._getframe().f_lineno
You can also make lambda function

I was also interested in a __LINE__ command in python.
My starting point was https://stackoverflow.com/a/6811020 and I extended it with a metaclass object. With this modification it has the same behavior like in C++.
import inspect
class Meta(type):
def __repr__(self):
# Inspiration: https://stackoverflow.com/a/6811020
callerframerecord = inspect.stack()[1] # 0 represents this line
# 1 represents line at caller
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
# print(info.filename) # __FILE__ -> Test.py
# print(info.function) # __FUNCTION__ -> Main
# print(info.lineno) # __LINE__ -> 13
return str(info.lineno)
class __LINE__(metaclass=Meta):
pass
print(__LINE__) # print for example 18

wow, 7 year old question :)
Anyway, taking Tugrul's answer, and writing it as a debug type method, it can look something like:
def debug(message):
import sys
import inspect
callerframerecord = inspect.stack()[1]
frame = callerframerecord[0]
info = inspect.getframeinfo(frame)
print(info.filename, 'func=%s' % info.function, 'line=%s:' % info.lineno, message)
def somefunc():
debug('inside some func')
debug('this')
debug('is a')
debug('test message')
somefunc()
Output:
/tmp/test2.py func=<module> line=12: this
/tmp/test2.py func=<module> line=13: is a
/tmp/test2.py func=<module> line=14: test message
/tmp/test2.py func=somefunc line=10: inside some func

import inspect
.
.
.
def __LINE__():
try:
raise Exception
except:
return sys.exc_info()[2].tb_frame.f_back.f_lineno
def __FILE__():
return inspect.currentframe().f_code.co_filename
.
.
.
print "file: '%s', line: %d" % (__FILE__(), __LINE__())

Here is a tool to answer this old yet new question!
I recommend using icecream!
Do you ever use print() or log() to debug your code? Of course, you
do. IceCream, or ic for short, makes print debugging a little sweeter.
ic() is like print(), but better:
It prints both expressions/variable names and their values.
It's 40% faster to type.
Data structures are pretty printed.
Output is syntax highlighted.
It optionally includes program context: filename, line number, and parent function.
For example, I created a module icecream_test.py, and put the following code inside it.
from icecream import ic
ic.configureOutput(includeContext=True)
def foo(i):
return i + 333
ic(foo(123))
Prints
ic| icecream_test.py:6 in <module>- foo(123): 456

To get the line number in Python without importing the whole sys module...
First import the _getframe submodule:
from sys import _getframe
Then call the _getframe function and use its' f_lineno property whenever you want to know the line number:
print(_getframe().f_lineno) # prints the line number
From the interpreter:
>>> from sys import _getframe
... _getframe().f_lineno # 2
Word of caution from the official Python Docs:
CPython implementation detail: This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.
In other words: Only use this code for personal testing / debugging reasons.
See the Official Python Documentation on sys._getframe for more information on the sys module, and the _getframe() function / submodule.
Based on Mohammad Shahid's answer (above).