Using Python 3, I have a console application that I am porting to a GUI. The code has a bunch of print statements, something like this:
print(f1(), f2(), f3(), sep=getsep(), end=getend())
I would like to convert these calls into something like:
GuiPrintLine(f1(), f2(), f3(), sep=getsep(), end=getend())
where each line is eventually rendered using some (undefined) GUI framework.
This is easy to do if I can convert the arguments to print into to the string that print would normally produce without the side-effect of actually printing to sysout. In other words, I need a function like this:
s = print_to_string(*args, **kwargs)
How do I format a set of parameters to print(...) into a single string that produces the same output as print() would produce?
I realize I could emulate print by concatenating all the args with sep and ends, but I would prefer to use a built-in solution if there is one.
Using print and redirecting sysout is not attractive since it requires modifying the global state of the app and sysout might be used for other diagnostics at the same time.
It seems like this should be trivial in Python, so maybe I'm just missing something obvious.
Thanks for any help!
Found the answer via string io. With this I don't have to emulate Print's handling of sep/end or even check for existence.
import io
def print_to_string(*args, **kwargs):
output = io.StringIO()
print(*args, file=output, **kwargs)
contents = output.getvalue()
output.close()
return contents
My solution :
def print_to_string(*args, **kwargs):
newstr = ""
for a in args:
newstr+=str(a)+' '
return newstr
Related
I am new to Stackoverflow so I will try to ask the question as detailed as possible. I am facing an issue where I call a function to write a value to a python file, and another function to return this written value. However, the function that supposed to return this written value does not return the most updated value after the write function.
I have done multiple researches and tried multiple methods from other questions posted in Stackoverflow but it still didn't work for me.
Here is what I currently have:
In test.py
import random
import string
from test_value import *
def write():
letters = string.ascii_letters
value_to_be_passed = ''.join(random.choice(letters) for i in range(10))
f = open("test_value.py", "w")
f.write('value="' + value_to_be_passed + '"')
f.close()
def read():
print(value)
write()
read()
In test_value.py
value="MutmHDlVQj"
If you run this file, it will generate a new string and store this new string into test_value.py. But when the function read() runs, it does not print out the newly generated string. It seem as though when I run this program, python will immediately store the value in test_value.py in it's memory and does not return the new value when called.
I have tried fsync, flush, with open but it still doesn't work. It won't be ideal to read the file as a plaintext as I might be storing multiple variables in test_value.py so that my program can call the value of the respective variables.
You can run the code in your ide, upon the first run it will print MutmHDlVQj in your terminal even though the goal is supposed to print the newly generated string.
Does anyone know how this could be solved? Thank you in advance.
Workaround Solution
Thanks to tripleee that gave me the idea of this cheap solution. However, I am sure there are better methods that can be done to solve this issue. May the experts give your opinions on this!
Instead of importing at compile time, I ran the importing when I really needed it.
def read():
from test_value import *
print(value)
The import happens when your script is compiled, before it runs and writes anything to the new file. If you want to import something during runtime, you can do that with importlib; but this seems fundamentally like an XY problem. If your task is to serialize some data, use pickle or simply save the data in a standard format like JSON.
from import method you can't pass a variable so simply for a return value, you must specify in test_value.py a return function.
here's the functioning code:
import random
import string
from test_value import value
def write():
letters = string.ascii_letters
value_to_be_passed = ''.join(random.choice(letters) for I in range(10))
f = open("test_value.py", "w")
f.write('value="' + value_to_be_passed + '"')
f.close()
def read():
returned_value = value()
print(returned_value)
write()
read()
in test_value.py:
def value():
value="SbjvLSYfNs"
return value
written like this I have a functioning code that prints the value returned from another python file like you asked.
hope it helps in the future, but remember that if you want to return a value, you must specify the return function
While loading a pretrained model with pytorch via
model.load_state_dict(torch.load(MODEL_PATH))
the console is flooded with output containing information about the model (which is annoying). Afaik there is no verbosity option, neither in model.load_state_dict nor in torch.load. Please let me know if I overlooked this parameter.
However, this led me to the question if there is any general way to force a function to be non verbose. Maybe something like:
with os.nonverbose():
model.load_state_dict(torch.load(MODEL_PATH))
Any ideas?
As #triplee commented, most libraries would use Python logging, which can be modified extensively. I haven't worked with PyTorch before but from https://github.com/pytorch/vision/issues/330 it looks like it's actaully using print (which is horrible if it is but okay).
In general, you can, however, suppress the stdout output of anything by redirecting stdout. Here's a good link https://wrongsideofmemphis.com/2010/03/01/store-standard-output-on-a-variable-in-python/
In your question you ask if this can be done with a context manager. I don't see why not and it seems appropriate given that you want to reset stdout after the function call. Something like this:
from io import StringIO # Python3
import sys
class SilencedStdOut:
def __enter__(self):
self.old_stdout = sys.stdout
self.result = StringIO()
sys.stdout = self.result
def __exit__(self, *args, **kwargs):
sys.stdout = self.old_stdout
result_string = self.result.getvalue() # use if you want or discard.
If you only ever want to supress a single function and never a block of code, however, then a decorator should work fine too.
I am looking for a way to quickly print a variable name and value while rapidly developing/debugging a small Python script on a Unix command line/ssh session.
It seems like a very common requirement and it seems wasteful (on keystrokes and time/energy) to duplicate the variable_names on every line which prints or logs its value. i.e. rather than
print 'my_variable_name:', my_variable_name
I want to be able to do the following for str, int, list, dict:
log(my_variable_name)
log(i)
log(my_string)
log(my_list)
and get the following output:
my_variable_name:some string
i:10
my_string:a string of words
my_list:[1, 2, 3]
Ideally the output would also log the function name.
I have seen some solutions attempting to use locals, globals, frames etc., But I have not yet seen something that works for ints, strings, lists, and works inside functions too.
Thanks!
Sorry to Necro this ancient thread, but this was surprisingly difficult to find a good answer for.
Using the '=' sign after the variable achieves this. For instance:
import pathlib as pl
import logging
logging.basicConfig(level=logging.DEBUG)
data_root = pl.Path("D:\Code_Data_Dev\Data\CSV_Workspace_Data")
logging.debug(f'{data_root=}')
This outputs
DEBUG:root:data_root=WindowsPath('D:/Code_Data_Dev/Data/CSV_Workspace_Data')
If the tool you need is only for developing and debugging, there's a useful package called q.
It has been submitted to pypi, it can be installed with pip install q or easy_install q.
import q; q(foo)
# use #q to trace a function's arguments and return value
#q
def bar():
...
# to start an interactive console at any point in your code:
q.d()
The results are output to the file /tmp/q (or any customized paths) by default, so they won't be mixed with stdout and normal logs. You can check the output with tail -f /tmp/q. The output is highlighted with different colors.
The author introduced his library in a lightning talk of PyconUS 2013. The video is here, begins at 25:15.
Here is another evil hack:
import inspect
def log(a):
call_line = inspect.stack()[1][4][0].strip()
assert call_line.strip().startswith("log(")
call_line = call_line[len("log("):][:-1]
print "Log: %s = %s" % (call_line, a)
b=3
log(b)
This obviously would need some range checks, better parsing, etc.
Also works only if the calls is made always in the same way and has probably more - unknown to me - assumptions...
I don't know any way to simply get the name of a variable of a string.
you can however get the list of argument of the current fonction log.
import inspect
def log(a):
frame = inspect.currentframe()
args, _, _, values = inspect.getargvalues(frame)
print "%s:%s" % (args[0], values[args[0]])
I'm trying to convert to get a command executed which is passed to the print statement. Eg:
print "exec(raw_input())"
Can I get to run the exec() in some way?
do this
command = raw_input("Command: ")
exec(command)
why are you trying to print it? be more clear if this isnt what you are looking for
I Guess this is what you are looking for ?
>>> exec("x = raw_input()")
23
>>> x
'23'
>>>
Are you asking for something simple like
aString = "raw_input()"
print "exec(" + aString + ")"
exec(aString)
from __future__ import print_function
def foo(x):
exec x
print = foo
print("exec(raw_input())")
Running
% test.py
kaboom
results in:
NameError: name 'kaboom' is not defined
From the tags you applied, it seems like you're looking for a magic string that will allow you to run any arbitrary code when Python passes that string to the print statement.
There are no such known strings. You might try very long strings, just over the obvious boundaries, looking for the traditional stack- and heap-overflow cases, but if you find one in Python X.Y.Z, chances are it was already fixed in X.Y.Z+1.
If the Python script is actually doing an exec or eval on your string, of course that's easy to exploit; if it's just doing a print, the contents never even get compiled, much less run; they're just a plain old string. The fact that the string happens to have dangerous-looking things like exec in it is irrelevant; it's not more exploitable than getting Python to print the string "rm -rf /".
Of course if you can arrange for the output of one Python interpreter to feed in as the input to an interactive session in another interpreter, then whatever you get the first interpreter to print will get executed in the second one… but in that case, you don't need the exec in the middle of the string anyway.
The print statement writes to sys.stdout by default, and sys.stdout is like a file. You can make a class that looks like a file, with a write method, which would be called by the print statement. I put together a script which demonstrates this, print_exec.py. Also note, this doesn't work if the code in the print statement contains print, itself. i.e., print "print 'foo'" won't work. So, in the example, I had to print to sys.stderr to actually see something happening. Hope this helps.
print_exec.py
import sys
class file_like(object):
def write(self, x):
exec(x)
sys.stdout = file_like()
y = "exec(raw_input())"
print "exec(y)"
Example running the print_exec.py:
>python print_exec.py
print "import sys; print >> sys.stderr, 'hi'"
hi
>
I have a bunch of print calls that I need to write to a file instead of stdout. (I don't need stdout at all.)
I am considering three approaches. Are there any advantages (including performance) to any one of them?
Full redirect, which I saw here:
import sys
saveout = sys.stdout
fsock = open('out.log', 'w')
sys.stdout = fsock
print(x)
# and many more print calls
# later if I ever need it:
# sys.stdout = saveout
# fsock.close()
Redirect in each print statement:
fsock = open('out.log', 'w')
print(x, file = fsock)
# and many more print calls
Write function:
fsock = open('out.log', 'w')
fsock.write(str(x))
# and many more write calls
I would not expect any durable performance differences among these approaches.
The advantage of the first approach is that any reasonably well-behaved code which you rely upon (modules you import) will automatically pick up your desired redirection.
The second approach has no advantage. It's only suitable for debugging or throwaway code ... and not even a good idea for that. You want your output decisions to be consolidated in a few well-defined places, not scattered across your code in every call to print(). In Python3 print() is a function rather than a statement. This allows you to re-define it, if you like. So you can def print(*args) if you want. You can also call __builtins__.print() if you need access to it, within the definition of your own custom print(), for example.
The third approach ... and by extension the principle that all of your output should be generated in specific functions and class methods that you define for that purpose ... is probably best.
You should keep your output and formatting separated from your core functionality as much as possible. By keeping them separate you allow your core to be re-used. (For example you might start with something that's intended to run from a text/shell console, and later need to provide a Web UI, a full-screen (curses) front end or a GUI for it. You may also build entirely different functionality around it ... in situations where the resulting data needs to be returned in its native form (as objects) rather than pulled in as text (output) and re-parsed into new objects.
For example I've had more than one occasional where something I wrote to perform some complex queries and data gathering from various sources and print a report ... say of the discrepancies ... later need to be adapted into a form which could spit out the data in some form (such as YAML/JSON) that could be fed into some other system (say, for reconciling one data source against another.
If, from the outset, you keep the main operations separate from the output and formatting then this sort of adaptation is relatively easy. Otherwise it entails quite a bit of refactoring (sometimes tantamount to a complete re-write).
From the filenames you're using in your question, it sounds like you're wanting to create a log file. Have you consider the Python logging module instead?
I think that semantics is imporant:
I would suggest first approach for situation when you printing the same stuff you would print to console. Semantics will be the same. For more complex situation I would use standard logging module.
The second and third approach are a bit different in case you are printing text lines. Second approach - print adds the newline and write does not.
I would use the third approach in writing mainly binary or non-textual format and I would use redirect in print statement in the most other cases.