I just want to make the print output redirect to a file. my code as below:
import sys
# define the log file that receives your log info
log_file = open("message.log", "w")
# redirect print output to log file
sys.stdout = log_file
print ("Now all print info will be written to message.log")
# any command line that you will execute
...
log_file.close()
print ("Now this will be presented on screen")
After execute the script, it comes an error:
[~/Liaohaifeng]$ python3 log.py
Traceback (most recent call last):
File "log.py", line 14, in <module>
print ("Now this will be presented on screen")
ValueError: I/O operation on closed file.
why does this happen? if I update my script as below:
import sys
# make a copy of original stdout route
stdout_backup = sys.stdout
# define the log file that receives your log info
log_file = open("message.log", "w")
# redirect print output to log file
sys.stdout = log_file
print ("Now all print info will be written to message.log"
# any command line that you will execute
...
log_file.close()
# restore the output to initial pattern
sys.stdout = stdout_backup
print ("Now this will be presented on screen")
It will be OK. So, could you please kindly tell me the theory in this issue?
As mentioned in comments, print does not print to a closed filehandle and you have closed sys.stdout, potentially breaking any prints evoked after it closed. Which may happen even without your knowledge, eg somewhere in the imported code. That's why you shouldn't fiddle with sys.* variables (or any variables you didn't create, really) unless you absolutely need to. There is a proper way to redirect print output to a file and it goes like this:
log_file = open('message.log', 'w')
print('Stuff to print in the log', file=log_file)
log_file.close()
Or even safer like this:
with open('message.log', 'w') as log_file:
# Do stuff
print('Stuff to print in the log', file=log_file)
The handle will automatically flush and close when the with block finishes.
Related
I am having trouble figuring out what the issue with my code snippet for writing print messages in my console to multiple log-files is doing.
The code snippet I have posted below is supposed to create a new directory test, then write 11 log-files, 1 global log file, and 10 loop log files to this directory. However, the 1st 2 print messages to my global log file is missing when I run this and I cannot figure out what the issue is?
import sys
import os
# Create a test folder to store these global and loop log files.
path = os.getcwd()
test_dir_name = 'test'
test_dir_path = os.path.join(path, test_dir_name)
os.mkdir(test_dir_path)
# Keep a reference to the original stdout.
orig_stdout = sys.stdout
# Define global logfile path.
global_log_name = "global-log.txt"
global_log_path = os.path.join(test_dir_path, global_log_name)
# Problematic code-snippet
sys.stdout = open(global_log_path, 'w')
print("This is a global log file.") # Why is my code omitting this line?
print("The loop is now creating 10 individual log files.") # And this one?
sys.stdout.close()
for i in range(10):
sys.stdout = open(global_log_path, 'w')
print("Creating loop log file {}...".format(i))
sys.stdout.close()
loop_log_name = "local-log-{}.txt".format(i)
loop_log_path = os.path.join(test_dir_path, loop_log_name)
sys.stdout = open(loop_log_path, 'w')
print("This is loop log file {}".format(i))
print("Closing this loop log file...")
sys.stdout.close()
sys.stdout = open(global_log_path, 'w')
print("Loops have concluded.") # But then it includes this line.
print("Now closing global log file.") # And this line in the global log file.
sys.stdout.close()
sys.stdout = orig_stdout
print("Back to original console.")
Some assistance would be greatly appreciated.
The principal issue with this code snippet is the inappropriate use of open(global_log_path, 'w') to append further print messages to global-log.txt. After you have initially executed:
sys.stdout = open(global_log_path, 'w')
print("This is a global log file.") # Why is my code omitting this line?
print("The loop is now creating 10 individual log files.") # And this one?
Subsequent redirections of stdout to global-log.txt instead require passing the argument a, standing for append to open() like so:
sys.stdout = open(global_log_path, 'a')
print("Creating loop log file {}...".format(i))
This prevents previously redirected text from being overwritten, which was happening with your code snippet.
I have a issue with my raspberry pi that starts up a python script.How do I save the printed output to a file when it is running on boot? I found script below on the internet but it doesn't seem to write the printed text,it creates the file but the content is empty.
sudo python /home/pi/python.py > /home/pi/output.log
It does write its output to the file but you cannot see it until the python file has finished executing due to buffer never flushed.
If you change the output to a file within your python script you can periodicity call flush in your code to push the output through to the file as and when you wish, something like this.
import sys
import time
outputFile = "output.txt";
with open(outputFile, "w+") as sys.stdout:
while True:
print("some output")
sys.stdout.flush() # force buffer content out to file
time.sleep(5) # wait 5 seconds
if you want to set the output back to the terminal, you may want to save a reference to the original stdout like this
import time
outputFile = "output.txt";
original_stdout = sys.stdout
with open(outputFile, "w+") as sys.stdout:
print("some output in file")
sys.stdout.flush()
time.sleep(5)
sys.stdout = original_stdout
print("back in terminal")
I am trying to redirect output to the file
Below code works fine
import sys
print('###################################Genarating Global Variables##########################')
storage_file = open("Storage-output.txt", 'w')
sys.stdout = storage_file
print ('whatever')
storage_file.close()
Below code is not working:
import sys
print('###################################Genarating Global Variables##########################')
storage_file = open("Storage-output.txt", 'w')
sys.stdout = storage_file
print ('what')
storage_file.close()
print('Completed Global variables creation')
Basically what I need is, some of print output should go to file and some of print output should display in terminal. For that I am planning to open file and closing file whenever required in script.
The error message is self-explained:
Traceback (most recent call last):
File "something.py", line 7, in <module>
print('Completed Global variables creation')
ValueError: I/O operation on closed file
You assign your file handler to stdout: sys.stdout = storage_file, but closed storage_file before the last print, which caused this problem. If you want last print to work like the first line again, you need to re-assign sys.stdout to its original value, by the way, when dealing with file, always consider to use with statement:
import sys
print('###################################Genarating Global Variables##########################')
stdout = sys.stdout
with open("Storage-output.txt", 'w') as storage_file:
sys.stdout = storage_file
print ('what')
sys.stdout = stdout
print('Completed Global variables creation')
you're closing the file but you are not changing sys.stdout back to it's original value; so this line print('Completed Global variables creation') is trying to print in storage_file, which is a closed file descriptor.
something like this should work. you can also save in a variable the original value o sys.stdout and then change it back instead of accessing sys.__stdout__
import sys
print('###################################Genarating Global Variables##########################')
storage_file = open("Storage-output.txt", 'w')
sys.stdout = storage_file
print ('what')
storage_file.close()
sys.stdout = sys.__stdout__ # return to its original value as stated in https://docs.python.org/2/library/sys.html#sys.__stdout__
print('Completed Global variables creation')
In Python 2.7 I have the following code inside certain loop
file = open("log.txt", 'a+')
last_position = file.tell()
subprocess.Popen(["os_command_producing_error"], stderr = file)
file.seek(last_position)
error = file.read()
print(error) # example of some action with the error
The intention is that the error that was just given by stderr gets, say printed, while file is keeping the whole record.
I am a beginner in Python and I am not clear what happens in the stderr = file.
My problem is that error keeps being empty, even though errors keep getting logged in the file.
Could someone explain why?
I have tried adding closing and opening the file again, or file.flush() right after the subprocess line. But still the same effect.
Edit: The code in the answer below makes sense to me and it seems to work for for the author of that post. For me (in Windows) it is not working. It gives an empty err and an empty file log.txt. If I run it line by line (e.g. debugging) it does work. How to understand and solve this problem?
Edit: I changed the Popen with call and now it works. I guess call waits for the subprocess to finish in order to continue with the script.
error is empty because you are reading too soon before the process has a chance to write anything to the file. Popen() starts a new process; it does not wait for it to finish.
call() is equivalent to Popen().wait() that does wait for the child process to exit that is why you should see non-empty error in this case (if the subprocess does write anything to stderr).
#!/usr/bin/env python
import subprocess
with open("log.txt", 'a+') as file:
subprocess.check_call(["os_command_producing_error"], stderr=file)
error = file.read()
print(error)
You should be careful with mixing buffered (.read()) and unbuffered I/O (subprocess).
You don't need the external file here, to read the error:
#!/usr/bin/env python
import subprocess
error = subprocess.check_output(["os_command_producing_error"],
stderr=subprocess.STDOUT)
print(error)
It merges stderr and stdout and returns the output.
If you don't want to capture stdout then to get only stderr, you could use Popen.communicate():
#!/usr/bin/env python
import subprocess
p = subprocess.Popen(["os_command_producing_error"], stderr=subprocess.PIPE)
error = p.communicate()[1]
print(error)
You could both capture stderr and append it to a file:
#!/usr/bin/env python
import subprocess
error = bytearray()
p = subprocess.Popen(["os_command_producing_error"],
stderr=subprocess.PIPE, bufsize=1)
with p.stderr as pipe, open('log.txt', 'ab') as file:
for line in iter(pipe.readline, b''):
error += line
file.write(line)
p.wait()
print(error)
See Python: read streaming input from subprocess.communicate().
Try these following codes:
file = open("log.txt", 'a+')
sys.stderr = file
last_position = file.tell()
try:
subprocess.call(["os_command_producing_error"])
except:
file.close()
err_file = open("log.txt", 'r')
err_file.seek(last_position)
err = err_file.read()
print err
err_file.close()
sys.stderr map the standard error message like sys.stdout(map standard output) and sys.stdin(map standard input).
And this will map the standard error to file. So all of the standard error will be write to the file log.txt.
import itertools
variations = itertools.product('abc', repeat=3)
for variations in variations:
variation_string = ""
for letter in variations:
variation_string += letter
print (variation_string)
How can I redirect output into a txt file (on windows platform)?
From the console you would write:
python script.py > out.txt
If you want to do it in Python then you would write:
with open('out.txt', 'w') as f:
f.write(something)
Obviously this is just a trivial example. You'd clearly do more inside the with block.
You may also redirect stdout to your file directly in your script as print writes by default to sys.stdout file handler. Python provides a simple way to to it:
import sys # Need to have acces to sys.stdout
fd = open('foo.txt','w') # open the result file in write mode
old_stdout = sys.stdout # store the default system handler to be able to restore it
sys.stdout = fd # Now your file is used by print as destination
print 'bar' # 'bar' is added to your file
sys.stdout=old_stdout # here we restore the default behavior
print 'foorbar' # this is printed on the console
fd.close() # to not forget to close your file
In window command prompt, this command will store output of program.py into file output.txt
python program.py > output.txt
If it were me, I would use David Heffernan's method above to write your variable to the text file (because other methods require the user to use a command prompt).
import itertools
file = open('out.txt', 'w')
variations = itertools.product('abc', repeat=3)
for variations in variations:
variation_string = ""
for letter in variations:
variation_string += letter
file.write(variation_string)
file.close()
you may use >>
log = open("test.log","w")
print >> log, variation_string
log.close()
Extension to David's answer
If you are using PyCharm,
Go to Run --> Edit Configurations --> Logs --> Check mark Save console
output to file --> Enter complete path --> Apply