I have this code which runs fine in Python 2.5 but not in 2.7:
import sys
import traceback
try:
from io import StringIO
except:
from StringIO import StringIO
def CaptureExec(stmt):
oldio = (sys.stdin, sys.stdout, sys.stderr)
sio = StringIO()
sys.stdout = sys.stderr = sio
try:
exec(stmt, globals(), globals())
out = sio.getvalue()
except Exception, e:
out = str(e) + "\n" + traceback.format_exc()
sys.stdin, sys.stdout, sys.stderr = oldio
return out
print "%s" % CaptureExec("""
import random
print "hello world"
""")
And I get:
string argument expected, got 'str'
Traceback (most recent call last):
File "D:\3.py", line 13, in CaptureExec
exec(stmt, globals(), globals())
File "", line 3, in
TypeError: string argument expected, got 'str'
io.StringIO is confusing in Python 2.7 because it's backported from the 3.x bytes/string world. This code gets the same error as yours:
from io import StringIO
sio = StringIO()
sio.write("Hello\n")
causes:
Traceback (most recent call last):
File "so2.py", line 3, in <module>
sio.write("Hello\n")
TypeError: string argument expected, got 'str'
If you are only using Python 2.x, then skip the io module altogether, and stick with StringIO. If you really want to use io, change your import to:
from io import BytesIO as StringIO
It's bad news
io.StringIO wants to work with unicode. You might think you can fix it by putting a u in front of the string you want to print like this
print "%s" % CaptureExec("""
import random
print u"hello world"
""")
however print is really broken for this as it causes 2 writes to the StringIO. The first one is u"hello world" which is fine, but then it follows with "\n"
so instead you need to write something like this
print "%s" % CaptureExec("""
import random
sys.stdout.write(u"hello world\n")
""")
Related
I'm trying to save stdout to a variable in python. My current solution works fine in a python shell, but when ran as a script it errors. Here is my current test code:
import yaml
import sys
from slackclient import SlackClient
from io import StringIO # Python3
slack_token = "SOME TOKEN HERE"
sc = SlackClient(slack_token)
old_stdout = sys.stdout
result = StringIO()
sys.stdout = result
row_message = '[{"title": "Test", "pretext": "this is an example message!", "text": "Super Awesome Test!"}]'
sc.api_call(
"chat.postMessage",
channel="#channel_name",
username="My User",
icon_url="http://some.url.here/",
attachments=row_message
)
sys.stdout = old_stdout
result_string = result.getvalue()
example = yaml.load(result_string)['ts']
print(example)
My output in the python shell is what i expect:
>>> print(example)
1489460593.207577
>>>
But when I save the code and run it as a script:
# ./test
Traceback (most recent call last):
File "./test", line 22, in <module>
example = yaml.load(result_string)['ts']
TypeError: 'NoneType' object is not subscriptable
Could anyone enlighten me as to why this works in the python shell but not as a script? What can I try to make this work? Thanks.
Thanks #zstewart, you were correct. I was just overcomplicating it. Here is what I am doing instead.
response = sc.api_call("chat.postMessage", channel="#channel", username="username", icon_url="http://some.url.here", attachments=row_message)
thread_val_list = [ val for key,val in response.items() if key=='ts' ]
thread_val = "".join(thread_val_list)
Roughly speaking, I want to port this to pure Python:
#!/bin/bash
{
python test.py
} &> /tmp/test.log
This didn't work for some unknown reasons:
import os.path, sys
import tempfile
with open(os.path.join(tempfile.gettempdir(), "test.log"), "a") as fp:
sys.stdout = sys.stderr = fp
raise Exception("I'm dying")
The resulting test.log was empty (and I didn't see anything on my console,) when I tested it with Python 2.6.6, Python 2.7.8 and Python 3.4.2 on CentOS x86_64.
But Ideally I'd like a solution for Python 2.6.
(For now, it's tolerable to clutter the log file with intermixed output from stdout and stderr or multithreading, as long as any data won't simply disappear into a blackhole.)
Show me a concise and portable solution which is confirmed to work with an exception stack trace on sys.stderr. (Preferably something other than os.dup2)
Remember that file objects are closed after with blocks :)
Use simply this:
sys.stdout = sys.stderr = open("test.log","w")
raise Exception("Dead")
Content of test.log after exit:
Traceback (most recent call last):
File "test.py", line 5, in <module>
raise Exception("Dead")
Exception: Dead
You can use a method like this one:
import traceback
import sys
from contextlib import contextmanager
#contextmanager
def output_to_file(filepath, write_mode='w'):
stdout_orig = None
stderr_orig = None
stdout_orig = sys.stdout
stderr_orig = sys.stderr
f = open(filepath, write_mode)
sys.stdout = f
sys.stderr = f
try:
yield
except:
info = sys.exc_info()
f.write('\n'.join(traceback.format_exception(*info)))
f.close()
sys.stdout = stdout_orig
sys.stderr = stderr_orig
And the the usage is:
with output_to_file('test.log'):
print('hello')
raise Exception('I am dying')
And the cat test.log produces:
hello
Traceback (most recent call last):
File "<ipython-input-3-a3b702c7b741>", line 20, in outputi_to_file
yield
File "<ipython-input-4-f879d82580b2>", line 3, in <module>
raise Exception('I am dying')
Exception: I am dying
This works for me:
#!/usr/bin/env python
from __future__ import print_function
import os, os.path, sys, tempfile
old_out = os.dup(sys.stdout.fileno())
old_err = os.dup(sys.stderr.fileno())
with open(os.path.join(tempfile.gettempdir(), "test.log"), "a") as fp:
fd = fp.fileno()
os.dup2(fd, sys.stdout.fileno())
os.dup2(fd, sys.stderr.fileno())
print("Testing")
print('testing errs', file=sys.stderr)
raise Exception("I'm dying")
The future is just for cleaner handling of Python2 or Python3 with the same example. (I've also changed the raise statement to instantiate an exception, strings as exceptions have been deprecated for a long time and they're not properly supported under Python3).
The old_* values are just if we wanted to restore our original stdout and/or stderr after using our redirected file.
I'm using python's ftplib to write a small FTP client, but some of the functions in the package don't return string output, but print to stdout. I want to redirect stdout to an object which I'll be able to read the output from.
I know stdout can be redirected into any regular file with:
stdout = open("file", "a")
But I prefer a method that doesn't uses the local drive.
I'm looking for something like the BufferedReader in Java that can be used to wrap a buffer into a stream.
from cStringIO import StringIO # Python3 use: from io import StringIO
import sys
old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()
# blah blah lots of code ...
sys.stdout = old_stdout
# examine mystdout.getvalue()
There is a contextlib.redirect_stdout() function in Python 3.4+:
import io
from contextlib import redirect_stdout
with io.StringIO() as buf, redirect_stdout(buf):
print('redirected')
output = buf.getvalue()
Here's a code example that shows how to implement it on older Python versions.
Just to add to Ned's answer above: you can use this to redirect output to any object that implements a write(str) method.
This can be used to good effect to "catch" stdout output in a GUI application.
Here's a silly example in PyQt:
import sys
from PyQt4 import QtGui
class OutputWindow(QtGui.QPlainTextEdit):
def write(self, txt):
self.appendPlainText(str(txt))
app = QtGui.QApplication(sys.argv)
out = OutputWindow()
sys.stdout=out
out.show()
print "hello world !"
A context manager for python3:
import sys
from io import StringIO
class RedirectedStdout:
def __init__(self):
self._stdout = None
self._string_io = None
def __enter__(self):
self._stdout = sys.stdout
sys.stdout = self._string_io = StringIO()
return self
def __exit__(self, type, value, traceback):
sys.stdout = self._stdout
def __str__(self):
return self._string_io.getvalue()
use like this:
>>> with RedirectedStdout() as out:
>>> print('asdf')
>>> s = str(out)
>>> print('bsdf')
>>> print(s, out)
'asdf\n' 'asdf\nbsdf\n'
Starting with Python 2.6 you can use anything implementing the TextIOBase API from the io module as a replacement.
This solution also enables you to use sys.stdout.buffer.write() in Python 3 to write (already) encoded byte strings to stdout (see stdout in Python 3).
Using StringIO wouldn't work then, because neither sys.stdout.encoding nor sys.stdout.buffer would be available.
A solution using TextIOWrapper:
import sys
from io import TextIOWrapper, BytesIO
# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)
# do something that writes to stdout or stdout.buffer
# get output
sys.stdout.seek(0) # jump to the start
out = sys.stdout.read() # read output
# restore stdout
sys.stdout.close()
sys.stdout = old_stdout
This solution works for Python 2 >= 2.6 and Python 3.
Please note that our new sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings.
This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes, which again often makes use of sys.stdout.buffer.
You can build a slight variation that accepts unicode and byte strings for write():
class StdoutBuffer(TextIOWrapper):
def write(self, string):
try:
return super(StdoutBuffer, self).write(string)
except TypeError:
# redirect encoded byte strings directly to buffer
return super(StdoutBuffer, self).buffer.write(string)
You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.
This method restores sys.stdout even if there's an exception. It also gets any output before the exception.
import io
import sys
real_stdout = sys.stdout
fake_stdout = io.BytesIO() # or perhaps io.StringIO()
try:
sys.stdout = fake_stdout
# do what you have to do to create some output
finally:
sys.stdout = real_stdout
output_string = fake_stdout.getvalue()
fake_stdout.close()
# do what you want with the output_string
Tested in Python 2.7.10 using io.BytesIO()
Tested in Python 3.6.4 using io.StringIO()
Bob, added for a case if you feel anything from the modified / extended code experimentation might get interesting in any sense, otherwise feel free to delete it
Ad informandum ... a few remarks from extended experimentation during finding some viable mechanics to "grab" outputs, directed by numexpr.print_versions() directly to the <stdout> ( upon a need to clean GUI and collecting details into debugging-report )
# THIS WORKS AS HELL: as Bob Stein proposed years ago:
# py2 SURPRISEDaBIT:
#
import io
import sys
#
real_stdout = sys.stdout # PUSH <stdout> ( store to REAL_ )
fake_stdout = io.BytesIO() # .DEF FAKE_
try: # FUSED .TRY:
sys.stdout.flush() # .flush() before
sys.stdout = fake_stdout # .SET <stdout> to use FAKE_
# ----------------------------------------- # + do what you gotta do to create some output
print 123456789 # +
import numexpr # +
QuantFX.numexpr.__version__ # + [3] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
QuantFX.numexpr.print_versions() # + [4] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
_ = os.system( 'echo os.system() redir-ed' )# + [1] via real_stdout + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
_ = os.write( sys.stderr.fileno(), # + [2] via stderr + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
b'os.write() redir-ed' )# *OTHERWISE, if via fake_stdout, EXC <_io.BytesIO object at 0x02C0BB10> Traceback (most recent call last):
# ----------------------------------------- # ? io.UnsupportedOperation: fileno
#''' ? YET: <_io.BytesIO object at 0x02C0BB10> has a .fileno() method listed
#>>> 'fileno' in dir( sys.stdout ) -> True ? HAS IT ADVERTISED,
#>>> pass; sys.stdout.fileno -> <built-in method fileno of _io.BytesIO object at 0x02C0BB10>
#>>> pass; sys.stdout.fileno()-> Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# io.UnsupportedOperation: fileno
# ? BUT REFUSES TO USE IT
#'''
finally: # == FINALLY:
sys.stdout.flush() # .flush() before ret'd back REAL_
sys.stdout = real_stdout # .SET <stdout> to use POP'd REAL_
sys.stdout.flush() # .flush() after ret'd back REAL_
out_string = fake_stdout.getvalue() # .GET string from FAKE_
fake_stdout.close() # <FD>.close()
# +++++++++++++++++++++++++++++++++++++ # do what you want with the out_string
#
print "\n{0:}\n{1:}{0:}".format( 60 * "/\\",# "LATE" deferred print the out_string at the very end reached -> real_stdout
out_string #
)
'''
PASS'd:::::
...
os.system() redir-ed
os.write() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.5
NumPy version: 1.10.4
Python version: 2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU? True
VML available? True
VML/MKL version: Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
>>>
EXC'd :::::
...
os.system() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version: 2.5
NumPy version: 1.10.4
Python version: 2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU? True
VML available? True
VML/MKL version: Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
Traceback (most recent call last):
File "<stdin>", line 9, in <module>
io.UnsupportedOperation: fileno
'''
In Python3.6, the StringIO and cStringIO modules are gone, you should use io.StringIO instead.So you should do this like the first answer:
import sys
from io import StringIO
old_stdout = sys.stdout
old_stderr = sys.stderr
my_stdout = sys.stdout = StringIO()
my_stderr = sys.stderr = StringIO()
# blah blah lots of code ...
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
// if you want to see the value of redirect output, be sure the std output is turn back
print(my_stdout.getvalue())
print(my_stderr.getvalue())
my_stdout.close()
my_stderr.close()
Use pipe() and write to the appropriate file descriptor.
https://docs.python.org/library/os.html#file-descriptor-operations
Here's another take on this. contextlib.redirect_stdout with io.StringIO() as documented is great, but it's still a bit verbose for every day use. Here's how to make it a one-liner by subclassing contextlib.redirect_stdout:
import sys
import io
from contextlib import redirect_stdout
class capture(redirect_stdout):
def __init__(self):
self.f = io.StringIO()
self._new_target = self.f
self._old_targets = [] # verbatim from parent class
def __enter__(self):
self._old_targets.append(getattr(sys, self._stream)) # verbatim from parent class
setattr(sys, self._stream, self._new_target) # verbatim from parent class
return self # instead of self._new_target in the parent class
def __repr__(self):
return self.f.getvalue()
Since __enter__ returns self, you have the context manager object available after the with block exits. Moreover, thanks to the __repr__ method, the string representation of the context manager object is, in fact, stdout. So now you have,
with capture() as message:
print('Hello World!')
print(str(message)=='Hello World!\n') # returns True
I'm new in python programming. When i try running a simple python script i get error like this in my terminal
root#bt:/tmp# python code.py
Traceback (most recent call last):
File "code.py", line 42, in <module>
print host+" -> Offline!"
NameError: name 'host' is not defined
I have been search in Google but im difficult to fix my problem because im new in this programming language. Can you help me?
This is my script like this :
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
from netaddr import IPNetwork
import urllib2
import urllib
import re
import getpass
import sys
import telnetlib
import time
import os
import socket
import sys
socket.setdefaulttimeout(4)
register_openers()
try:
os.remove("rom-0")
except:
pass
try:
host=str(sys.argv[1])
urllib.urlretrieve ("http://"+host+"/rom-0", "rom-0")
datagen, headers = multipart_encode({"uploadedfile": open("rom-0")})
request = urllib2.Request("http://localhost/decoded.php", datagen, headers)
str1 = urllib2.urlopen(request).read()
m = re.search('rows=10>(.*)', str1)
if m:
found = m.group(1)
tn = telnetlib.Telnet(host, 23, 3)
tn.read_until("Password: ")
tn.write(found + "\n")
tn.write("set lan dhcpdns 8.8.8.8\n")
tn.write("sys password admin\n")
print host+" -> Success"
tn.write("exit\n")
except:
print host+" -> Offline!"
How i can fix error like this.?
Thanks
If i put : host=str(sys.argv[1]) before try.except show error like this :
Traceback (most recent call last):
File "code.py", line 17, in
host=str(sys.argv[1])
IndexError: list index out of range
And this is my input :
from netaddr import IPNetwork
import os
for ip in IPNetwork ('41.108.48.1/24'):
os.system("python code.py "+str(ip))
Your except clause will catch any error in any line of code in the try block. If you don't specify enough arguments on the command line, the line host = str(sys.argv[1]) will fail, leaving host unassigned, which then causes the error you are seeing when you try to print it.
You should take most of the code out of your try block, really, and/or create multiple try blocks that catch errors in much smaller chunks of code. Furthermore, you should specify the actual exception type you want to handle with each except instead of trying to handle all of them. Bare except: catches things you probably don't want caught, such as KeyboardInterrupt and SystemExit. If you must catch most exceptions, use except Exception: instead of just except:.
it seem that your script expects an input parameter
host=str(sys.argv[1])
in case that parameter is not supplied, as shown in your post, an exception raised and been caught in the except clause before the host parameter was defined
try to declare host before the try/except block
you are defining host in the first line of try/except
i believe the error is in that first line.
to debug this take remove the try/except to see what the actual error is.
I have a problem with printing my output from muscle aligning in python. My code is:
from Bio.Align.Applications import MuscleCommandline
from StringIO import StringIO
from Bio import AlignIO
def align_v1 (Fasta):
muscle_cline = MuscleCommandline(input="hiv_protease_sequences_w_wt.fasta")
stdout, stderr = muscle_cline()
MultipleSeqAlignment = AlignIO.read(StringIO(stdout), "fasta")
print MultipleSeqAlignment
Any help?
It would be nice to know what error you received, but the following should solve your problem:
from Bio.Align.Applications import MuscleCommandline
from StringIO import StringIO
from Bio import AlignIO
muscle_exe = r"C:\muscle3.8.31_i86win32.exe" #specify the location of your muscle exe file
input_sequences = "hiv_protease_sequences_w_wt.fasta"
output_alignment = "output_alignment.fasta"
def align_v1 (Fasta):
muscle_cline = MuscleCommandline(muscle_exe, input=Fasta, out=output_alignment)
stdout, stderr = muscle_cline()
MultipleSeqAlignment = AlignIO.read(output_alignment, "fasta")
print MultipleSeqAlignment
align_v1(input_sequences)
In my case I received a ValueError:
>>> AlignIO.read(StringIO(stdout), "fasta")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\WinPython-64bit-3.3.2.3\python-3.3.2.amd64\lib\site-packages\Bio\AlignIO\__init__.py", line 427, in read
raise ValueError("No records found in handle")
ValueError: No records found in handle
This could be avoided by saving the output and reopening with AlignIO.read.
I also received a FileNotFoundError that could be avoided by specifying the location of the muscle exe file. eg:
muscle_exe = r"C:\muscle3.8.31_i86win32.exe"
The instructions for this are shown in help(MuscleCommandline), but this is not currently in the Biopython tutorial page.
Finally, I am assuming you want to run the command using different input sequences, so I modifed the function to the format “function_name(input_file).”
I used python 3.3. Hopefully the code above is for python 2.x as in your original post. For python 3.x, change "from StringIO import StringIO" to "from io import StringIO" and of course “print MultipleSeqAlignment” to “print(MultipleSeqAlignment)”.