When I run the following code:
import shelve
input = open("input.txt",)
shelveFile = shelve.open("myData")
shelveFile["inputFile"] = input
input.close()
shelveFile.close()
I expect the shelve file myData to hold the file object input. Instead, running the code produces the following error:
Traceback (most recent call last):
File "/Users/ashutoshmishra/Documents/Sandbox/Sandbox3.py", line 5, in <module>
shelveFile["inputFile"] = input
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/shelve.py", line 124, in __setitem__
p.dump(value)
TypeError: cannot serialize '_io.TextIOWrapper' object
I was wondering why I could not save the file object input to the shelve file myData?
The following answer is taken from #DanD.'s comment above.
Read the file: shelveFile["inputFile"] = input.read()
Related
(I'm doing all this in python 3.10.4 using pycryptodome)
I'm trying to do this process:
Get a hash of a file
Save that hash somewhere
Load that hash and perform RSA signing using a private key
I'm having a problem in step 3 where to save the hash, I have to save it as a string which doesn't work in Step 3.
I've tried using pickle but I'm getting
"ctypes objects containing pointers cannot be pickled"
Code generating the hash:
sha256 = SHA256.new()
with open(fileDir, 'rb') as f:
while True:
data = f.read(BUF_SIZE)
if not data:
break
sha256.update(data)
Code to perform the signing:
get_file(fileName + '.hash', directory)
with open(currentDir + '/client_files/downloaded/' + fileName + '.hash', 'r') as f:
hash_data = f.read()
with open(currentDir + '/client_files/private_key.pem', 'rb') as f:
private_key = RSA.importKey(f.read())
print(private_key)
signer = PKCS1_v1_5.new(private_key)
signature = signer.sign(hash_data)
The error I'm getting:
Traceback (most recent call last):
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 168, in <module>
main()
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 163, in main
sign(fileName, 'worker_test_files')
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 120, in sign
signature = signer.sign(hash_data)
File "C:\Users\User\anaconda3\envs\nscc_project\lib\site-packages\Crypto\Signature\pkcs1_15.py", line 77, in sign
em = _EMSA_PKCS1_V1_5_ENCODE(msg_hash, k)
File "C:\Users\User\anaconda3\envs\nscc_project\lib\site-packages\Crypto\Signature\pkcs1_15.py", line 191, in _EMSA_PKCS1_V1_5_ENCODE
digestAlgo = DerSequence([ DerObjectId(msg_hash.oid).encode() ])
AttributeError: 'str' object has no attribute 'oid'
Note that I'm currently saving the original hash as a string to a text file. If I try to use pickle to save the object as a whole I get this error
with open(currentDir + '/worker_files/sha256.pickle', 'wb') as f:
pickle.dump(sha256, f)
Traceback (most recent call last):
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 188, in <module>
main()
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 179, in main
hash_file(fileName, 'worker_test_files')
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 55, in hash_file
pickle.dump(sha256, f)
ValueError: ctypes objects containing pointers cannot be pickled
Thanks to #Topaco. Changing to using Cyptography for both hashing and signing seemed to work.
Hashing with Cryptography, dumping to a file with pickle, then load and sign with Cryptography again.
i want to read arff file but i am getting this error. Could any one help me with this
import arff, numpy as np
dataset = arff.load(open('ckdfull.arff', 'rb'))
data = np.array(dataset['data'])
Traceback (most recent call last):
File "C:/Users/username/PycharmProjects/ckd/ckd.py", line 2, in <module>
dataset = arff.load(open('ckdfull.arff', 'rb'))
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 896, in load
return_type=return_type)
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 739, in decode
matrix_type=return_type)
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 651, in _decode
row = row.strip(' \r\n')
TypeError: a bytes-like object is required, not 'str'
Thank You
I am trying to modify my file after fetching it from HDFS using pyspark and then i want to save it in HDFS for that i have written below code.
Code:
import subprocess
from subprocess import Popen, PIPE
from pyspark import SparkContext
cat = sc.textFile("/user/root/parsed.txt")
hrk = "#"
for line in cat.collect():
if (code == "ID"):
line =line.strip() + "|"+hrk
line.saveAsTextFile("/user/root/testsprk")
print(line)
But when i run the code i am getting below error.
Error:
Traceback (most recent call last):
File "<stdin>", line 30, in <module>
AttributeError: 'unicode' object has no attribute 'saveAsTextFile'
I know there is some issue with my line variable but i am not able to fix it.
It because you are collecting all data, it means that collection is not RDD, but normal list and line is just one string.
You shouldn't collect all data on driver. Instead, use RDD.map and then RDD.saveAsTextFile
def add_hrk_on_id(line):
if (code == "ID"):
return line.strip() + "|"+hrk
else
return line
cat.map(add_hrk_on_id).saveAsTextFile(path)
I am partially able to work with json saved as file:
#! /usr/bin/python3
import json
from pprint import pprint
json_file='a.json'
json_data=open(json_file)
data = json.load(json_data)
json_data.close()
print(data[10])
But I am trying to achieve the same from data directly from web. I am trying with the accepted answer here:
#! /usr/bin/python3
from urllib.request import urlopen
import json
from pprint import pprint
jsonget=urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")
data = json.load(jsonget)
pprint(data)
which is giving me error:
Traceback (most recent call last):
File "i.py", line 10, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib64/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
What is going wrong here?
Changing the code as par Charlie's reply to:
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee"))
data = json.load(jsonget)
pprint(jsonget)
breaks at json.load:
Traceback (most recent call last):
File "i.py", line 9, in <module>
data = json.load(jsonget)
File "/usr/lib64/python3.5/json/__init__.py", line 265, in load
return loads(fp.read(),
AttributeError: 'str' object has no attribute 'read'
It's actually telling you the answer: you're getting back a byte array, where in Python 3 a string is different because of dealing with unicode. In Python 2.7, it would work. You should be able to fix it by converting your bytes explicitly to a string with
jsonget=str(urlopen("http://api.crossref.org/works?query.author=Rudra+Banerjee")_
Given below is the code that I am using to read values form a file
import linecache
line_number=line_number+1
linecache.getline("write.txt", line_number)
I am using py2exe to convert the file into an executable. The problem is after conversion to exe it cannot read the file.
The error message is:
Could not convert string to a float
As suggested in the comment the full error trace is:
Traceback (most recent call last):
File "logic.py", line 437, in <module>
File "logic.py", line 224, in readdata
ValueError: could not convert string to float:
Where am I going wrong?