We Keep Coding

How to save SHA256 object to a file?

(I'm doing all this in python 3.10.4 using pycryptodome)
I'm trying to do this process:
Get a hash of a file
Save that hash somewhere
Load that hash and perform RSA signing using a private key
I'm having a problem in step 3 where to save the hash, I have to save it as a string which doesn't work in Step 3.
I've tried using pickle but I'm getting
"ctypes objects containing pointers cannot be pickled"
Code generating the hash:
sha256 = SHA256.new()
with open(fileDir, 'rb') as f:
while True:
data = f.read(BUF_SIZE)
if not data:
break
sha256.update(data)
Code to perform the signing:
get_file(fileName + '.hash', directory)
with open(currentDir + '/client_files/downloaded/' + fileName + '.hash', 'r') as f:
hash_data = f.read()
with open(currentDir + '/client_files/private_key.pem', 'rb') as f:
private_key = RSA.importKey(f.read())
print(private_key)
signer = PKCS1_v1_5.new(private_key)
signature = signer.sign(hash_data)
The error I'm getting:
Traceback (most recent call last):
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 168, in <module>
main()
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 163, in main
sign(fileName, 'worker_test_files')
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\client\client.py", line 120, in sign
signature = signer.sign(hash_data)
File "C:\Users\User\anaconda3\envs\nscc_project\lib\site-packages\Crypto\Signature\pkcs1_15.py", line 77, in sign
em = _EMSA_PKCS1_V1_5_ENCODE(msg_hash, k)
File "C:\Users\User\anaconda3\envs\nscc_project\lib\site-packages\Crypto\Signature\pkcs1_15.py", line 191, in _EMSA_PKCS1_V1_5_ENCODE
digestAlgo = DerSequence([ DerObjectId(msg_hash.oid).encode() ])
AttributeError: 'str' object has no attribute 'oid'
Note that I'm currently saving the original hash as a string to a text file. If I try to use pickle to save the object as a whole I get this error
with open(currentDir + '/worker_files/sha256.pickle', 'wb') as f:
pickle.dump(sha256, f)
Traceback (most recent call last):
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 188, in <module>
main()
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 179, in main
hash_file(fileName, 'worker_test_files')
File "c:\Users\User\Documents\Coding\VSCode Projects\practiceGround\sec_cloud_project\worker\worker.py", line 55, in hash_file
pickle.dump(sha256, f)
ValueError: ctypes objects containing pointers cannot be pickled

Thanks to #Topaco. Changing to using Cyptography for both hashing and signing seemed to work.
Hashing with Cryptography, dumping to a file with pickle, then load and sign with Cryptography again.

"[WinError 6] The handle is invalid" with Urllib

import urllib.request
def Download(url, file_name):
urllib.request.urlretrieve(url, file_name)
f = open("links.txt", "r")
lines = f.readlines()
for line in lines:
for x in range (1, 5):
filenaame = x
cut_string = line.split('?$')
new_string = cut_string[0]
numerator = new_string.split('/1/')
separator = ''
link = (separator.join(numerator[0] + "/{}/".format(x) + numerator[1]))
file_name = link.split('/{}/'.format(x))
file_name = file_name[1]
file_name = file_name.split('.')
file_name = (separator.join(file_name[0] + "{}".format(filenaame)))
filenaame =+ 1
print("Downloading: {}".format(file_name))
Download(link, filenaame)
Error:
Traceback (most recent call last):
File "C:\python\downloader\rr.py", line 29, in <module>
Download(link, filenaame)
File "C:\python\downloader\rr.py", line 5, in Download
urllib.request.urlretrieve(url, file_name)
File "C:\python\lib\urllib\request.py", line 258, in urlretrieve
tfp = open(filename, 'wb')
OSError: [WinError 6] The handle is invalid
I have googled a lot about this and in every result I found the person was using the subprocess module, which I'm not, which makes it even more difficult.
The code is for downloading images. It does download the first one successfully, then it crashes. Anyone know what's causing the error? I'm still a beginner.

you're passing your counter as a second parameter. In the end the function uses that as a filename to copy the downloaded data to.
By passing an integer you enable the "pass a lowlevel file descriptor" to open:
file is either a string or bytes object giving the pathname (absolute or relative to the current working directory) of the file to be opened or an integer file descriptor of the file to be wrapped.
But the file descriptor doesn't exist (fortunately!)
Let's reproduce the issue here:
>>> open(4,"rb")
Traceback (most recent call last):
File "<string>", line 301, in runcode
File "<interactive input>", line 1, in <module>
OSError: [Errno 9] Bad file descriptor
the fix is obviously:
Download(link, file_name)
(I'd suggest that you rename your filenaame counter for something more meaningful, it avoids mistakes like that)

At this line,
urllib.request.urlretrieve(url, file_name)
here file_name should be a string so before calling Download(link, filenaame) make integer filenaame to string try.
Download(link, str(filenaame))
Sample working for downloading an image
def download_img(url):
name = random.randrange(100,1000)
full_name = str(name)+ ".jpg"
urllib.request.urlretrieve(url, full_name)
download_img("http://images.freeimages.com/images/small-previews/25d/eagle-1523807.jpg")

How to download file from a batch file?

I have some files that contain one URL per line, like
https://url/url/url.com/page-1.jpg
https://url/url/url.com/a.mp3
https://url/url/url.com/b.mp3
....
I try to code for:
import wget
with open ("5074_url.txt", encoding='utf-8', mode = 'r') as f:
for line in list(f): # OR f.readlines()
filename = wget.download(line)
print (filename)
but raise Error msg:
Traceback (most recent call last):
File ".\Geturl2.py\", line 33, in <module>
filename = wget.download(line)
File "C:\Program Files (x86)\Python\lib\site-packages\wget.py", line 506, in download
(fd, tmpfile) = tempfile.mkstemp(".tmp", prefix=prefix, dir=".")
File "C:\Program Files (x86)\Python\lib\tempfile.py", line 342, in mkstemp
return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "C:\Program Files (x86)\Python\lib\tempfile.py", line 260, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
OSError: [Errno 22] Invalid argument: '.\\page-1.jpg\ngjf6wrvy.tmp'
\ngjf6wrvy.tmp What's this ? There's no this in file, I'm sure that.
I'm using Python 3.6.5 on windows 10
I know it could use urllib.request.urlretrieve(url, filename) ,
but it has filename option. I don't wanna change filename.
So How don't change filename ?
[Solved]
import wget
with open ("5074_url.txt", encoding='utf-8', mode = 'r') as f:
for url in f.readlines():
filename = wget.download(url.strip())
print (filename)
Thank you so much for help !!!

I don't know what you're trying to achieve exactly, but here's an example with requests:
#!/usr/bin/env python
import os
import requests
with open('test.txt', 'r') as f:
for url in f.readlines():
r = requests.get(url.strip())
print(r)
This will "download" each URL contained in test.txt and store in memory. The variable r contains the Response object.

Trying to get an image with python

I am trying to get an image from a website and I don't know what i am doing wrong.
Here is my code:
import httplib2
h = httplib2.Http('.cache')
response, content = h.request('http://1.bp.blogspot.com/-KSBzFF0bIyU/TtFyj-KgC0I/AAAAAAAABEo/o43IoY_9Bec/s1600/praia-de-ponta-verde-maceio.jpg')
print(response.status)
with open('maceio.jpg', 'wb') as f:
print(content, file = f)
--------------------------------------------------------------------------------
200
Traceback (most recent call last):
File "/home/matheus/workspace/Get Link/new_method_v2.py", line 12, in <module>
print(content, file = f)
TypeError: 'str' does not support the buffer interface

The error is caused by the following line:
print(content, file = f)
print implicitely converts the bytes object named content to a string (str object), which cannot be written to a file in binary mode, since Python does not know which character encoding to use.
Why are you taking the print detour at all? Just write the contents to the file using the file.write() method:
with open('maceio.jpg', 'wb') as f:
f.write(content)

EOFError whilst pickling in Python

Here's the problem - I'm trying to pickle, and then unpickle hiscores. When I use pickle.load, Python seems to think that I'm trying to load a file called 'Pickle' that I have. Here's the code:
def recieve_hiscores():
hiscores_file = open("hiscores_file.dat", "rb")
for i in hiscores_file:
hiscores = pickle.load(hiscores_file)
hiscores = str(hiscores)
print(hiscores)
Here's the pickling code:
def send_hiscores(score):
hiscores_file = open("hiscores_file.dat", "ab")
pickle.dump(score, hiscores_file)
hiscores_file.close()
And here's the error message:
Traceback (most recent call last):
File "C:\Python31\My Updated Trivia Challenge.py", line 106, in <module>
main()
File "C:\Python31\My Updated Trivia Challenge.py", line 104, in main
recieve_hiscores()
File "C:\Python31\My Updated Trivia Challenge.py", line 56, in recieve_hiscores
hiscores = pickle.load(hiscores_file)
File "C:\Python31\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
EOFError
Don't worry if there's any other mistakes, I'm still learning, but I can't work this out.

When you iterate over the file, you get newline separated lines. This is NOT how you get a series of pickles. The end of file error is raised because the first line has a partial pickle.
Try this:
def recieve_hiscores():
highscores = []
with open("hiscores_file.dat", "rb") as hiscores_file:
try:
while True:
hiscore = pickle.load(hiscores_file)
hiscore = str(hiscore)
print(hiscore)
highscores.append(hiscore)
except EOFError:
pass
return highscores