I'm able to embed the original filename using python-gnupg when encrypting a file using below:
filename = 'test.tim'
with open(filename, 'rb) as fh:
status = gpg.encrypt_file(fh, recipients='somerecipient', output='test.tim.gpg',\
sign='somesignature', extra_args=['--set-filename', os.path.basename(filename)])
This can be verified by using gpg from the command line:
$ gpg2 --list-packets test.tim.gpg | grep name
I am however unable to preserve the original filename when decrypting the file:
with open(filename, 'rb') as fh:
status = gpg.decrypt_file(fh, extra_args['--use-embedded-filename'])
I am aware about the output parameter (which specifies the filename to save contents to) in the decrypt_file function, but i want to preserve the original filename (which i won't always know)
It seems the decrypt_file function always passes the --decrypt flag to gpg which always ouputs the contents to stdout (unless used in conjunction with output parameter) as in:
$ gpg --decrypt --use-embedded-filename test.tim.gpg
Below command will decrypt and save output to original filename:
$ gpg --use-embedded-filename test.tim.gpg
Any ideas?
Tim
The functionality to do what you want doesn't exist in the original python-gnupg.
There's a modified version here by isislovecruft (which is what you get if you pip install gnupg) that adds support for --list-packets with gpg.listpackets but still doesn't support --use-embeded-file-name
So my approach, if I were to insist on using python only, would probably be to start with isislovecruft's version and then subclass GPG like this:
import gnupg
import os
GPGBINARY = os.environ.get('GPGBINARY', 'gpg')
hd = os.path.join(os.getcwd(), 'keys')
class myGPG(gnupg.GPG):
def decrypt_file_original_name(self, file, always_trust=False, passphrase=None, extra_args=None):
args = ["--use-embedded-filename"]
output = calculate_the_file_name_using_list_packets()
self.set_output_without_confirmation(args, output)
if always_trust: # pragma: no cover
args.append("--always-trust")
if extra_args:
args.extend(extra_args)
result = self.result_map['crypt'](self)
self._handle_io(args, file, result, passphrase, binary=True)
# logger.debug('decrypt result: %r', result.data)
return result
gpg = myGPG(gnupghome=hd, gpgbinary=GPGBINARY)
Bear in mind, at this point it is almost certainly much easier to just use subprocess and run the gpg binary directly from the shell, especially as you don't care about the output.
Anyway, I got this far and have run out of time for now, so I leave implementing calculate_the_file_name_using_list_packets up to you, if you choose to go the 'pure python' route. Hopefully it is a bit easier now you have gpg.list-packets. Good luck!
Related
I have been trying to append my output of a command to a temporary file in python and later doing some operations. Not able to append the data to a temporary file. Any help is appreciated! My sample code as follows.
Getting the error like this.
with open(temp1 , 'r') as f:
TypeError: expected str, bytes or os.PathLike object, not _TemporaryFileWrapper
import tempfile
import os
temp1 = tempfile.NamedTemporaryFile()
os.system("echo Hello world | tee temp1")
with open(temp1 , 'r') as f:
a = f.readlines()[-1]
print(a)
import tempfile
import os
# Opening in update-text mode to avoid encoding the data written to it
temp1 = tempfile.NamedTemporaryFile("w+")
# popen opens a pipe from the command, allowing one to capture its output
output = os.popen("echo Hello world")
# Write the command output to the temporary file
temp1.write(output.read())
# Reset the stream position at the beginning of the file, if you want to read its contents
temp1.seek(0)
print(temp1.read())
Check out subprocess.Popen for more powerful subprocess communication.
Whatever you're trying to do isn't right. It appears that you are trying to have a system call write to a file, and then you want to read that file in your Python code. You're creating a temporary file, but then your system call is writing to a statically named file, named 'temp1' rather than to the temporary file you've opened. So it's unclear if you want/need to use a computed temporary file name or if using temp1 is OK. The easiest way to fix your code to do what I think you want is like this:
import os
os.system("echo Hello world | tee temp1")
with open('temp1' , 'r') as f:
a = f.readlines()[-1]
print(a)
If you need to create a temporary file name in your situation, then you have to be careful if you are at all concerned about security or thread safety. What you really want to do is have the system create a temporary directory for you, and then create a statically named file in that directory. Here's your code reworked to do that:
import tempfile
import os
with tempfile.TemporaryDirectory() as dir:
tempfile = os.path.join(dir, "temp1")
os.system("echo Hello world /tmp > " + tempfile)
with open(tempfile) as f:
buf = f.read()
print(buf)
This method has the added benefit of automatically cleaning up for you.
UPDATE: I have now seen #UlisseBordingnon's answer. That's a better solution overall. Using os.system() is discouraged. I would have gone a bit different of a way by using the subprocess module, but what they suggest is 100% valid, and is thread and security safe. I guess I'll leave my answer here as maybe you or other readers need to use os.system() or otherwise have the shell process you execute write directly to a file.
As others have suggested, you should use the subprocess module instead of os.system. However from subprocess you can use the most recent interface (and by most recent, I believe this was adding in Python 3.4) of subprocess.run.
The neat thing about using .run is that you can pass any file-like object to stdout and the stdout stream will automatically redirect to that file.
import tempfile
import subprocess
with tempfile.NamedTemporaryFile("w+") as f:
subprocess.run(["echo", "hello world"], stdout=f)
# command has finished running, let's check the file
f.seek(0)
print(f.read())
# hello world
If you are using python 3.5 or later (as with most of us), then use subprocess.run is better because you do not need a temporary file:
import subprocess
completed_process = subprocess.run(
["echo", "hello world"],
capture_output=True,
encoding="utf-8",
)
print(completed_process.stdout)
Notes
The capture_output parameter tells run() to save the output to the .stdout and .stderr attributes
The encoding parameter will convert the output from bytes to string
Depending on your needs, if your print your output, a quickier way, but maybe not exactly what you are looking for is to redirect the output to a file, at the command line level
Example(egfile.py):
import os
os.system("echo Hello world")
At command level you can simply do:
python egfile.py > file.txt
The output of the file will be redirected to the file instead to the screen
The php code is
'''
$input_file = "a.txt";
$source = file_get_contents($input_file);
$source = gzcompress($source);
file_put_contents("php.txt",$source)
'''
The python code is
'''
testFile = "a.txt"
content = None
with open(testFile,"rb") as f:
content = f.read()
outContent = zlib.compress(content)
with open("py.txt","wb") as f:
f.write(outContent)
'''
The python3 version is [Python 3.6.9]
The php version is [PHP 7.2.17]
I need the same result for same md5.
The problem is not in PHP or Python, but rather in your "need". You cannot expect to get the same result, unless the two environments happen to be using the same version of the same compression code with the same settings. Since you do not have control of the version of code being used, your "need" can never be guaranteed to be met.
You should instead be doing your md5 on the decompressed data, not the compressed data.
I find the solution.
The code is
compress = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,zlib.DEFLATED, 15, 9)
outContent = compress.compress(content)
outContent += compress.flush()
The python zlib provide a interface "zlib.compressobj",which returns a compressobj,and the parameters decide the result.
You can adjust parameters to make sure the python's result is same with php's
I'm trying to achieve the functionality provided by the following bash command in Python.
echo "$DATA" | gpg --symmetric --armor --batch --passphrase "${KEY}"
So far I've tried to use subprocess but am having difficulty passing in the data. I tried giving it as a command in the list of parameters to send to subprocess but that just effectively echos the the entire thing.
cmd = f"| gpg --symmetric --armor --batch --passphrase {key}".split()
temp = ["echo", f"\"{data}\""]
temp.extend(cmd)
res = subprocess.run(temp, stdout=subprocess.PIPE, universal_newlines=True)
encrypted = res.stdout.strip()
I'm also interested in using the python-gnupg module but have not yet figured out how to replicate the above with it either.
Thanks in advance for any help!
You can use the input argument to run()/check_output():
from getpass import getpass
import subprocess
key = getpass("KEY: ")
data = b'Symmetric Encryption with GPG and Subprocess'
command = ["gpg", "--symmetric", "--armor", "--batch", "--passphrase", key]
out = subprocess.check_output(command, input=data, universal_newlines=False)
Note that GNU echo will, by default, append a newline. Use echo -n to not print the trailing \n. Either way, you'll want to be careful to mimic this in Python.
In case anyone was wondering, I also got the python-gnupg module to work for my application. I am sticking with the subprocess answer since that reduces dependencies but wanted to share this as well.
gpg = gnupg.GPG()
encrypted = str(gpg.encrypt(data, recipients=None, symmetric=True, passphrase=key, extra_args=["--batch"]))
The python-gnupg module has a long history with serious security flaws, many of which it's more likely to be affected by due to the decision to use subproess to call an external binary executable.
Instead, the recommendation of the GnuPG Project is to use the CPython bindings for the GPGME C API, which ship with the GPGME source code.
import gpg
from getpass import getpass
key = getpass("KEY: ")
c = gpg.Context(armor=True)
data = b"Symmetric encryption with GPGME."
ciphertext, result, sign_result = c.encrypt(data, sign=False, passphrase=key)
with open("some_file.txt.asc", "wb") as f:
f.write(ciphertext)
Because this uses symmetric encryption, there won't be a digital signature included and there are no recipient keys to check for. Which means both result and sign_result will return None. Only ciphertext contains anything and that's the ASCII armoured excrypted data, which can be written to a file as above, or you can do something else with it.
The documentation for this, far superior, module is included with the GPGME source, but an online draft version is available here.
I am creating an ZIP file with ZipFile in Python 2.5, it works OK so far:
import zipfile, os
locfile = "test.txt"
loczip = os.path.splitext (locfile)[0] + ".zip"
zip = zipfile.ZipFile (loczip, "w")
zip.write (locfile)
zip.close()
But I couldn't find how to encrypt the files in the ZIP file.
I could use system and call PKZIP -s, but I suppose there must be a more "Pythonic" way. I'm looking for an open source solution.
I created a simple library to create a password encrypted zip file in python. - here
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt", "dst.zip", "password", compression_level)
The library requires zlib.
I have checked that the file can be extracted in WINDOWS/MAC.
This thread is a little bit old, but for people looking for an answer to this question in 2020/2021.
Look at pyzipper
A 100% API compatible replacement for Python’s zipfile that can read and write AES encrypted zip files.
7-zip is also a good choice, but if you do not want to use subprocess, go with pyzipper...
The duplicate question: Code to create a password encrypted zip file? has an answer that recommends using 7z instead of zip. My experience bears this out.
Copy/pasting the answer by #jfs here too, for completeness:
To create encrypted zip archive (named 'myarchive.zip') using open-source 7-Zip utility:
rc = subprocess.call(['7z', 'a', '-mem=AES256', '-pP4$$W0rd', '-y', 'myarchive.zip'] +
['first_file.txt', 'second.file'])
To install 7-Zip, type:
$ sudo apt-get install p7zip-full
To unzip by hand (to demonstrate compatibility with zip utility), type:
$ unzip myarchive.zip
And enter P4$$W0rd at the prompt.
Or the same in Python 2.6+:
>>> zipfile.ZipFile('myarchive.zip').extractall(pwd='P4$$W0rd')
pyminizip works great in creating a password protected zip file. For unziping ,it fails at some situations. Tested on python 3.7.3
Here, i used pyminizip for encrypting the file.
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt",'src', "dst.zip", "password", compression_level)
For unzip, I used zip file module:
from zipfile import ZipFile
with ZipFile('/home/paulsteven/dst.zip') as zf:
zf.extractall(pwd=b'password')
You can use pyzipper for this task and it will work great when you want to encrypt a zip file or generate a protected zip file.
pip install pyzipper
import pyzipper
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA,
encryption=pyzipper.WZ_AES) as zf:
zf.setpassword(secret_password)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
The strength of the AES encryption can be configure to be 128, 192 or 256 bits. By default it is 256 bits. Use the setencryption() method to specify the encryption kwargs:
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA) as zf:
zf.setpassword(secret_password)
zf.setencryption(pyzipper.WZ_AES, nbits=128)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
Official Python ZipFile documentation is available here: https://docs.python.org/3/library/zipfile.html
#tripleee's answer helped me, see my test below.
This code works for me on python 3.5.2 on Windows 8.1 ( 7z path added to system).
rc = subprocess.call(['7z', 'a', output_filename + '.zip', '-mx9', '-pSecret^)'] + [src_folder + '/'])
With two parameters:
-mx9 means max compression
-pSecret^) means password is Secret^). ^ is escape for ) for Windows OS, but when you unzip, it will need type in the ^.
Without ^ Windows OS will not apply the password when 7z.exe creating the zip file.
Also, if you want to use -mhe switch, you'll need the file format to be in 7z instead of zip.
I hope that may help.
2022 answer:
I believe this is an utterly mundane task and therefore should be oneliner. I abstracted away all the frevolous details in a library that is as powerfull as a bash terminal.
from crocodile.toolbox import Path
file = Path(r'my_string_path')
result_file = file.zip(pwd="lol", use_7z=True)
when the 7z flag is raised, it gets called behind the scenes.
You don't need to learn 7z command line syntax.
You don't need to worry about installing 7z, does that automatically if it's not installed. (tested on windows so far)
You can use the Chilkat library. It's commercial, but has a free evaluation and seems pretty nice.
Here's an example I got from here:
import chilkat
# Demonstrates how to create a WinZip-compatible 128-bit AES strong encrypted zip
zip = chilkat.CkZip()
zip.UnlockComponent("anything for 30-day trial")
zip.NewZip("strongEncrypted.zip")
# Set the Encryption property = 4, which indicates WinZip compatible AES encryption.
zip.put_Encryption(4)
# The key length can be 128, 192, or 256.
zip.put_EncryptKeyLength(128)
zip.SetPassword("secret")
zip.AppendFiles("exampleData/*",True)
zip.WriteZip()
a way of checking if a file has been modified, is calculating and storing a hash (or checksum) for the file. Then at any point the hash can be re-calculated and compared against the stored value.
I'm wondering if there is a way to store the hash of a file in the file itself? I'm thinking text files.
The algorithm to calculate the hash should be iterative and consider that the hash will be added to the file the hash is being calculated for... makes sense? Anything available?
Thanks!
edit:
https://security.stackexchange.com/questions/3851/can-a-file-contain-its-md5sum-inside-it
from Crypto.Hash import HMAC
secret_key = "Don't tell anyone"
h = HMAC.new(secret_key)
text = "whatever you want in the file"
## or: text = open("your_file_without_hash_yet").read()
h.update(text)
with open("file_with_hash") as fh:
fh.write(text)
fh.write(h.hexdigest())
Now, as some people tried to point out, though they seemed confused - you need to remember that this file has the hash on the end of it and that the hash is itself not part of what gets hashed. So when you want to check the file, you would do something along the lines of:
end_len = len(h.hex_digest())
all_text = open("file_with_hash").read()
text, expected_hmac = all_text[:end_len], all_text[end_len:]
h = HMAC.new(secret_key)
h.update(text)
if h.hexdigest() != expected_hmac:
raise "Somebody messed with your file!"
It should be clear though that this alone doesn't ensure your file hasn't been changed; the typical use case is to encrypt your file, but take the hash of the plaintext. That way, if someone changes the hash (at the end of the file) or tries changing any of the characters in the message (the encrypted portion), things will mismatch and you will know something was changed.
A malicious actor won't be able to change the file AND fix the hash to match because they would need to change some data, and then rehash everything with your private key. So long as no one knows your private key, they won't know how to recreate the correct hash.
This is an interesting question. You can do it if you adopt a proper convention for hashing and verifying the integrity of the files. Suppose you have this file, namely, main.py:
#!/usr/bin/env python
# encoding: utf-8
print "hello world"
Now, you could append an SHA-1 hash to the python file as a comment:
(printf '#'; cat main.py | sha1sum) >> main.py
Updated main.py:
#!/usr/bin/env python
# encoding: utf-8
print "hello world"
#30e3b19d4815ff5b5eca3a754d438dceab9e8814 -
Hence, to verify if the file was modified you can do this in Bash:
if [ "$(printf '#';head -n-1 main.py | sha1sum)" == "$(tail -n1 main.py)" ]
then
echo "Unmodified"
else
echo "Modified"
fi
Of course, someone could try to fool you by changing the hash string manually. In order to stop these bad guys, you can improve the system by tempering the file with a secret string before adding the hash to the last line.
Improved version
Add the hash in the last line including your secret string:
(printf '#';cat main.py;echo 'MyUltraSecretTemperString12345') | sha1sum >> main.py
For checking if the file was modified:
if [ "$(printf '#';(head -n-1 main.py; echo 'MyUltraSecretTemperString12345') | sha1sum)" == "$(tail -n1 main.py)" ]
then
echo "Unmodified"
else
echo "Modified"
fi
Using this improved version, the bad guys only can fool you if they find your ultra secret key first.
EDIT: This is a rough implementation of the keyed-hash message authentication code (HMAC).
Well although it looks like a strange idea, it could be an application of a little used but very powerful property of windows NTFS file system: the File Streams.
It allows to add many streams to a file without changing the content of the default stream. For example:
echo foo > foo.text
echo bar > foo.text:alt
type foo.text
=> foo
more < foo.text:alt
=> bar
But when listing the directory, you can only see one single file: foo.txt
So in your use case, you could write the hash of main stream in stream named hash, and later compare the content of the hash stream with the hash of the main stream.
Just a remark: for a reason I do not know, type foo.text:alt generates the following error:
"The filename, directory name, or volume label syntax is incorrect."
that's why my example uses more < as recommended in the Using streams page on MSDN
So assuming you have a myhash function that gives the hash for a file (you can easily build one by using the hashlib module):
def myhash(filename):
# compute the hash of the file
...
return hash_string
You can do:
def store_hash(filename):
hash_string = myhash(filename)
with open(filename + ":hash") as fd:
fd.write(hash_string)
def compare_hash(filename):
hash_string = myhash(filename)
with open(filename + ":hash") as fd:
orig = fd.read()
return (hash_string == orig)