Related
I have got a python script which is creating an ODBC connection. The ODBC connection is generated with a connection string. In this connection string I have to include the username and password for this connection.
Is there an easy way to obscure this password in the file (just that nobody can read the password when I'm editing the file) ?
Base64 encoding is in the standard library and will do to stop shoulder surfers:
>>> import base64
>>> print(base64.b64encode("password".encode("utf-8")))
cGFzc3dvcmQ=
>>> print(base64.b64decode("cGFzc3dvcmQ=").decode("utf-8"))
password
Here is a simple method:
Create a python module - let's call it peekaboo.py.
In peekaboo.py, include both the password and any code needing that password
Create a compiled version - peekaboo.pyc - by importing this module (via python commandline, etc...).
Now, delete peekaboo.py.
You can now happily import peekaboo relying only on peekaboo.pyc. Since peekaboo.pyc is byte compiled it is not readable to the casual user.
This should be a bit more secure than base64 decoding - although it is vulnerable to a py_to_pyc decompiler.
Douglas F Shearer's is the generally approved solution in Unix when you need to specify a password for a remote login.
You add a --password-from-file option to specify the path and read plaintext from a file.
The file can then be in the user's own area protected by the operating system.
It also allows different users to automatically pick up their own own file.
For passwords that the user of the script isn't allowed to know - you can run the script with elavated permission and have the password file owned by that root/admin user.
If you are working on a Unix system, take advantage of the netrc module in the standard Python library. It reads passwords from a separate text file (.netrc), which has the format decribed here.
Here is a small usage example:
import netrc
# Define which host in the .netrc file to use
HOST = 'mailcluster.loopia.se'
# Read from the .netrc file in your home directory
secrets = netrc.netrc()
username, account, password = secrets.authenticators( HOST )
print username, password
How about importing the username and password from a file external to the script? That way even if someone got hold of the script, they wouldn't automatically get the password.
The best solution, assuming the username and password can't be given at runtime by the user, is probably a separate source file containing only variable initialization for the username and password that is imported into your main code. This file would only need editing when the credentials change. Otherwise, if you're only worried about shoulder surfers with average memories, base 64 encoding is probably the easiest solution. ROT13 is just too easy to decode manually, isn't case sensitive and retains too much meaning in it's encrypted state. Encode your password and user id outside the python script. Have he script decode at runtime for use.
Giving scripts credentials for automated tasks is always a risky proposal. Your script should have its own credentials and the account it uses should have no access other than exactly what is necessary. At least the password should be long and rather random.
base64 is the way to go for your simple needs. There is no need to import anything:
>>> 'your string'.encode('base64')
'eW91ciBzdHJpbmc=\n'
>>> _.decode('base64')
'your string'
A way that I have done this is as follows:
At the python shell:
>>> from cryptography.fernet import Fernet
>>> key = Fernet.generate_key()
>>> print(key)
b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
>>> cipher = Fernet(key)
>>> password = "thepassword".encode('utf-8')
>>> token = cipher.encrypt(password)
>>> print(token)
b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
Then, create a module with the following code:
from cryptography.fernet import Fernet
# you store the key and the token
key = b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
token = b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
# create a cipher and decrypt when you need your password
cipher = Fernet(key)
mypassword = cipher.decrypt(token).decode('utf-8')
Once you've done this, you can either import mypassword directly or you can import the token and cipher to decrypt as needed.
Obviously, there are some shortcomings to this approach. If someone has both the token and the key (as they would if they have the script), they can decrypt easily. However it does obfuscate, and if you compile the code (with something like Nuitka) at least your password won't appear as plain text in a hex editor.
for python3 obfuscation using base64 is done differently:
import base64
base64.b64encode(b'PasswordStringAsStreamOfBytes')
which results in
b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM='
note the informal string representation, the actual string is in quotes
and decoding back to the original string
base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
b'PasswordStringAsStreamOfBytes'
to use this result where string objects are required the bytes object can be translated
repr = base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
secret = repr.decode('utf-8')
print(secret)
for more information on how python3 handles bytes (and strings accordingly) please see the official documentation.
This is a pretty common problem. Typically the best you can do is to either
A) create some kind of ceasar cipher function to encode/decode (just not rot13)
or
B) the preferred method is to use an encryption key, within reach of your program, encode/decode the password. In which you can use file protection to protect access the key.
Along those lines if your app runs as a service/daemon (like a webserver) you can put your key into a password protected keystore with the password input as part of the service startup. It'll take an admin to restart your app, but you will have really good pretection for your configuration passwords.
Your operating system probably provides facilities for encrypting data securely. For instance, on Windows there is DPAPI (data protection API). Why not ask the user for their credentials the first time you run then squirrel them away encrypted for subsequent runs?
Here is my snippet for such thing. You basically import or copy the function to your code. getCredentials will create the encrypted file if it does not exist and return a dictionaty, and updateCredential will update.
import os
def getCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
try:
with open(directory+'\\Credentials.txt', 'r') as file:
cred = file.read()
file.close()
except:
print('I could not file the credentials file. \nSo I dont keep asking you for your email and password everytime you run me, I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
return {'lanid':base64.b64decode(bytes(cred.split(splitter)[0], encoding='utf-8')).decode('utf-8'),
'email':base64.b64decode(bytes(cred.split(splitter)[1], encoding='utf-8')).decode('utf-8'),
'password':base64.b64decode(bytes(cred.split(splitter)[2], encoding='utf-8')).decode('utf-8')}
def updateCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
print('I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
cred = getCredentials()
updateCredentials()
Place the configuration information in a encrypted config file. Query this info in your code using an key. Place this key in a separate file per environment, and don't store it with your code.
More homegrown appraoch rather than converting authentication / passwords / username to encrytpted details. FTPLIB is just the example.
"pass.csv" is the csv file name
Save password in CSV like below :
user_name
user_password
(With no column heading)
Reading the CSV and saving it to a list.
Using List elelments as authetntication details.
Full code.
import os
import ftplib
import csv
cred_detail = []
os.chdir("Folder where the csv file is stored")
for row in csv.reader(open("pass.csv","rb")):
cred_detail.append(row)
ftp = ftplib.FTP('server_name',cred_detail[0][0],cred_detail[1][0])
Do you know pit?
https://pypi.python.org/pypi/pit (py2 only (version 0.3))
https://github.com/yoshiori/pit (it will work on py3 (current version 0.4))
test.py
from pit import Pit
config = Pit.get('section-name', {'require': {
'username': 'DEFAULT STRING',
'password': 'DEFAULT STRING',
}})
print(config)
Run:
$ python test.py
{'password': 'my-password', 'username': 'my-name'}
~/.pit/default.yml:
section-name:
password: my-password
username: my-name
If running on Windows, you could consider using win32crypt library. It allows storage and retrieval of protected data (keys, passwords) by the user that is running the script, thus passwords are never stored in clear text or obfuscated format in your code. I am not sure if there is an equivalent implementation for other platforms, so with the strict use of win32crypt your code is not portable.
I believe the module can be obtained here: http://timgolden.me.uk/pywin32-docs/win32crypt.html
You could also consider the possibility of storing the password outside the script, and supplying it at runtime
e.g. fred.py
import os
username = 'fred'
password = os.environ.get('PASSWORD', '')
print(username, password)
which can be run like
$ PASSWORD=password123 python fred.py
fred password123
Extra layers of "security through obscurity" can be achieved by using base64 (as suggested above), using less obvious names in the code and further distancing the actual password from the code.
If the code is in a repository, it is often useful to store secrets outside it, so one could add this to ~/.bashrc (or to a vault, or a launch script, ...)
export SURNAME=cGFzc3dvcmQxMjM=
and change fred.py to
import os
import base64
name = 'fred'
surname = base64.b64decode(os.environ.get('SURNAME', '')).decode('utf-8')
print(name, surname)
then re-login and
$ python fred.py
fred password123
Why not have a simple xor?
Advantages:
looks like binary data
noone can read it without knowing the key (even if it's a single char)
I get to the point where I recognize simple b64 strings for common words and rot13 as well. Xor would make it much harder.
There are several ROT13 utilities written in Python on the 'Net -- just google for them. ROT13 encode the string offline, copy it into the source, decode at point of transmission.But this is really weak protection...
This doesn't precisely answer your question, but it's related. I was going to add as a comment but wasn't allowed.
I've been dealing with this same issue, and we have decided to expose the script to the users using Jenkins. This allows us to store the db credentials in a separate file that is encrypted and secured on a server and not accessible to non-admins.
It also allows us a bit of a shortcut to creating a UI, and throttling execution.
import base64
print(base64.b64encode("password".encode("utf-8")))
print(base64.b64decode(b'cGFzc3dvcmQ='.decode("utf-8")))
I have a Python script that contains a plain text password used to bind Mac to AD.
I need to deploy this script to few Macs.
How can I secure password which is in plain text, so that no one can see the password?
I don't want to use base64, I'd need something stronger than base64.
You can use encode and decode functions.
def encode(key, clear):
enc = []
for i in range(len(clear)):
key_c = key[i % len(key)]
enc_c = chr((ord(clear[i]) + ord(key_c)) % 256)
enc.append(enc_c)
return base64.urlsafe_b64encode("".join(enc).encode()).decode()
def decode(key, enc):
dec = []
enc = base64.urlsafe_b64decode(enc).decode()
for i in range(len(enc)):
key_c = key[i % len(key)]
dec_c = chr((256 + ord(enc[i]) - ord(key_c)) % 256)
dec.append(dec_c)
return "".join(dec)
Usage: encode("mykey", "password")
(You need to import base64)
If you want high secured password, i coded a script but i tried to decode the string and i can not do it.
Try storing the password in a variable in a separate Python script.
pass.py
password = 'PASSWORD'
Import the variable from this script separately.
main.py
from pass import password
This will create a bytecode file named pass.pyc. Now you can delete pass.py and continue using pass.pyc, which isn't human-readable and much harder to decompile.
Edit: If somebody has access to your source code, they can still see the password with a print statement!
Edit 2: As Giacomo Alzetta commented, the password is still human-readable here, but it comes down to the "attack model" here. If someone really wants to, they can still dig around and find it. Other alternatives are encryption algorithms available in OpenSSL. Symmetric key algorithms might still not be useful for you since you'd probably have to store the encryption key within the source code. Asymmetric key algorithms might work.
In Python I'm using pdfminer to read the text from a pdf with the code below this message. I now get an error message saying:
File "/usr/local/lib/python2.7/dist-packages/pdfminer/pdfpage.py", line 124, in get_pages
raise PDFTextExtractionNotAllowed('Text extraction is not allowed: %r' % fp)
PDFTextExtractionNotAllowed: Text extraction is not allowed: <cStringIO.StringO object at 0x7f79137a1
ab0>
When I open this pdf with Acrobat Pro it turns out it is secured (or "read protected"). From this link however, I read that there's a multitude of services which can disable this read-protection easily (for example pdfunlock.com. When diving into the source of pdfminer, I see that the error above is generated on these lines.
if check_extractable and not doc.is_extractable:
raise PDFTextExtractionNotAllowed('Text extraction is not allowed: %r' % fp)
Since there's a multitude of services which can disable this read-protection within a second, I presume it is really easy to do. It seems that .is_extractable is a simple attribute of the doc, but I don't think it is as simple as changing .is_extractable to True..
Does anybody know how I can disable the read protection on a pdf using Python? All tips are welcome!
================================================
Below you will find the code with which I currently extract the text from non-read protected.
def getTextFromPDF(rawFile):
resourceManager = PDFResourceManager(caching=True)
outfp = StringIO()
device = TextConverter(resourceManager, outfp, codec='utf-8', laparams=LAParams(), imagewriter=None)
interpreter = PDFPageInterpreter(resourceManager, device)
fileData = StringIO()
fileData.write(rawFile)
for page in PDFPage.get_pages(fileData, set(), maxpages=0, caching=True, check_extractable=True):
interpreter.process_page(page)
fileData.close()
device.close()
result = outfp.getvalue()
outfp.close()
return result
I had some issues trying to get qpdf to behave in my program. I found a useful library, pikepdf, that is based on qpdf and automatically converts pdfs to be extractable.
The code to use this is pretty straightforward:
import pikepdf
pdf = pikepdf.open('unextractable.pdf')
pdf.save('extractable.pdf')
As far as I know, in most cases the full content of the PDF is actually encrypted, using the password as the encryption key, and so simply setting .is_extractable to True isn't going to help you.
Per this thread:
Does a library exist to remove passwords from PDFs programmatically?
I would recommend removing the read-protection with a command-line tool such as qpdf (easily installable, e.g. on Ubuntu use apt-get install qpdf if you don't have it already):
qpdf --password=PASSWORD --decrypt SECURED.pdf UNSECURED.pdf
Then open the unlocked file with pdfminer and do your stuff.
For a pure-Python solution, you can try using PyPDF2 and its .decrypt() method, but it doesn't work with all types of encryption, so really, you're better off just using qpdf - see:
https://github.com/mstamy2/PyPDF2/issues/53
I used below code using pikepdf and able to overwrite.
import pikepdf
pdf = pikepdf.open('filepath', allow_overwriting_input=True)
pdf.save('filepath')
In my case there was no password, but simply setting check_extractable=False circumvented the PDFTextExtractionNotAllowed exception for a problematic file (that opened fine in other viewers).
Full disclosure, I am one of the maintainers of pdfminer.six. It is a community-maintained version of pdfminer for python 3.
This issue was fixed in 2020 by disabling the check_extractable by default. It now shows a warning instead of raising an error.
Similar question and answer here.
The 'check_extractable=True' argument is by design.
Some PDFs explicitly disallow to extract text, and PDFMiner follows the directive. You can override it (giving check_extractable=False), but do it at your own risk.
If you want to unlock all pdf files in a folder without renaming them, you may use this code:
import glob, os, pikepdf
p = os.getcwd()
for file in glob.glob('*.pdf'):
file_path = os.path.join(p, file).replace('\\','/')
init_pdf = pikepdf.open(file_path)
new_pdf = pikepdf.new()
new_pdf.pages.extend(init_pdf.pages)
new_pdf.save(str(file))
In pikepdf library it is impossible to overwrite the existing file by saving it with the same name. In contrast, you would like to copy the pages to the newly created empty pdf file, and save it.
I too faced the same problem of parsing the secured pdf but it has got resolved using pikepdf library. I tried this library on my jupyter notebbok and on windows os but it gave errors but it worked smoothly on Ubuntu
If you've forgotten the password to your PDF, below is a generic script which tries a LOT of password combinations on the same PDF. It uses pikepdf, but you can update the function check_password to use something else.
Usage example:
I used this when I had forgotten a password on a bank PDF. I knew that my bank always encrypts these kind of PDFs with the same password-structure:
Total length = 8
First 4 characters = an uppercase letter.
Last 4 characters = a number.
I call script as follows:
check_passwords(
pdf_file_path='/Users/my_name/Downloads/XXXXXXXX.pdf',
combination=[
ALPHABET_UPPERCASE,
ALPHABET_UPPERCASE,
ALPHABET_UPPERCASE,
ALPHABET_UPPERCASE,
NUMBER,
NUMBER,
NUMBER,
NUMBER,
]
)
Password-checking script:
(Requires Python3.8, with libraries numpy and pikepdf)
from typing import *
from itertools import product
import time, pikepdf, math, numpy as np
from pikepdf import PasswordError
ALPHABET_UPPERCASE: Sequence[str] = tuple('ABCDEFGHIJKLMNOPQRSTUVWXYZ')
ALPHABET_LOWERCASE: Sequence[str] = tuple('abcdefghijklmnopqrstuvwxyz')
NUMBER: Sequence[str] = tuple('0123456789')
def as_list(l):
if isinstance(l, (list, tuple, set, np.ndarray)):
l = list(l)
else:
l = [l]
return l
def human_readable_numbers(n, decimals: int = 0):
n = round(n)
if n < 1000:
return str(n)
names = ['', 'thousand', 'million', 'billion', 'trillion', 'quadrillion']
n = float(n)
idx = max(0,min(len(names)-1,
int(math.floor(0 if n == 0 else math.log10(abs(n))/3))))
return f'{n/10**(3*idx):.{decimals}f} {names[idx]}'
def check_password(pdf_file_path: str, password: str) -> bool:
## You can modify this function to use something other than pike pdf.
## This function should throw return True on success, and False on password-failure.
try:
pikepdf.open(pdf_file_path, password=password)
return True
except PasswordError:
return False
def check_passwords(pdf_file_path, combination, log_freq: int = int(1e4)):
combination = [tuple(as_list(c)) for c in combination]
print(f'Trying all combinations:')
for i, c in enumerate(combination):
print(f"{i}) {c}")
num_passwords: int = np.product([len(x) for x in combination])
passwords = product(*combination)
success: bool | str = False
count: int = 0
start: float = time.perf_counter()
for password in passwords:
password = ''.join(password)
if check_password(pdf_file_path, password=password):
success = password
print(f'SUCCESS with password "{password}"')
break
count += 1
if count % int(log_freq) == 0:
now = time.perf_counter()
print(f'Tried {human_readable_numbers(count)} ({100*count/num_passwords:.1f}%) of {human_readable_numbers(num_passwords)} passwords in {(now-start):.3f} seconds ({human_readable_numbers(count/(now-start))} passwords/sec). Latest password tried: "{password}"')
end: float = time.perf_counter()
msg: str = f'Tried {count} passwords in {1000*(end-start):.3f}ms ({count/(end-start):.3f} passwords/sec). '
msg += f"Correct password: {success}" if success is not False else f"All {num_passwords} passwords failed."
print(msg)
Comments
Obviously, don't use this to break into PDFs which are not your own. I hold no responsibility over how you use this script or any consequences of using it.
A lot of optimizations can be made.
Right now check_password uses pikepdf, which loads the file from disk for every "check". This is really slow, ideally it should run against an in-memory copy. I haven't figured out a way to do that, though.
You can probably speed this up a LOT by calling qpdf directly using C++, which is much better than Python for this kind of stuff.
I would avoid multi-processing here, since we're calling the same qpdf binary (which is normally a system-wide installation), which might become the bottleneck.
I'm trying to understand how does Linux encrypt our password on the etc/shadow file, so I've dont a new virtual 'test' user to make some test:
user: newuser
password: usrpw123
Generated salt: Ii4CGbr7
So the OS makes me the following line on the etc/shadow file, using a SHA512 encryptation system ($6$): newuser:$6$Ii4CGbr7$IOua8/oPV79Yp.BwzpxlSHjmCvRfTomZ.bhEvjZV2x5qhrvk82lZVrEtWQQej2pOWMdN7hvKwNgvCXKFQm5CB/:15069:0:99999:7:::
Now, I take the SHA512 module from python and try this:
import hashlib
m = hashlib.sha512()
m.update('Ii4CGbr7'+'usrpw123')
print m.hexdigest
This gives me the following hash as a result:
c73156daca3e31125ce457f1343201cc8a26400b2974440af2cc72687922b48b6631d21c186796ea2756ad987a996d2b261fe9ff3af4cc81e14c3029eac5df55
As you can see, it's different than the other one on the /etc/shadow file, and I dont know why if I'm using the same salt+password to generate the hash.
Can someone give me a hand and explain me more or less why this happens?
And also, why does the /etc/shadow files generates a hash with some dots (.)?
Thanks
The fields in /etc/shadow are not built or interpreted the way you think they are. You'll want to read the man page for details, but the most obvious difference is that it uses an unusual base64 encoding for both the salt and the hash.
There is an algorithm for generating the password hashes found in /etc/shadow.
See this document for an explanation:
http://www.akkadia.org/drepper/SHA-crypt.txt
There's an implementation of this in python here:
http://packages.python.org/passlib/lib/passlib.hash.sha512_crypt.html
I fell into the same trap as everything that I read lead me to believe you could retrieve the results the same way you have it written.
I was able to determine the password by using the salt and password using crypt.crypt()
import crypt
crypt.crypt(password, salt)
salt: $6$Ii4CGbr7
password: usrpw123
doesn't exactly use the hashlib library but it works.
I'm making a program in Python to be distributed to windows users via an installer.
The program needs to be able to download a file every day encrypted with the user's public key and then decrypt it.
So I need to find a Python library that will let me generate public and private PGP keys, and also decrypt files encrypted with the public key.
Is this something pyCrypto will do (documentation is nebulous)? Are there other pure Python libraries? How about a standalone command line tool in any language?
All I saw so far was GNUPG but installing that on Windows does stuff to the registry and throws dll's everywhere, and then I have to worry about whether the user already has this installed, how to backup their existing keyrings, etc. I'd rather just have a python library or command line tool and mange the keys myself.
Update: pyME might work but it doesn't seem to be compatible with Python 2.4 which I have to use.
You don't need PyCrypto or PyMe, fine though those packages may be - you will have all kinds of problems building under Windows. Instead, why not avoid the rabbit-holes and do what I did? Use gnupg 1.4.9. You don't need to do a full installation on end-user machines - just gpg.exe and iconv.dll from the distribution are sufficient, and you just need to have them somewhere in the path or accessed from your Python code using a full pathname. No changes to the registry are needed, and everything (executables and data files) can be confined to a single folder if you want.
There's a module GPG.py which was originally written by Andrew Kuchling, improved by Richard Jones and improved further by Steve Traugott. It's available here, but as-is it's not suitable for Windows because it uses os.fork(). Although originally part of PyCrypto, it is completely independent of the other parts of PyCrypto and needs only gpg.exe/iconv.dll in order to work.
I have a version (gnupg.py) derived from Traugott's GPG.py, which uses the subprocess module. It works fine under Windows, at least for my purposes - I use it to do the following:
Key management - generation, listing, export etc.
Import keys from an external source (e.g. public keys received from a partner company)
Encrypt and decrypt data
Sign and verify signatures
The module I've got is not ideal to show right now, because it includes some other stuff which shouldn't be there - which means I can't release it as-is at the moment. At some point, perhaps in the next couple of weeks, I hope to be able to tidy it up, add some more unit tests (I don't have any unit tests for sign/verify, for example) and release it (either under the original PyCrypto licence or a similar commercial-friendly license). If you can't wait, go with Traugott's module and modify it yourself - it wasn't too much work to make it work with the subprocess module.
This approach was a lot less painful than the others (e.g. SWIG-based solutions, or solutions which require building with MinGW/MSYS), which I considered and experimented with. I've used the same (gpg.exe/iconv.dll) approach with systems written in other languages, e.g. C#, with equally painless results.
P.S. It works with Python 2.4 as well as Python 2.5 and later. Not tested with other versions, though I don't foresee any problems.
After a LOT of digging, I found a package that worked for me. Although it is said to support the generation of keys, I didn't test it. However I did manage to decrypt a message that was encrypted using a GPG public key. The advantage of this package is that it does not require a GPG executable file on the machine, and is a Python based implementation of the OpenPGP (rather than a wrapper around the executable).
I created the private and public keys using GPG4win and kleopatra for windows
See my code below.
import pgpy
emsg = pgpy.PGPMessage.from_file(<path to the file from the client that was encrypted using your public key>)
key,_ = pgpy.PGPKey.from_file(<path to your private key>)
with key.unlock(<your private key passpharase>):
print (key.decrypt(emsg).message)
Although the question is very old. I hope this helps future users.
PyCrypto supports PGP - albeit you should test it to make sure that it works to your specifications.
Although documentation is hard to come by, if you look through Util/test.py (the module test script), you can find a rudimentary example of their PGP support:
if verbose: print ' PGP mode:',
obj1=ciph.new(password, ciph.MODE_PGP, IV)
obj2=ciph.new(password, ciph.MODE_PGP, IV)
start=time.time()
ciphertext=obj1.encrypt(str)
plaintext=obj2.decrypt(ciphertext)
end=time.time()
if (plaintext!=str):
die('Error in resulting plaintext from PGP mode')
print_timing(256, end-start, verbose)
del obj1, obj2
Futhermore, PublicKey/pubkey.py provides for the following relevant methods:
def encrypt(self, plaintext, K)
def decrypt(self, ciphertext):
def sign(self, M, K):
def verify (self, M, signature):
def can_sign (self):
"""can_sign() : bool
Return a Boolean value recording whether this algorithm can
generate signatures. (This does not imply that this
particular key object has the private information required to
to generate a signature.)
"""
return 1
PyMe does claim full compatibility with Python 2.4, and I quote:
The latest version of PyMe (as of this
writing) is v0.8.0. Its binary
distribution for Debian was compiled
with SWIG v1.3.33 and GCC v4.2.3 for
GPGME v1.1.6 and Python v2.3.5,
v2.4.4, and v2.5.2 (provided in
'unstable' distribution at the time).
Its binary distribution for Windows
was compiled with SWIG v1.3.29 and
MinGW v4.1 for GPGME v1.1.6 and Python
v2.5.2 (although the same binary get
installed and works fine in v2.4.2 as
well).
I'm not sure why you say "it doesn't seem to be compatible with Python 2.4 which I have to use" -- specifics please?
And yes it does exist as a semi-Pythonic (SWIGd) wrapper on GPGME -- that's a popular way to develop Python extensions once you have a C library that basically does the job.
PyPgp has a much simpler approach -- that's why it's a single, simple Python script: basically it does nothing more than "shell out" to command-line PGP commands. For example, decryption is just:
def decrypt(data):
"Decrypt a string - if you have the right key."
pw,pr = os.popen2('pgpv -f')
pw.write(data)
pw.close()
ptext = pr.read()
return ptext
i.e., write the encrypted cyphertext to the standard input of pgpv -f, read pgpv's standard output as the decrypted plaintext.
PyPgp is also a very old project, though its simplicity means that making it work with modern Python (e.g., subprocess instead of now-deprecated os.popen2) would not be hard. But you still do need PGP installed, or PyPgp won't do anything;-).
M2Crypto has PGP module, but I have actually never tried to use it. If you try it, and it works, please let me know (I am the current M2Crypto maintainer). Some links:
Module sources
Demo Script
unit tests
Update: The PGP module does not provide ways to generate keys, but presumably these could be created with the lower level RSA, DSA etc. modules. I don't know PGP insides, so you'd have to dig up the details. Also, if you know how to generate these using openssl command line commands, it should be reasonably easy to convert that to M2Crypto calls.
As other have noted, PyMe is the canonical solution for this, since it's based on GpgME, which is part of the GnuPG ecosystem.
For Windows, I strongly recommend to use Gpg4win as the GnuPG distribution, for two reasons:
It's based on GnuPG 2, which, among other things, includes gpg2.exe, which can (finally, I might add :) start gpg-agent.exe on-demand (gpg v1.x can't).
And secondly, it's the only official Windows build by the GnuPG developers. E.g. it's entirely cross-compiled from Linux to Windows, so not a iota of non-free software was used in preparing it (quite important for a security suite :).
To sign with only the exported public key file without a keyring.
With PGPy 0.5.2 (pure Python GPG RFC implementation):
key_fpath = './recipient-PUB.gpg'
rsa_pub, _ = pgpy.PGPKey.from_file(rkey_fpath)
rkey = rsa_pub.subkeys.values()[0]
text_message = pgpy.PGPMessage.new('my msg')
encrypted_message = rkey.encrypt(text_message)
print encrypted_message.__bytes__()
With gpg 1.10.0 (gpgme Python bindings - former PyME):
rkey_fpath = './recipient-PUB.gpg'
cg = gpg.Context()
rkey = list(cg.keylist(source = rkey_fpath))
ciphertext, result, sign_result = cg.encrypt('my msg', recipients=rkey, sign=False, always_trust=True)
print ciphertext
A simple benchmark in a for loop shows me that for this simple operation on my system PGPy is ~3x time faster than gpgme Python bindings (please do not take this statement as X is faster than Y: I will invite you to test in your environment).
Here's a full script that will:
Attempt to decrypt all the files in a given folder that were encrypted with your public key.
Write the new files to a specified folder.
Move the encrypted files to a specified folder.
The script also has everything you need to create and store your own private and public keys, check out the "First time set up" section below.
The idea is that you can schedule this script to run as often as you like, and it'll automatically decrypt data found and store it for you.
I hope this helps someone, this was a tricky project to figure out.
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ Introduction, change log and table of contents
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Purpose: This script is used to decrypt files that are passed to us from ICF.
#
# Change date Changed by Description
# 2022-10-03 Ryan Bradley Initial draft
# 2022-10-12 Ryan Bradley Cleaned up some comments and table of contents.
#
# Table of Contents
# [1.0] Hard-coded variables
# [1.1] Load packages and custom functions
# [1.3] First time set up
# [1.4] Define custom functions
# [2.0] Load keys and decrypt files
#
# Sources used to create this script, and for further reading:
# https://github.com/SecurityInnovation/PGPy/
# https://stackoverflow.com/questions/1020320/how-to-do-pgp-in-python-generate-keys-encrypt-decrypt
# https://pypi.org/project/PGPy/
# https://betterprogramming.pub/creating-a-pgp-encryption-tool-with-python-19bae51b7fd
# https://pgpy.readthedocs.io/en/latest/examples.html
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ [1.1] Load packages
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import glob
import pgpy
import shutil
import io
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ [1.2] Hard-coded variables
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Define the paths to public and private keys
path_public_key = r'YOUR PATH HERE'
path_private_key = r'YOUR PATH HERE'
# Define paths to files you want to try decrypting
path_original_files = r'YOUR PATH HERE'
path_decrypted_files = r'YOUR PATH HERE'
path_encrypted_files= r'YOUR PATH HERE'
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ [1.3] First time set up
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# IMPORTANT WARNINGS!!!!
# - Do NOT share your private key with anyone else.
# - You MUST have the associated private key that is is generated along with a public key
# if you want to be able to decrypt anything that is encryped with that public key. Do
# not overwrite the existing keys unless you will never need any of the previously
# encryped data.
# - Do not generate new public and private keys unless you have a good reason to.
#
# The following steps will walk you through how to create and write public and private keys to
# a network location. Be very careful where you store this information. Anyone with access
# to your private key can decrypt anything that was encryped with your public key.
#
# These steps only need to be performed one time when the script is first being
# created. They are commented out intentionally, as they shouldn't need to be performed
# every time the script is ran.
#
# Here's the a link to the documentation on this topic:
# https://pgpy.readthedocs.io/en/latest/examples.html
# # Load the extra things we need to define a new key
# from pgpy.constants import PubKeyAlgorithm, KeyFlags, HashAlgorithm, SymmetricKeyAlgorithm, CompressionAlgorithm
# # Gerate a new a primary key. For this example, we'll use RSA, but it could be DSA or ECDSA as well
# key = pgpy.PGPKey.new(PubKeyAlgorithm.RSAEncryptOrSign, 4096)
# # Define a new user
# uid = pgpy.PGPUID.new('SA_CODA_Admin', comment='Customer Strategy and Data Analytics service account.', email='CustomerDataAnalytics#cmsenergy.com')
# # Add the new user id to the key, and define all the key preferences.
# key.add_uid(uid, usage={KeyFlags.Sign, KeyFlags.EncryptCommunications, KeyFlags.EncryptStorage},
# hashes=[HashAlgorithm.SHA256, HashAlgorithm.SHA384, HashAlgorithm.SHA512, HashAlgorithm.SHA224],
# ciphers=[SymmetricKeyAlgorithm.AES256, SymmetricKeyAlgorithm.AES192, SymmetricKeyAlgorithm.AES128],
# compression=[CompressionAlgorithm.ZLIB, CompressionAlgorithm.BZ2, CompressionAlgorithm.ZIP, CompressionAlgorithm.Uncompressed]
# , is_compressed = True)
# # Write the ASCII armored public key to a network location.
# text_file = open(path_public_key, 'w')
# text_file.write(str(key.pubkey))
# text_file.close()
# # Write the ASCII armored private key to a network location.
# text_file = open(path_private_key, 'w')
# text_file.write(str(key))
# text_file.close()
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ [1.4] Define custom functions
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def file_encrypt(path_original_file, path_encrypted_file, key_public):
"""
A function that encrypts the content of a file at the given path and
creates an ecryped version file at the new location using the specified
public key.
"""
# Create a PGP file, compressed with ZIP DEFLATE by default unless otherwise specified
pgp_file = pgpy.PGPMessage.new(path_original_file, file=True)
# Encrypt the data with the public key
encrypted_data = key_public.encrypt(pgp_file)
# Write the encryped data to the encrypted destination
text_file = open(path_encrypted_file, 'w')
text_file.write(str(encrypted_data))
text_file.close()
def file_decrypt(path_encrypted_file, path_decrypted_file, key_private):
"""
A function that decrypts the content of a file at path path and
creates a decrypted file at the new location using the given
private key.
"""
# Load a previously encryped message from a file
pgp_file = pgpy.PGPMessage.from_file(path_encrypted_file)
# Decrypt the data with the given private key
decrypted_data = key_private.decrypt(pgp_file).message
# Read in the bytes of the decrypted data
toread = io.BytesIO()
toread.write(bytes(decrypted_data))
toread.seek(0) # reset the pointer
# Write the data to the location
with open(path_decrypted_file, 'wb') as f:
shutil.copyfileobj(toread, f)
f.close()
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#~~ [2.0] Load keys and decrypt files
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Load your pre-generated public key from the network
key_public, _ = pgpy.PGPKey.from_file(path_public_key)
# Load your pre-generated public key from the network
key_private, _ = pgpy.PGPKey.from_file(path_private_key)
# Find and process any encrypted files in the landing folder
for file in glob.glob(path_original_files + '\*.pgp'):
# Get the path to the file we need to decrypt
path_encrypted_file = str(file)
# Extract the file name
parts = path_encrypted_file.split('\\')
str_file_name = parts[len(parts)-1]
str_clean_file_name = str_file_name[:-4]
# Extract the file exension
str_extension = str_clean_file_name.split('.')
str_extension = str_extension[len(str_extension) - 1]
# Create the path to the new decryped file, dropping the ".pgp" extension
path_decrypted_file = path_decrypted_files + '\\' + str_clean_file_name
# Create the path to the place we'll store the encryped file
path_archived_encrypted_file = path_encrypted_files + '\\' + str_file_name
# Decrypt the file
try:
file_decrypt(path_encrypted_file, path_decrypted_file, key_private)
# Move the encryped file to its new location
shutil.move(path_encrypted_file, path_archived_encrypted_file)
except:
print('DECRYPTION ERROR!')
print(f'Unable to decrypt {path_encrypted_file}')
# Just for reference, here's how you would call the function to encrypt a file:
# file_encrypt(path_original_file, path_encrypted_file, key_public)