I'm trying to achieve the functionality provided by the following bash command in Python.
echo "$DATA" | gpg --symmetric --armor --batch --passphrase "${KEY}"
So far I've tried to use subprocess but am having difficulty passing in the data. I tried giving it as a command in the list of parameters to send to subprocess but that just effectively echos the the entire thing.
cmd = f"| gpg --symmetric --armor --batch --passphrase {key}".split()
temp = ["echo", f"\"{data}\""]
temp.extend(cmd)
res = subprocess.run(temp, stdout=subprocess.PIPE, universal_newlines=True)
encrypted = res.stdout.strip()
I'm also interested in using the python-gnupg module but have not yet figured out how to replicate the above with it either.
Thanks in advance for any help!
You can use the input argument to run()/check_output():
from getpass import getpass
import subprocess
key = getpass("KEY: ")
data = b'Symmetric Encryption with GPG and Subprocess'
command = ["gpg", "--symmetric", "--armor", "--batch", "--passphrase", key]
out = subprocess.check_output(command, input=data, universal_newlines=False)
Note that GNU echo will, by default, append a newline. Use echo -n to not print the trailing \n. Either way, you'll want to be careful to mimic this in Python.
In case anyone was wondering, I also got the python-gnupg module to work for my application. I am sticking with the subprocess answer since that reduces dependencies but wanted to share this as well.
gpg = gnupg.GPG()
encrypted = str(gpg.encrypt(data, recipients=None, symmetric=True, passphrase=key, extra_args=["--batch"]))
The python-gnupg module has a long history with serious security flaws, many of which it's more likely to be affected by due to the decision to use subproess to call an external binary executable.
Instead, the recommendation of the GnuPG Project is to use the CPython bindings for the GPGME C API, which ship with the GPGME source code.
import gpg
from getpass import getpass
key = getpass("KEY: ")
c = gpg.Context(armor=True)
data = b"Symmetric encryption with GPGME."
ciphertext, result, sign_result = c.encrypt(data, sign=False, passphrase=key)
with open("some_file.txt.asc", "wb") as f:
f.write(ciphertext)
Because this uses symmetric encryption, there won't be a digital signature included and there are no recipient keys to check for. Which means both result and sign_result will return None. Only ciphertext contains anything and that's the ASCII armoured excrypted data, which can be written to a file as above, or you can do something else with it.
The documentation for this, far superior, module is included with the GPGME source, but an online draft version is available here.
Related
I have got a python script which is creating an ODBC connection. The ODBC connection is generated with a connection string. In this connection string I have to include the username and password for this connection.
Is there an easy way to obscure this password in the file (just that nobody can read the password when I'm editing the file) ?
Base64 encoding is in the standard library and will do to stop shoulder surfers:
>>> import base64
>>> print(base64.b64encode("password".encode("utf-8")))
cGFzc3dvcmQ=
>>> print(base64.b64decode("cGFzc3dvcmQ=").decode("utf-8"))
password
Here is a simple method:
Create a python module - let's call it peekaboo.py.
In peekaboo.py, include both the password and any code needing that password
Create a compiled version - peekaboo.pyc - by importing this module (via python commandline, etc...).
Now, delete peekaboo.py.
You can now happily import peekaboo relying only on peekaboo.pyc. Since peekaboo.pyc is byte compiled it is not readable to the casual user.
This should be a bit more secure than base64 decoding - although it is vulnerable to a py_to_pyc decompiler.
Douglas F Shearer's is the generally approved solution in Unix when you need to specify a password for a remote login.
You add a --password-from-file option to specify the path and read plaintext from a file.
The file can then be in the user's own area protected by the operating system.
It also allows different users to automatically pick up their own own file.
For passwords that the user of the script isn't allowed to know - you can run the script with elavated permission and have the password file owned by that root/admin user.
If you are working on a Unix system, take advantage of the netrc module in the standard Python library. It reads passwords from a separate text file (.netrc), which has the format decribed here.
Here is a small usage example:
import netrc
# Define which host in the .netrc file to use
HOST = 'mailcluster.loopia.se'
# Read from the .netrc file in your home directory
secrets = netrc.netrc()
username, account, password = secrets.authenticators( HOST )
print username, password
How about importing the username and password from a file external to the script? That way even if someone got hold of the script, they wouldn't automatically get the password.
The best solution, assuming the username and password can't be given at runtime by the user, is probably a separate source file containing only variable initialization for the username and password that is imported into your main code. This file would only need editing when the credentials change. Otherwise, if you're only worried about shoulder surfers with average memories, base 64 encoding is probably the easiest solution. ROT13 is just too easy to decode manually, isn't case sensitive and retains too much meaning in it's encrypted state. Encode your password and user id outside the python script. Have he script decode at runtime for use.
Giving scripts credentials for automated tasks is always a risky proposal. Your script should have its own credentials and the account it uses should have no access other than exactly what is necessary. At least the password should be long and rather random.
base64 is the way to go for your simple needs. There is no need to import anything:
>>> 'your string'.encode('base64')
'eW91ciBzdHJpbmc=\n'
>>> _.decode('base64')
'your string'
A way that I have done this is as follows:
At the python shell:
>>> from cryptography.fernet import Fernet
>>> key = Fernet.generate_key()
>>> print(key)
b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
>>> cipher = Fernet(key)
>>> password = "thepassword".encode('utf-8')
>>> token = cipher.encrypt(password)
>>> print(token)
b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
Then, create a module with the following code:
from cryptography.fernet import Fernet
# you store the key and the token
key = b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
token = b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
# create a cipher and decrypt when you need your password
cipher = Fernet(key)
mypassword = cipher.decrypt(token).decode('utf-8')
Once you've done this, you can either import mypassword directly or you can import the token and cipher to decrypt as needed.
Obviously, there are some shortcomings to this approach. If someone has both the token and the key (as they would if they have the script), they can decrypt easily. However it does obfuscate, and if you compile the code (with something like Nuitka) at least your password won't appear as plain text in a hex editor.
for python3 obfuscation using base64 is done differently:
import base64
base64.b64encode(b'PasswordStringAsStreamOfBytes')
which results in
b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM='
note the informal string representation, the actual string is in quotes
and decoding back to the original string
base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
b'PasswordStringAsStreamOfBytes'
to use this result where string objects are required the bytes object can be translated
repr = base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
secret = repr.decode('utf-8')
print(secret)
for more information on how python3 handles bytes (and strings accordingly) please see the official documentation.
This is a pretty common problem. Typically the best you can do is to either
A) create some kind of ceasar cipher function to encode/decode (just not rot13)
or
B) the preferred method is to use an encryption key, within reach of your program, encode/decode the password. In which you can use file protection to protect access the key.
Along those lines if your app runs as a service/daemon (like a webserver) you can put your key into a password protected keystore with the password input as part of the service startup. It'll take an admin to restart your app, but you will have really good pretection for your configuration passwords.
Your operating system probably provides facilities for encrypting data securely. For instance, on Windows there is DPAPI (data protection API). Why not ask the user for their credentials the first time you run then squirrel them away encrypted for subsequent runs?
Here is my snippet for such thing. You basically import or copy the function to your code. getCredentials will create the encrypted file if it does not exist and return a dictionaty, and updateCredential will update.
import os
def getCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
try:
with open(directory+'\\Credentials.txt', 'r') as file:
cred = file.read()
file.close()
except:
print('I could not file the credentials file. \nSo I dont keep asking you for your email and password everytime you run me, I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
return {'lanid':base64.b64decode(bytes(cred.split(splitter)[0], encoding='utf-8')).decode('utf-8'),
'email':base64.b64decode(bytes(cred.split(splitter)[1], encoding='utf-8')).decode('utf-8'),
'password':base64.b64decode(bytes(cred.split(splitter)[2], encoding='utf-8')).decode('utf-8')}
def updateCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
print('I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
cred = getCredentials()
updateCredentials()
Place the configuration information in a encrypted config file. Query this info in your code using an key. Place this key in a separate file per environment, and don't store it with your code.
More homegrown appraoch rather than converting authentication / passwords / username to encrytpted details. FTPLIB is just the example.
"pass.csv" is the csv file name
Save password in CSV like below :
user_name
user_password
(With no column heading)
Reading the CSV and saving it to a list.
Using List elelments as authetntication details.
Full code.
import os
import ftplib
import csv
cred_detail = []
os.chdir("Folder where the csv file is stored")
for row in csv.reader(open("pass.csv","rb")):
cred_detail.append(row)
ftp = ftplib.FTP('server_name',cred_detail[0][0],cred_detail[1][0])
Do you know pit?
https://pypi.python.org/pypi/pit (py2 only (version 0.3))
https://github.com/yoshiori/pit (it will work on py3 (current version 0.4))
test.py
from pit import Pit
config = Pit.get('section-name', {'require': {
'username': 'DEFAULT STRING',
'password': 'DEFAULT STRING',
}})
print(config)
Run:
$ python test.py
{'password': 'my-password', 'username': 'my-name'}
~/.pit/default.yml:
section-name:
password: my-password
username: my-name
If running on Windows, you could consider using win32crypt library. It allows storage and retrieval of protected data (keys, passwords) by the user that is running the script, thus passwords are never stored in clear text or obfuscated format in your code. I am not sure if there is an equivalent implementation for other platforms, so with the strict use of win32crypt your code is not portable.
I believe the module can be obtained here: http://timgolden.me.uk/pywin32-docs/win32crypt.html
You could also consider the possibility of storing the password outside the script, and supplying it at runtime
e.g. fred.py
import os
username = 'fred'
password = os.environ.get('PASSWORD', '')
print(username, password)
which can be run like
$ PASSWORD=password123 python fred.py
fred password123
Extra layers of "security through obscurity" can be achieved by using base64 (as suggested above), using less obvious names in the code and further distancing the actual password from the code.
If the code is in a repository, it is often useful to store secrets outside it, so one could add this to ~/.bashrc (or to a vault, or a launch script, ...)
export SURNAME=cGFzc3dvcmQxMjM=
and change fred.py to
import os
import base64
name = 'fred'
surname = base64.b64decode(os.environ.get('SURNAME', '')).decode('utf-8')
print(name, surname)
then re-login and
$ python fred.py
fred password123
Why not have a simple xor?
Advantages:
looks like binary data
noone can read it without knowing the key (even if it's a single char)
I get to the point where I recognize simple b64 strings for common words and rot13 as well. Xor would make it much harder.
There are several ROT13 utilities written in Python on the 'Net -- just google for them. ROT13 encode the string offline, copy it into the source, decode at point of transmission.But this is really weak protection...
This doesn't precisely answer your question, but it's related. I was going to add as a comment but wasn't allowed.
I've been dealing with this same issue, and we have decided to expose the script to the users using Jenkins. This allows us to store the db credentials in a separate file that is encrypted and secured on a server and not accessible to non-admins.
It also allows us a bit of a shortcut to creating a UI, and throttling execution.
import base64
print(base64.b64encode("password".encode("utf-8")))
print(base64.b64decode(b'cGFzc3dvcmQ='.decode("utf-8")))
I don't understand what I'm doing wrong here. As far as I can tell from the python gnupg documentation this encrypt_file call should have content like the encrypt call, but everything I'm trying is resulting in a GPG blob that decrypts to a blank file. No error, no indication that there was any problem with the operation, just... emptyness, like it was compiled with sartre=1...
Am I missing some key to the incantation?
Python 3.6.5,
python-gnupg==0.4.7
from gnupg import GPG
from io import StringIO
gpg = GPG(gpgbinary='/usr/local/bin/gpg', gnupghome='/tmp/gpg/')
keyserver = <insert keyserver URI>
fp = <insert GPG key fingerprint>
gpg.recv_keys(keyserver, fp)
sio = StringIO('\n'.join([f"Hello world {n}" for n in range(120)]))
c0 = gpg.encrypt(sio.getvalue(), recipients=fp)
len(str(c0))
1252
c1 = gpg.encrypt_file(sio, recipients=fp)
len(str(c1))
858
sio = StringIO('\n'.join([f"Hello world {n}" for n in range(1200)]))
c0 = gpg.encrypt(sio.getvalue(), recipients=fp)
len(str(c0))
4559
c1 = gpg.encrypt_file(sio, recipients=fp)
len(str(c1))
858
Seems the following were the operative issues:
The file-like object, it seems, has to be binary; e.g. a BytesIO or a file pointer opened in binary mode.
You have to seek(0) to write the full content of the file-like object. (Kind of obvious but I overlooked it; I probably kept checking encrypt() first then encrypt_file() without seeking back to the start. It probably didn't help that this doesn't make it work on a StringIO, so I may have mentally ruled it out as the issue early on.)
I'm able to embed the original filename using python-gnupg when encrypting a file using below:
filename = 'test.tim'
with open(filename, 'rb) as fh:
status = gpg.encrypt_file(fh, recipients='somerecipient', output='test.tim.gpg',\
sign='somesignature', extra_args=['--set-filename', os.path.basename(filename)])
This can be verified by using gpg from the command line:
$ gpg2 --list-packets test.tim.gpg | grep name
I am however unable to preserve the original filename when decrypting the file:
with open(filename, 'rb') as fh:
status = gpg.decrypt_file(fh, extra_args['--use-embedded-filename'])
I am aware about the output parameter (which specifies the filename to save contents to) in the decrypt_file function, but i want to preserve the original filename (which i won't always know)
It seems the decrypt_file function always passes the --decrypt flag to gpg which always ouputs the contents to stdout (unless used in conjunction with output parameter) as in:
$ gpg --decrypt --use-embedded-filename test.tim.gpg
Below command will decrypt and save output to original filename:
$ gpg --use-embedded-filename test.tim.gpg
Any ideas?
Tim
The functionality to do what you want doesn't exist in the original python-gnupg.
There's a modified version here by isislovecruft (which is what you get if you pip install gnupg) that adds support for --list-packets with gpg.listpackets but still doesn't support --use-embeded-file-name
So my approach, if I were to insist on using python only, would probably be to start with isislovecruft's version and then subclass GPG like this:
import gnupg
import os
GPGBINARY = os.environ.get('GPGBINARY', 'gpg')
hd = os.path.join(os.getcwd(), 'keys')
class myGPG(gnupg.GPG):
def decrypt_file_original_name(self, file, always_trust=False, passphrase=None, extra_args=None):
args = ["--use-embedded-filename"]
output = calculate_the_file_name_using_list_packets()
self.set_output_without_confirmation(args, output)
if always_trust: # pragma: no cover
args.append("--always-trust")
if extra_args:
args.extend(extra_args)
result = self.result_map['crypt'](self)
self._handle_io(args, file, result, passphrase, binary=True)
# logger.debug('decrypt result: %r', result.data)
return result
gpg = myGPG(gnupghome=hd, gpgbinary=GPGBINARY)
Bear in mind, at this point it is almost certainly much easier to just use subprocess and run the gpg binary directly from the shell, especially as you don't care about the output.
Anyway, I got this far and have run out of time for now, so I leave implementing calculate_the_file_name_using_list_packets up to you, if you choose to go the 'pure python' route. Hopefully it is a bit easier now you have gpg.list-packets. Good luck!
I am creating an ZIP file with ZipFile in Python 2.5, it works OK so far:
import zipfile, os
locfile = "test.txt"
loczip = os.path.splitext (locfile)[0] + ".zip"
zip = zipfile.ZipFile (loczip, "w")
zip.write (locfile)
zip.close()
But I couldn't find how to encrypt the files in the ZIP file.
I could use system and call PKZIP -s, but I suppose there must be a more "Pythonic" way. I'm looking for an open source solution.
I created a simple library to create a password encrypted zip file in python. - here
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt", "dst.zip", "password", compression_level)
The library requires zlib.
I have checked that the file can be extracted in WINDOWS/MAC.
This thread is a little bit old, but for people looking for an answer to this question in 2020/2021.
Look at pyzipper
A 100% API compatible replacement for Python’s zipfile that can read and write AES encrypted zip files.
7-zip is also a good choice, but if you do not want to use subprocess, go with pyzipper...
The duplicate question: Code to create a password encrypted zip file? has an answer that recommends using 7z instead of zip. My experience bears this out.
Copy/pasting the answer by #jfs here too, for completeness:
To create encrypted zip archive (named 'myarchive.zip') using open-source 7-Zip utility:
rc = subprocess.call(['7z', 'a', '-mem=AES256', '-pP4$$W0rd', '-y', 'myarchive.zip'] +
['first_file.txt', 'second.file'])
To install 7-Zip, type:
$ sudo apt-get install p7zip-full
To unzip by hand (to demonstrate compatibility with zip utility), type:
$ unzip myarchive.zip
And enter P4$$W0rd at the prompt.
Or the same in Python 2.6+:
>>> zipfile.ZipFile('myarchive.zip').extractall(pwd='P4$$W0rd')
pyminizip works great in creating a password protected zip file. For unziping ,it fails at some situations. Tested on python 3.7.3
Here, i used pyminizip for encrypting the file.
import pyminizip
compression_level = 5 # 1-9
pyminizip.compress("src.txt",'src', "dst.zip", "password", compression_level)
For unzip, I used zip file module:
from zipfile import ZipFile
with ZipFile('/home/paulsteven/dst.zip') as zf:
zf.extractall(pwd=b'password')
You can use pyzipper for this task and it will work great when you want to encrypt a zip file or generate a protected zip file.
pip install pyzipper
import pyzipper
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA,
encryption=pyzipper.WZ_AES) as zf:
zf.setpassword(secret_password)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
The strength of the AES encryption can be configure to be 128, 192 or 256 bits. By default it is 256 bits. Use the setencryption() method to specify the encryption kwargs:
def encrypt_():
secret_password = b'your password'
with pyzipper.AESZipFile('new_test.zip',
'w',
compression=pyzipper.ZIP_LZMA) as zf:
zf.setpassword(secret_password)
zf.setencryption(pyzipper.WZ_AES, nbits=128)
zf.writestr('test.txt', "What ever you do, don't tell anyone!")
with pyzipper.AESZipFile('new_test.zip') as zf:
zf.setpassword(secret_password)
my_secrets = zf.read('test.txt')
Official Python ZipFile documentation is available here: https://docs.python.org/3/library/zipfile.html
#tripleee's answer helped me, see my test below.
This code works for me on python 3.5.2 on Windows 8.1 ( 7z path added to system).
rc = subprocess.call(['7z', 'a', output_filename + '.zip', '-mx9', '-pSecret^)'] + [src_folder + '/'])
With two parameters:
-mx9 means max compression
-pSecret^) means password is Secret^). ^ is escape for ) for Windows OS, but when you unzip, it will need type in the ^.
Without ^ Windows OS will not apply the password when 7z.exe creating the zip file.
Also, if you want to use -mhe switch, you'll need the file format to be in 7z instead of zip.
I hope that may help.
2022 answer:
I believe this is an utterly mundane task and therefore should be oneliner. I abstracted away all the frevolous details in a library that is as powerfull as a bash terminal.
from crocodile.toolbox import Path
file = Path(r'my_string_path')
result_file = file.zip(pwd="lol", use_7z=True)
when the 7z flag is raised, it gets called behind the scenes.
You don't need to learn 7z command line syntax.
You don't need to worry about installing 7z, does that automatically if it's not installed. (tested on windows so far)
You can use the Chilkat library. It's commercial, but has a free evaluation and seems pretty nice.
Here's an example I got from here:
import chilkat
# Demonstrates how to create a WinZip-compatible 128-bit AES strong encrypted zip
zip = chilkat.CkZip()
zip.UnlockComponent("anything for 30-day trial")
zip.NewZip("strongEncrypted.zip")
# Set the Encryption property = 4, which indicates WinZip compatible AES encryption.
zip.put_Encryption(4)
# The key length can be 128, 192, or 256.
zip.put_EncryptKeyLength(128)
zip.SetPassword("secret")
zip.AppendFiles("exampleData/*",True)
zip.WriteZip()
I am trying to copy the contents of a variable to the clipboard automatically within a python script. So, a variable is created that holds a string, and I'd like to copy that string to the clipboard.
Is there a way to do this with Pyclips or
os.system("echo '' | pbcopy")
I've tried passing the variable where the string should go, but that doesn't work which makes sense to me.
Have you tried this?
import os
def addToClipBoard(text):
command = 'echo ' + text.strip() + '| clip'
os.system(command)
Read more solutions here.
Edit:
You may call it as:
addToClipBoard(your_variable)
Since you mentioned PyCLIPS, it sounds like 3rd-party packages are on the table. Let me thrown a recommendation for pyperclip. Full documentation can be found on GitHub, but here's an example:
import pyperclip
variable = 'Some really "complex" string with\na bunch of stuff in it.'
pyperclip.copy(variable)
While the os.system(...'| pbcopy') examples are also good, they could give you trouble with complex strings and pyperclip provides the same API cross-platform.
The accepted answer was not working for me as the output had quotes, apostrophes and $$ signs which were interpreted and replaced by the shell.
I've improved the function based on answer. This solution uses temporary file instead of echoing the string in the shell.
def add_to_clipboard(text):
import tempfile
with tempfile.NamedTemporaryFile("w") as fp:
fp.write(text)
fp.flush()
command = "pbcopy < {}".format(fp.name)
os.system(command)
Replace the pbcopy with clip for Windows or xclip for Linux.
For X11 (Unix/Linux):
os.system('echo "%s" | xsel -i' % variable)
xsel also gives you a choice of writing to:
the primary selection (default)
the secondary selection (-s option), or
the clipboard (-b option).
If xsel doesn't work as you expect, it is probably because you are using the wrong selection/clipboard.
In addition, with the -a option, you can append to the clipboard instead of overwrite. With -c, the clipboard is cleared.
Improvement
The module subprocess provides a more secure way to do the same thing:
from subprocess import Popen, PIPE
Popen(('xsel', '-i'), stdin=PIPE).communicate(variable)