I have got a python script which is creating an ODBC connection. The ODBC connection is generated with a connection string. In this connection string I have to include the username and password for this connection.
Is there an easy way to obscure this password in the file (just that nobody can read the password when I'm editing the file) ?
Base64 encoding is in the standard library and will do to stop shoulder surfers:
>>> import base64
>>> print(base64.b64encode("password".encode("utf-8")))
cGFzc3dvcmQ=
>>> print(base64.b64decode("cGFzc3dvcmQ=").decode("utf-8"))
password
Here is a simple method:
Create a python module - let's call it peekaboo.py.
In peekaboo.py, include both the password and any code needing that password
Create a compiled version - peekaboo.pyc - by importing this module (via python commandline, etc...).
Now, delete peekaboo.py.
You can now happily import peekaboo relying only on peekaboo.pyc. Since peekaboo.pyc is byte compiled it is not readable to the casual user.
This should be a bit more secure than base64 decoding - although it is vulnerable to a py_to_pyc decompiler.
Douglas F Shearer's is the generally approved solution in Unix when you need to specify a password for a remote login.
You add a --password-from-file option to specify the path and read plaintext from a file.
The file can then be in the user's own area protected by the operating system.
It also allows different users to automatically pick up their own own file.
For passwords that the user of the script isn't allowed to know - you can run the script with elavated permission and have the password file owned by that root/admin user.
If you are working on a Unix system, take advantage of the netrc module in the standard Python library. It reads passwords from a separate text file (.netrc), which has the format decribed here.
Here is a small usage example:
import netrc
# Define which host in the .netrc file to use
HOST = 'mailcluster.loopia.se'
# Read from the .netrc file in your home directory
secrets = netrc.netrc()
username, account, password = secrets.authenticators( HOST )
print username, password
How about importing the username and password from a file external to the script? That way even if someone got hold of the script, they wouldn't automatically get the password.
The best solution, assuming the username and password can't be given at runtime by the user, is probably a separate source file containing only variable initialization for the username and password that is imported into your main code. This file would only need editing when the credentials change. Otherwise, if you're only worried about shoulder surfers with average memories, base 64 encoding is probably the easiest solution. ROT13 is just too easy to decode manually, isn't case sensitive and retains too much meaning in it's encrypted state. Encode your password and user id outside the python script. Have he script decode at runtime for use.
Giving scripts credentials for automated tasks is always a risky proposal. Your script should have its own credentials and the account it uses should have no access other than exactly what is necessary. At least the password should be long and rather random.
base64 is the way to go for your simple needs. There is no need to import anything:
>>> 'your string'.encode('base64')
'eW91ciBzdHJpbmc=\n'
>>> _.decode('base64')
'your string'
A way that I have done this is as follows:
At the python shell:
>>> from cryptography.fernet import Fernet
>>> key = Fernet.generate_key()
>>> print(key)
b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
>>> cipher = Fernet(key)
>>> password = "thepassword".encode('utf-8')
>>> token = cipher.encrypt(password)
>>> print(token)
b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
Then, create a module with the following code:
from cryptography.fernet import Fernet
# you store the key and the token
key = b'B8XBLJDiroM3N2nCBuUlzPL06AmfV4XkPJ5OKsPZbC4='
token = b'gAAAAABe_TUP82q1zMR9SZw1LpawRLHjgNLdUOmW31RApwASzeo4qWSZ52ZBYpSrb1kUeXNFoX0tyhe7kWuudNs2Iy7vUwaY7Q=='
# create a cipher and decrypt when you need your password
cipher = Fernet(key)
mypassword = cipher.decrypt(token).decode('utf-8')
Once you've done this, you can either import mypassword directly or you can import the token and cipher to decrypt as needed.
Obviously, there are some shortcomings to this approach. If someone has both the token and the key (as they would if they have the script), they can decrypt easily. However it does obfuscate, and if you compile the code (with something like Nuitka) at least your password won't appear as plain text in a hex editor.
for python3 obfuscation using base64 is done differently:
import base64
base64.b64encode(b'PasswordStringAsStreamOfBytes')
which results in
b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM='
note the informal string representation, the actual string is in quotes
and decoding back to the original string
base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
b'PasswordStringAsStreamOfBytes'
to use this result where string objects are required the bytes object can be translated
repr = base64.b64decode(b'UGFzc3dvcmRTdHJpbmdBc1N0cmVhbU9mQnl0ZXM=')
secret = repr.decode('utf-8')
print(secret)
for more information on how python3 handles bytes (and strings accordingly) please see the official documentation.
This is a pretty common problem. Typically the best you can do is to either
A) create some kind of ceasar cipher function to encode/decode (just not rot13)
or
B) the preferred method is to use an encryption key, within reach of your program, encode/decode the password. In which you can use file protection to protect access the key.
Along those lines if your app runs as a service/daemon (like a webserver) you can put your key into a password protected keystore with the password input as part of the service startup. It'll take an admin to restart your app, but you will have really good pretection for your configuration passwords.
Your operating system probably provides facilities for encrypting data securely. For instance, on Windows there is DPAPI (data protection API). Why not ask the user for their credentials the first time you run then squirrel them away encrypted for subsequent runs?
Here is my snippet for such thing. You basically import or copy the function to your code. getCredentials will create the encrypted file if it does not exist and return a dictionaty, and updateCredential will update.
import os
def getCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
try:
with open(directory+'\\Credentials.txt', 'r') as file:
cred = file.read()
file.close()
except:
print('I could not file the credentials file. \nSo I dont keep asking you for your email and password everytime you run me, I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
return {'lanid':base64.b64decode(bytes(cred.split(splitter)[0], encoding='utf-8')).decode('utf-8'),
'email':base64.b64decode(bytes(cred.split(splitter)[1], encoding='utf-8')).decode('utf-8'),
'password':base64.b64decode(bytes(cred.split(splitter)[2], encoding='utf-8')).decode('utf-8')}
def updateCredentials():
import base64
splitter='<PC+,DFS/-SHQ.R'
directory='C:\\PCT'
if not os.path.exists(directory):
os.makedirs(directory)
print('I will be saving an encrypted file at {}.\n'.format(directory))
lanid = base64.b64encode(bytes(input(' LanID: '), encoding='utf-8')).decode('utf-8')
email = base64.b64encode(bytes(input(' eMail: '), encoding='utf-8')).decode('utf-8')
password = base64.b64encode(bytes(input(' PassW: '), encoding='utf-8')).decode('utf-8')
cred = lanid+splitter+email+splitter+password
with open(directory+'\\Credentials.txt','w+') as file:
file.write(cred)
file.close()
cred = getCredentials()
updateCredentials()
Place the configuration information in a encrypted config file. Query this info in your code using an key. Place this key in a separate file per environment, and don't store it with your code.
More homegrown appraoch rather than converting authentication / passwords / username to encrytpted details. FTPLIB is just the example.
"pass.csv" is the csv file name
Save password in CSV like below :
user_name
user_password
(With no column heading)
Reading the CSV and saving it to a list.
Using List elelments as authetntication details.
Full code.
import os
import ftplib
import csv
cred_detail = []
os.chdir("Folder where the csv file is stored")
for row in csv.reader(open("pass.csv","rb")):
cred_detail.append(row)
ftp = ftplib.FTP('server_name',cred_detail[0][0],cred_detail[1][0])
Do you know pit?
https://pypi.python.org/pypi/pit (py2 only (version 0.3))
https://github.com/yoshiori/pit (it will work on py3 (current version 0.4))
test.py
from pit import Pit
config = Pit.get('section-name', {'require': {
'username': 'DEFAULT STRING',
'password': 'DEFAULT STRING',
}})
print(config)
Run:
$ python test.py
{'password': 'my-password', 'username': 'my-name'}
~/.pit/default.yml:
section-name:
password: my-password
username: my-name
If running on Windows, you could consider using win32crypt library. It allows storage and retrieval of protected data (keys, passwords) by the user that is running the script, thus passwords are never stored in clear text or obfuscated format in your code. I am not sure if there is an equivalent implementation for other platforms, so with the strict use of win32crypt your code is not portable.
I believe the module can be obtained here: http://timgolden.me.uk/pywin32-docs/win32crypt.html
You could also consider the possibility of storing the password outside the script, and supplying it at runtime
e.g. fred.py
import os
username = 'fred'
password = os.environ.get('PASSWORD', '')
print(username, password)
which can be run like
$ PASSWORD=password123 python fred.py
fred password123
Extra layers of "security through obscurity" can be achieved by using base64 (as suggested above), using less obvious names in the code and further distancing the actual password from the code.
If the code is in a repository, it is often useful to store secrets outside it, so one could add this to ~/.bashrc (or to a vault, or a launch script, ...)
export SURNAME=cGFzc3dvcmQxMjM=
and change fred.py to
import os
import base64
name = 'fred'
surname = base64.b64decode(os.environ.get('SURNAME', '')).decode('utf-8')
print(name, surname)
then re-login and
$ python fred.py
fred password123
Why not have a simple xor?
Advantages:
looks like binary data
noone can read it without knowing the key (even if it's a single char)
I get to the point where I recognize simple b64 strings for common words and rot13 as well. Xor would make it much harder.
There are several ROT13 utilities written in Python on the 'Net -- just google for them. ROT13 encode the string offline, copy it into the source, decode at point of transmission.But this is really weak protection...
This doesn't precisely answer your question, but it's related. I was going to add as a comment but wasn't allowed.
I've been dealing with this same issue, and we have decided to expose the script to the users using Jenkins. This allows us to store the db credentials in a separate file that is encrypted and secured on a server and not accessible to non-admins.
It also allows us a bit of a shortcut to creating a UI, and throttling execution.
import base64
print(base64.b64encode("password".encode("utf-8")))
print(base64.b64decode(b'cGFzc3dvcmQ='.decode("utf-8")))
I want to encrypt a .zip file using AES256 in Python. I am aware of the Python cryptography module, in particular the example given at:
https://cryptography.io/en/latest/fernet/
However, I have needs that are a bit different:
I want to output binary data (because I want a small encrypted file). How can I output in binary instead of armored ASCII?
I do not want to have the plaintext timestamp. Any way to remove it?
If I cannot fix those points I will use another method. Any suggestions? I was considering issuing gpg commands through subprocess.
Looking at Fernet module, seems it encrypts and authenticates the data. Actually its safer than only encrypting (see here). However, removing the timestamp, in the case of this module, doesn't make sense if you also want to authenticate.
Said that, seems you want to risky and only encrypt instead of encrypt and authenticate. You might follow the examples of the same module found at https://cryptography.io/en/latest/hazmat/primitives/symmetric-encryption/. Just make sure this is what you really want.
As you're worried about size and want to use AES, you could try AES in CTR mode, which does not need padding, avoiding extra bytes at the end.
import os
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend
backend = default_backend()
key = os.urandom(32)
nonce = os.urandom(16)
cipher = Cipher(algorithms.AES(key), modes.CTR(nonce), backend=backend)
encryptor = cipher.encryptor()
ct = encryptor.update(b"a secret message") + encryptor.finalize()
print(ct)
decryptor = cipher.decryptor()
print(decryptor.update(ct) + decryptor.finalize())
So, answering your questions:
(1) The update method already returns a byte array.
(2) This way there will be no plaintext data automatically appended to the ciphertext (but be aware of the security implications about not authenticating the data). However, you'll need to pass the IV anyway, what you would have to do in either case.
I'm trying to write a simple Python solution to encrypt a file securely using a passphrase. I figured I would use something like bcrypt or pbkdf2 so that as time goes on, I could make my password hashes more and more difficult to brute force. I also figured I would use AES for the actual encryption, as it's a pretty safe standard. I'm not fixed on the encryption cipher, but I really like bcrypt.
I'm having quite a difficult time figuring out how to actually perform the encryption. Let's say I have a passphrase and a file I'd like to encrypt. I'd assume that I essentially need to do something like this:
from Crypto.Cipher import AES
from bcrypt import gensalt, hashpw
from hashlib import sha256
def encryptify(passphrase, file_name):
target_file = open(file_name, 'r')
# generate password, takes time
passphrase_rounds = 15
passphrase_salt = gensalt(rounds)
passphrase = sha256(hashpw(passphrase, passphrase_salt)).hexdigest()
# encrypt the file
encrypted_file = AES.new(passphrase, AES.MODE_CBC).encrypt(target_file.read())
At the final step, it fails with a ValueError, telling me that my key must be 16, 24, or 32 bytes long. What I'm not understanding is if what I'm doing is secure and why the last step is failing. I thought that SHA256 outputs 32 characters of data?
I'm particularly concerned about taking a bcrypt passphrase and throwing it through sha256, are there any potential security risks by doing this? I wouldn't imagine so, but then again, I'm not a cryptographer.
I can't comment about safety, but if you want your actual 32 bytes of SHA256, you need to call digest, not hexdigest. hexdigest returns a hexadecimal string representation (that would be 64 characters).
I'm trying to understand how does Linux encrypt our password on the etc/shadow file, so I've dont a new virtual 'test' user to make some test:
user: newuser
password: usrpw123
Generated salt: Ii4CGbr7
So the OS makes me the following line on the etc/shadow file, using a SHA512 encryptation system ($6$): newuser:$6$Ii4CGbr7$IOua8/oPV79Yp.BwzpxlSHjmCvRfTomZ.bhEvjZV2x5qhrvk82lZVrEtWQQej2pOWMdN7hvKwNgvCXKFQm5CB/:15069:0:99999:7:::
Now, I take the SHA512 module from python and try this:
import hashlib
m = hashlib.sha512()
m.update('Ii4CGbr7'+'usrpw123')
print m.hexdigest
This gives me the following hash as a result:
c73156daca3e31125ce457f1343201cc8a26400b2974440af2cc72687922b48b6631d21c186796ea2756ad987a996d2b261fe9ff3af4cc81e14c3029eac5df55
As you can see, it's different than the other one on the /etc/shadow file, and I dont know why if I'm using the same salt+password to generate the hash.
Can someone give me a hand and explain me more or less why this happens?
And also, why does the /etc/shadow files generates a hash with some dots (.)?
Thanks
The fields in /etc/shadow are not built or interpreted the way you think they are. You'll want to read the man page for details, but the most obvious difference is that it uses an unusual base64 encoding for both the salt and the hash.
There is an algorithm for generating the password hashes found in /etc/shadow.
See this document for an explanation:
http://www.akkadia.org/drepper/SHA-crypt.txt
There's an implementation of this in python here:
http://packages.python.org/passlib/lib/passlib.hash.sha512_crypt.html
I fell into the same trap as everything that I read lead me to believe you could retrieve the results the same way you have it written.
I was able to determine the password by using the salt and password using crypt.crypt()
import crypt
crypt.crypt(password, salt)
salt: $6$Ii4CGbr7
password: usrpw123
doesn't exactly use the hashlib library but it works.
I have a 'public key' in a variable named varkey, for getting the public key I used the urllib and stored that public key in a variable. Now I want to encrypt a msg/string using the public key.
It's ok if somebody could lead me to some library.
My blog post (the passingcuriosity.com link in John Boker's answer) does AES -- a symmetric encryption algorithm -- using the M2Crypto library. M2Crypto is a Python wrapper around OpenSSL. The API is pretty much a straight translation of OpenSSL's into Python, so the somewhat sketchy documentation shouldn't be too much of a problem. If the public key encryption algorithm you need to use is supported by M2Crypto, then you could very well use it to do your public key cryptography.
I found the M2Crypto test suite to be a useful example of using its API. In particular, the RSA (in test_rsa.py), PGP (in test_pgp.py), and EVP (in test_evp.py) tests will help you figure out how to set up and use the library. Do be aware that they are unit-tests, so it can be a little tricky to figure out exactly what code is necessary and what is an artefact of being a test.
PS: As I'm new, my posts can only contain one link so I had to delete most of them. Sorry.
Example
from M2Crypto import RSA
rsa = RSA.load_pub_key('rsa.pub.pem')
encrypted = rsa.public_encrypt('your message', RSA.pkcs1_oaep_padding)
print encrypted.encode('base64')
Output
X3iTasRwRdW0qPRQBXiKN5zvPa3LBiCDnA3HLH172wXTEr4LNq2Kl32PCcXpIMxh7j9CmysLyWu5
GLQ18rUNqi9ydG4ihwz3v3xeNMG9O3/Oc1XsHqqIRI8MrCWTTEbAWaXFX1YVulVLaVy0elODECKV
4e9gCN+5dx/aG9LtPOE=
Here's the script that demonstrates how to encrypt a message using M2Crypto ($ easy_install m2crypto) given that public key is in varkey variable:
#!/usr/bin/env python
import urllib2
from M2Crypto import BIO, RSA
def readkey(filename):
try:
key = open(filename).read()
except IOError:
key = urllib2.urlopen(
'http://svn.osafoundation.org/m2crypto/trunk/tests/' + filename
).read()
open(filename, 'w').write(key)
return key
def test():
message = 'disregard the -man- (I mean file) behind curtain'
varkey = readkey('rsa.pub.pem')
# demonstrate how to load key from a string
bio = BIO.MemoryBuffer(varkey)
rsa = RSA.load_pub_key_bio(bio)
# encrypt
encrypted = rsa.public_encrypt(message, RSA.pkcs1_oaep_padding)
print encrypted.encode('base64')
del rsa, bio
# decrypt
read_password = lambda *args: 'qwerty'
rsa = RSA.load_key_string(readkey('rsa.priv2.pem'), read_password)
decrypted = rsa.private_decrypt(encrypted, RSA.pkcs1_oaep_padding)
print decrypted
assert message == decrypted
if __name__ == "__main__":
test()
Output
gyLD3B6jXspHu+o7M/TGLAqALihw7183E2effp9ALYfu8azYEPwMpjbw9nVSwJ4VvX3TBa4V0HAU
n6x3xslvOnegv8dv3MestEcTH9b3r2U1rsKJc1buouuc+MR77Powj9JOdizQPT22HQ2VpEAKFMK+
8zHbkJkgh4K5XUejmIk=
disregard the -man- (I mean file) behind curtain
From my recent python experience, python doesn't do encryption natively. You need to use an external (3rd party) package. Each of these, obviously, offers a different experience. Which are you using? This will probably determine how your syntax will vary.
You might want to have a look at:
http://www.example-code.com/python/encryption.asp
or this
http://passingcuriosity.com/2009/aes-encryption-in-python-with-m2crypto/
Have you ever heard about "RSAError: data too large for key size"?
Try your sample with more long message:
encrypted = rsa.public_encrypt('My blog post (the passingcuriosity.com link in John Boker's answer) does AES -- a symmetric encryption algorithm -- using the M2Crypto library', RSA.pkcs1_oaep_padding)
You could use MD5 or SHA1 hashing along with your key...