I've been tasked with password-protecting a Java app with minimal concern for real security. So it seemed sensible to store username/password pairs in a text file and then encrypt it. For the encryption, it seemed appropriate to use XOR ciphers because they're easy and fast (remember--it just has to discourage the casual user, not be bulletproof).
I wrote all the appropriate Java, and then realized I needed a way to encrypt the config file. I wrote an additional method, but it was clunky to use more than once or twice (and seemed to only work for some inputs), so I decided it was best to write something in Python, to be played with at a REPL.
This is what I ended up with:
from itertools import izip, cycle
KEY = "stackoverflow"
def encrypt(text):
return ''.join(chr(ord(x) ^ ord(y)) for (x,y) in izip(text,cycle(KEY)))
def decrypt(text):
return encrypt(text)
def export(users, file):
with open(file, "w") as f:
for user, password in users.items():
f.write(encrypt('"%s" "%s"'%(user, password)) + "\n")
def import_data(file):
with open(file) as f:
return [decrypt(i) for i in f.readlines()]
On the surface, it works:
>>> x = encrypt("Hello world!")
>>> x
';\x11\r\x0f\x04O\x01\n\x00\n\x08N'
>>> decrypt(x)
'Hello world!'
But then things start to fall apart:
>>> export({"foo" : "bar", "baz" : "quux", "spam" : "eggs"}, "users.dat")
>>> import_data("users.dat")
['"foo" "bar"e', '"baz" "quux"}', '"spam" "eggs"y']
And here's how vim reads it -
And then:
>>> export({"what" : "not", "this" : "that", "admin_istrator" : "quux"}, "users2.dat")
>>> import_data("users2.dat")
['"thi', "k97$ma{~'l", '"what" "not"}', '"admin_istrator" "quux', '7~']
Vim:
It occurred to me that I might be having a problem with a character's encrypted form being a newline, but as far as I see that doesn't explain the wacky behavior in the first example or all of the wacky behavior in the second one.
Regarding the newlines, my Plan B is to encrypt the entire file--newlines and all--and then slurp it back up, decrypt it, split it on "\n", and proceed with my line-based parsing.
Thanks in advance.
Update: Here's my implementation of Plan B (described two paragraphs ago).
def import2(file):
with open(file) as f:
return decrypt(f.read())
and then:
>>> export({"foo" : "bar", "this" : "that", "admin_istrator" : "letmein"}, "users2.dat")
>>> import2("users2.dat")
'"this" "that"y%smg&91uux!}"admin_istrator" "letmein"y'
Update Two: Binary.
[Code is the same as above, except that all opens are open(file, "rb") or open(file, "wb").]
>>> export({"foo" : "bar", "this" : "that", "admin_istrator" : "letmein"}, "users2.dat")
>>> import2("users2.dat")
'"this" "that"y%smg&91uux!}"admin_istrator" "letmein"y'
>>> import_data("users2.dat")
['"t', "k97$ma{~'", '"foo" "bar"', '"admin_istrator" "letmein"']
Final update: Base 64, other shenanigans.
def import2(file):
with open(file, "rb") as f:
return filter(str.strip, [decrypt(i) for i in f.readlines()])
where encrypt and decrypt encode in/decode base 64.
You are trying to store binary in text mode file. Use open(file, "wb") for writing and open(file, "rb") for reading to open file in binary mode and fix an issue.
In text mode every "\r", "\n" and "\r\n" sequences are treated as newlines so they are converted into your local OS line ending convention ("\r\n" for Windows, "\n" for Unix, "\r" for old Macs). In case you read them from text file, you will always get "\n" instead, in case you write them, I don't remember actual behavior, but you will surely also get mess instead of your data :)
And with XOR encryption it's very likely you'll run into this kind of stuff :)
If you are forced not to use binary files, try base64 encoding (e.g. "some\0te\n\nxt with bi\x01naries".encode('base64')). To decode use .decode (thanks, Captain Obvious!).
The problem is you are not reading the same data you codified (you add a '\n' after the encryption), just do a rstrip() of the data you read:
def import_data(file):
with open(file) as f:
return [decrypt(i.rstrip()) for i in f.readlines()]
You can fix it by encrypting the newlines and not resetting the key between lines
from itertools import izip, cycle
KEY = "stackoverflow"
def encrypt(text):
return ''.join(chr(ord(x) ^ ord(y)) for (x,y) in izip(text,key))
def decrypt(text):
return encrypt(text)
def export(users, file):
with open(file, "w") as f:
for user, password in users.items():
f.write(encrypt('"%s" "%s"\n'%(user, password)))
def import_data(file):
with open(file) as f:
return [decrypt(i) for i in f]
key = cycle(KEY)
export({"foo" : "bar", "baz" : "quux", "spam" : "eggs"}, "users.dat")
key = cycle(KEY)
for row in import_data("users.dat"):
print row
This should be turned into a class, and key would be an instance variable instead of a global as it is here
Related
I'm attempting to anonymize a file so that all the content except certain keywords are replaced with gibberish, but the format is kept the same (including punctuation, length of string and capitalization). For example:
I am testing this, check it out! This is a keyword: long
Wow, another line.
should turn in to:
T ad ehistmg ptrs, erovj qo giw! Tgds ar o qpyeogf: long
Yeg, rmbjthe yadn.
I am attempting to do this in python, but i'm having no luck in finding a solution. I have tried replacing via tokenization and writing to another file, but without much success.
Initially let's disregard the fact that we have to preserve some keywords. We will fix that later.
The easiest way to perform this kind of 1-to-1 mapping is to use the method str.translate. The string module also contains constants that contain all ASCII lowercase and uppercase characters, and random.shuffle can be used to obtain a random permutation.
import string
import random
random_caps = list(string.ascii_uppercase)
random_lows = list(string.ascii_lowercase)
random.shuffle(random_caps)
random.shuffle(random_lows)
all_random_chars = ''.join(random_lows + random_caps)
translation_table = str.maketrans(string.ascii_letters, all_random_chars)
with open('the-file-i-want.txt', 'r') as f:
contents = f.read()
translated_contents = contents.translate(translation_table)
with open('the-file-i-want.txt', 'w') as f:
f.write(translated_contents)
In python 2 the str.maketrans is a function in the string module instead of a static method of str.
The translation_table is a mapping from characters to characters, so it will map every single ASCII character to an other one. The translate method simply applies this table to each character in the string.
Important note: the above method is actually reversible, because each letter its mapped to a unique other letter. This means that using a simple analysis over the frequency of the symbols it's possible to reverse it.
If you want to make this harder or impossible, you could re-create the translation_table for every line:
import string
import random
random_caps = list(string.ascii_uppercase)
random_lows = list(string.ascii_lowercase)
with open('the-file-i-want.txt', 'r') as f:
translated_lines = []
for line in f:
random.shuffle(random_lows)
random.shuffle(random_caps)
all_random_chars = ''.join(random_lows + random_caps)
translation_table = str.maketrans(string.ascii_letters, all_random_chars)
translated_lines.append(line.translate(translation_table))
with open('the-file-i-want.txt', 'w') as f:
f.writelines(translated_lines)
Also note that you could translate and save the file line by line:
with open('the-file-i-want.txt', 'r') as f, open('output.txt', 'w') as o:
for line in f:
random.shuffle(random_lows)
random.shuffle(random_caps)
all_random_chars = ''.join(random_lows + random_caps)
translation_table = str.maketrans(string.ascii_letters, all_random_chars)
o.write(line.translate(translation_table))
Which means you can translate huge files with this code, as far as the lines themselves are not insanely long.
The code above messing all characters, without taking into account such keywords.
The simplest way to handle the requirement is to simply check for each line whether one of keywords occur and "reinsert" it there:
import re
import string
import random
random_caps = list(string.ascii_uppercase)
random_lows = list(string.ascii_lowercase)
keywords = ['long'] # add all the possible keywords in this list
keyword_regex = re.compile('|'.join(re.escape(word) for word in keywords))
with open('the-file-i-want.txt', 'r') as f, open('output.txt', 'w') as o:
for line in f:
random.shuffle(random_lows)
random.shuffle(random_caps)
all_random_chars = ''.join(random_lows + random_caps)
translation_table = str.maketrans(string.ascii_letters, all_random_chars)
matches = keyword_regex.finditer(line)
translated_line = list(line.translate(translation_table))
for match in matches:
translated_line[match.start():match.end()] = match.group()
o.write(''.join(translated_line))
Sample usage (using the version that prevserves keywords):
$ echo 'I am testing this, check it out! This is a keyword: long
Wow, another line.' > the-file-i-want.txt
$ python3 trans.py
$ cat output.txt
M vy hoahitc hfia, ufoum ih pzh! Hfia ia v modjpel: long
Ltj, fstkwzb hdsz.
Note how long is preserved.
I have a file which contains a lot of Strings. I am trying to compute SHA1 hashes of these strings individually and store those
import hashlib
inp = open("inp.txt" , "r")
outputhash = open("outputhashes.txt", "w")
for eachpwd in inp:
sha_1 = hashlib.sha1()
sha_1.update(eachpwd)
outputhash.write(sha_1.hexdigest())
outputhash.write("\n")
The issue I am facing is once a strings SHA1 is computed the next string is being appended(I feel this is why I am not getting the correct hashes) and its hash is being computed. Hence I am not getting the correct hashes. I am new to python. I know what to do but don't know how to do it. Can you point me in the right direction to go about this?
You're iterating over a file, which is going to return the lines, including the line terminator (a \n character at the end of the string)
You should remove it:
import hashlib
inp = open("inp.txt" , "r")
outputhash = open("outputhashes.txt", "w")
for line in inp: # Change this
eachpwd = line.strip() # Change this
# Add this to understand the problem:
print repr(line)
sha_1 = hashlib.sha1()
sha_1.update(eachpwd)
outputhash.write(sha_1.hexdigest())
outputhash.write("\n")
For my project, I need to be able to store random byte strings in a file and read the byte string again later. For example, I want to store randomByteString from the following code:
>>> from os import urandom
>>> randomByteString=urandom(8)
>>> randomByteString
b'zOZ\x84\xfb\xceM~'
What would be the proper way to do this?
Edit: Forgot to mention that I also want to store 'normal' string alongside the byte strings.
Code like:
>>> fh = open("e:\\test","wb")
>>> fh.write(randomByteString)
8
>>> fh.close()
Operate the file as binary mode. Also, you could do it in a better manner if the file operations are near one place (Thanks to #Blender):
>>> with open("e:\\test","wb") as fh:
fh.write(randomByteString)
Update: if you want to strong normal strings, you could encode it and then write it like:
>>> "test".encode()
b'test'
>>> fh.write("test".encode())
Here the fh means the same file handle opened previously.
Works just fine. You can't expect the output to make much sense though.
>>> import os
>>> with open("foo.txt", "wb") as fh:
... fh.write(os.urandom(8))
...
>>> fh.close()
>>> with open("foo.txt", "r") as fh:
... for line in fh.read():
... print line
...
^J^JM-/
^O
R
M-9
J
~G
Would using the pickle function be the fastest and most robust way to write an integer to a text file?
Here is the syntax I have so far:
import pickle
pickle.dump(obj, file)
If there is a more robust alternative, please feel free to tell me.
My use case is writing an user input:
n=int(input("Enter a number: "))
Yes, A human will need to read it and maybe edit it
There will be 10 numbers in the file
Python may need to read it back later.
I think it's simpler doing:
number = 1337
with open('filename.txt', 'w') as f:
f.write('%d' % number)
But it really depends on your use case.
Write
result = 1
f = open('output1.txt','w') # w : writing mode / r : reading mode / a : appending mode
f.write('{}'.format(result))
f.close()
Read
f = open('output1.txt', 'r')
input1 = f.readline()
f.close()
print(input1)
With python 2, you can also do:
number = 1337
with open('filename.txt', 'w') as f:
print >>f, number
I personally use this when I don't need formatting.
I just encountered a similar problem.
I used a simple approach, saving the integer in a variable, and writing the variable to the file as a string. If you need to add more variables you can always use "a+" instead of "w" to append instead of write.
f = open("sample.txt", "w")
integer = 10
f.write(str(integer))
f.close()
Later you can use float to read the file and you wont throw and error.
The following opens a while and appends the following number to it.
def writeNums(*args):
with open("f.txt", "a") as f:
f.write("\n".join([str(n) for n in args]) + "\n")
writeNums(input("Enter a numer:"))
I am trying to write a dictionary to a txt file. Then read the dict values by typing the keys with raw_input. I feel like I am just missing one step but I have been looking for a while now.
I get this error
File "name.py", line 24, in reading
print whip[name]
TypeError: string indices must be integers, not str
My code:
#!/usr/bin/env python
from sys import exit
class Person(object):
def __init__(self):
self.name = ""
self.address = ""
self.phone = ""
self.age = ""
self.whip = {}
def writing(self):
self.whip[p.name] = p.age, p.address, p.phone
target = open('deed.txt', 'a')
target.write(str(self.whip))
print self.whip
def reading(self):
self.whip = open('deed.txt', 'r').read()
name = raw_input("> ")
if name in self.whip:
print self.whip[name]
p = Person()
while True:
print "Type:\n\t*read to read data base\n\t*write to write to data base\n\t*exit to exit"
action = raw_input("\n> ")
if "write" in action:
p.name = raw_input("Name?\n> ")
p.phone = raw_input("Phone Number?\n> ")
p.age = raw_input("Age?\n> ")
p.address = raw_input("Address?\n>")
p.writing()
elif "read" in action:
p.reading()
elif "exit" in action:
exit(0)
Have you tried the json module? JSON format is very similar to python dictionary. And it's human readable/writable:
>>> import json
>>> d = {"one":1, "two":2}
>>> json.dump(d, open("text.txt",'w'))
This code dumps to a text file
$ cat text.txt
{"two": 2, "one": 1}
Also you can load from a JSON file:
>>> d2 = json.load(open("text.txt"))
>>> print d2
{u'two': 2, u'one': 1}
Your code is almost right! You are right, you are just missing one step. When you read in the file, you are reading it as a string; but you want to turn the string back into a dictionary.
The error message you saw was because self.whip was a string, not a dictionary. So you need to convert the string to a dictionary.
Example
Here is the simplest way: feed the string into eval(). Like so:
def reading(self):
s = open('deed.txt', 'r').read()
self.whip = eval(s)
You can do it in one line, but I think it looks messy this way:
def reading(self):
self.whip = eval(open('deed.txt', 'r').read())
But eval() is sometimes not recommended. The problem is that eval() will evaluate any string, and if someone tricked you into running a really tricky string, something bad might happen. In this case, you are just running eval() on your own file, so it should be okay.
But because eval() is useful, someone made an alternative to it that is safer. This is called literal_eval and you get it from a Python module called ast.
import ast
def reading(self):
s = open('deed.txt', 'r').read()
self.whip = ast.literal_eval(s)
ast.literal_eval() will only evaluate strings that turn into the basic Python types, so there is no way that a tricky string can do something bad on your computer.
EDIT
Actually, best practice in Python is to use a with statement to make sure the file gets properly closed. Rewriting the above to use a with statement:
import ast
def reading(self):
with open('deed.txt', 'r') as f:
s = f.read()
self.whip = ast.literal_eval(s)
In the most popular Python, known as "CPython", you usually don't need the with statement as the built-in "garbage collection" features will figure out that you are done with the file and will close it for you. But other Python implementations, like "Jython" (Python for the Java VM) or "PyPy" (a really cool experimental system with just-in-time code optimization) might not figure out to close the file for you. It's good to get in the habit of using with, and I think it makes the code pretty easy to understand.
To store Python objects in files, use the pickle module:
import pickle
a = {
'a': 1,
'b': 2
}
with open('file.txt', 'wb') as handle:
pickle.dump(a, handle)
with open('file.txt', 'rb') as handle:
b = pickle.loads(handle.read())
print a == b # True
Notice that I never set b = a, but instead pickled a to a file and then unpickled it into b.
As for your error:
self.whip = open('deed.txt', 'r').read()
self.whip was a dictionary object. deed.txt contains text, so when you load the contents of deed.txt into self.whip, self.whip becomes the string representation of itself.
You'd probably want to evaluate the string back into a Python object:
self.whip = eval(open('deed.txt', 'r').read())
Notice how eval sounds like evil. That's intentional. Use the pickle module instead.
Hi there is a way to write and read the dictionary to file you can turn your dictionary to JSON format and read and write quickly just do this :
To write your date:
import json
your_dictionary = {"some_date" : "date"}
f = open('destFile.txt', 'w+')
f.write(json.dumps(your_dictionary))
and to read your data:
import json
f = open('destFile.txt', 'r')
your_dictionary = json.loads(f.read())
I created my own functions which work really nicely:
def writeDict(dict, filename, sep):
with open(filename, "a") as f:
for i in dict.keys():
f.write(i + " " + sep.join([str(x) for x in dict[i]]) + "\n")
It will store the keyname first, followed by all values. Note that in this case my dict contains integers so that's why it converts to int. This is most likely the part you need to change for your situation.
def readDict(filename, sep):
with open(filename, "r") as f:
dict = {}
for line in f:
values = line.split(sep)
dict[values[0]] = {int(x) for x in values[1:len(values)]}
return(dict)
You can iterate through the key-value pair and write it into file
pair = {'name': name,'location': location}
with open('F:\\twitter.json', 'a') as f:
f.writelines('{}:{}'.format(k,v) for k, v in pair.items())
f.write('\n')