We Keep Coding

How to avoid/bypass Python Fernet "InvalidToken"

I want to decrypt many files and it's working perfectly (encrypt, decrypt).
My problem is that, if any file exists which is not is encrypted, I get an "invalidToken" error message, and the program is unable to decrypt the remaining encrypted files.
Here is my code
import os
from cryptography.fernet import Fernet
import sys
import warnings
InvalidToken = False
key = b'SpecialKindKey'
fernet = Fernet(key)
encr_file_list = []
path ="C:/users/NineTail/desktop/pokemon/go/"
#we shall store all the file names in this list
filelist = []
for root, dirs, files in os.walk(path):
for file in files:
#append the file name to the list
filelist.append(os.path.join(root,file))
#print all the file names
for name in filelist:
print(name)
for file in filelist:
with open(file, 'rb') as f:
file_content = f.read()
decrypted_content = fernet.decrypt(file_content)
with open(file, 'wb') as f:
f.write(decrypted_content)
Error message :
Traceback (most recent call last):
File "r3.py", line 36, in <module>
decrypted_content = fernet.decrypt(file_content)
File "C:\Program Files\Python38\lib\site-packages\cryptography\fernet.py", line 85, in decrypt
timestamp, data = Fernet._get_unverified_token_data(token)
File "C:\Program Files\Python38\lib\site-packages\cryptography\fernet.py", line 121, in _get_unverified_token_data
raise InvalidToken
cryptography.fernet.InvalidToken

Use try/except to try to decrypt. If decrypting fails, continue without writing.
...
from cryptography.fernet import Fernet, InvalidToken
...
for file in filelist:
with open(file, 'rb') as f:
file_content = f.read()
try:
decrypted_content = fernet.decrypt(file_content)
except InvalidToken:
continue
with open(file, 'wb') as f:
f.write(decrypted_content)
...

Perhaps try having a loop that repeats the number of times of files you have, and calls each file separately from the rest
Define a list of files, and store that in a variable
say:
num = #number of files
files = [file1.txt,file2.txt]
For i in range(num):
files[num] = open(item, 'r')
num += 1

Do not exit if the file is not found python3

Hi I have a list of filenames
When reading a file, it comes out with an error
Traceback (most recent call last):
File "test.py", line 229, in <module>
f = open(i, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'directory_home.json'
How can I skip the error and complete the rest of the files?
code
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)

Something like this. Note that 'pass' goes to next iteration. 'Continue' would repeat iteration (in this case useless). 'Break' would end iteration.
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
try:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)
f.colse()
except FileNotFoundError:
print("File does not exist")
pass

Two options.
Option 1: os.path.isfile
import os
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
if os.path.isfile(file):
f = open(i)
data = json.load(f)
for i in data['name']:
print(i)
else:
continue # or whatever you want to do.
Option 2: Exception handling
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
try:
f = open(i, 'r')
except FileNotFoundError:
continue # or whatever
data = json.load(f)
for i in data['name']:
print(i)

You should pur your code in a try-except statement, like this
try:
file = ["directory_home.json", "directory_ever.json", "home.json", "home.json"]
for i in file:
f = open(i, 'r')
data = json.load(f)
for i in data['name']:
print(i)
except FileNotFoundError as e:
# log exception and do whatever you want
pass
In this way, if your code raises an exception because the file si not present, you are able ti handle It.
Please remember that in the code above, just the exception due to a not existing file si catch. This means that if another exception occurs the code will raise a traceback

Reading from multiple files and storing data in a list

I am trying to read print search for all files in a directory and store contents in each file in a list to be used.
My problem is when i use print to debug if the file exists, it prints out the current file or first file in the list. However, It complains that file is not found when i try to read from this file
import re
import os
# Program to extract emails from text files
def path_file():
#path = raw_input("Please enter path to file:\n> ")
path = '/home/holy/thinker/leads/'
return os.listdir('/home/holy/thinker/leads') # returns a list like ["file1.txt", 'image.gif'] # need to remove trailing slashes
# read a file as 1 big string
def in_file():
print path_file()
content = []
for a_file in path_file(): # ['add.txt', 'email.txt']
print a_file
fin = open(a_file, 'r')
content.append(fin.read()) # store content of each file
print content
fin.close()
return content
print in_file()
# this is the error i get
""" ['add.txt', 'email.txt']
add.txt
Traceback (most recent call last):
File "Extractor.py", line 24, in <module>
print in_file()
File "Extractor.py", line 17, in in_file
fin = open(a_file, 'r')
IOError: [Errno 2] No such file or directory: 'add.txt'
"""
The error I get is aboive

os.listdir will return you only file name. You have to directory name on before that file name.
Its trying to open add.txt in same directory where you ran your program. Please add directory name before file name.
def path_file():
#path = raw_input("Please enter path to file:\n> ")
path = '/home/holy/thinker/leads/'
return [os.path.join(path, x) for x in os.listdir(path)]

you should use the full path of the file you want to read.
so please do fin = open(os.path.join(r'/home/holy/thinker/leads/', a_file), 'r')

Here's a rewrite using glob to limit which files are considered;
import glob
import os
import re
import sys
if sys.hexversion < 0x3000000:
# Python 2.x
inp = raw_input
else:
# Python 3.xrange
inp = input
def get_dir(prompt):
while True:
dir_name = inp(prompt)
dir_name = os.path.join(os.getcwd(), dir_name)
if os.path.isdir(dir_name):
return dir_name
else:
print("{} does not exist or is not a directory".format(dir_name))
def files_in_dir(dir_name, file_spec="*.txt"):
return glob.glob(os.path.join(dir_name, file_spec))
def file_iter(files):
for fname in files:
with open(fname) as inf:
yield fname, inf.read()
def main():
email_dir = get_dir("Please enter email directory: ")
email_files = files_in_dir(email_dir, "*.eml")
print(email_files)
content = [txt for fname,txt in file_iter(email_files)]
print(content)
if __name__=="__main__":
main()
and a trial run looks like
Please enter email directory: c:\temp
['c:\\temp\\file1.eml', 'c:\\temp\\file2.eml']
['file1 line one\nfile1 line two\nfile1 line three',
'file2 line one\nfile2 line two']

Gzip problem,traceback and IOError: [Errno 2] No such file or directory

I'm new to python and bioinformatics field. I'm using python-2.6. Now I'm trying to select all fastq.gz files, then gzip.open(just a few lines because it's too huge and time-wasting), then count 'J' , then pick out those files with 'J' count NOT equal to 0.
The following is my code:
#!/usr/bin/python
import os,sys,re,gzip
path = "/home/XXX/nearline"
for file in os.listdir(path):
if re.match('.*\.recal.fastq.gz', file):
text = gzip.open(file,'r').readlines()[:10]
word_list = text.split()
number = word_list.count('J') + 1
if number !== 0:
print file
But I got some errors:
Traceback (most recent call last):
File "fastqfilter.py", line 9, in <module>
text = gzip.open(file,'r').readlines()[:10]
File "/share/lib/python2.6/gzip.py", line 33, in open
return GzipFile(filename, mode, compresslevel)
File "/share/lib/python2.6/gzip.py", line 79, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'ERR001268_1.recal.fastq.gz'
What's this traceback: File......
Is there anything wrong with gzip here?
And why can't it find ERR001268_1.recal.fastq.gz? It's the first fastq file in the list, and DOES exist there.
Hope give me some clues, and any point out any other errors in the script.
THanks a lot.
Edit: thx everyone. I followed Dan's suggestion. And I tried on ONE fastq file first. My script goes like:
#!/usr/bin/python
import os,sys
import gzip
import itertools
file = gzip.open('/home/xug/nearline/ERR001274_1.recal.fastq.gz','r')
list(itertools.islice(file.xreadlines(),10))
word_list = list.split()
number = word_list.count('J') + 1
if number != 0:
print 'ERR001274_1.recal.fastq.gz'
Then errors are:
Traceback (most recent call last):
File "try2.py", line 8, in <module>
list(itertools.islice(text.xreadlines(),10))
AttributeError: GzipFiles instance has no attribute 'xreadlines'
Edit again: Thx Dan, I've solved the problem yesterday. Seems GzipFiles don't support xreadlines. So I tried the similar way as you suggested later. And it works. See below:
#!/usr/bin/python
import os,sys,re
import gzip
from itertools import islice
path = "/home/XXXX/nearline"
for file in os.listdir(path):
if re.match('.*\.recal.fastq.gz', file):
fullpath = os.path.join(path, file)
myfile = gzip.open(fullpath,'r')
head = list(islice(myfile,1000))
word_str = ";".join(str(x) for x in head)
number = word_str.count('J')
if number != 0:
print file

on this line:
text = gzip.open(file,'r').read()
file is a filename not a full path so
fullpath = os.path.join(path, file)
text = gzip.open(fullpath,'r').read()
about F.readlines()[:10] will read the whole file in to a list of lines and then take the first 10
import itertools
list(itertools.islice(F.xreadlines(),10))
this will not read the whole file into memory and will only read the first 10 lines into a list
but as gzip.open returns an object that doesn't have .xreadlines() and but as files are iterable on their lines just:
list(itertools.islice(F,10))
would work as this test shows:
>>> import gzip,itertools
>>> list(itertools.islice(gzip.open("/home/dan/Desktop/rp718.ps.gz"),10))
['%!PS-Adobe-2.0\n', '%%Creator: dvips 5.528 Copyright 1986, 1994 Radical Eye Software\n', '%%Title: WLP-94-int.dvi\n', '%%CreationDate: Mon Jan 16 16:24:41 1995\n', '%%Pages: 6\n', '%%PageOrder: Ascend\n', '%%BoundingBox: 0 0 596 842\n', '%%EndComments\n', '%DVIPSCommandLine: dvips -f WLP-94-int.dvi\n', '%DVIPSParameters: dpi=300, comments removed\n']

Change your code to:
#!/usr/bin/python
import os,sys,re,gzip
path = "/home/XXX/nearline"
for file in os.listdir(path):
if re.match('.*\.recal.fastq.gz', file):
text = gzip.open(os.path.join(path,file),'r').readlines()[:10]
word_list = text.split()
number = word_list.count('J') + 1
if number !== 0:
print file

It's trying to open ERR001268_1.recal.fastq.gz from the working directory, not from /home/XXX/nearline.

Bad file descriptor error

If I try executing the following code
f = file('test','rb')
fout = file('test.out','wb')
for i in range(10):
a = f.read(1)
fout.write(a)
f.close()
f = fout
f.seek(4)
print f.read(4)
Where 'test' is any arbitrary file, I get:
Traceback (most recent call last):
File "testbad.py", line 12, in <module>
print f.read(4)
IOError: [Errno 9] Bad file descriptor
If however, I change just the fout line to use a temporary file:
import tempfile
f = file('test','rb')
fout = tempfile.NamedTemporaryFile()
for i in range(10):
a = f.read(1)
fout.write(a)
f.close()
f = fout
f.seek(4)
print f.read(4)
There are no errors. Does anyone know why this is? I would have expected the first case to work, but I must be doing something wrong.
Thanks in advance for any help!

you've only opened the file fout for writing, not reading. To open for both use
fout = file('test.out','r+b')