bencode including some binary data - python

I am using bencode library and I am trying to encode and decode binary data:
>>> import bencode
>>> bencode.Bencoder.decode(bencode.Bencoder.encode({'x': b'xyz'}))
Traceback (most recent call last):
File "<input>", line 1, in <module>
bencode.Bencoder.decode(bencode.Bencoder.encode({'x': b'xyz'}))
File "/home/pt12lol/virtualenv-python3/lib/python3.5/site-packages/bencode/__init__.py", line 228, in decode
return _decode(becode_string)
File "/home/pt12lol/virtualenv-python3/lib/python3.5/site-packages/bencode/__init__.py", line 197, in _decode
data = _decode_item(pf, six.next(src))
File "/home/pt12lol/virtualenv-python3/lib/python3.5/site-packages/bencode/__init__.py", line 178, in _decode_item
tok = next()
File "/home/pt12lol/virtualenv-python3/lib/python3.5/site-packages/bencode/__init__.py", line 147, in _tokenizer
s = m.group(m.lastindex) #
http://stackoverflow.com/questions/22489243/re-in-python-lastindex-attribute
AttributeError: 'NoneType' object has no attribute 'group'
How can I manage it?

Related

Parsing GFF files throws TypeError

I am trying to parse a gif file with Biopython, and am using the sample code from their website. This is the code:
from BCBio import GFF
in_file = "infile.gff"
in_handle = open(in_file)
for rec in GFF.parse(in_handle):
print(rec)
in_handle.close()
When I run the code I get the following error:
Traceback (most recent call last):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/Bio/SeqIO/Interfaces.py", line 47, in __init__
self.stream = open(source, "r" + mode)
TypeError: expected str, bytes or os.PathLike object, not FakeHandle
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "get_genes_dpt.py", line 37, in <module>
for rec in GFF.parse(in_handle):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 746, in parse
target_lines):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 322, in parse_in_parts
for results in self.parse_simple(gff_files, limit_info, target_lines):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 343, in parse_simple
for results in self._gff_process(gff_files, limit_info, target_lines):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 637, in _gff_process
for out in self._lines_to_out_info(line_gen, limit_info, target_lines):
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 699, in _lines_to_out_info
fasta_recs = self._parse_fasta(FakeHandle(line_iter))
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/BCBio/GFF/GFFParser.py", line 560, in _parse_fasta
return list(SeqIO.parse(in_handle, "fasta"))
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/Bio/SeqIO/__init__.py", line 607, in parse
return iterator_generator(handle)
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/Bio/SeqIO/FastaIO.py", line 183, in __init__
super().__init__(source, mode="t", fmt="Fasta")
File "/Users/juliofdiaz/anaconda2/envs/python37/lib/python3.7/site-packages/Bio/SeqIO/Interfaces.py", line 51, in __init__
if source.read(0) != "":
TypeError: read() takes 1 positional argument but 2 were given
I am not sure how to fix the error as it seems I am passing a str and not a FakeHandle. I am running biopython 1.78 with conda.

Error extracting an archive using `tarfile`

I am getting an error while trying to extract a .tar.gz archive using the tarfile library.
Here is the relevant code snippet:
# `gzip_archive_bytes_content` is the content of the gzip archive, in "bytes" format
repo_sources_file_object = io.BytesIO(gzip_archive_bytes_content)
repo_sources_tar_object = tarfile.TarFile(fileobj=repo_sources_file_object)
repo_sources_tar_object.extractall(path="/tmp/")
This the error I get:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/tarfile.py", line 186, in nti
s = nts(s, "ascii", "strict")
File "/usr/local/lib/python3.7/tarfile.py", line 170, in nts
return s.decode(encoding, errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9a in position 1: ordinal not in range(128)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/tarfile.py", line 2289, in next
tarinfo = self.tarinfo.fromtarfile(self)
File "/usr/local/lib/python3.7/tarfile.py", line 1095, in fromtarfile
obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
File "/usr/local/lib/python3.7/tarfile.py", line 1037, in frombuf
chksum = nti(buf[148:156])
File "/usr/local/lib/python3.7/tarfile.py", line 189, in nti
raise InvalidHeaderError("invalid header")
tarfile.InvalidHeaderError: invalid header
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/site-packages/my-package/__main__.py", line 87, in <module>
function(**function_args)
File "/usr/local/lib/python3.7/site-packages/my-package/chart.py", line 107, in reinstall
install()
File "/usr/local/lib/python3.7/site-packages/my-package/chart.py", line 89, in install
repo_sources_tar_object = tarfile.TarFile(fileobj=repo_sources_file_object)
File "/usr/local/lib/python3.7/tarfile.py", line 1484, in __init__
self.firstmember = self.next()
File "/usr/local/lib/python3.7/tarfile.py", line 2301, in next
raise ReadError(str(e))
tarfile.ReadError: invalid header
Python version: 3.7
I switched from instanciating directly a tarfile.TarFile object to using the tarfile.open() constructor, and it fixed it:
repo_sources_tar_object = tarfile.open(fileobj=repo_sources_file_object)
There is actually a warning on this in the documentation, here:
Do not use this class directly: use tarfile.open() instead.
The best practice is to use a context manager in order to automatically close the file when the job is done.
One could write:
import contextlib
import io
import tarfile
gzip_archive_bytes_content = b"..."
repo_sources_file_object = io.BytesIO(gzip_archive_bytes_content)
with contextlib.closing(tarfile.open(fileobj=repo_sources_file_object)) as arch:
arch.extractall(path="/tmp/")
This is available with tarfile.TarFile, but not with tarfile.open().
So you could write:
with tarfile.TarFile(...) as arch:
...

Output parameter in "qgis:heatmapkerneldensityestimation" is not working

This is the code:
import processing
import os
#defines the folder
folder="C:/Users/Pueyo/Google Drive/Consultoria/Mapa escolar/Dades UMAT/heatmap"
#capts all the files in the folder
filelist=os.listdir(folder)
feedback = QgsProcessingFeedback()
#if the file is a shapefile, run the algortihm
for file in filelist:
if file.endswith('.shp'):
layer=QgsVectorLayer(folder+file,file,'ogr')
file2 = file.replace(".shp",""
output=str(folder + "/hm200_" + file2 + ".tif")
parameters={'INPUT':layer,'RADIUS':200, 'PIXEL_SIZE':5,'OUTPUT':output}
processing.runAndLoadResults('qgis:heatmapkerneldensityestimation', parameters, feedback=feedback)
A screen capture:
https://i.stack.imgur.com/2vLxn.png
This is the error I get:
Traceback (most recent call last):
File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 1, in <module>
File "<string>", line 15, in <module>
File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-
ltr/./python/plugins\processing\tools\general.py", line 138, in runAndLoadResults
return Processing.runAlgorithm(alg, parameters=parameters, onFinish=handleAlgorithmResults, feedback=feedback, context=context)
File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins\processing\core\Processing.py", line 183, in runAlgorithm raise QgsProcessingException(msg)
_core.QgsProcessingException: There were errors executing the algorithm.
The problem must be the definition of the output string, because I tried the same code but writing a route directly on the parameters list and it worked.

liac-arff to read arff file TypeError

i want to read arff file but i am getting this error. Could any one help me with this
import arff, numpy as np
dataset = arff.load(open('ckdfull.arff', 'rb'))
data = np.array(dataset['data'])
Traceback (most recent call last):
File "C:/Users/username/PycharmProjects/ckd/ckd.py", line 2, in <module>
dataset = arff.load(open('ckdfull.arff', 'rb'))
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 896, in load
return_type=return_type)
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 739, in decode
matrix_type=return_type)
File "C:\Users\username\PycharmProjects\ckd\venv\lib\site-packages\arff.py", line 651, in _decode
row = row.strip(' \r\n')
TypeError: a bytes-like object is required, not 'str'
Thank You

AttributeError: 'file' object has no attribute 'encode'

Traceback (most recent call last):
File "E:\blahblahblah\emailsend.py", line 26, in <module>
msg.attach(MIMEText(file))
File "E:\blahblahblah\Python 2.7.11\lib\email\mime\text.py", line 30, in __init__
self.set_payload(_text, _charset)
File "E:\blahblahblah\Python 2.7.11\lib\email\message.py", line 226, in set_payload
self.set_charset(charset)
File "E:\blahblahblah\Python 2.7.11\lib\email\message.py", line 268, in set_charset
cte(self)
File "E:\blahblahblah\Python 2.7.11\lib\email\encoders.py", line 73, in encode_7or8bit
orig.encode('ascii')
AttributeError: 'file' object has no attribute 'encode'https://stackoverflow.com/questions/ask#
I've been looking this up a lot but I haven't found an answer.
The only important parts of the code is this:
file = open('newfile.txt')
msg.attach(MIMEText(file))
There are other parts but I've debugged it and I get the error at the 'msg.attach(MIMEText(file))' line.
Any help?
MIMEText takes the content of the file, not the file object.
msg.attach(MIMEText(open("newfile.txt").read()))

Categories