We Keep Coding

'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte django

so i have variable contains in bytes and i want to save it to str. but how to do it? i tried various kinds of ways, but always got error
'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte
i want to save the result encrypt file to text file. so i have the proof it the file was encrypt. here;s my ways 1:
def create_file(f):
response = HttpResponse(content_type="text/plain")
response['Content-Disposition'] = 'attachment; filename=file.txt'
filename = f
print(filename)
name_byte = codecs.decode(filename)
print(name_byte)
return response
my ways 2 :
def create_file(enc):
with open("file.txt", "w", encoding="utf-8") as f:
enc = enc.decode('utf-8')
f.write(enc)
my ways 3:
def create_file(f):
file = open("file.txt", "w")
f = f.decode('utf-8')
download = file.write(f)
file.close()
print(download)
return download
f = b'\xa0\x0e\xdc\x14'
f is the return result of encrypt
i called the function :
#in views
download = create_file(enc)
print(download)
#in urls
path("create_file", views.create_file, name="create_file"),
#in html
<a href="{% url 'create_file' %}">

The phrase "have variable contains in bytes and i want to save it to str" makes no sense at all. bytes and str are different data types serving different purposes. bytes holds a byte stream while str hold abstract text. Yes, bytes can also hold textual info encoded in some text encoding like UTF-8, but that is only a specific case. If, for example, your bytes hold a JPEG image then converting it to str does not make any sense simply because it's not text. So if your encrypted file contains a byte stream you need to treat it as such. It's not text any more until you decrypt it.
So the only thing you can do is to save your byte stream as is using the binary file mode:
v = b'\xa0\x0e\xdc\x14'
with open ('/path', 'wb') as fo:
fo.write(v)
The same applies to sending the encrypted file as Django response:
response = HttpResponse(v, content_type="application/octet-stream")
response['Content-Disposition'] = 'attachment; filename=file.bin'

If you just want to write the entire line to file then all you have to do is convert to string.
So this would work:
v = b'\xa0\x0e\xdc\x14'
with open("C:/test/output.txt", 'w') as file:
file.write(str(v))
The result is a text file like this:

Python reading a PE file and changing resource section

I am trying to open a Windows PE file and alter some strings in the resource section.
f = open('c:\test\file.exe', 'rb')
file = f.read()
if b'A'*10 in file:
s = file.replace(b'A'*10, newstring)
In the resource section I have a string that is just:
AAAAAAAAAA
And I want to replace that with something else. When I read the file I get:
\x00A\x00A\x00A\x00A\x00A\x00A\x00A\x00A\x00A\x00A
I have tried opening with UTF-16 and decoding as UTF-16 but then I run into a error:
UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 1604-1605: illegal encoding
Everyone I seen who had the same issue fixed by decoding to UTF-16. I am not sure why this doesn't work for me.

If resource inside binary file is encoded to utf-16, you shouldn't change encoding.
try this
f = open('c:\\test\\file.exe', 'rb')
file = f.read()
unicode_str = u'AAAAAAAAAA'
encoded_str = unicode_str.encode('UTF-16')
if encoded_str in file:
s = file.replace(encoded_str, new_utf_string.encode('UTF-16'))
inside binary file everything is encoded, keep in mind

UnicodeDecodeError when import json file

I want to open a json file in python and I have the error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 64864: ordinal not in range(128)
my code is quite simple:
# -*- coding: utf-8 -*-
import json
with open('birdw3l2.json') as data_file:
data = json.load(data_file)
print(data)
Someone can help me? Thanks!

Try the following code.
import json
with open('birdw3l2.json') as data_file:
data = json.load(data_file).decode('utf-8')
print(data)

You should specify your encoding format when you load your json file. like this:
data = json.load(data_file, encoding='utf-8')
The encoding depends on your file encoding.

Python 3: JSON File Load with Non-ASCII Characters

just trying to load this JSON file(with non-ascii characters) as a python dictionary with Unicode encoding but still getting this error:
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 90: ordinal not in range(128)
JSON file content = "tooltip":{
"dxPivotGrid-sortRowBySummary": "Sort\"{0}\"byThisRow",}
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
for line in f:
data.append(json.loads(line.encode('utf-8','replace')))

You have several problems as near as I can tell. First, is the file encoding. When you open a file without specifying an encoding, the file is opened with whatever sys.getfilesystemencoding() is. Since that may vary (especially on Windows machines) its a good idea to explicitly use encoding="utf-8" for most json files. Because of your error message, I suspect that the file was opened with an ascii encoding.
Next, the file is decoded from utf-8 into python strings as it is read by the file system object. The utf-8 line has already been decoded to a string and is already ready for json to read. When you do line.encode('utf-8','replace'), you encode the line back into a bytes object which the json loads (that is, "load string") can't handle.
Finally, "tooltip":{ "navbar":"Operações de grupo"} isn't valid json, but it does look like one line of a pretty-printed json file containing a single json object. My guess is that you should read the entire file as 1 json object.
Putting it all together you get:
import json
with open('/Users/myvb/Desktop/Automation/pt-PT.json', encoding="utf-8") as f:
data = json.load(f)
From its name, its possible that this file is encoded as a Windows Portugese code page. If so, the "cp860" encoding may work better.

I had the same problem, what worked for me was creating a regular expression, and parsing every line from the json file:
REGEXP = '[^A-Za-z0-9\'\:\.\;\-\?\!]+'
new_file_line = re.sub(REGEXP, ' ', old_file_line).strip()

Having a file with content similar to yours I can read the file in one simple shot:
>>> import json
>>> fname = "data.json"
>>> with open(fname) as f:
... data = json.load(f)
...
>>> data
{'tooltip': {'navbar': 'Operações de grupo'}}

You don't need to read each line. You have two options:
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
data.append(json.load(f))
Or, you can load all lines and pass them to the json module:
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
data.append(json.loads(''.join(f.readlines())))
Obviously, the first suggestion is the best.

Python file chunk in json

I have some service api and need to upload file by chunks in json format.
Here is example of code:
for chunk in file_content.chunks():
chunkSize = len(chunk)
d = {'chunksize': chunkSize, 'chunk': chunk}
dump = json.dumps(d)
But getting:
'utf8' codec can't decode byte 0x93 in position 11: invalid start byte
How can i fix it or maybe ignore?
Note: I cannot change the api

json.dumps takes a number of arguments, setting ensure_ascii to false should do the trick.
dump = json.dumps(d, ensure_ascii=False)