We Keep Coding

Decode double quotes in a json string to convert to json Python

Faced the following problem, I have a following string like this:
mystr = '[\\x22https://dosprn.co.il/\\x22, \\x22DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. \\\\\\x22Tools\\\\\\x22 Create Bootable USB Drive\\x22]'
I am trying to convert this string to JSON. However, this is not possible:
json.loads(mystr)
Traceback (most recent call last):
File "/Users/Tokyonight/PycharmProjects/WebCeoDev/venv/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-57-6f4efa0d20c6>", line 1, in <module>
json.loads(mystr)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
import html, json
# Escape html objects. And unescape unicode.
mystr = '[\\x22https://dosprn.co.il/\\x22, \\x22DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. \\\\\\x22Tools\\\\\\x22 Create Bootable USB Drive\\x22]'
mystr = mystr.encode('ascii', 'xmlcharrefreplace').decode('unicode_escape')
mystr_to_json = json.loads(mystr)
print(mystr_to_json)
#['https://dosprn.co.il/', 'DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. "Tools" Create Bootable USB Drive']
mystr_to_json[1] = html.unescape(mystr_to_json[1])
print(mystr_to_json)
# ['https://dosprn.co.il/', 'DOSPrn - הדפסה בעברית ב-DOS מדפסת Windows DOSPrn. "Tools" Create Bootable USB Drive']
It works. But I think it's too cumbersome. I'm pretty sure there is an easier way to do this.
Anyone have any ideas how to do it better?
Thanks.

Why doesn't this JSON load in python?

For a couple of days i've been trying to learn how to use JSON in python, but even if i copy everything from tutorials it still won't load.
import json
data = '''
{
"people":[
{
"name": "veljko",
"age":"20",
"email": veljkov02#gail.com,
"educated": "true"
} ,
{
"name":"aleksandar",
"age":"24",
"email":"marko99#gmail.com",
"educated":"false"
}
]
}
'''
men = json.loads(data)
print(men)
(Ignore the random words used in code, it's just how i learn)
It just keeps giving the same error messages, i've tried with the most simple JSON but it's still the same.
Traceback (most recent call last):
File "C:\Users\*********\Desktop\code3\json.py", line 21, in <module>
men = json.loads(data)
File "C:\Users\************\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\*************\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode.
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\************\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 14 (char 75)
Process finished with exit code 1
Please help, i'm a begginer and am not sure how to solve this problem.

You are missing quotes around veljkov02#gail.com. You need to change it to "veljkov02#gail.com", so that it is interpreted as a string. The exception tells you which line and character the error is at, so double check that line when you get errors like these.

Download file from GitLab using Python HTTP requests

I'd like to download a YAML file from Gitlab using Python requests. I'm almost there, but I havent quite got to the cigar stage.
I am doing the following :-
GITLAB_FILE ="https://my_url/api/v4/projects/my_id/repository/files/path_and_filename.yaml/raw?ref=master&private_token=mytoken"
g=requests.request("GET", GITLAB_FILE, verify=False)
print(g.json())
Now, it all works in as much as I can get to the file ok, but when it comes to accessing the data the print(g.json()) throws an error, but then continues to print out the file contents as I'd hoped. The error is :-
Traceback (most recent call last):
File "/home/myproj/edm/lib/python3.7/site-packages/requests/models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./http_test.py", line 73, in <module>
print (g.json())
File "/home/myproj/edm/lib/python3.7/site-packages/requests/models.py", line 917, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value]
Once this is printed, it then proceeds to print out the contents of my yaml file correctly.
I suspect its something to do with the
print(g.json())
expecting a json format, and encountering a yaml file?
Any pointers as to how I can get the file contents error free would be helpful.

I've answered my own question.
Quite simply instead of
print(g.json())
I used
print(g.text)
Which did exactly what it said on the tin.

Viusal studio python error when using requests package (error : json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0))

Im getting the error whenever i try to run this code in visual studio code.
Image of code
Error message:
Traceback (most recent call last):
File "c:\Users\climia1\Downloads\hangman.py", line 3, in <module>
r = requests.get('https://api.hypixel.net/skyblock/bazaar?key=29d9ecf6-71bb-49b1-b931-e0ed5f00f31d').json()
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\climia1\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[Done] exited with code=1 in 18.946 seconds
it displays the same error on repl.it but it goes away after one run. with visual studio it stays. any help?

Your json file must be empty or isn't formatted properly
if you are reading or writing to a empty Json file make sure it contains {}
if you are writing or reading a json file with lot of data make sure all the tags are closed

Reading scraped json file into python

I'm trying to read a JSON file I've scraped from a website into Python, with the eventual aim of turning it into a csv file for later statistical analysis.
An example of the file I want to read is:
globals.jsonpCallback('/feed/match/1-1-8vOt3JMq-1-2-yj0a9.dat', {"s":1,"d":{"bt":1,"sc":2,"st":{"notice":"finished","status-name":"Finished","result-name":"<p id=\"event-status\" class=\"result\"><span class=\"bold\" id=\"event-status-finished\">Final result 3:1<\/span><\/p>"},"oddsdata":{"back":{"E-1-2-0-0-0":{"handicapType":0,"handicapValue":"0.00","odds":{"49":{"0":1.32,"2":7,"1":4.8},"46":{"0":1.33,"2":7.3,"1":4.5},"9":{"0":1.25,"2":12,"1":5},"16":{"0":1.29,"2":11,"1":5},"164":{"0":1.31,"2":7.59,"1":4.5},"18":{"0":1.24,"2":11.6,"1":5.25},"23":{"0":1.27,"2":11.23,"1":4.75},"26":{"0":1.3,"2":10.5,"1":5},"5":{"0":1.23,"2":12,"1":5.75},"44":{"0":1.14,"2":4,"1":2.52},"1":{"0":1.3,"2":8.5,"1":4.4},"8":{"0":1.25,"2":9,"1":5},"3":{"0":1.25,"2":11,"1":4.5},"31":{"0":1.25,"2":9,"1":4.5},"60":{"0":1.29,"2":8.5,"1":4.5},"15":{"0":1.28,"2":10,"1":5.4},"34":{"0":1.25,"2":10,"1":5},"111":{"0":1.25,"2":10,"1":4.84},"68":{"0":1.26,"2":9.75,"1":4.84},"105":{"0":1.28,"2":8,"1":4.5},"71":{"0":1.3,"2":9,"1":5},"28":{"0":1.25,"2":12,"1":5.5},"33":{"0":1.3,"2":9.25,"1":4.84},"14":{"0":1.22,"2":10.93,"1":5.05},"21":{"0":1.33,"2":7.5,"1":4.33},"30":{"0":1.22,"2":8.5,"1":5.5},"24":{"0":1.3,"2":9,"1":4.95},"32":{"0":1.25,"2":9.5,"1":4.9},"2":{"0":1.25,"2":10,"1":5},"90":{"0":1.28,"2":10.25,"1":4.8},"56":{"0":1.3,"2":9.8,"1":4.8},"41":{"0":1.3,"2":8.9,"1":4.5},"128":{"0":1.27,"2":12,"1":5.5},"75":{"0":1.2,"2":10.5,"1":5.2}},"mixedParameterId":0,"OutcomeID":{"0":"s3nfxv464x0x1o7d5","2":"s3nfxv464x0x1o7d6","1":"s3nfxv498x0x0"},"opening_odds":{"49":{"0":null,"2":null,"1":null},"46":{"0":null,"2":null,"1":null},"9":{"0":null,"2":null,"1":null},"16":{"0":1.22,"2":null,"1":null},"164":{"0":null,"2":null,"1":null},"18":{"0":1.31,"2":9.8,"1":5.65},"23":{"0":null,"2":4.75,"1":11.23},"26":{"0":1.25,"2":10,"1":5.25},"5":{"0":1.22,"2":12.5,"1":5.5},"44":{"0":null,"2":null,"1":null},"1":{"0":null,"2":null,"1":null},"8":{"0":null,"2":null,"1":null},"3":{"0":null,"2":null,"1":null},"31":{"0":null,"2":null,"1":null},"60":{"0":null,"2":null,"1":null},"15":{"0":1.25,"2":null,"1":4.5},"34":{"0":null,"2":null,"1":null},"111":{"0":null,"2":null,"1":null},"68":{"0":1.27,"2":9.15,"1":4.65},"105":{"0":null,"2":null,"1":null},"71":{"0":1.29,"2":8,"1":4.5},"28":{"0":1.22,"2":10,"1":5},"33":{"0":1.4,"2":6.65,"1":4.25},"14":{"0":1.32,"2":7.8,"1":4.42},"21":{"0":null,"2":null,"1":null},"30":{"0":null,"2":null,"1":null},"24":{"0":null,"2":null,"1":null},"32":{"0":1.3,"2":8.5,"1":4.75},"2":{"0":1.35,"2":7.75,"1":4.3},"90":{"0":null,"2":null,"1":null},"56":{"0":null,"2":11,"1":4.59},"41":{"0":null,"2":null,"1":null},"128":{"0":1.25,"2":null,"1":5},"75":{"0":1.25,"2":8.9,"1":4.8}},"opening_change_time":{"49":{"0":false,"2":false,"1":false},"46":{"0":false,"2":false,"1":false},"9":{"0":false,"2":false,"1":false},"16":{"0":false,"2":false,"1":false},"164":{"0":false,"2":false,"1":false},"18":{"0":false,"2":false,"1":false},"23":{"0":false,"2":false,"1":false},"26":{"0":false,"2":false,"1":false},"5":{"0":false,"2":false,"1":false},"44":{"0":false,"2":false,"1":false},"1":{"0":false,"2":false,"1":false},"8":{"0":false,"2":false,"1":false},"3":{"0":false,"2":false,"1":false},"31":{"0":false,"2":false,"1":false},"60":{"0":false,"2":false,"1":false},"15":{"0":false,"2":false,"1":false},"34":{"0":false,"2":false,"1":false},"111":{"0":false,"2":false,"1":false},"68":{"0":false,"2":false,"1":false},"105":{"0":false,"2":false,"1":false},"71":{"0":false,"2":false,"1":false},"28":{"0":false,"2":false,"1":false},"33":{"0":false,"2":false,"1":false},"14":{"0":false,"2":false,"1":false},"21":{"0":false,"2":false,"1":false},"30":{"0":false,"2":false,"1":false},"24":{"0":false,"2":false,"1":false},"32":{"0":false,"2":false,"1":false},"2":{"0":false,"2":false,"1":false},"90":{"0":false,"2":false,"1":false},"56":{"0":false,"2":false,"1":false},"41":{"0":false,"2":false,"1":false},"128":{"0":false,"2":false,"1":false},"75":{"0":false,"2":false,"1":false}},"opening_volume":{"49":{"0":null,"2":null,"1":null},"46":{"0":null,"2":null,"1":null},"9":{"0":null,"2":null,"1":null},"16":{"0":null,"2":null,"1":null},"164":{"0":null,"2":null,"1":null},"18":{"0":null,"2":null,"1":null},"23":{"0":null,"2":null,"1":null},"26":{"0":null,"2":null,"1":null},"5":{"0":null,"2":null,"1":null},"44":{"0":null,"2":null,"1":null},"1":{"0":null,"2":null,"1":null},"8":{"0":null,"2":null,"1":null},"3":{"0":null,"2":null,"1":null},"31":{"0":null,"2":null,"1":null},"60":{"0":null,"2":null,"1":null},"15":{"0":null,"2":null,"1":null},"34":{"0":null,"2":null,"1":null},"111":{"0":null,"2":null,"1":null},"68":{"0":null,"2":null,"1":null},"105":{"0":null,"2":null,"1":null},"71":{"0":null,"2":null,"1":null},"28":{"0":null,"2":null,"1":null},"33":{"0":null,"2":null,"1":null},"14":{"0":null,"2":null,"1":null},"21":{"0":null,"2":null,"1":null},"30":{"0":null,"2":null,"1":null},"24":{"0":null,"2":null,"1":null},"32":{"0":null,"2":null,"1":null},"2":{"0":null,"2":null,"1":null},"90":{"0":null,"2":null,"1":null},"56":{"0":null,"2":null,"1":null},"41":{"0":null,"2":null,"1":null},"128":{"0":null,"2":null,"1":null},"75":{"0":null,"2":null,"1":null}},"volume":[],"change_time":{"49":{"0":1192344925,"2":1192344925,"1":1192344925},"46":{"0":1192356640,"2":1192356640,"1":1192356640},"9":{"0":1192374117,"2":1192374117,"1":1192374117},"16":{"0":1192643462,"2":1192643462,"1":1192643462},"164":{"0":1192356640,"2":1192356640,"1":1192356640},"18":{"0":1192643462,"2":1192642538,"1":1192643462},"23":{"0":1192528549,"2":1192533516,"1":1192533516},"26":{"0":1192643462,"2":1192527200,"1":1192643462},"5":{"0":1192617648,"2":1192617648,"1":1192617648},"44":{"0":1192054308,"2":1192054308,"1":1192054308},"1":{"0":1192383076,"2":1192383076,"1":1192383076},"8":{"0":1192364254,"2":1192364254,"1":1192364254},"3":{"0":1192430433,"2":1192430433,"1":1192430433},"31":{"0":1192365159,"2":1192365159,"1":1192365159},"60":{"0":1192389374,"2":1192389374,"1":1192389374},"15":{"0":1192610985,"2":1192310256,"1":1192610985},"34":{"0":1192366959,"2":1192366959,"1":1192366959},"111":{"0":1192487538,"2":1192487538,"1":1192487538},"68":{"0":1192473161,"2":1192473161,"1":1192473161},"105":{"0":1192354376,"2":1192354376,"1":1192354376},"71":{"0":1192610985,"2":1192610985,"1":1192610985},"28":{"0":1192570418,"2":1192570418,"1":1192570418},"33":{"0":1192397461,"2":1192397461,"1":1192397461},"14":{"0":1192641649,"2":1192641649,"1":1192641649},"21":{"0":1192362474,"2":1192362474,"1":1192362474},"30":{"0":1192356171,"2":1192356171,"1":1192356171},"24":{"0":1192419956,"2":1192419956,"1":1192419956},"32":{"0":1192635347,"2":1192635347,"1":1192635347},"2":{"0":1192449725,"2":1192449725,"1":1192449725},"90":{"0":1192426261,"2":1192426261,"1":1192426261},"56":{"0":1192366052,"2":1192621850,"1":1192621850},"41":{"0":1192436180,"2":1192436180,"1":1192436180},"128":{"0":1192624523,"2":1192441597,"1":1192624523},"75":{"0":1192437084,"2":1192437084,"1":1192437084}},"st":{"49":[3,60],"46":[3,60],"9":[3,60],"16":[3,60],"164":[3,60],"18":[3,60],"23":[3,60],"26":[3,60],"5":[3,60],"44":[3,60],"1":[3,60],"8":[3,60],"3":[3,60],"31":[3,60],"60":[3,60],"15":[3,60],"34":[3,60],"111":[3,60],"68":[3,60],"105":[3,60],"71":[3,60],"28":[3,60],"33":[3,60],"14":[3,60],"21":[3,60],"30":[3,60],"24":[3,60],"32":[3,60],"2":[3,60],"90":[3,60],"56":[3,60],"41":[3,60],"128":[3,60],"75":[3,60]},"bs":[],"act":{"49":true,"46":true,"9":true,"16":true,"164":true,"18":true,"23":true,"26":true,"5":true,"44":true,"1":true,"8":true,"3":true,"31":true,"60":true,"15":true,"34":false,"111":true,"68":true,"105":true,"71":true,"28":true,"33":true,"14":true,"21":true,"30":false,"24":true,"32":true,"2":true,"90":true,"56":true,"41":true,"128":true,"75":true},"actEx":{"44":{"0":true,"2":true,"1":true}}}},"lay":[]},"history":{"back":null,"lay":null},"hcl":true,"time-base":1192644000,"nav":{"1":{"2":["1","2","3","5","8","9","14","15","16","18","21","23","24","26","28","30","31","32","33","34","41","44","46","49","56","60","68","71","75","90","105","111","128","164"]},"4":{"2":["2","5","9","23","46","49","164"]}},"hasLiveOdds":false,"brokenParser":["30","34","103","154"],"hash":"c9fff233b9f89d1da10e81a9fdae7c14"},"refresh":16});
My current code looks like this (previous copy and paste job was inexplicably wrong):
import json
from pprint import pprint
json_data = open('/home/readejj/Documents/data/1-1-jFEwehyT-1-2-yjaf9.dat').read()
json_stuff = json_data[json_data.find('{'):json_data.rfind('}')+1]
json_stuff = json_stuff.replace("\"","'")
print "json_stuff",type(json_stuff), len(json_stuff), json_stuff[1:2000]
d = json.loads(json_stuff)
pprint(d)
This is adopted from: Reading JSON from a file?
The error I'm getting is:
Traceback (most recent call last):
File "read_json.py", line 16, in <module>
d = json.loads(json_stuff)
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)
If anyone can help that would be greatly appreciated - my apologies if it's a very basic error, I am not an experienced programmer.
UPDATE: It's been pointed out my replace line is bad - if removed, I get a different error:
Traceback (most recent call last):
File "read_oddsportal_json.py", line 19, in <module>
d = json.loads(json_stuff)
File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 99 (char 99)

This line:
json_stuff = json_stuff.replace("\"","'")
is bad. Delete it.
The JSON spec requires that strings are double-quote (") delimited, not single-quote.