We Keep Coding

Path inside the dictionary from the variable

I have a code:
def replaceJSONFilesList(JSONFilePath, JSONsDataPath, newJSONData):
JSONFileHandleOpen = open(JSONFilePath, 'r')
ReadedJSONObjects = json.load(JSONFileHandleOpen)
JSONFileHandleOpen.close()
ReadedJSONObjectsModifyingSector = ReadedJSONObjects[JSONsDataPath]
for newData in newJSONData:
ReadedJSONObjectsModifyingSector.append(newData)
JSONFileHandleWrite = open(JSONFilePath, 'w')
json.dump(ReadedJSONObjects, JSONFileHandleWrite)
JSONFileHandleWrite.close()
def modifyJSONFile(Path):
JSONFilePath = '/path/file'
JSONsDataPath = "['first']['second']"
newJSONData = 'somedata'
replaceJSONFilesList(JSONFilePath, JSONsDataPath, newJSONData)
Now I have an error:
KeyError: "['first']['second']"
But if I try:
ReadedJSONObjectsModifyingSector = ReadedJSONObjects['first']['second']
Everything is okay.
How I should send the path to the list from the JSON's dictionary — from one function to other?

You cannot pass language syntax elements as if they were data strings. Similarly, you could not pass the string "2 > 1 and False", and expect the function to be able to insert that into an if condition.
Instead, extract the data items and pass them as separate strings (which matches their syntax in the calling routine), or as a tuple of strings. For instance:
JSONsDataPath = ('first', 'second')
...
Then, inside the function ...
ReadedJSONObjects[JSONsDataPath[0]][JSONsDataPath[1]]
If you have a variable sequence of indices, then you need to write code to handle that case; research that on Stack Overflow.
The iterative way to handle an unknown quantity of indices is like this:
obj = ReadedJSONObjects
for index in JSONsDataPath:
obj = obj[index]

Python: how to load json with dict containing range

my python code is about generating a sequence number from a dict keys, and my dict keys are defined with a range using cycle package in itertools module.
working example:
from itertools import cycle
e = {'Apple': cycle(range(1,999)),'Orange': cycle(range(1,999)),'Banana': cycle(range(1,999))}
def SequenceNum(f):
return f'{next(e[f])}'.zfill(3)
X = SequenceNum('Apple')
print(X)
output
001 --> it keeps incrementing in the range specified above in dict `e`
Challenge:
My requirement is to convert this dict of e into a json file. So it will load keys and values by parsing json file.
cat test.json
{
"DATA": {
"Apple": "cycle(range(1,999))",
"Orange": "cycle(range(1,999))",
"Banana": "cycle(range(1,999))"
}
}
(i had to put the dict values inside double quotes to avoid json file loading error.)
code
import json
from itertools import cycle
with open('test.json') as f:
FromJson = json.load(f)
d = FromJson['DATA']
print(d)
def SequenceNum(f):
return f'{next(d[f])}'.zfill(3)
X = SequenceNum('Apple')
i = 1
while i <= 10:
print(i, SequenceNum('Apple'))
i += 1
here new dict is d that loads json file and it will load the values in single quotes.
output
{'Apple': 'cycle(range(1,999))', 'Orange': 'cycle(range(1,999))', 'Banana': 'cycle(range(1,999))'} #THIS IS OUTPUT of 'd' after loading json file
Traceback (most recent call last):
File "c:\Users\chandu\Documents\test.py", line 14, in <module>
print(i, SequenceNum('Apple'))
File "c:\Users\chandu\Documents\test.py", line 12, in SequenceNum
return f'{next(d[f])}'.zfill(3)
TypeError: 'str' object is not an iterator
it is giving error because my dict values are not properly iterable by cycle itertools modules, since they are in quotes. i dont know if there is any other cause for this error.
please help to resolve this error,
Thanks in advance.

If you are sure what each value is, you can do eval with care:
def SequenceNum(f):
return f'{next(eval(d[f]))}'.zfill(3)
Note this is very dangerous to use, because eval evaluates anything that is passed into it and can cause harm.
This also will always fetch first value from iterator as it is evaluated new everytime. To solve, you can:
def SequenceNum(f):
return eval(d[f])
i = 1
seq_iter = SequenceNum('Apple')
while i <= 10:
print(i, f'{next(seq_iter)}'.zfill(3))
i += 1

Python 3 Read a json file with missing objects within lines

I'm reading a json file with the structure below:
[{"id":1,"gender":"Male","first_name":"Andrew","last_name":"Scott","email":"ascott0#shutterfly.com","ville":"Connecticut"},
{"id":3,"first_name":"Mary","last_name":"Richards","email":"mrichards2#japanpost.jp","ville":"Minnesota"}]
So, as you can see in the second "line" the field "gender" it'is not present.I realize that because my code to read the file got wrong in this line.
my code:
import json
def jsonreader():
##Reader for json files
##Open files using json library
with open('cust_data.json') as file:
data = json.load(file)
resultlist = list()
for line in data:
print(line["id"],line["gender"])
I got the error:-
C:/xxxxx/x.py
1 Male
Traceback (most recent call last):
2 Female
File "C:/xxxxx/x", line 67, in <module>
jsonreader()
File "C:/xxxxx/x", line 56, in jsonreader
print(line["id"],line["gender"])
KeyError: 'gender'
Before answer guys, you should know that I have a method to define the default value in "gender", voila my method:
def definegender(x):
if x is None:
x = 'unknown'
return x
elif (x =='Male') or (x=='Female'):#not None:
return {
'Male':'M',
'Female': 'F'
}.get(x)
else:
return x
So, in this case, I could not use something like a default value reading the values because I need to send some value to my method.
Some one of you guys would know how should be the best way to read this kind of files when we have missing objects. Thanks

why not using a default value for your dictionary in dict.get?
print(line["id"],line.get("gender","unknown"))
And since you want to transform input further, you could nest two dict.get together, the first one with None as default value, and a new table, like this:
gender_dict = {"Male":"M", "Female":"F", None : "unknown"}
print(line["id"],gender_dict.get(line.get("gender")))
(note that you don't need your overcomplex gender conversion method anymore)

Although this already has a perfect answer, my point of view is that there can be alternatives too. So here it is:
for line in data:
try:
print(line["id"],line["gender"])
except KeyError:
print(line["id"],"Error!!! no gender!")
This is called ErrorHandling. Read the docs here:
https://docs.python.org/3.6/tutorial/errors.html
update: Do you mean this?
update2 corrected misstake
try:
gender = definegender(line["gender"])
except KeyError:
gender = definegender(None)
print(line["id"],gender)
update3: (for future purposes)
as .get() returns None by default the most simple solution would be
gender = definegender(line.get("gender"))
print(line["id"],gender)

Why not simplify this with an if-statement?
for line in data:
if "gender" in line:
print(line)

TypeError: string indices must be integers when parsing JSON

This code throws a TypeError: string indices must be integers when parsing JSON. Why is that?
ids=[]
for line in f:
k = json.loads(line)
if "person" in k:
for id in ids:
if k["iden"] == id[0]: #Exception raised here
#Do some processing
else:
ids += [(k["iden"], 1)]

json.loads(line) gives you a string (in your case), not a dictionary. If you've got a dictionary, you can use dictionary['whatever'], but your_str['other_str'] won't work. Check your json file, it may be containing some unwanted data.
Here's the documentation on json.loads(s):
Deserialize s (a str or unicode instance containing a JSON document) to a Python object.
In your case, that Python object is a string, not a dictionary.

I'll guess f is your file.readlines(), and you have something like this before:
my_json_file = open('/path/to/file.json')
f = my_json_file.readlines()
my_json_file.close()
Try, instead of doing readlines(), to pass the file directly to json.load:
my_json_file = open('/path/to/file.json')
k = json.loads(my_json_file)
my_json_file.close()

Json in Python: Receive/Check duplicate key error

The json module of python acts a little of the specification when having duplicate keys in a map:
import json
>>> json.loads('{"a": "First", "a": "Second"}')
{u'a': u'Second'}
I know that this behaviour is specified in the documentation:
The RFC specifies that the names within a JSON object should be
unique, but does not specify how repeated names in JSON objects should
be handled. By default, this module does not raise an exception;
instead, it ignores all but the last name-value pair for a given name:
For my current project, I absolutely need to make sure that no duplicate keys are present in the file and receive an error/exception if this is the case? How can this be accomplished?
I'm still stuck on Python 2.7, so a solution which also works with older versions would help me most.

Well, you could try using the JSONDecoder class and specifying a custom object_pairs_hook, which will receive the duplicates before they would get deduped.
import json
def dupe_checking_hook(pairs):
result = dict()
for key,val in pairs:
if key in result:
raise KeyError("Duplicate key specified: %s" % key)
result[key] = val
return result
decoder = json.JSONDecoder(object_pairs_hook=dupe_checking_hook)
# Raises a KeyError
some_json = decoder.decode('''{"a":"hi","a":"bye"}''')
# works
some_json = decoder.decode('''{"a":"hi","b":"bye"}''')

The below code will take any json that has repeated keys and put them into an array. for example takes this json string '''{"a":"hi","a":"bye"}''' and gives {"a":['hi','bye']} as output
import json
def dupe_checking_hook(pairs):
result = dict()
for key,val in pairs:
if key in result:
if(type(result[key]) == dict):
temp = []
temp.append(result[key])
temp.append(val)
result[key] = temp
else:
result[key].append(val)
else:
result[key] = val
return result
decoder = json.JSONDecoder(object_pairs_hook=dupe_checking_hook)
#it will not raise error
some_json = decoder.decode('''{"a":"hi","a":"bye"}''')