We Keep Coding

Python parsing json from string

I need to parse json from a partial string I get back from a web service. I have the following snippet of code which is working fine but is extremely ugly. Is there a better or cleaner way to do this?
x = '"1":{"name":"item one","code":"1"},"2":{"name":"item two","code":"2"},"3":{"name":"item three","code":"3"}'
split = x.split('},')
index = 0
for s in split:
split[index] = '{' + s + '}}'
index += 1
joined = ','.join(split)
joined = '[' + joined[:-1] + ']'
j = json.loads(joined)
print(j)
Here is the result:
[{'1': {'name': 'item one', 'code': '1'}},
{'2': {'name': 'item two', 'code': '2'}},
{'3': {'name': 'item three', 'code': '3'}}]

You can use the following snippet:
>>> [dict([t]) for t in json.loads(f"{{{x}}}").items()]
[{'1': {'name': 'item one', 'code': '1'}},
{'2': {'name': 'item two', 'code': '2'}},
{'3': {'name': 'item three', 'code': '3'}}]

You can fix the inconsistency by hand (add the missing braces) and use json module to parse:
data = json.loads('{' + x + '}')
Then you can convert the parsed data to the desired representation:
[{item[0]: item[1]} for item in data.items()]
#[{'1': {'name': 'item one', 'code': '1'}},
# {'2': {'name': 'item two', 'code': '2'}},
# {'3': {'name': 'item three', 'code': '3'}}]
Otherwise, you will end up implementing your own JSON parser, which is not trivial.

How delete all keys in a nested dictionary if the key equals a given key

I have a nested dictionary that's updated dynamically so I never know how many levels there are. What I need to do is delete all entries in the dictionary that equal a given key like "command" for example.
I've tried looping through the dict but I found that the number of levels change at runtime so that didn't work. I was thinking that maybe this should use recursion but I would like to avoid that if I can.
I have include a sample of a mock dict, what I want is all keys that = command to be removed.
data = {
'id': 1,
'name': 'Option 1',
'command': do_something,
'sub_opt': {
'id': 10,
'name': 'Sub Option',
'command': do_something_more,
'sub_sub_opt': {
'id': 100,
'name': 'Sub Sub Option',
'command': do_something_crazy,
}
}
}

I know you're trying to avoid recursion, but the code isn't all that bad. Here's an example. (I changed the the values of 'command' keys to strings.)
def delete(data, key):
data.pop(key, None)
for k, v in data.items():
if isinstance(v, dict):
delete(v, key)
delete(data, 'command')
print(data)
{'id': 1, 'name': 'Option 1', 'command': 'do_something', 'sub_opt': {'id': 10, 'name': 'Sub Option', 'command': 'do_something_more', 'sub_sub_opt': {'id': 100, 'name': 'Sub Sub Option', 'command': 'do_something_crazy'}}}
{'id': 1, 'name': 'Option 1', 'sub_opt': {'id': 10, 'name': 'Sub Option', 'sub_sub_opt': {'id': 100, 'name': 'Sub Sub Option'}}}

Mongo Distinct Query with full row object

first of all i'm new to mongo so I don't know much and i cannot just remove duplicate rows due to some dependencies.
I have following data stored in mongo
{'id': 1, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'},
{'id': 2, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'},
{'id': 3, 'key': 'pehnvosjijipehnvosjijipehnvosjijipehnvosjijipehnvosjiji', 'name': 'some name', 'country': 'IN'},
{'id': 4, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'},
{'id': 5, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'}
you can see some of the rows are duplicate with different id
as long as it will take to solve this issue from input I must tackle it on output.
I need the data in the following way:
{'id': 1, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'},
{'id': 3, 'key': 'pehnvosjijipehnvosjijipehnvosjijipehnvosjijipehnvosjiji', 'name': 'some name', 'country': 'IN'},
{'id': 4, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'}
My query
keys = db.collection.distinct('key', {})
all_data = db.collection.find({'key': {$in: keys}})
As you can see it takes two queries for a same result set Please combine it to one as the database is very large
I might also create a unique key on the key but the value is so long (152 characters) that it will not help me.
Or it will??

You need to use the aggregation framework for this. There are multiple ways to do this, the solution below uses the $$ROOT variable to get the first document for each group:
db.data.aggregate([{
"$sort": {
"_id": 1
}
}, {
"$group": {
"_id": "$key",
"first": {
"$first": "$$ROOT"
}
}
}, {
"$project": {
"_id": 0,
"id":"$first.id",
"key":"$first.key",
"name":"$first.name",
"country":"$first.country"
}
}])

Python dictionary like Json

I have several lists of data which looks like this:
ISIN Currency Rates
26545555 Eur 0.12345
56554455 Eur 0.25665
75884554 Eur 0.89654
I want to save this data into a dictionary or json like format.
So I am trying to store the following data:
id: 0, ISIN: 26545555, Currency: Eur, Rates: 0.12345
id: 1, ISIN: 56554455, Currency: Eur, Rates: 0.25665
The problem is I am trying to use the following dictionary:
dict_data = {'id': '', 'ISIN': '', 'Currency': ''}
But when I try to append data it doesn't store all of the data.
I am getting the data from an Excel sheet using Pandas. If you think I should use something else, please let me know.

You should use list of dicts:
[
{'id': 0, 'ISIN': '', 'Currency': ''},
{'id': 1, 'ISIN': '', 'Currency': ''}
]
or if you want dict:
{
0: {'ISIN': '', 'Currency': ''},
1: {'ISIN': '', 'Currency': ''}
}
or(partly based on Marco's suggestion):
{
0: [26545555, 'Eur', 0.12345],
1: [26545555, 'Eur', 0.12345]
}
But personally I prefer second variant if you will need to access elements by ID.

One item is a dict, and group all data in a list
[
{'id': 0, 'ISIN': '', 'Currency': ''},
{'id': 1, 'ISIN': '', 'Currency': ''}
]

How to make one-line choices generator from nested dictionaries?

I have such structure:
actions = {
'cat1': {
'visit': {
'id': 1,
'description': 'Desc 1',
'action': 'Act 1',
},
},
'cat2': {
'download': {
'id': 2,
'description': 'Desc 2',
'action': 'Act 2',
},
'click': {
'id': 3,
'description': 'Desc 3',
'action': 'Act 3',
},
...
},
...
}
And following code for generating tuple of tuples for django choice field:
CHOICES = []
for a in actions.values():
for c in a.values():
CHOICES.append((c['id'], c['description']))
Is it possible to write above code in one line nested for loop?

CHOICES = [(c['id'], c['description']) for a in actions.values() for c in a.values()]

With map and reduce:
map(lambda x : [x['id'],x['description']],reduce(lambda x,y:x+y.values(),actions.values(),[]))