I have a long string with some data… I need to split it by & and then each of it by = and create from it pair like key: value from the last split… Is it possible to do without big loops? Something like:
video_data = video_data.split('&')
video_data = {key:value for value.split('=') in video_data.iteritems()}
It looks like you're trying to parse a query string. Python already has a method for this, and that will also handle multiple values for keys and automatically create a dict of key->list of values for you:
from urlparse import parse_qs
s = 'a=3&b=5&a=4'
qs = parse_qs(s)
# {'a': ['3', '4'], 'b': ['5']}
As noted by J.F. Sebastian in comments:
Note: it does more than just splits on & and = e.g :
parse_qs("a=%21&b=urlencoded")
# {'a': ['!'], 'b': ['urlencoded']}
Either urlparse.parse_qs() or urlparse.parse_qsl() would do this job for you, better, faster and more robustly:
>>> example = 'foo=bar&ham=eggs&answer=42'
>>> from urlparse import parse_qs, parse_qsl
>>> parse_qs(example)
{'answer': ['42'], 'foo': ['bar'], 'ham': ['eggs']}
>>> parse_qsl(example)
[('foo', 'bar'), ('ham', 'eggs'), ('answer', '42')]
>>> dict(parse_qsl(example))
{'answer': '42', 'foo': 'bar', 'ham': 'eggs'}
Use one or the other depending on how much you need to support keys appearing multiple times in the query string.
But you really wanted to do this yourself with a dict comprehension, you need to nest the .split() call into a tuple:
video_data = {key: value for item in video_data.split('&') for key, value in (item.split('='),)}
but the same parse is just easier without a dict comprehension; using a generator expression instead to produce a sequence of key-value pairs for the dict() factory instead:
video_data = dict(item.split('=') for item in video_data.split('&'))
Demo:
>>> example = 'foo=bar&ham=eggs&answer=42'
>>> {key: value for item in example.split('&') for key, value in (item.split('='),)}
{'answer': '42', 'foo': 'bar', 'ham': 'eggs'}
>>> dict(item.split('=') for item in example.split('&'))
{'answer': '42', 'foo': 'bar', 'ham': 'eggs'}
Related
I have a json below, and I want to parse out value from this dict.
I can do something like this to get one specific value
print(abc['everything']['A']['1']['tree']['value'])
But, what is best way to parse out all "value?"
I want to output good, bad, good.
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
If you are willing to use pandas, you could just use pd.json_normalize, which is actually quite fast:
import pandas as pd
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
df = pd.json_normalize(abc)
print(df.values[0])
['good' 'bad' 'good']
Without any extra libraries, you will have to iterate through your nested dictionary:
values = [abc['everything'][e][k][k1]['value'] for e in abc['everything'] for k in abc['everything'][e] for k1 in abc['everything'][e][k]]
print(values)
['good', 'bad', 'good']
Provided your keys and dictionaries have a value somewhere, you can try this:
Create a function (or reuse the code) that gets the first element of the dictionary until the value key exists, then return that. Note that there are other ways of doing this.
Iterate through, getting the result under each value key and return.
# Define function
def get(d):
while not "value" in d:
d = list(d.values())[0]
return d["value"]
# Get the results from your example
results = [get(v) for v in list(abc["everything"].values())]
['good', 'bad', 'good']
A Recursive way:
def fun(my_dict, values=[]):
if not isinstance(my_dict, dict):
return values
for i, j in my_dict.items():
if i == 'value':
values.append(j)
else:
values = fun(j, values)
return values
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
data = fun(abc)
print(data)
Output:
['good', 'bad', 'good']
Firstly, the syntax you are using is incorrect.
If you are using pandas, you can code like
import pandas as pd
df4 = pd.DataFrame({"TreeType": ["Tree1", "Tree2", "Tree3"],
"Values": ["Good", "Bad","Good"]})
df4.index = ["A","B","C"]
next just run the code df4, you would get the correct output.
output:
TreeType Values
A Tree1 Good
B Tree2 Bad
C Tree3 Good
*edit
I make
word=['I','love','hello','world','love','I']
when I convert to set, It change the order to
print(set(word))
output: {'world', 'I', 'hello', 'love'}
How to sort the set again to be
{'I', 'love', 'hello', 'world'}
Sets are unordered. If you want order, convert back to a list.
E.g.
print(sorted(set(word)))
sorted will sort your items and return a list.
However, if you want to retain the order of your elements rather than sort them, you can use a set for deduplication and a list for ordering, something like this:
def unique(items):
seen = set()
result = []
for item in items:
if item not in seen:
seen.add(item)
result.append(item)
return result
and use it as:
>>> word = ['I','love','hello','world','love','I']
>>> print(unique(word))
['I', 'love', 'hello', 'world']
If you just want an ordered collection of unique values, you can create a dict from the list, either with a dict comprehension or dict.fromkeys. In Python 3, dictionaries will retain insertion order; for older versions, use collections.OrderedDict. The dict will have values besides the keys, but you can just ignore those.
>>> word = ['a','b','c','c','b','e']
>>> {k: None for k in word}
{'a': None, 'b': None, 'c': None, 'e': None}
>>> dict.fromkeys(word)
{'a': None, 'b': None, 'c': None, 'e': None}
Other than sorted, this also works if the original order is different than the sorted order.
>>> word = ['f','a','b','c','c','b','e']
>>> dict.fromkeys(word)
{'f': None, 'a': None, 'b': None, 'c': None, 'e': None}
You can then either convert the result to list or keep it a dict and add more values, but if you make it a set, the order will be lost again. Like a set, the dict also allows fast O(1) lookup, but no set operations like intersection or union.
I am trying to find the output of this list of dict using Python. Because foo and data are unique keys, the output should merge them.
input = [{'foo': 'foo-main-123'}, {'foo': 'foo-main-345'}, {'data': 'data-main-111'}]
output = {'foo', ['foo-main-123', 'foo-main-345'], 'data': ['data-main-111']}
you can use defaultdict(list), then iterate each k,v pair in each dict in input.
try this:
from collections import defaultdict
input = [{'foo': 'foo-main-123'}, {'foo': 'foo-main-345'}, {'data': 'data-main-111'}]
output = defaultdict(list)
for d in input:
for k,v in d.items():
output[k].append(v)
output=dict(output)
print(output)
Output:
{'foo': ['foo-main-123', 'foo-main-345'], 'data': ['data-main-111']}
I may have understood this wrong but looking at the examples found in "Learning Python" by O'Reilly I tried to do the following:
>>> d={}
>>> d['h']='GG'
>>> d['f']='JJ'
>>> d['h']='PP'
>>> print d
{'h': 'PP', 'f': 'JJ'}
Now instead of the 'key' 'h' having two entries 'GG' and 'PP' it only has the last entry, the last one replacing the first one.
I want BOTH in the same key.
>>> d['h']+='RR'
>>> print d
{'h': 'PPRR', 'f': 'JJ'}
Again this doesn't work, what I wanted was not a concatenated string but comma-separated entires.
I am confused why this does not work.
Your use-case is handled nicely by the collections.defaultdict() type instead:
>>> from collections import defaultdict
>>> d = defaultdict(list)
>>> d['h'].append('GG')
>>> d['f'].append('JJ')
>>> d['h'].append('PP')
>>> d
defaultdict(<type 'list'>, {'h': ['GG', 'PP'], 'f': ['JJ']})
A regular dictionary maps one key to one value, if you want that value to be a list, then you should make it a list, and append to the list instead.
You don't have to use a defaultdict() object, you can always make your values explicit lists:
>>> d = {}
>>> d['h'] = ['GG']
>>> d['f'] = ['JJ']
>>> d['h'].append('PP')
>>> print d
{'h': ['GG', 'PP'], 'f': ['JJ']}
but now you need to create the lists explicitly. The latter problem can then be circumvented again by using dict.setdefault():
>>> d = {}
>>> d.setdefault('h', []).append('GG')
>>> d.setdefault('f', []).append('JJ')
>>> d.setdefault('h', []).append('PP')
which is just a more verbose way of using what defaultdict() objects can provide directly.
It sounds like you want your dictionary to have 'h' map to a list of strings, which you can do as follows:
>>> d={}
>>> d['f']='JJ'
>>> d['h']=['PP']
>>> d['h'].append( 'RR' )
>>> d
{'h': ['PP', 'RR'], 'f': 'JJ'}
If you want all the keys of your dictionary to map to a list (instead of just 'h'), you can use collection.defaultdict as demonstrated in #MartijnPieters's answer.
I have a dictionary which contains dictionaries, which may also contain dictionaries, e.g.
dictionary = {'ID': 0001, 'Name': 'made up name', 'Transactions':
{'Transaction Ref': 'a1', 'Transaction Details':
{'Bill To': 'abc', 'Ship To': 'def', 'Product': 'Widget A'
...} ...} ... }
Currently I'm unpacking to get the 'Bill To' for ID 001, 'Transaction Ref' a1 as follows:
if dictionary['ID'] == 001:
transactions = dictionary['Transactions']
if transactions['Transaction Ref'] == 'a1':
transaction_details = transactions['Transaction Details']
bill_to = transaction_details['Bill To']
I can't help but think this is is a little clunky, especially the last two lines - I feel like something along the lines of the following should work:
bill_to = transactions['Transaction Details']['Bill To']
Is there a simpler approach for drilling down into nested dictionaries without having to unpack into interim variables?
You can use something like this:
>>> def lookup(dic, key, *keys):
... if keys:
... return lookup(dic.get(key, {}), *keys)
... return dic.get(key)
...
>>> d = {'a':{'b':{'c':5}}}
>>> print lookup(d, 'a', 'b', 'c')
5
>>> print lookup(d, 'a', 'c')
None
Additionally, if you don't want to define your search keys as individual parameters, you can just pass them in as a list like this:
>>> print lookup(d, *['a', 'b', 'c'])
5
>>> print lookup(d, *['a', 'c'])
None
bill_to = transactions['Transaction Details']['Bill To']
actually works. transactions['Transaction Details'] is an expression denoting a dict, so you can do lookup in it. For practical programs, I would prefer an OO approach to nested dicts, though. collections.namedtuple is particularly useful for quickly setting up a bunch of classes that only contain data (and no behavior of their own).
There's one caveat: in some settings, you might want to catch KeyError when doing lookups, and in this setting, that works too, it's hard to tell which dictionary lookup failed:
try:
bill_to = transactions['Transaction Details']['Bill To']
except KeyError:
# which of the two lookups failed?
# we don't know unless we inspect the exception;
# but it's easier to do the lookup and error handling in two steps
Following is another way of accessing nested dictionaries
>>> dbo={'m':{'d':{'v':{'version':1}}}}
>>> name='m__d__v__version' # it'll refer to 'dbo['m']['d']['v']['version']', '__' is the separator
>>> version = reduce(dict.get, name.split('__'), dbo)
>>> print version
1
>>>
Here, variable 'name' refers to 'dbo['m']['d']['v']['version']', which seems much shorter and neat.
This method will not throw KeyError. If a key is not found then you'll get 'None'.
Ref.: http://code.activestate.com/recipes/475156-using-reduce-to-access-deeply-nested-dictionaries/
You can access nested dictionaries with a tuple using NestedDict.
>>> from ndicts.ndicts import NestedDict
>>> nested_dict = {"a": {"a": 0, "b": 1},
... "b": {"a": 2, "b": 3}}
>>> nd = NestedDict(nested_dict)
>>> nd["a", "a"]
0
>>> nd["b", "a"]
2
>>> nd["a"]
{"a": 0, "b": 1}
To install ndicts:
pip install ndicts