We Keep Coding

Creating a list by iterating over a dictionary

I defined a dictionary like this (list is a list of integers):
my_dictionary = {'list_name' : list, 'another_list_name': another_list}
Now, I want to create a new list by iterating over this dictionary. In the end, I want it to look like this:
my_list = [list_name_list_item1, list_name_list_item2,
list_name_list_item3, another_list_name_another_list_item1]
And so on.
So my question is: How can I realize this?
I tried
for key in my_dictionary.keys():
k = my_dictionary[key]
for value in my_dictionary.values():
v = my_dictionary[value]
v = str(v)
my_list.append(k + '_' + v)
But instead of the desired output I receive a Type Error (unhashable type: 'list') in line 4 of this example.

You're trying to get a dictionary item by it's value whereas you already have your value.
Do it in one line using a list comprehension:
my_dictionary = {'list_name' : [1,4,5], 'another_list_name': [6,7,8]}
my_list = [k+"_"+str(v) for k,lv in my_dictionary.items() for v in lv]
print(my_list)
result:
['another_list_name_6', 'another_list_name_7', 'another_list_name_8', 'list_name_1', 'list_name_4', 'list_name_5']
Note that since the order in your dictionary is not guaranteed, the order of the list isn't either. You could fix the order by sorting the items according to keys:
my_list = [k+"_"+str(v) for k,lv in sorted(my_dictionary.items()) for v in lv]

Try this:
my_list = []
for key in my_dictionary:
for item in my_dictionary[key]:
my_list.append(str(key) + '_' + str(item))
Hope this helps.

Your immediate problem is that dict().values() is a generator yielding the values from the dictionary, not the keys, so when you attempt to do a lookup on line 4, it fails (in this case) as the values in the dictionary can't be used as keys. In another case, say {1:2, 3:4}, it would fail with a KeyError, and {1:2, 2:1} would not raise an error, but likely give confusing behaviour.
As for your actual question, lists do not attribute any names to data, like dictionaries do; they simply store the index.
def f()
a = 1
b = 2
c = 3
l = [a, b, c]
return l
Calling f() will return [1, 2, 3], with any concept of a, b, and c being lost entirely.
If you want to simply concatenate the lists in your dictionary, making a copy of the first, then calling .extend() on it will suffice:
my_list = my_dictionary['list_name'][:]
my_list.extend(my_dictionary['another_list_name'])
If you're looking to keep the order of the lists' items, while still referring to them by name, look into the OrderedDict class in collections.

You've written an outer loop over keys, then an inner loop over values, and tried to use each value as a key, which is where the program failed. Simply use the dictionary's items method to iterate over key,value pairs instead:
["{}_{}".format(k,v) for k,v in d.items()]
Oops, failed to parse the format desired; we were to produce each item in the inner list. Not to worry...
d={1:[1,2,3],2:[4,5,6]}
list(itertools.chain(*(
["{}_{}".format(k,i) for i in l]
for (k,l) in d.items() )))
This is a little more complex. We again take key,value pairs from the dictionary, then make an inner loop over the list that was the value and format those into strings. This produces inner sequences, so we flatten it using chain and *, and finally save the result as one list.
Edit: Turns out Python 3.4.3 gets quite confused when doing this nested as generator expressions; I had to turn the inner one into a list, or it would replace some combination of k and l before doing the formatting.
Edit again: As someone posted in a since deleted answer (which confuses me), I'm overcomplicating things. You can do the flattened nesting in a chained comprehension:
["{}_{}".format(k,v) for k,l in d.items() for v in l]
That method was also posted by Jean-François Fabre.

Use list comprehensions like this
d = {"test1":[1,2,3,],"test2":[4,5,6],"test3":[7,8,9]}
new_list = [str(item[0])+'_'+str(v) for item in d.items() for v in item[1]]
Output：
new_list:
['test1_1',
'test1_2',
'test1_3',
'test3_7',
'test3_8',
'test3_9',
'test2_4',
'test2_5',
'test2_6']

Let's initialize our data
In [1]: l0 = [1, 2, 3, 4]
In [2]: l1 = [10, 20, 30, 40]
In [3]: d = {'name0': l0, 'name1': l1}
Note that in my example, different from yours, the lists' content is not strings... aren't lists heterogeneous containers?
That said, you cannot simply join the keys and the list's items, you'd better cast these value to strings using the str(...) builtin.
Now it comes the solution to your problem... I use a list comprehension
with two loops, the outer loop comes first and it is on the items (i.e., key-value couples) in the dictionary, the inner loop comes second and it is on the items in the corresponding list.
In [4]: res = ['_'.join((str(k), str(i))) for k, l in d.items() for i in l]
In [5]: print(res)
['name0_1', 'name0_2', 'name0_3', 'name0_4', 'name1_10', 'name1_20', 'name1_30', 'name1_40']
In [6]:
In your case, using str(k)+'_'+str(i) would be fine as well, but the current idiom for joining strings with a fixed 'text' is the 'text'.join(...) method. Note that .join takes a SINGLE argument, an iterable, and hence in the list comprehension I used join((..., ...))
to collect the joinands in a single argument.

Iterating over first half a dictionary in python

How to iterate over first half of dictionary in python
This iterates over all values in the dictionary
for key, value in checkbox_dict.iteritems():
print key,value
But I want to iterate over the first half of the dictionary only.
one way is to do it like this
for key, value in dict(checkbox_dict.items()[:11]).iteritems():
print key,value
Is there any better way also ?

If you mean: over half of the items here's a way:
for key, value in checkbox_dict.items()[:int(len(checkbox_dict)/2)]:
pass
… But be aware: the elements in a normal dictionary don't necessarily keep the same iteration order that was used for inserting the elements … unless you use an OrderedDict.

You can use itertools.islice to slice the dict.iteritems iterator, unlike dict.items() with slice this won't create an intermediate list in memory.
>>> from itertools import islice
>>> d = dict.fromkeys('abcdefgh')
>>> for k, v in islice(d.iteritems(), len(d)/2):
print k, v
...
a None
c None
b None
e None
Note that normal dictionaries are unordered, so the items are returned in arbitrary order.

Why can't I iterate over a Counter in Python?

Why is that when I try to do the below, I get the need more than 1 value to unpack?
for key,value in countstr:
print key,value
for key,value in countstr:
ValueError: need more than 1 value to unpack
However this works just fine:
for key,value in countstr.most_common():
print key,value
I don't understand, aren't countstr and countstr.most_common() equivalent?
EDIT:
Thanks for the below answers, then I guess what I don't understand is: If countstr is a mapping what is countstr.most_common()? -- I'm really new to Python, sorry if I am missing something simple here.

No, they're not. Iterating over a mapping (be it a collections.Counter or a dict or ...) iterates only over the mapping's keys.
And there's another difference: iterating over the keys of a Counter delivers them in no defined order. The order returned by most_common() is defined (sorted in reverse order of value).

No, they aren't equivalent. countstr is a Counter which is a dictionary subclass. Iterating over it yields 1 key at a time. countstr.most_common() is a list which contains 2-tuples (ordered key-value pairs).

A countstr is a Counter which is a subclass for counting hashable objects. It is an unordered collection where elements are stored as dictionary keys and their counts are stored as dictionary values.
>>> c = Counter(a=4, b=2, c=0, d=-2)
>>> list(c.elements())
['a', 'a', 'a', 'a', 'b', 'b']

You can't iterate the counter values/keys directly but you can copy them into a list and iterate the list.
s = ["bcdef", "abcdefg", "bcde", "bcdef"]
import collections
counter=collections.Counter(s)
vals = list(counter.values())
keys = list(counter.keys())
vals[0]
keys[0]
Output:
2
'bcdef'

Get elements of a tuple-indexed dictionary specifying only one field of the tuple

I've got a big (20k+) set of data in a form of a dictionary indexed by tuple, e.g.
a = {(1,'000200','l1p'): 53, (15,'230512','l3c'): 81, ...}
I would like to filter that dictionary providing only one field of that tuple, e.g.
a[(_,_,'l1p')]`, or `a[(:,:,'l1p')]
Is there any better way than creating a list, like
[i for i in a.keys() if 'l1p' in i]
As I said, I'm trying to avoid copying elements as there are many entries in the dictionary.
EDIT: Is there any other way of obtaining the elements with 'l1p' in the key-tuple than iterating over the whole dictionary? I would like to perform a recursive least-square fitting on resultant sub-list.

First of all, what you have is a dictionary, not a list (and definitely not a tuple). Lists and tuples are just sequences of values numbered 0, 1, 2, ..., etc., while a dictionary is an unordered set of values each labelled & accessed with a unique key (in this case, the tuples).
With that out of the way, to get all of the values of a where the third element of the key is 'l1p', you can just do:
[v for k,v in a.items() if k[2] == 'l1p']
If you're concerned about saving memory and won't be trying to evaluate the entire result at once, this can be rewritten as a generator expression:
(v for k,v in a.items() if k[2] == 'l1p')
Note that, if you're using Python 2, a.items() will need to be changed to a.iteritems(), or the change to a generator will have been for naught.
Alternatively, if you want to instead get a sub-dictionary that includes the matching keys, do:
{k: v for k,v in a.items() if k[2] == 'l1p'}
Note that this is not a memory-friendly option. The closest analogue using a generator would be to create a generator of (key, value) pairs rather than a proper dictionary:
((k,v) for k,v in a.items() if k[2] == 'l1p')

Storing and updating lists in Python dictionaries: why does this happen?

I have a list of data that looks like the following:
// timestep,x_position,y_position
0,4,7
0,2,7
0,9,5
0,6,7
1,2,5
1,4,7
1,9,0
1,6,8
... and I want to make this look like:
0, (4,7), (2,7), (9,5), (6,7)
1, (2,5), (4,7), (9,0), (6.8)
My plan was to use a dictionary, where the value of t is the key for the dictionary, and the value against the key would be a list. I could then append each (x,y) to the list. Something like:
# where t = 0, c = (4,7), d = {}
# code 1
d[t].append(c)
Now this causes IDLE to fail. However, if I do:
# code 2
d[t] = []
d[t].append(c)
... this works.
So the question is: why does code 2 work, but code 1 doesn't?
PS Any improvement on what I'm planning on doing would be of great interest!! I think I will have to check the dictionary on each loop through the input to see if the dictionary key already exists, I guess by using something like max(d.keys()): if it is there, append data, if not create the empty list as the dictionary value, and then append data on the next loop through.

Let's look at
d[t].append(c)
What is the value of d[t]? Try it.
d = {}
t = 0
d[t]
What do you get? Oh. There's nothing in d that has a key of t.
Now try this.
d[t] = []
d[t]
Ahh. Now there's something in d with a key of t.
There are several things you can do.
Use example 2.
Use setdefault. d.setdefault(t,[]).append(c).
Use collections.defaultdict. You'd use a defaultdict(list) instead of a simple dictionary, {}.
Edit 1. Optimization
Given input lines from a file in the above form: ts, x, y, the grouping process is needless. There's no reason to go from a simple list of ( ts, x, y ) to a more complex
list of ( ts, (x,y), (x,y), (x,y), ... ). The original list can be processed exactly as it arrived.
d= collections.defaultdict(list)
for ts, x, y in someFileOrListOrQueryOrWhatever:
d[ts].append( (x,y) )
Edit 2. Answer Question
"when initialising a dictionary, you need to tell the dictionary what the key-value data structure will look like?"
I'm not sure what the question means. Since, all dictionaries are key-value structures, the question's not very clear. So, I'll review the three alternatives, which may answer the question.
Example 2.
Initialization
d= {}
Use
if t not in d:
d[t] = list()
d[t].append( c )
Each dictionary value must be initialized to some useful structure. In this case, we check to see if the key is present; when the key is missing, we create the key and assign an empty list.
Setdefault
Initialization
d= {}
Use
d.setdefault(t,list()).append( c )
In this case, we exploit the setdefault method to either fetch a value associated with a key or create a new value associated with a missing key.
default dict
Initialization
import collections
d = collections.defaultdict(list)
Use
d[t].append( c )
The defaultdict uses an initializer function for missing keys. In this case, we provide the list function so that a new, empty list is created for a missing key.

I think you want to use setdefault. It's a bit weird to use but does exactly what you need.
d.setdefault(t, []).append(c)
The .setdefault method will return the element (in our case, a list) that's bound to the dict's key t if that key exists. If it doesn't, it will bind an empty list to the key t and return it. So either way, a list will be there that the .append method can then append the tuple c to.

dict=[] //it's not a dict, it's a list, the dictionary is dict={}
elem=[1,2,3]
dict.append(elem)
you can access the single element in this way:
print dict[0] // 0 is the index
the output will be:
[1, 2, 3]

In the case your data is not already sorted by desired criteria, here's the code that might help to group the data:
#!/usr/bin/env python
"""
$ cat data_shuffled.txt
0,2,7
1,4,7
0,4,7
1,9,0
1,2,5
0,6,7
1,6,8
0,9,5
"""
from itertools import groupby
from operator import itemgetter
# load the data and make sure it is sorted by the first column
sortby_key = itemgetter(0)
data = sorted((map(int, line.split(',')) for line in open('data_shuffled.txt')),
key=sortby_key)
# group by the first column
grouped_data = []
for key, group in groupby(data, key=sortby_key):
assert key == len(grouped_data) # assume the first column is 0,1, ...
grouped_data.append([trio[1:] for trio in group])
# print the data
for i, pairs in enumerate(grouped_data):
print i, pairs
Output:
0 [[2, 7], [4, 7], [6, 7], [9, 5]]
1 [[4, 7], [9, 0], [2, 5], [6, 8]]