I am looking for an efficient python method to utilise a hash table that has two keys:
E.g.:
(1,5) --> {a}
(2,3) --> {b,c}
(2,4) --> {d}
Further I need to be able to retrieve whole blocks of entries, for example all entries that have "2" at the 0-th position (here: (2,3) as well as (2,4)).
In another post it was suggested to use list comprehension, i.e.:
sum(val for key, val in dict.items() if key[0] == 'B')
I learned that dictionaries are (probably?) the most efficient way to retrieve a value from an object of key:value-pairs. However, calling only an incomplete tuple-key is a bit different than querying the whole key where I either get a value or nothing. I want to ask if python can still return the values in a time proportional to the number of key:value-pairs that match? Or alternatively, is the tuple-dictionary (plus list comprehension) better than using pandas.df.groupby() (but that would occupy a bit much memory space)?
The "standard" way would be something like
d = {(randint(1,10),i):"something" for i,x in enumerate(range(200))}
def byfilter(n,d):
return list(filter(lambda x:x==n, d.keys()))
byfilter(5,d) ##returns a list of tuples where x[0] == 5
Although in similar situations I often used next() to iterate manually, when I didn't need the full list.
However there may be some use cases where we can optimize that. Suppose you need to do a couple or more accesses by key first element, and you know the dict keys are not changing meanwhile. Then you can extract the keys in a list and sort it, and make use of some itertools functions, namely dropwhile() and takewhile():
ls = [x for x in d.keys()]
ls.sort() ##I do not know why but this seems faster than ls=sorted(d.keys())
def bysorted(n,ls):
return list(takewhile(lambda x: x[0]==n, dropwhile(lambda x: x[0]!=n, ls)))
bysorted(5,ls) ##returns the same list as above
This can be up to 10x faster in the best case (i=1 in my example) and more or less take the same time in the worst case (i=10) because we are trimming the number of iterations needed.
Of course you can do the same for accessing keys by x[1], you just need to add a key parameter to the sort() call
I have a list of dicts. Among other elements, each dict in the list has a date and time element which looks like - 2018-08-14T14:42:14. I have written a function which compares two such strings and returns the most recent one. How do I use this to sort the list (most recent first)? Also, each dict is quite big in size hence, if possible, I would like to get the indices of array sorted (according to the time element) rather than the whole array. I have seen other similar questions on this site but all of them tell about sorting basing on a known data type like int or string.
The dates written in ISO format have one nice property - if you sort them alphabetically, you sort them according to date values also (if the belong to one timezone, of course). Just use list.sort() function to do that. That will sort list in-place. Anyway you should not worry about memory, since creating the second sorted list will not take much memory since it holds references to dictionaries in the first list.
a = [
{'time': '2018-01-02T00:00:00Z'},
{'time': '2018-01-01T00:00:00Z'},
]
a.sort(key=lambda x: x['time'])
print(a)
We sort on the time by converting it to a python datetime object, which has natural ordering like int. So, you need not worry about the format of the time string.
# l is the list of the dicts, each dict contains a key "time".
l = sorted(l, key=lambda x: datetime.datetime.strptime(x["time"], '%Y-%m-%dT%H:%M:%S'))
My problem is understanding why these certain lines of code do what they do. Basically why it works logically. I am using PyCharm python 3 I think.
house_Number = {
"Luca": 1, "David": 2, "Alex": 3, "Kaden": 4, "Kian": 5
}
for item in house_Number:
print(house_Number[item]) # Why does this print the values tied with the key?
print(item) # Why does this print the key?
This is my first question so sorry I don't know how to format the code to make it look nice. My question is why when you use the for loop to print the dictionary key or value the syntax to print the key is to print every item? And what does it even mean to print(house_Number[item]).
They both work to print key or value but I really want to know a logical answer as to why it works this way. Thanks :D
I'm not working on any projects just starting to learn off of codeacademey.
In Python, iteration over a dictionary (for item in dict) is defined as iteration over that dictionary's keys. This is simply how the language was designed -- other languages and collection classes do it differently, iterating, for example, over key-value tuples, templated Pair<X,Y> objects, or what have you.
house_Number[item] accesses the value in house_Number referenced by the key item. [...] is the syntax for indexing in Python (and most other languages); an_array[2] gives the third element of an_array and house_Number[item] gives the value corresponding to the key item in the dictionary house_Number.
Just a side note: Python naming conventions would dictate house_number, not house_Number. Capital letters are generally only used in CamelCasedClassNames and CONSTANTS.
In python values inside a dictionary object are accessed using dictionay_name['KEY']
In your case you are iterating over the keys of dictionary
Hope this helps
for item in dic:
print(item) # key
print(dic[item]) # value
Dictionaries are basically containers containing some items (keys) which are stored by hashing method. These keys just map to the values (dic[key]).
Like in set, if you traverse using for loop, you get the keys from it (in random order since they are hashed). Similarly, dictionaries are just sets with a value associated with it. it makes more sense to iterate the keys as in sets (too in random order).
Read more about dicionaries here https://docs.python.org/3/tutorial/datastructures.html#dictionaries and hopefully that will answer your question. Specifically, look at the .items() method of the dictionary object.
When you type for item in house_Number, you don’t specify whether item is the key or value of house_Number. Then python just thinks that you meant the key of house_Number.
So when you do the function print(house_Number[item]), you’re printing the value because your taking the key and finding the value. In other words, you taking each key once, and finding their values, which are 1, 2, 3, 4, 5, 6
The print(item) is just to print the item, which are the keys, "Luca", "David", "Alex", "Kaden", "Kian"
Because the print(house_Number[item]) and print(item) alternating, you get the keys and values alternating, each on a new line.
When should I use a dictionary, list or set?
Are there scenarios that are more suited for each data type?
A list keeps order, dict and set don't: when you care about order, therefore, you must use list (if your choice of containers is limited to these three, of course ;-) ).
dict associates each key with a value, while list and set just contain values: very different use cases, obviously.
set requires items to be hashable, list doesn't: if you have non-hashable items, therefore, you cannot use set and must instead use list.
set forbids duplicates, list does not: also a crucial distinction. (A "multiset", which maps duplicates into a different count for items present more than once, can be found in collections.Counter -- you could build one as a dict, if for some weird reason you couldn't import collections, or, in pre-2.7 Python as a collections.defaultdict(int), using the items as keys and the associated value as the count).
Checking for membership of a value in a set (or dict, for keys) is blazingly fast (taking about a constant, short time), while in a list it takes time proportional to the list's length in the average and worst cases. So, if you have hashable items, don't care either way about order or duplicates, and want speedy membership checking, set is better than list.
Do you just need an ordered sequence of items? Go for a list.
Do you just need to know whether or not you've already got a particular value, but without ordering (and you don't need to store duplicates)? Use a set.
Do you need to associate values with keys, so you can look them up efficiently (by key) later on? Use a dictionary.
When you want an unordered collection of unique elements, use a set. (For example, when you want the set of all the words used in a document).
When you want to collect an immutable ordered list of elements, use a tuple. (For example, when you want a (name, phone_number) pair that you wish to use as an element in a set, you would need a tuple rather than a list since sets require elements be immutable).
When you want to collect a mutable ordered list of elements, use a list. (For example, when you want to append new phone numbers to a list: [number1, number2, ...]).
When you want a mapping from keys to values, use a dict. (For example, when you want a telephone book which maps names to phone numbers: {'John Smith' : '555-1212'}). Note the keys in a dict are unordered. (If you iterate through a dict (telephone book), the keys (names) may show up in any order).
Use a dictionary when you have a set of unique keys that map to values.
Use a list if you have an ordered collection of items.
Use a set to store an unordered set of items.
In short, use:
list - if you require an ordered sequence of items.
dict - if you require to relate values with keys
set - if you require to keep unique elements.
Detailed Explanation
List
A list is a mutable sequence, typically used to store collections of homogeneous items.
A list implements all of the common sequence operations:
x in l and x not in l
l[i], l[i:j], l[i:j:k]
len(l), min(l), max(l)
l.count(x)
l.index(x[, i[, j]]) - index of the 1st occurrence of x in l (at or after i and before j indeces)
A list also implements all of the mutable sequence operations:
l[i] = x - item i of l is replaced by x
l[i:j] = t - slice of l from i to j is replaced by the contents of the iterable t
del l[i:j] - same as l[i:j] = []
l[i:j:k] = t - the elements of l[i:j:k] are replaced by those of t
del l[i:j:k] - removes the elements of s[i:j:k] from the list
l.append(x) - appends x to the end of the sequence
l.clear() - removes all items from l (same as del l[:])
l.copy() - creates a shallow copy of l (same as l[:])
l.extend(t) or l += t - extends l with the contents of t
l *= n - updates l with its contents repeated n times
l.insert(i, x) - inserts x into l at the index given by i
l.pop([i]) - retrieves the item at i and also removes it from l
l.remove(x) - remove the first item from l where l[i] is equal to x
l.reverse() - reverses the items of l in place
A list could be used as stack by taking advantage of the methods append and pop.
Dictionary
A dictionary maps hashable values to arbitrary objects. A dictionary is a mutable object. The main operations on a dictionary are storing a value with some key and extracting the value given the key.
In a dictionary, you cannot use as keys values that are not hashable, that is, values containing lists, dictionaries or other mutable types.
Set
A set is an unordered collection of distinct hashable objects. A set is commonly used to include membership testing, removing duplicates from a sequence, and computing mathematical operations such as intersection, union, difference, and symmetric difference.
For C++ I was always having this flow chart in mind: In which scenario do I use a particular STL container?, so I was curious if something similar is available for Python3 as well, but I had no luck.
What you need to keep in mind for Python is: There is no single Python standard as for C++. Hence there might be huge differences for different Python interpreters (e.g. CPython, PyPy). The following flow chart is for CPython.
Additionally I found no good way to incorporate the following data structures into the diagram: bytes, byte arrays, tuples, named_tuples, ChainMap, Counter, and arrays.
OrderedDict and deque are available via collections module.
heapq is available from the heapq module
LifoQueue, Queue, and PriorityQueue are available via the queue module which is designed for concurrent (threads) access. (There is also a multiprocessing.Queue available but I don't know the differences to queue.Queue but would assume that it should be used when concurrent access from processes is needed.)
dict, set, frozen_set, and list are builtin of course
For anyone I would be grateful if you could improve this answer and provide a better diagram in every aspect. Feel free and welcome.
PS: the diagram has been made with yed. The graphml file is here
Although this doesn't cover sets, it is a good explanation of dicts and lists:
Lists are what they seem - a list of values. Each one of them is
numbered, starting from zero - the first one is numbered zero, the
second 1, the third 2, etc. You can remove values from the list, and
add new values to the end. Example: Your many cats' names.
Dictionaries are similar to what their name suggests - a dictionary.
In a dictionary, you have an 'index' of words, and for each of them a
definition. In python, the word is called a 'key', and the definition
a 'value'. The values in a dictionary aren't numbered - tare similar
to what their name suggests - a dictionary. In a dictionary, you have
an 'index' of words, and for each of them a definition. The values in
a dictionary aren't numbered - they aren't in any specific order,
either - the key does the same thing. You can add, remove, and modify
the values in dictionaries. Example: telephone book.
http://www.sthurlow.com/python/lesson06/
In combination with lists, dicts and sets, there are also another interesting python objects, OrderedDicts.
Ordered dictionaries are just like regular dictionaries but they remember the order that items were inserted. When iterating over an ordered dictionary, the items are returned in the order their keys were first added.
OrderedDicts could be useful when you need to preserve the order of the keys, for example working with documents: It's common to need the vector representation of all terms in a document. So using OrderedDicts you can efficiently verify if a term has been read before, add terms, extract terms, and after all the manipulations you can extract the ordered vector representation of them.
May be off topic in terms of the question OP asked-
List: A unhashsable collection of ordered, mutable objects.
Tuple: A hashable collection of ordered, immutable objects, like
list.
Set: An unhashable collection of unordered, mutable and distinct
objects.
Frozenset: A hashable collection of unordered, immutable and
distinct objects.
Dictionary : A unhashable,unordered collection of mutable objects
that maps hashable values to arbitrary values.
To compare them visually, at a glance, see the image-
Lists are what they seem - a list of values. Each one of them is numbered, starting from zero - the first one is numbered zero, the second 1, the third 2, etc. You can remove values from the list, and add new values to the end. Example: Your many cats' names.
Tuples are just like lists, but you can't change their values. The values that you give it first up, are the values that you are stuck with for the rest of the program. Again, each value is numbered starting from zero, for easy reference. Example: the names of the months of the year.
Dictionaries are similar to what their name suggests - a dictionary. In a dictionary, you have an 'index' of words, and for each of them a definition. In python, the word is called a 'key', and the definition a 'value'. The values in a dictionary aren't numbered - tare similar to what their name suggests - a dictionary. In a dictionary, you have an 'index' of words, and for each of them a definition. In python, the word is called a 'key', and the definition a 'value'. The values in a dictionary aren't numbered - they aren't in any specific order, either - the key does the same thing. You can add, remove, and modify the values in dictionaries. Example: telephone book.
When use them, I make an exhaustive cheatsheet of their methods for your reference:
class ContainerMethods:
def __init__(self):
self.list_methods_11 = {
'Add':{'append','extend','insert'},
'Subtract':{'pop','remove'},
'Sort':{'reverse', 'sort'},
'Search':{'count', 'index'},
'Entire':{'clear','copy'},
}
self.tuple_methods_2 = {'Search':'count','index'}
self.dict_methods_11 = {
'Views':{'keys', 'values', 'items'},
'Add':{'update'},
'Subtract':{'pop', 'popitem',},
'Extract':{'get','setdefault',},
'Entire':{ 'clear', 'copy','fromkeys'},
}
self.set_methods_17 ={
'Add':{['add', 'update'],['difference_update','symmetric_difference_update','intersection_update']},
'Subtract':{'pop', 'remove','discard'},
'Relation':{'isdisjoint', 'issubset', 'issuperset'},
'operation':{'union' 'intersection','difference', 'symmetric_difference'}
'Entire':{'clear', 'copy'}}
Dictionary: A python dictionary is used like a hash table with key as index and object as value.
List: A list is used for holding objects in an array indexed by position of that object in the array.
Set: A set is a collection with functions that can tell if an object is present or not present in the set.
Dictionary: When you want to look up something using something else than indexes. Example:
dictionary_of_transport = {
"cars": 8,
"boats": 2,
"planes": 0
}
print("I have the following amount of planes:")
print(dictionary_of_transport["planes"])
#Output: 0
List and sets: When you want to add and remove values.
Lists: To look up values using indexes
Sets: To have values stored, but you cannot access them using anything.