We Keep Coding

Pass same keyword argument to a function twice

This is two questions, and I'd be happy with either being answered (unless one method is preferred to another).
I have a function, say
def my_func(a, b, c, d = None):
print(a)
print(f"I like {b}")
print(f"I am {c}")
if d:
print(d)
I have a dictionary of keywords my_dict = {'a': 'Hello', 'b': 'Dogs', 'c': 'Happy', 'd': 10} which are always passed as inputs to the function my_func(**kwargs).
My questions are:
If I want to input a different value, say a='Goodbye', is there a way I can input the same argument twice overriding the first entry with the second instance of it?
Alternatively, is there something comparable to my_dict.update({'a': 'Hello'}) that won't change the values in the dictionary permanently, but will let me pass in a different value for a specific keyword?
I know I can create a new dictionary and pass that in, but I think it would be cleaner if I could do it without needing to do this (though feel free to correct me if I'm wrong!).
Edit: I'm using Python 3.8.

On Python 3.5 and up, you can unpack my_dict into a new dict and override the 'a' entry:
my_func(**{**my_dict, 'a': 'Goodbye'})
On Python 3.9 and up, you can use the | operator to create a new dict by merging entries from two dicts. Values for duplicated keys will be taken from the second dict:
my_func(**my_dict | {'a': 'Goodbye'})

Since Python 3.3, you can use collections.ChainMap:
A ChainMap groups multiple dicts or other mappings together to create
a single, updateable view.[...]
Lookups search the underlying mappings successively until a key is
found. [...]
A ChainMap incorporates the underlying mappings by reference. So, if
one of the underlying mappings gets updated, those changes will be
reflected in ChainMap.
So, your code could be:
from collections import ChainMap
def my_func(a, b, c):
print(a)
print(f"I like {b}")
print(f"I am {c}")
my_dict = {'a': 'Hello', 'b': 'Dogs', 'c': 'Happy'}
new = {'a':'Goodbye' }
my_func(**ChainMap(new, my_dict))
#Goodbye
#I like Dogs
#I am Happy
Note that new must come before my_dict, as the value will be taken from the first dict that contains the key.
Since Python 3.10, you can use the | operator:
d | other
Create a new dictionary with the merged keys and values of d
and other, which must both be dictionaries. The values of other take
priority when d and other share keys.
So, you could call your function with
my_func(**(my_dict | new))
Note that in this case, new must come after `mydict.

The following code is gives the output as i = 0, 1, 2, 3, 4 for some reason. Can anyone explain how this is happening? [duplicate]

Let's say we have a Python dictionary d, and we're iterating over it like so:
for k, v in d.iteritems():
del d[f(k)] # remove some item
d[g(k)] = v # add a new item
(f and g are just some black-box transformations.)
In other words, we try to add/remove items to d while iterating over it using iteritems.
Is this well defined? Could you provide some references to support your answer?
See also How to avoid "RuntimeError: dictionary changed size during iteration" error? for the separate question of how to avoid the problem.

Alex Martelli weighs in on this here.
It may not be safe to change the container (e.g. dict) while looping over the container.
So del d[f(k)] may not be safe. As you know, the workaround is to use d.copy().items() (to loop over an independent copy of the container) instead of d.iteritems() or d.items() (which use the same underlying container).
It is okay to modify the value at an existing index of the dict, but inserting values at new indices (e.g. d[g(k)] = v) may not work.

It is explicitly mentioned on the Python doc page (for Python 2.7) that
Using iteritems() while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries.
Similarly for Python 3.
The same holds for iter(d), d.iterkeys() and d.itervalues(), and I'll go as far as saying that it does for for k, v in d.items(): (I can't remember exactly what for does, but I would not be surprised if the implementation called iter(d)).

You cannot do that, at least with d.iteritems(). I tried it, and Python fails with
RuntimeError: dictionary changed size during iteration
If you instead use d.items(), then it works.
In Python 3, d.items() is a view into the dictionary, like d.iteritems() in Python 2. To do this in Python 3, instead use d.copy().items(). This will similarly allow us to iterate over a copy of the dictionary in order to avoid modifying the data structure we are iterating over.

I have a large dictionary containing Numpy arrays, so the dict.copy().keys() thing suggested by #murgatroid99 was not feasible (though it worked). Instead, I just converted the keys_view to a list and it worked fine (in Python 3.4):
for item in list(dict_d.keys()):
temp = dict_d.pop(item)
dict_d['some_key'] = 1 # Some value
I realize this doesn't dive into the philosophical realm of Python's inner workings like the answers above, but it does provide a practical solution to the stated problem.

The following code shows that this is not well defined:
def f(x):
return x
def g(x):
return x+1
def h(x):
return x+10
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[g(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[h(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
The first example calls g(k), and throws an exception (dictionary changed size during iteration).
The second example calls h(k) and throws no exception, but outputs:
{21: 'axx', 22: 'bxx', 23: 'cxx'}
Which, looking at the code, seems wrong - I would have expected something like:
{11: 'ax', 12: 'bx', 13: 'cx'}

Python 3 you should just:
prefix = 'item_'
t = {'f1': 'ffw', 'f2': 'fca'}
t2 = dict()
for k,v in t.items():
t2[k] = prefix + v
or use:
t2 = t1.copy()
You should never modify original dictionary, it leads to confusion as well as potential bugs or RunTimeErrors. Unless you just append to the dictionary with new key names.

This question asks about using an iterator (and funny enough, that Python 2 .iteritems iterator is no longer supported in Python 3) to delete or add items, and it must have a No as its only right answer as you can find it in the accepted answer. Yet: most of the searchers try to find a solution, they will not care how this is done technically, be it an iterator or a recursion, and there is a solution for the problem:
You cannot loop-change a dict without using an additional (recursive) function.
This question should therefore be linked to a question that has a working solution:
How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete")
Also helpful as it shows how to change the items of a dict on the run: How can I replace a key:value pair by its value wherever the chosen key occurs in a deeply nested dictionary? (= "replace").
By the same recursive methods, you will also able to add items as the question asks for as well.
Since my request to link this question was declined, here is a copy of the solution that can delete items from a dict. See How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete") for examples / credits / notes.
import copy
def find_remove(this_dict, target_key, bln_overwrite_dict=False):
if not bln_overwrite_dict:
this_dict = copy.deepcopy(this_dict)
for key in this_dict:
# if the current value is a dict, dive into it
if isinstance(this_dict[key], dict):
if target_key in this_dict[key]:
this_dict[key].pop(target_key)
this_dict[key] = find_remove(this_dict[key], target_key)
return this_dict
dict_nested_new = find_remove(nested_dict, "sub_key2a")
The trick
The trick is to find out in advance whether a target_key is among the next children (= this_dict[key] = the values of the current dict iteration) before you reach the child level recursively. Only then you can still delete a key:value pair of the child level while iterating over a dictionary. Once you have reached the same level as the key to be deleted and then try to delete it from there, you would get the error:
RuntimeError: dictionary changed size during iteration
The recursive solution makes any change only on the next values' sub-level and therefore avoids the error.

I got the same problem and I used following procedure to solve this issue.
Python List can be iterate even if you modify during iterating over it.
so for following code it will print 1's infinitely.
for i in list:
list.append(1)
print 1
So using list and dict collaboratively you can solve this problem.
d_list=[]
d_dict = {}
for k in d_list:
if d_dict[k] is not -1:
d_dict[f(k)] = -1 # rather than deleting it mark it with -1 or other value to specify that it will be not considered further(deleted)
d_dict[g(k)] = v # add a new item
d_list.append(g(k))

Today I had a similar use-case, but instead of simply materializing the keys on the dictionary at the beginning of the loop, I wanted changes to the dict to affect the iteration of the dict, which was an ordered dict.
I ended up building the following routine, which can also be found in jaraco.itertools:
def _mutable_iter(dict):
"""
Iterate over items in the dict, yielding the first one, but allowing
it to be mutated during the process.
>>> d = dict(a=1)
>>> it = _mutable_iter(d)
>>> next(it)
('a', 1)
>>> d
{}
>>> d.update(b=2)
>>> list(it)
[('b', 2)]
"""
while dict:
prev_key = next(iter(dict))
yield prev_key, dict.pop(prev_key)
The docstring illustrates the usage. This function could be used in place of d.iteritems() above to have the desired effect.

How to remove a key/value pair in python dictionary?

Say I have a dictionary like this :
d = {'ben' : 10, 'kim' : 20, 'bob' : 9}
Is there a way to remove a pair like ('bob',9) from the dictionary?
I already know about d.pop('bob') but that will remove the pair even if the value was something other than 9.
Right now the only way I can think of is something like this :
if (d.get('bob', None) == 9):
d.pop('bob')
but is there an easier way? possibly not using if at all

pop also returns the value, so performance-wise (as neglectable as it may be) and readability-wise it might be better to use del.
Other than that I don't think there's something easier/better you can do.
from timeit import Timer
def _del():
d = {'a': 1}
del d['a']
def _pop():
d = {'a': 1}
d.pop('a')
print(min(Timer(_del).repeat(5000, 5000)))
# 0.0005624240000000613
print(min(Timer(_pop).repeat(5000, 5000)))
# 0.0007729860000003086

You want to perform two operations here
1) You want to test the condition d['bob']==9.
2) You want to remove the key along with value if the 1st answer is true.
So we can not omit the testing part, which requires use of if, altogether. But we can certainly do it in one line.
d.pop('bob') if d.get('bob')==9 else None

Creating a dict containing a sub-dict as the new value with the index as the key

I have a dictionary currently setup as
{'name': 'firm', 'name':'firm', etc},
Where keys are analyst names and values are analyst firms.
I am trying to create a new dictionary where the new values are the old k,v pairs and the associated key is simply the index (1, 2, 3, 4, etc).
Current code is below:
num_analysts = len(analysts.keys())
for k,v in analysts.items():
analysts_dict = dict.fromkeys(range(num_analysts), [k,v])
Current result
Each numeric key is getting given the same value (old k,v pair). What is wrong with my expression?

You can enumerate the items and convert them to a dictionary. However, dictionaries, in general, are not ordered. This means that the keys may be assigned essentially randomly.
dict(enumerate(analysts.items(), 1))
#{1: ('name1', 'firm1'), 2: ('name2', 'firm2')}

Enumerate and dictionary comprehension for this
d = {'name1': 'firm1', 'name2': 'firm2'}
d2 = {idx: '{}, {}'.format(item, d[item]) for idx, item in enumerate(d, start = 1)}
{1: 'name1, firm1', 2: 'name2, firm2'}

There are already effective answer posted by others. So I may just put the reason why your own solution does't work properly. It may caused by lazy binding. There are good resource on: http://quickinsights.io/python/python-closures-and-late-binding/
Because late binding will literally pick up the last one in dictionary you created. But this last one is not "virtually last one", it is determined by the OS. (Other people already give some explanation on dict data-structure.)
For each time you run in python command line the result may change. If you put the code in .py file, For each time you run in IDE, the result will be same.(always the last one in dict)

During each iteration, analysts_dict is assigned value based on the result of dict.items().
However, you should use comprehension to generate the final result in one line,
E.g. [{i: e} for i, e in enumerate(analysts.items())]
analysts = {
"a": 13,
"b": 123,
"c": 1234
}
num_analysts = len(analysts.keys())
analysts_dict = [{i: e} for i, e in enumerate(analysts.items())]
print(analysts_dict)
>> [{0: ('a', 13)}, {1: ('b', 123)}, {2: ('c', 1234)}]

This code
for k,v in analysts.items():
analysts_dict = dict.fromkeys(range(num_analysts), [k,v])
loops over the original dict and on each loop iteration it creates a new dict using the range numbers as the keys. By the way, every item in that dict shares a reference to a single [k, v] list object. That's generally a bad idea. You should only use an immutable object (eg None, a number, or a string) as the value arg to the dict.fromkeys method. The purpose of the method is to allow you to create a dict with a simple default value for the keys you supply, you can't use it to make a dict with lists as the values if you want those lists to be separate lists.
The new dict object is bound to the name analysts_dict. On the next loop iteration, a new dict is created and bound to that name, replacing the one just created on the previous loop, and the replaced dict is destroyed.
So you end up with an analysts_dict containing a bunch of references to the final [k, v] pair read from the original dict.
To get your desired result, you should use DYZ's code, which I won't repeat here. Note that it stores the old name & firm info in tuples, which is better than using lists for this application.

Python: cycling/scanning though fields in an object

I have a JSON file named MyFile.json that contains this structure:
[{u'randomName1': {u'A': 16,u'B': 20,u'C': 71},u'randomName2': {u'A': 12,u'B': 17,u'C': 47}},...]
I can open the file and load it like this:
import json
with open('MyFile.json') as data_file:
data = json.load(data_file)
And I can access the values in the first element like this:
data[0]["randomName1"][A]
data[0]["randomName1"][B]
data[0]["randomName1"][C]
data[0]["randomName2"][A]
data[0]["randomName2"][B]
data[0]["randomName2"][C]
The A B C keys are always named A B C (and there are always exactly 3 of them, so that's no problem.
The problem is:
1) I don't know how many elements are in the list, and
2) I don't know how many "randomName" keys are in each element, and
3) I don't know the names of the randomName keys.
How do I scan/cycle through the entire file, getting all the elements, and getting all the key names and associated key values for each element?
I don't have the knowledge or desire to write a complicated parsing script of my own. I was expecting that there's a way for the json library to provide this information.
For example (and this is not a perfect analogy I realize) if I am given an array X in AWK, I can scan all the index/name pairs by using
for(index in X){print index, X[index]);
Is there something like this in Python?
---------------- New info below this line -------------
Thank you Padraic and E.Gordon. That goes a long way toward solving the problem.
In an attempt to make my initial post as concise as possible, I simplified my JSON data example too much.
My JSON data actually looks this this:
data=[
{ {u'X': u'randomName1': {u'A': 11,u'B': 12,u'C': 13}, u'randomName2': {u'A': 21,u'B': 22,u'C': 23}, ... }, u'Y': 101, u'Z': 102 },
.
.
.
]
The ellipses represent arbitrary repetition, as described in the original post. The X Y Z keys are always named X Y Z (and there are always exactly 3 of them).
Using your posts as a starting point, I've been working on this for a couple of hours, but being new to Python I'm stumped. I cannot figure out how to add the extra loop to work with that data. I would like the output stream to look something like this:
Z,102,Y,101,randomName1,A,11,B,12,C,13,randomName2,A,21,B,22,C,23,...
.
.
.
Thanks for your help.
-
----------------- 3/23/16 update below --------------
Again, thanks for the help. Here's what I finally came up with. It does what I need:
import json
with open('MyFile.json') as data_file:
data = json.load(data_file)
for record in data:
print record['Z'],record['Y']
for randomName in record['X']:
print randomName, randomName['A'], randomName['B'],randomName['C']
...

You can print the items in the dicts:
js = [{u'randomName1': {u'A': 16,u'B': 20,u'C': 71},u'randomName2': {u'A': 12,u'B': 17,u'C': 47}}]
for dct in js:
for k, v in dct.items():
print(k, v)
Which gives you the key/inner dict pairings:
randomName1 {'B': 20, 'A': 16, 'C': 71}
randomName2 {'B': 17, 'A': 12, 'C': 47}
If you want the values from the inner dicts you can add another loop
for dct in js:
for k1, d in dct.items():
print(k1)
for k2,v in d.items():
print(k2,v)
Which will give you:
randomName1
A 16
B 20
C 71
randomName2
A 12
B 17
C 47
If you have arbitrary levels of nesting we will have to do it recursively.

You can use the for element in list construct to loop over all the elements in a list, without having to know its length.
The iteritems() dictionary method provides a convenient way to get the key-value pairs from a dictionary, again without needing to know how many there are or what the keys are called.
For example:
import json
with open('MyFile.json') as data_file:
data = json.load(data_file)
for element in data:
for name, values in element.iteritems():
print("%s has A=%d, B=%d and C=%d" % (name,
values["A"],
values["B"],
values["C"]))