We Keep Coding

How to reset value of multiple dictionaries elegantly in python

I am working on a code which pulls data from database and based on the different type of tables , store the data in dictionary for further usage.
This code handles around 20-30 different table so there are 20-30 dictionaries and few lists which I have defined as class variables for further usage in code.
for example.
class ImplVars(object):
#dictionary capturing data from Asset-Feed table
general_feed_dict = {}
ports_feed_dict = {}
vulns_feed_dict = {}
app_list = []
...
I want to clear these dictionaries before I add data in it.
Easiest or common way is to use clear() function but this code is repeatable as I will have to write for each dict.
Another option I am exploring is with using dir() function but its returning variable names as string.
Is there any elegant method which will allow me to fetch all these class variables and clear them ?

You can use introspection as you suggest:
for d in filter(dict.__instancecheck__, ImplVars.__dict__.values()):
d.clear()
Or less cryptic, covering lists and dicts:
for obj in ImplVars.__dict__.values():
if isinstance(obj, (list, dict)):
obj.clear()
But I would recommend you choose a bit of a different data structure so you can be more explicit:
class ImplVars(object):
data_dicts = {
"general_feed_dict": {},
"ports_feed_dict": {},
"vulns_feed_dict": {},
}
Now you can explicitly loop over ImplVars.data_dicts.values and still have other class variables that you may not want to clear.

code:
a_dict = {1:2}
b_dict = {2:4}
c_list = [3,6]
vars_copy = vars().copy()
for variable, value in vars_copy.items():
if variable.endswith("_dict"):
vars()[variable] = {}
elif variable.endswith("_list"):
vars()[variable] = []
print(a_dict)
print(b_dict)
print(c_list)
result:
{}
{}
[]

Maybe one of the easier kinds of implementation would be to create a list of dictionaries and lists you want to clear and later make the loop clear them all.
d = [general_feed_dict, ports_feed_dict, vulns_feed_dict, app_list]
for element in d:
element.clear()
You could also use list comprehension for that.

How to access elements of a dictionary in python where the keys are bytes instead of strings

Sorry if it's too much of a noob question.
I have a dictionary where the keys are bytes (like b'access_token' ) instead of strings.
{
b'access_token': [b'b64ssscba8c5359bac7e88cf5894bc7922xxx'],
b'token_type': [b'bearer']
}
usually I access the elements of a dictionary by data_dict.get('key'), but in this case I was getting NoneType instead of the actual value.
How do I access them or is there a way to convert this bytes keyed dict to string keyed dict?
EDIT: I actually get this dict from parsing a query string like this access_token=absdhasd&scope=abc by urllib.parse.parse_qs(string)

You can use str.encode() and bytes.decode() to swap between the two (optionally, providing an argument that specifies the encoding. 'UTF-8' is the default). As a
result, you can take your dict:
my_dict = {
b'access_token': [b'b64ssscba8c5359bac7e88cf5894bc7922xxx'],
b'token_type': [b'bearer']
}
and just do a comprehension to swap all the keys:
new_dict = {k.decode(): v for k,v in my_dict.items()}
# {
# 'access_token': [b'b64ssscba8c5359bac7e88cf5894bc7922xxx'],
# 'token_type': [b'bearer']
# }
Similarly, you can just use .encode() when accessing the dict in order to get a bytes object from your string:
my_key = 'access_token'
my_value = my_dict[my_key.encode()]
# [b'b64ssscba8c5359bac7e88cf5894bc7922xxx']

Most probably, you are making some silly mistake.
It is working fine in my tests.
Perhaps you forgot to add the prefix b when trying to index the dictionary
d={
b'key1': [b'val1'],
b'key2': [b'val2']
}
d[b'key1'] # --> returns [b'val1']
d.get(b'key2') # --> returns [b'val2']

Perhaps this could be something you're looking for?
dict = {
b'access_token': [b'b64ssscba8c5359bac7e88cf5894bc7922xxx'],
b'token_type': [b'bearer']
}
print(dict.get( b'access_token'))

Python set dictionary nested key with dot delineated string

If I have a dictionary that is nested, and I pass in a string like "key1.key2.key3" which would translate to:
myDict["key1"]["key2"]["key3"]
What would be an elegant way to be able to have a method where I could pass on that string and it would translate to that key assignment? Something like
myDict.set_nested('key1.key2.key3', someValue)

Using only builtin stuff:
def set(my_dict, key_string, value):
"""Given `foo`, 'key1.key2.key3', 'something', set foo['key1']['key2']['key3'] = 'something'"""
# Start off pointing at the original dictionary that was passed in.
here = my_dict
# Turn the string of key names into a list of strings.
keys = key_string.split(".")
# For every key *before* the last one, we concentrate on navigating through the dictionary.
for key in keys[:-1]:
# Try to find here[key]. If it doesn't exist, create it with an empty dictionary. Then,
# update our `here` pointer to refer to the thing we just found (or created).
here = here.setdefault(key, {})
# Finally, set the final key to the given value
here[keys[-1]] = value
myDict = {}
set(myDict, "key1.key2.key3", "some_value")
assert myDict == {"key1": {"key2": {"key3": "some_value"}}}
This traverses myDict one key at a time, ensuring that each sub-key refers to a nested dictionary.
You could also solve this recursively, but then you risk RecursionError exceptions without any real benefit.

There are a number of existing modules that will already do this, or something very much like it. For example, the jmespath module will resolve jmespath expressions, so given:
>>> mydict={'key1': {'key2': {'key3': 'value'}}}
You can run:
>>> import jmespath
>>> jmespath.search('key1.key2.key3', mydict)
'value'
The jsonpointer module does something similar, although it likes / for a separator instead of ..
Given the number of pre-existing modules I would avoid trying to write your own code to do this.

EDIT: OP's clarification makes it clear that this answer isn't what he's looking for. I'm leaving it up here for people who find it by title.
I implemented a class that did this a while back... it should serve your purposes.
I achieved this by overriding the default getattr/setattr functions for an object.
Check it out! AndroxxTraxxon/cfgutils
This lets you do some code like the following...
from cfgutils import obj
a = obj({
"b": 123,
"c": "apple",
"d": {
"e": "nested dictionary value"
}
})
print(a.d.e)
>>> nested dictionary value

Python Reference Dictionary Key in same Dictionary

I'm trying to access a key in a dictionary before "declaring" it.
Similar to this:
test_dict = {'path': '/root/secret/', 'path2': test_dict['path']+'meow/'}
I am aware that I could accomplish this by doing in the next line, like:
test_dict['path2'] = test_dict['path']+'meow'
however for readability i would prefer writing all the keys in the dict for a config file.
Is this possible in Python?

Convince yourself that this is not possible. You cannot refer to an object that hasn't even been created. What you can, however, do, is use a string variable. This should do what you want relatively easily.
p = '/root/secret/'
test_dict = {'path' : p, 'path2' : os.path.join(p, 'meow')}
Also, it's good practice to use os.path.join when concatenating sub-paths together.

#cᴏʟᴅsᴘᴇᴇᴅ, I think this is more readable, imagine if OP were to add 15 paths.
p = '/root/secret/'
# initiate dict
test_dict = {}
# assign values
test_dict['path'] = p
test_dict['path2'] = os.path.join(p, 'meow')

how to create a dictionary from a set of properly formatted tuples in python

Is there a simple way to create a dictionary from a list of formatted tuples. e.g. if I do something like:
d={"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
This creates a dictionary called d. However, if I want to create a dictionary from a string which contains the same string, I can't do that
res=<some command that returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}>
print res
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
d=dict(res)
This throws an error that says:
ValueError: dictionary update sequence element #0 has length 1; 2 is required

I strongly strongly suspect that you have json on your hands.
import json
d = json.loads('{"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}')
would give you what you want.

Use dict(zip(tuples))
>>> u = ("foo", "bar")
>>> v = ("blah", "zoop")
>>> d = dict(zip(u, v))
>>> d
{'foo': 'blah', 'bar': 'zoop'}
Note, if you have an odd number of tuples this will not work.

Based on what you gave is, res is
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
So the plan is to grab the string starting at the curly brace to the end and use json to decode it:
import json
# Discard the text before the curly brace
res = res[res.index('{'):]
# Turn that text into a dictionary
d = json.loads(res)

All you need to do in your particular case is
d = eval(res)
And please keep security in mind when using eval, especially if you're mixing it with ajax/json.
UPDATE
Since others pointed out you might be getting this data over the web and it isn't just a "how to make this work" question, use this:
import json
json.loads(res)