We Keep Coding

Implement hmset in python with dictionary of list and nested dictionary

I was trying to implement below redis code into python django application
hmset test_template:TEMPLATE_ID test_tags "[{\"key\":\"test_manual_entry_1\",\"value\":\"Some_value_1\"},{\"key\":\"test_manual_entry_2\",\"value\":\"Some_value_2\"}]"
I have tried hset and hmset functions but both are giving the error. Below is sample of my code looks like this
class RedisUtil:
def hset(self, key_name, data):
key_name = "test_template:TEMPLATE_ID"
list_data = [{"key": "test_manual_entry_1", "value": "Some_value1"}, {"key": "test_manual_entry_2", "value": "Some_value2"}]
data = {"test_tags": [json.dumps(d) for d in list_data]} # output list: ['{"key": "test_manual_entry_1", "value": "Some_value1"}', '{"key": "test_manual_entry_2", "value": "Some_value2"}']
I have tried below methods to save but all methods are giving me error
# Method 1
self.redis_client.hset(key_name, data) # Exception: redis.exceptions.DataError: Invalid input of type: 'dict'. Convert to a bytes, string, int or float first.
#Method 2
self.redis_client.hset(key_name, "test_tag", data["test_tags"]) # Exception: redis.exceptions.DataError: Invalid input of type: 'list'. Convert to a bytes, string, int or float first.
Also, I would like add there that there may be case where my list will be empty, this could be an edge case.
Thanks in advance for any help.

Here's Python Redis hset doc: https://redis.readthedocs.io/en/stable/commands.html?highlight=hset#redis.commands.core.CoreCommands.hset
The function signature is hset(name, key=None, value=None, mapping=None, items=None).
For method 1, You passed data as key. Besides, I presume data is a dict, which is differ from string.
For method 2, you use data["test_tags"] as value, But still, data["test_tags"] isn't a string but a list.
If you want to implement hmset, may you should use this one instead(but already deprecated, not recommended)?

Yet another Python looping over JSON array

I spent several hours on this, tried everything I found online, pulled some of the hair left on my head...
I have this JSON sent to a Flask webservice I'm writing :
{'jsonArray': '[
{
"nom":"0012345679",
"Start":"2018-08-01",
"Finish":"2018-08-17",
"Statut":"Validee"
},
{
"nom":"0012345679",
"Start":"2018-09-01",
"Finish":"2018-09-10",
"Statut":"Demande envoyée au manager"
},
{
"nom":"0012345681",
"Start":"2018-04-01",
"Finish":"2018-04-08",
"Statut":"Validee"
},
{
"nom":"0012345681",
"Start":"2018-07-01",
"Finish":"2018-07-15",
"Statut":"Validee"
}
]'}
I want to simply loop through the records :
app = Flask(__name__)
#app.route('/graph', methods=['POST'])
def webhook():
if request.method == 'POST':
req_data = request.get_json()
print(req_data) #-> shows JSON that seems to be right
##print(type(req_data['jsonArray']))
#j1 = json.dumps(req_data['jsonArray'])
#j2 = json.loads(req_data['jsonArray'])
#data = json.loads(j1)
#for rec in data:
# print(rec) #-> This seems to consider rec as one of the characters of the whole JSON string, and prints every character one by one
#for key in data:
# value = data[key]
# print("The key and value are ({}) = ({})".format(key, value)) #-> TypeError: string indices must be integers
for record in req_data['jsonArray']:
for attribute, value in rec.items(): #-> Gives error 'str' object has no attribute 'items'
print(attribute, value)
I believe I am lost between JSON object, python dict object, strings, but I don't know what I am missing. I really tried to put the JSON received through json.dumps and json.loads methods, but still nothing. What am I missing ??
I simply want to loop through each record to create another python object that I will feed to a charting library like this :
df = [dict(Task="0012345678", Start='2017-01-01', Finish='2017-02-02', Statut='Complete'),
dict(Task="0012345678", Start='2017-02-15', Finish='2017-03-15', Statut='Incomplete'),
dict(Task="0012345679", Start='2017-01-17', Finish='2017-02-17', Statut='Not Started'),
dict(Task="0012345679", Start='2017-01-17', Finish='2017-02-17', Statut='Complete'),
dict(Task="0012345680", Start='2017-03-10', Finish='2017-03-20', Statut='Not Started'),
dict(Task="0012345680", Start='2017-04-01', Finish='2017-04-20', Statut='Not Started'),
dict(Task="0012345680", Start='2017-05-18', Finish='2017-06-18', Statut='Not Started'),
dict(Task="0012345681", Start='2017-01-14', Finish='2017-03-14', Statut='Complete')]

The whole thing is wrapped in single quotes, meaning it's a string and you need to parse it.
for record in json.loads(req_data['jsonArray']):
Looking at your commented code, you did this:
j1 = json.dumps(req_data['jsonArray'])
data = json.loads(j1)
Using json.dumps on a string is the wrong idea, and moreover json.loads(json.dumps(x)) is just the same as x, so that just got you back where you started, i.e. data was the same thing as req_data['jsonArray'] (a string).
This was the right idea:
j2 = json.loads(req_data['jsonArray'])
but you never used j2.
As you've seen, iterating over a string gives you each character of the string.

how to add variable to json

im trying to add a variable passed into a function into json but i keep getting an error, i used this question here, this is my code:
#!/usr/bin/env python
import requests
def interact(token):
token_string = str(token)
print token_string
headers = {{'X-Username':'user','X-Token':'{0}'}}.format(token_string)
print "token:"
print headers
# main
login()
get_token = login()
interact(get_token)
this is the error:
TypeError: unhashable type: 'dict'

You have to apply format to the string, not the dict that contains the string:
headers = {
{'X-Username':'user',
'X-Token':'{0}'.format(token_string)
}
}
And, you can't put a dict in a set; you'll have to use a list.
headers = [
{'X-Username':'user',
'X-Token':'{0}'.format(token_string)
}
]

You are getting that error because you are trying to put a dict in a Set.
Consider:
foo = {"a"} # type(foo) <class 'set'>
foo.add({"b": 1}) # throws unhashable error
{"a", {"b": 1}} # equivalent to the above 2 lines
This is a fairly common error since set literals and dict literals both use curly braces:
bar = {"a", 1, "b", 2} # oops! 4 element set instead of dict with 2 k/v pairs

Parsing Python JSON with multiple same strings with different values

I am stuck on an issue where I am trying to parse for the id string in JSON that exists more than 1 time. I am using the requests library to pull json from an API. I am trying to retrieve all of the values of "id" but have only been able to successfully pull the one that I define. Example json:
{
"apps": [{
"id": "app1",
"id": "app2",
"id": "new-app"
}]
}
So what I have done so far is turn the json response into dictionary so that I am actually parse the first iteration of "id". I have tried to create for loops but have been getting KeyError when trying to find string id or TypeError: list indices must be integers or slices, not str. The only thing that I have been able to do successfully is define which id locations to output.
(data['apps'][N]['id']) -> where N = 0, 1 or 2
This would work if there was only going to be 1 string of id at a time but will always be multiple and the location will change from time to time.
So how do return the values of all strings for "id" from this single json output? Full code below:
import requests
url = "http://x.x.x.x:8080/v2/apps/"
response = requests.get(url)
#Error if not 200 and exit
ifresponse.status_code!=200:
print("Status:", response.status_code, "CheckURL.Exiting")
exit()
#Turn response into a dict and parse for ids
data = response.json()
for n in data:
print(data['apps'][0]['id'])
OUTPUT:
app1
UPDATE:
Was able to get resolution thanks to Robᵩ. Here is what I ended up using:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
print(value)
data = response.json(object_pairs_hook = list_hook)
Also The API that I posted as example is not a real API. It was just supposed to be a visual representation of what I was trying to achieve. I am actually using Mesosphere's Marathon API . Trying to build a python listener for port mapping containers.

Your best choice is to contact the author of the API and let him know that his data format is silly.
Your next-best choice is to modify the behavior of the the JSON parser by passing in a hook function. Something like this should work:
def list_hook(pairs):
result = {}
for name, value in pairs:
if name == 'id':
result.setdefault(name, []).append(value)
else:
result[name] = value
return result
data = response.json(object_pairs_hook = list_hook)
for i in range(3):
print(i, data['apps'][0]['id'][i])

Flask jsonify: key starting with digit

Say i want to create the following JSON document in Flask:
{"1": {"name": "Tom"}}
I couldn't just call return jsonify(**kwargs), because Python forbids naming variables starting with a number, like 1.
Is it possible to create such a JSON document with Flask's jsonify? If not, what are my workarounds?
Clarification: I want to create a JSON document, which contains dicts indexed by ids, and ids have the form of 24 hexadecimal values (ObjectId from MongoDB). It means that most of the time ids will start with a digit.

Conveniently, you can use... **kwargs!
>>> def echo(**kwargs):
... print kwargs
...
>>> echo(**{"1":{"name": "Tom"}})
{'1': {'name': 'Tom'}}
Yes, you can't manually specify an individual keyword argument named 1... but you can pass a keyword argument named 1 via the ** keyword expansion operator.

Since flask.jsonify takes the same parameters as Python's dict, you can give it a dict as a parameter and let it sort out the rest:
#app.route('/')
def hello_world():
# note key is an int, not just a string with leading digits
# in your case this could be a mongodb object
return jsonify({1: {'name': 'bob'}})
Returns:
{
"1": {
"name": "bob"
}
}