I am currently in the process of using python to transmit a python dictionary from one raspberry pi to another over a 433Mhz link, using virtual wire (vw.py) to send data.
The issue with vw.py is that data being sent is in string format.
I am successfully receiving the data on PI_no2, and now I am trying to reformat the data so it can be placed back in a dictionary.
I have created a small snippet to test with, and created a temporary string in the same format it is received as from vw.py
So far I have successfully split the string at the colon, and I am now trying to get rid of the double quotes, without much success.
my_status = {}
#temp is in the format the data is recieved
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = temp.split(':')
print key
print value
key = key.replace("'",'')
value = value.replace("'",'')
my_status.update({key:value})
print my_status
Gives the result
'mycode'
['1','2','firstname','Lastname']
{'mycode': '[1,2,firstname,Lastname]'}
I require the value to be in the format
['1','2','firstname','Lastname']
but the strip gets rid of all the single speech marks.
You can use ast.literal_eval
import ast
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = map(ast.literal_eval, temp.split(':'))
status = {key: value}
Will output
{'mycode': ['1', '2', 'firstname', 'Lastname']}
This shouldn't be hard to solve. What you need to do is strip away the [ ] in your list string, then split by ,. Once you've done this, iterate over the elements are add them to a list. Your code should look like this:
string = "[1,2,firstname,lastname]"
string = string.strip("[")
string = string.strip("]")
values = string.split(",")
final_list = []
for val in values:
final_list.append(val)
print final_list
This will return:
> ['1','2','firstname','lastname']
Then take this list and insert it into your dictionary:
d = {}
d['mycode'] = final_list
The advantage of this method is that you can handle each value independently. If you need to convert 1 and 2 to int then you'll be able to do that while leaving the other two as str.
Alternatively to cricket_007's suggestion of using a syntax tree parser - you're format is very similar to the standard yaml format. This is a pretty lightweight and intutive framework so I'll suggest it
a = "'mycode':['1','2','firstname','Lastname']"
print yaml.load(a.replace(":",": "))
# prints the dictionary {'mycode': ['1', '2', 'firstname', 'Lastname']}
The only thing that's different between your format and yaml is the colon needs a space
It also will distinguish between primitive data types for you, if that's important. Drop the quotes around 1 and 2 and it determines that they're numerical.
Tadhg McDonald-Jensen suggested pickling in the comments. This will allow you to store more complicated objects, though you may lose the human-readable format you've been experimenting with
Is there a simple way to create a dictionary from a list of formatted tuples. e.g. if I do something like:
d={"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
This creates a dictionary called d. However, if I want to create a dictionary from a string which contains the same string, I can't do that
res=<some command that returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}>
print res
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
d=dict(res)
This throws an error that says:
ValueError: dictionary update sequence element #0 has length 1; 2 is required
I strongly strongly suspect that you have json on your hands.
import json
d = json.loads('{"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}')
would give you what you want.
Use dict(zip(tuples))
>>> u = ("foo", "bar")
>>> v = ("blah", "zoop")
>>> d = dict(zip(u, v))
>>> d
{'foo': 'blah', 'bar': 'zoop'}
Note, if you have an odd number of tuples this will not work.
Based on what you gave is, res is
# returns {"responseStatus":"SUCCESS","sessionId":"01234","userId":2000004904}
So the plan is to grab the string starting at the curly brace to the end and use json to decode it:
import json
# Discard the text before the curly brace
res = res[res.index('{'):]
# Turn that text into a dictionary
d = json.loads(res)
All you need to do in your particular case is
d = eval(res)
And please keep security in mind when using eval, especially if you're mixing it with ajax/json.
UPDATE
Since others pointed out you might be getting this data over the web and it isn't just a "how to make this work" question, use this:
import json
json.loads(res)
I am trying to convert a string to a dictionary with dict function, like this
import json
p = "{'id':'12589456'}"
d = dict(p)
print d['id']
But I get the following error
ValueError: dictionary update sequence element #0 has length 1; 2 is required
Why does it fail? How can I fix this?
What you have is a string, but dict function can only iterate over tuples (key-value pairs) to construct a dictionary. See the examples given in the dict's documentation.
In this particular case, you can use ast.literal_eval to convert the string to the corresponding dict object, like this
>>> p = "{'id':'12589456'}"
>>> from ast import literal_eval
>>> d = literal_eval(p)
>>> d['id']
'12589456'
Since p is a string containing JSON (ish), you have to load it first to get back a Python dictionary. Then you can access items within it:
p = '{"id":"12589456"}'
d = json.loads(p)
print d["id"]
However, note that the value in p is not actually JSON; JSON demands (and the Python json module enforces) that strings are quoted with double-quotes, not single quotes. I've updated it in my example here, but depending on where you got your example from, you might have more to do.
I am reading from a text file which has the format below:
0.000 ff:dd ff:ff 4 126 48000
0.001 sd:fg er:sd 5 125 67000
0.002 qw:er ff:dd 5 127 90000
0.003 xc:sd ff:dd 5 127 90000
0.004 io:uy gh:ij 4 126 56000
In the fourth column, 4 indicates request and 5 indicates response. i should form a dictionary with second column as the key if that row represents a request.
If the fourth column value is 5, it indicates that the row corresponds to the response. In this case, look at the 3rd column of that response, and if that 3rd column is there as the dictionary key, add 2rd column as an value of that corresponding key.
In the above example, the desired result is:
{'ff:dd': 1, 2, 48000, qw:er, xc:sd}, {'io:uv': 1, 0, 56000}
For ff:dd, 1 indicates that there is only 1 request from the ff:dd; 2 indicates there are 2 responses to ff:dd and 48000 is the 6th column value of the request corresponding to ff:dd. I hope you understood the question. Please ask for any clarifications.
For io:uv, since there are no responses, 1 indicates number of requests, 0 indicates number of responses and 56000 is the 6th column value for this request.
I am doing all this to analyze network traffic.
I don't know how to dynamically add the values. If there are fixed number of values, i can manage, but this is a tricky situation. I a using python2.6. Help is much appreciated. Thanks in advance!
Use a dictionary with keys being strings and the values being a tuple containing (requests, responses, 6th column, [list of response keys])
Let's make sure first that we are clear about what a dictionary is and what it can be used for (and hope I don't put my foot in my mouth - I am fairly new to Python myself).
About Dictionaries
In Python, a dict maps single keys to single values. You can read the documentation if you want, but one issue here is that you seem to want to map a single key to multiple values in your desired result:
{'ff:dd': 1, 2, 48000, qw:er, xc:sd}, {'io:uv': 1, 0, 56000}
This shows two dictionaries. Looking at the first dictionary, {'ff:dd': 1, maps the key 'ff:dd' to the value 1 only; the comma says, move on to the next key:value pair. So the rest is interpreted as keys 2 and 48000 mapped to no values (throws SyntaxError), and undefined names qw and xc mapped to undefined names er and sd (would throw NameError if you got that far). You probably meant for qw:er and xc:sd to be strings, in which case they would be seen as keys that are not mapped to any value , like the numbers 2 and 48000 before them. You can test this out in the shell:
>>> {'key':'value'} # A dict with one key, that has a value
{'key': 'value'}
>>> {'key'} # Curly braces can make a dict or a set
set(['key'])
>>> {'key1':'val1', 'key2'} # A dict needs to have values for each key
SyntaxError: invalid syntax
>>> {'key1':'val1', 'key2':} # Even an "empty" value has to be explicit
SyntaxError: invalid syntax
If you did for some reason want to define your keys before you know their values, it could be workable to use a zero or empty string, but the "correct" way to do it is probably to use dict.fromkeys() :
>>> {'key1':None, 'key2':'', 'key3':0} # One of these might do the job...
{'key3': 0, 'key2': '', 'key1': None}
>>> dict.fromkeys(['key1','key2', 'key3']) # ...but this is probably better.
{'key3': None, 'key2': None, 'key1': None}
(As a side note, you probably don't want to talk about "appending" to a dictionary. That's something you would do with an ordered list, e.g. by mylist.append(myvalue). There is no append() method for a dictionary; what you do is set a key, and if the key doesn't exist, it is created. "Append" means adding to the end, but dictionaries are unordered so they have no "end" as such.)
Storing Your Data
Now I'm going to make an assumption about what you're trying to do with qw:er, xc:sd, because it's not totally clear about your question. My assumption is that you simply want to have a list of the responses that were sent to 'ff:dd'. If you wanted, you can do other things, but with that assumption I'll try to shed some light on how to do something like what you want to do. It looks like your desired result is something like:
traffic = { 'ff:dd': { 'reqfrom':1, 'respto':2, 'reqvals':[48000],
'responders':['qw:er', 'xc:sd'] },
'io:uv': { 'reqfrom':1, 'respto':0, 'reqvals':[56000],
'responders':[] }
}
At the top level of the traffic dictionary, there are two keys: 'ff:dd' and 'io:uv'. The value of each key is another dictionary, so that you can access the number of requests from a key, responses to a key, and other values associated with that particular address, as follows:
>>> traffic['io:uv']['reqfrom'] # How many requests from 'io:uv'?
1
>>> traffic['ff:dd']['responders'] # What are the responses to 'ff:dd'?
['qw:er', 'xc:sd']
So, how do you dynamically store these values? Normally, you would simply assign a value to a key, like mydict['key'] = 'value'. The value of the key will be updated if the key already exists; if not, the key-value pair will be added to the dictionary. But since the values of your first keys are themselves dictionaries, it's a bit trickier.
Try this...
Here's an example of one possible approach, using the above assumed structure. I won't go into too much detail because more experienced Python users can probably show you better ways to do the same thing. Try this code on for size - run it, read it, break it, etc. You should be able to figure out how to adapt it for your purposes.
traffic = {}
packets = (('0.000', 'ff:dd', 'ff:ff', '4', '126', '48000'),
('0.001', 'sd:fg', 'er:sd', '5', '125', '67000'),
('0.002', 'qw:er', 'ff:dd', '5', '127', '90000'),
('0.003', 'xc:sd', 'ff:dd', '5', '127', '90000'),
('0.004', 'io:uv', 'gh:ij', '4', '126', '56000'))
def record_packet(packet):
if packet[3] == '4': # Request
# Set up the key-value if it doesn't exist
if packet[1] not in traffic:
traffic[packet[1]] = {'reqfrom':0,
'respto':0,
'reqvals':[],
'responders':[]
}
traffic[packet[1]]['reqfrom'] += 1
traffic[packet[1]]['reqvals'].append(packet[5])
elif packet[3] == '5': # Response
# Record the response IFF there has been a request
if packet[2] in traffic:
traffic[packet[2]]['respto'] += 1
traffic[packet[2]]['responders'].append(packet[1])
else:
# Handle weirdness here
pass
for packet in packets:
record_packet(packet)
for key in traffic:
for item in traffic[key]:
print "traffic['{0}']['{1}'] has value: {2}".format(key, item, traffic[key][item])
Let's try a data structure which makes things easier.
If I understand you correctly, you'd like these addresses to be the keys of your dictionary, and you'd like the data stored under these keys to be:
The number of requests sent from this address
The number of requests received at this address
A list of addresses which sent requests to this address
This would probably best be formatted like so:
{address:[sent, recv, [address, address, ...]], ...}
So, you may read the text file like so:
with open('myfile.txt', 'r') as myfile:
dct = {}
for line in myfile:
splt = line.split()
Now you are iterating over every line in the file and splitting it by the columns. Next, we'd like to determine if it was a request or a response:
if splt[3] == '4': # request
# we're about to fill this in
elif split[3] == '5': # response
# we're about to fill this in
This checks the 4th column for either '4' or '5'. You will run into problems if you have another value there (i.e. you have poorly formatted data).
Now, if we are handling a request, we'd do the following:
if splt[1] not in dct:
dct[splt[1]] = [1, 0, []]
else:
dct[splt[1]][0] += 1
This increments the number of requests sent by 1; if the address is not in the dictionary yet, then we add it. We use splt[1] here because we are talking about the sender's address, not the receiver's.
If we are handling a response, we'd act differently, of course. I am assuming here that you will never send a response without a request first being made... but just in case, I've put that case in there, with pass to just ignore it. It's up to you to figure out what you want to do with that...
if splt[2] not in dct:
pass
else:
dct[splt[2]][1] += 1
if splt[1] not in dct[splt[2]][2]:
dct[splt[2]][2].append(splt[1])
Here, we increment the number of responses received by 1, and add the responder's address to our list. We use splt[2] as the key instead because we are talking about the receiver's address, not the sender's.
This ought to be the crux for what you are trying to accomplish - although I don't understand what the fourth and fifth columns do, so I've omitted them.