I currently have a python dictionary where the keys are strings representing URLs, and the values are also string URLs.
#socketio.on('blacklist', namespace='/update')
def add_to_blacklist(message):
stored_blacklist.clear()
for key, val in message.items():
stored_blacklist[key] = val
lookup = {'name':'blacklist'}
resp = patch_internal('blacklist', payload={"sites":stored_blacklist}, **lookup)
However there is an issue with how python is interpreting the insertions. For example:
stored_blacklist["example.com"] = "thankspython.edu"
My desired behavior is that stored_blacklist maps "example.com" to "thankspython.edu" like so:
{'example.com':'thankspython.edu'}
However, printing stored_blacklist gives me this instead:
{'example': {'com': 'thankspython.edu'}}
How could I get the desired behavior where a string with a period character in it could be read as a normal string instead of automatically creating some pseudo-JSON object?
Related
So in my python script I have the following dictionary, except it's listed in string form:
{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}
I however would like to have this code as a dictionary of lists, as it clearly is. I tried the following code in orderto attempt to convert the string:
try:
dictimports = ast.literal_eval(stris)
print(dictimports)
except:
print("dict convert failed")
However it hits the except everytime :(
So to reiterate, I would like the keys to be say 'KERNEL32.DLL', and then those keys to have the list as the contents of the values, so have a list with the values ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'] in this instance.
stris = {'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}
stris is a dictionary. what seems to be the problem?
type(stris)
dict
stris.keys()
dict_keys(['MSVCRT.dll', 'KERNEL32.DLL', 'SHLWAPI.dll', 'USER32.dll'])
if your stris is a string - in which case you'd have
stris = "{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}"
and you will convert it to a dict
ast.literal_eval(stris)
{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree','ExitProcess','VirtualProtect','LoadLibraryA','VirtualAlloc',
'GetProcAddress'],'SHLWAPI.dll': ['PathFileExistsA'],'USER32.dll':['wsprintfA']}
You could use eval() to convert the string to a dict.
The expression argument is parsed and evaluated as a Python expression
eval(stris) will execute the executions given as string an in your case return the parsed dictionary.
But be aware of this: Using python's eval() vs. ast.literal_eval()?
I have result returned as from ZrangebyScore function as [b'101']. I would like to extract only 101 value and discard other additional characters. It is in byte form. How to convert it in Integer format using Python.
If you are using Py3 try this:
mylist = [b'101']
val = int(mylist[0].decode())
I'm trying to process a log from Symphony using Pandas, but have some trouble with a malformed JSON which I can't parse.
An example of the log :
'{id:46025,
work_assignment:43313=>43313,
declaration:<p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>H </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>=><p><strong>Bijkomende interventie.</strong></p>\r\n\r\n<p>He </p>\r\n\r\n<p><strong><em>Vaststellingen.</em></strong></p>\r\n\r\n<p><strong><em>CV. </em></strong>De.</p>,conclusions:<p>H </p>=><p>H </p>}'
What is the best way to process this?
For each part (id/work_assignment/declaration/etc) I would like to retrieve the old and new value (which are separated by "=>").
Use the following code:
def clean(my_log):
my_log.replace("{", "").replace("}", "") # Removes the unneeded { }
my_items = list(my_log.split(",")) # Split at the comma to get the pairs
my_dict = {}
for i in my_items:
key, value = i.split(":") # Split at the colon to separate the key and value
my_dict[key] = value # Add to the dictionary
return my_dict
Function returns a Python dictionary, which can then be converted to JSON using a serializer if needed, or directly used.
Hope I helped :D
I am currently in the process of using python to transmit a python dictionary from one raspberry pi to another over a 433Mhz link, using virtual wire (vw.py) to send data.
The issue with vw.py is that data being sent is in string format.
I am successfully receiving the data on PI_no2, and now I am trying to reformat the data so it can be placed back in a dictionary.
I have created a small snippet to test with, and created a temporary string in the same format it is received as from vw.py
So far I have successfully split the string at the colon, and I am now trying to get rid of the double quotes, without much success.
my_status = {}
#temp is in the format the data is recieved
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = temp.split(':')
print key
print value
key = key.replace("'",'')
value = value.replace("'",'')
my_status.update({key:value})
print my_status
Gives the result
'mycode'
['1','2','firstname','Lastname']
{'mycode': '[1,2,firstname,Lastname]'}
I require the value to be in the format
['1','2','firstname','Lastname']
but the strip gets rid of all the single speech marks.
You can use ast.literal_eval
import ast
temp = "'mycode':['1','2','firstname','Lastname']"
key,value = map(ast.literal_eval, temp.split(':'))
status = {key: value}
Will output
{'mycode': ['1', '2', 'firstname', 'Lastname']}
This shouldn't be hard to solve. What you need to do is strip away the [ ] in your list string, then split by ,. Once you've done this, iterate over the elements are add them to a list. Your code should look like this:
string = "[1,2,firstname,lastname]"
string = string.strip("[")
string = string.strip("]")
values = string.split(",")
final_list = []
for val in values:
final_list.append(val)
print final_list
This will return:
> ['1','2','firstname','lastname']
Then take this list and insert it into your dictionary:
d = {}
d['mycode'] = final_list
The advantage of this method is that you can handle each value independently. If you need to convert 1 and 2 to int then you'll be able to do that while leaving the other two as str.
Alternatively to cricket_007's suggestion of using a syntax tree parser - you're format is very similar to the standard yaml format. This is a pretty lightweight and intutive framework so I'll suggest it
a = "'mycode':['1','2','firstname','Lastname']"
print yaml.load(a.replace(":",": "))
# prints the dictionary {'mycode': ['1', '2', 'firstname', 'Lastname']}
The only thing that's different between your format and yaml is the colon needs a space
It also will distinguish between primitive data types for you, if that's important. Drop the quotes around 1 and 2 and it determines that they're numerical.
Tadhg McDonald-Jensen suggested pickling in the comments. This will allow you to store more complicated objects, though you may lose the human-readable format you've been experimenting with
I have a large file, that contains valid nested json on each line, each json looks like (real data is much bigger, so this peace of json will be shown for illustration just):
{"location":{"town":"Rome","groupe":"Advanced",
"school":{"SchoolGroupe":"TrowMet", "SchoolName":"VeronM"}},
"id":"145",
"Mother":{"MotherName":"Helen","MotherAge":"46"},"NGlobalNote":2,
"Father":{"FatherName":"Peter","FatherAge":"51"},
"Study":[{
"Teacher":["MrCrock","MrDaniel"],
"Field":{"Master1":["Marketing", "Politics", "Philosophy"],
"Master2":["Economics", "Management"], "ExamCode": "1256"}
}],
"season":["summer","spring"]}
I need to parse this file, in order to extract only some key-values from every json, to obtain the dataframe that should look like:
Groupe Id MotherName FatherName Master2
Advanced 56 Laure James Economics, Management
Middle 11 Ann Nicolas Web-development
Advanced 6 Helen Franc Literature, English Language
I use method proposed me in the other question .get but it doesn't work with nested json, so for instance if I try:
def extract_data(data):
""" convert 1 json dict to records for import"""
dummy = {}
jfile = json.loads(data.strip())
return (
jfile.get('Study', dummy).get('Field', np.nan).get('Master1',np.nan),
jfile.get('location', dummy).get('groupe', np.nan))
for this line jfile.get('Study', dummy).get('Field', np.nan).get('Master1', np.nan) it throws me an error:
AttributeError: 'list' object has no attribute 'get'
obviously it happens because the value of "Study" is not a dictionary, neither list, but a valid json! how can I deal with this problem? Does exist a method that works like .get, but for json? I guess there is another option : decode this json and then parse it with .get, but the problem that it is in the core of another json, so I have no clue how to decode it!
Data is a valid JSON formatted string. JSON contains four basic elements:
Object: defined with curly braces {}
Array: defined with braces []
Value: can be a string, a number, an object, an array, or the literals true, false or null
String: defined by double quotes and contain Unicode characters or common backslash escapes
Using json.loads will convert the string into a python object recursively. It means that every inner JSON element will be represented as a python object.
Therefore:
jfile.get('Study') ---> python list
To retrieve Field you should iterate over the study list:
file = json.loads(data.strip())
study_list = jfile.get('Study', []) # don't set default value with different type
for item in study_list:
print item.get('Field')