I have a list of objects, that has been stringified:
u'[<object: objstuff1, objstuff2>, <object: objstuff1, objstuff2>]'
I want to convert this back into a list:
[<object: objstuff1, objstuff2>, <object: objstuff1, objstuff2>]
I've tried using ast.literal_eval(), but unfortunately, it doesn't seem to work if the elements are objects, and I get a SyntaxError.
Is there any way I can reconvert my string representation of the list of objects back into a list?
You need to have a look at the pickle module to do this.
Basically, dump your objects using pickle.dumps, and load them back using pickle.loads.
ast.literal_eval doesn't work obviously, because there is a lot of information related to the objects (like attributes, and values) which is simply not captured in that string. Also note that you will be able to resurrect only the pickled data, if all you have are those string representations right now, you won't be able to create the objects back from them because of the information loss.
Related
I have an application where I need to convert Python shelve pickled dictionary files to JSON files.
import ujson, shelve
with open("shelveFile", "r") as sfile:
shelve_dict=shelve.open(sfile)
py_dict= dict(shelve_dict)
with open("jsonfile.json","w") as jsonfile:
ujson.dump(py_dict, jsonfile)
with open("jsonfile.json",'r') as readJSONfile:
ujson.loads(readJSONfile.read())
Note: If I use ujson.load(jsonfile2) I get a serialization error.
The issue I have: the shelve file uses Python tuple datatypes for some of the dictionary keys. I am able to use ujson.dump to save as JSON but when I try to use ujson.load(jsonfile) the keys get loaded as strings and not tuples.
Short of using a dictionary comprehension to convert the keys(unsure of that exact syntax), is there a library that would convert a shelve file to a JSON file that I could load back into a Python dictionary object?
When loaded by ujson.loads(fileobj.read()) method:
{"('tuplekey1','tuplekey2')": value,}
Should be:
{('tuplekey1','tuplekey2'): value,}
(please do not down-vote if question is not clear, I will do my best to clarify if needed... I do not post questions here often.)
Tuples cannot be used as dictionary keys when you want to save data with shelve or json. So in your case string representation of tuples was used instead of actual tuples - repr(('tuplekey1', 'tuplekey2')) gives "('tuplekey1', 'tuplekey2')".
So you need additional step to parse strings into tuples. For example, eval("('tuplekey1', 'tuplekey2')") or tuple(val.strip("()' ") for val in a.split(',')). Of course, you need to know (or recognize) what strings should be parsed into tuples.
The problem may be avoided - data should stored in another manner - another representation of tuples or another structure of data.
Is there an easy and more or less standard way to dump all the variables into a file, something like stacktrace but with the variables names and values? The ones that are in locals(), globals() and maybe dir().
I can't find an easy way, here's my code for "locals()" which doesn't work because the keys can be of different types:
vars1 = list(filter(lambda x: len(x) > 2 and locals()[x][:2] != "__", locals()))
And without filtering, when trying to dump the variables I get an error:
f.write(json.dumps(locals()))
# =>
TypeError: <filter object at 0x7f9bfd02b710> is not JSON serializable
I think there must be something better that doing it manually.
To start, in your non-working example, you don't exactly filter the keys (which should normally only be strings even if it's not technically required); locals()[x] is the values.
But even if you did filter the keys in some way, you don't generally know that all of the remaining values are JSON serialisable. Therefore, you either need to filter the values to keep only types that can be mapped to JSON, or you need a default serialiser implementation that applies some sensible serialisation to any value. The simplest thing would be to just use the built-in string representation as a fall-back:
json.dumps(locals(), default=repr)
By the way, there's also a more direct and efficient way of dumping JSON to a file (note the difference between dump and dumps):
json.dump(locals(), f, default=repr)
Say I have a dict at hand at runtime, is there an easy way to create code that defines the dict. For example it should output the string
"d = {'string_attr1' : 'value1', 'bool_attr1': True}"
Of course it would be possible to write a converter function by hand, which iterates over the key-value pairs and puts together the string. Would still require to handle special cases to decide if values have to be quoted or not, etc.
More generally: Is there a built in way or a library to generate variable declarations from runtime data structures?
Context: I would like to use a list of dicts as input for a code generator. The content of the dicts would be queried from an SQL database. I don't want to tightly couple code generation to the querying of the SQL database, so I think it would be convenient to go with generating a python source file defining a list of dictionaries, which can be used as an input to the code generator.
>>> help(repr)
Help on built-in function repr in module __builtin__:
repr(...)
repr(object) -> string
Return the canonical string representation of the object.
For most object types, eval(repr(object)) == object.
I'm trying to get a list of the names in a table using Django. The field I'm searching for is "name", and I print out my response, which gives the following:
[u"name1", u"name2"]
However, when I send that to a website in javascript, I see that the length is 16, though console.log shows the same result as the python print statements. When I try to iterate over the list that prints as above, I get the integers 0-15 (the loop I am using is
for (var name in names)).
Why is the string representation of this list so much different than the actual representation, and how do I get a representation that matches the print representation if I can't iterate over it or anything?
This is because names is actually a string within your javascript. You need to pass back the json list or convert the stringified json into objects. This second part can be done with JSON.parse(). Unfortunately, your question doesn't show how you're returning the data or how you're handling the data within javascript, so I can't help you any further than this for now.
I am using easyzone and dnspython to extract DNS records from a zone file. When extracting A records I am given back a string and an object in a tuple. I am new to Python coming from PHP and am not quite sure how to get at this object to get the value of it? I had no problems getting the string value in the tuple.
In this code snippet I iterate through the A records and write the values into a CSV:
# Write all A records
for a in z.names.items():
c.writerow([domain, 'A', a.__getitem__(0), a])
a contains the following:
('www.121dentalcare.com.', <easyzone.easyzone.Name object at 0x1012dd190>)
How would I access this object within a which is in the 2nd half of this tuple??
You can use indices to get items from a tuple:
sometuple[1]
just as you can do with lists and strings (see sequence types).
The documentation of easyzone is a little on the thin side, but from looking at the source code it appears the easyzone.easyzone.Name objects have .name, .soa and .ttl attributes:
print sometuple[1].name
The .soa attribute is another custom class, with .mname, .rname, .serial, .refresh, .retry, .expire and .minttl properties.