I have a JSON file that looks something like this:
{ data:
{ 123:
{ 212:
{ 343:
In python I load the JSON using json.loads(r.text). The strings inside the data object are not guaranteed to be those but can be any number. What I want to be able to do is to be able to get those numbers so I can store then in an array. In this example I want the array to look like [123,212,343]. Is there anyway to do this since they are nested objects and not a JSON array?
Thanks
Very briefly:
#!/usr/bin/env python
import json
foo = json.loads('{"123": null, "234": null, "456": null}')
print map(int, foo.keys())
The list foo.keys() will not be in the same order as presented in the JSON object (or its string representation).
If you need to preserve ordering, you might try the following modification:
#!/usr/bin/env python
import json, collections
foo = json.loads('{"123": null, "234": null, "456": null}', object_pairs_hook=collections.OrderedDict)
print map(int, foo.keys())
As you iterate over the list of keys, you can do a reverse lookup on an individual key in the usual way.
Note that you will likely want to convert the integer-key back to a Python string with str(), in order to retrieve its associated value. If you just want the list of keys for lookups, however, and you don't really need the actual integer values, you can skip the initial map call and just preserve the keys as strings.
Related
I have a dictionary full of unicode keys/values due to importing JSON through json.loads().
dictionaryName = {u'keyName' : u'valueName'}
I'm trying to access values inside the dictionary as follows:
accessValueName = dictionaryName.get('keyName')
This returns None, assumedly because it is looking for the String 'keyName' and the list is full of unicode values. I tried sticking a 'u' in front of my keyName when making the call, but it still returns none.
accessValueName = dictionaryName.get(u'keyName')
I also found several seemingly outdated methods to convert the entire dictionary to string values instead of unicode, however, they did not work, and I am not sure that I need the entire thing converted.
How can I either convert the entire dictionary from Unicode to String or just access the values using the keyname?
EDIT:
I just realized that I was trying to access a value from a nested dictionary that I did not notice was nested.
The solution is indeed:
accessValueName = dictionaryName.get('keyName')
Dictionaries store values in a hash table using the hash values of the object.
print(hash(u"example"))
print(hash("example"))
Yields the same result. Therefore the same dictionary value should be accessible with both.
I'm new to python and even newer to SQL and have just run into the following problem:
I want to insert a list (or actually, a list containing one or more dictionaries) into a single cell in my SQL database. This is one row of my data:
[a,b,c,[{key1: int, key2: int},{key1: int, key2: int}]]
As the number of dictionaries inside the lists varies and I want to iterate through the elements of the list later on, I thought it would make sense to keep it in one place (thus not splitting the list into its single elements). However, when trying to insert the list as it is, I get the following error:
sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.
How can this kind of list be inserted into a single cell of my SQL database?
SQLite has no facility for a 'nested' column; you'd have to store your list as text or binary data blob; serialise it on the way in, deserialise it again on the way out.
How you serialise to text or binary data depends on your use-cases. JSON (via the json module could be suitable if your lists and dictionaries consist only of text, numbers, booleans and None (with the dictionaries only using strings as keys). JSON is supported by a wide range of other languages, so you keep your data reasonably compatible. Or you could use pickle, which lets you serialise to a binary format and can handle just about anything Python can throw at it, but it's specific to Python.
You can then register an adapter to handle converting between the serialisation format and Python lists:
import json
import sqlite
def adapt_list_to_JSON(lst):
return json.dumps(lst).encode('utf8')
def convert_JSON_to_list(data):
return json.loads(data.decode('utf8'))
sqlite3.register_adapter(list, adapt_list_to_JSON)
sqlite3.register_converter("json", convert_JSON_to_list)
then connect with detect_types=sqlite3.PARSE_DECLTYPES and declare your column type as json, or use detect_types=sqlite3.PARSE_COLNAMES and use [json] in a column alias (SELECT datacol AS "datacol [json]" FROM ...) to trigger the conversion on loading.
How can I store python 'list' values into MySQL and access it later from the same database like a normal list?
I tried storing the list as a varchar type and it did store it. However, while accessing the data from MySQL I couldn't access the same stored value as a list, but it instead it acts as a string. So, accessing the list with index was no longer possible. Is it perhaps easier to store some data in the form of sets datatype? I see the MySQL datatype 'set' but i'm unable to use it from python. When I try to store set from python into MySQL, it throws the following error: 'MySQLConverter' object has no attribute '_set_to_mysql'. Any help is appreciated
P.S. I have to store co-ordinate of an image within the list along with the image number. So, it is going to be in the form [1,157,421]
Use a serialization library like json:
import json
l1 = [1,157,421]
s = json.dumps(l1)
l2 = json.loads(s)
Are you using an ORM like SQLAlchemy?
Anyway, to answer your question directly, you can use json or pickle to convert your list to a string and store that. Then to get it back, you can parse it (as JSON or a pickle) and get the list back.
However, if your list is always a 3 point coordinate, I'd recommend making separate x, y, and z columns in your table. You could easily write functions to store a list in the correct columns and convert the columns to a list, if you need that.
I have a dict, ast that is stored as:
{u'databaseConnections': {u'shard1': {u'username': u'user'}}}
I want to convert this into JSON so I can do something like:
user = dbConf['databaseConnections']['shard1']['username']
I've tried using json.dumps(ast, ensure_ascii=False), but this just gives me the error:
print dbConf['databaseConnections']['shard1']['username']
TypeError: string indices must be integers
What am I doing wrong?
Converting this thing to JSON is unnecessary and counterproductive; you can just do
user = ast['databaseConnections']['shard1']['username']
with your ast dict directly. dicts are key-value mappings; JSON is text. You're trying to access your data as a key-value mapping, not serialize it and send it over the internet.
I need to do the following in my code:
Read data from a text file
Convert the data into Json
Upsert the data into MongoDB
Here is an example of how the content of text file is going to look:
{
"S": "someString" <- Type String when inerted in mongodb
"N": 123 <- Type Int32
"F": 12.3 <- Type Double
"D": ? <- Need to be Type DateTime when inerted in mongodb
}
I don't know what I am suppose to have in the place of the "?" so when I use bson.json_util.loads function in python it can properly convert the text file into Json which later can be inserted into mongoDB.
Here is the code that does the load and insert:
with open('data.txt') as f:
data = json_util.loads(f.read())
db[dbName][colName].update({'_id': id}, data, upsert=True,safe=True)
I would appreciate it if someone can give an example of how the file should be formatted. (If your example can include even more complex Bson types like type "binary" or "code" that would be nice too :) )
Mongo's representation of datetimes is {"$date": number-of-milliseconds-since-epoch}. In your example:
{
"S": "someString",
"N": 123,
"F": 12.3,
"D": {"$date": 1352540684243}
}
D will generate a datetime field when written to mongo.
See the documentation for mongo json extensions.
You can also easily extend json_util to program your own extensions, for example, for ISO-formatted datetimes:
import json, dateutil.parser, bson.json_util
a = """{
"mydate": {"$isodate": "2012-11-01T20:19:55.782Z"}
}"""
def my_hook(dct):
if '$isodate' in dct:
return dateutil.parser.parse(dct['$isodate'])
return bson.json_util.object_hook(dct)
obj = json.loads(a, object_hook=my_hook)
JSON doesn't know anything about dates, so it's up to you to decide upon a standard format to serialize to and deserialize from. It'll be stored as a string in MongoDB, so just make sure you use the same format string when you call strftime and strptime in your Python code.
Why not use a timestamp. They are very easy to work with and require no real special handling besides in your actual program code or can easily be left alone as the integer that they are. I have dealt with format strings and there really isn't a nice way to ensure they are the same everywhere, hence why I have moved towards using a timestamp whenever possible.