Basically I need to connect to MongoDB documents records and put into values into dict.
**MongoDB Values**
{ "_id" : "LAC1397", "code" : "MIS", "label" : "Marshall Islands", "mappingName" : "RESIDENTIAL_COUNTRY" }
{ "_id" : "LAC1852", "code" : "COP", "label" : "Colombian peso", "mappingName" : "FOREIGN_CURRENCY_CODE"}
How do i map it to dict in the below fashion in python
**syntax :**
dict = {"mappingName|Code" : "Value" }
**Example :**
dict = { "RESIDENTIAL_COUNTRY|MIS" : "Marshall Islands" , "FOREIGN_CURRENCY_CODE|COP" : "Colombian peso" , "COMM_LANG|ENG" : "English" }
**Python Code**
from pymongo import MongoClient
client = MongoClient('localhost', 27017)
db = client.mongo
collection = db.masters
for post in collection.find():
Got stuck after this , not sure how to put into dict in the mentioned method
post will be a dict with the values from mongo, so you can loop the records and append to a new dictionary. As the comments mention, any duplicates would be overridden by the last found value. If this might be an issue, consider a sort() on the find() function.
Sample code:
from pymongo import MongoClient
db = MongoClient()['mydatabase']
db.mycollection.insert_one({ "_id" : "LAC1397", "code" : "MIS", "label" : "Marshall Islands", "mappingName" : "RESIDENTIAL_COUNTRY" })
db.mycollection.insert_one({ "_id" : "LAC1852", "code" : "COP", "label" : "Colombian peso", "mappingName" : "FOREIGN_CURRENCY_CODE"})
mydict = {}
for post in db.mycollection.find():
k = f"{post.get('mappingName')}|{post.get('code')}"
mydict[k] = post.get('label')
print(mydict)
Gives:
{'RESIDENTIAL_COUNTRY|MIS': 'Marshall Islands', 'FOREIGN_CURRENCY_CODE|COP': 'Colombian peso'}
Related
I am reading all collections of a mongodb database and the same time looping collection name in for loop to get each collection data dynamically using pymongo.
this is running on python 3.7 and mongodb 3.4 with pymongo
from pymongo import MongoClient
from bson.objectid import ObjectId
import json
client = MongoClient("localhost", 27017, maxPoolSize=50)
#print(client)
mydatabase = client["testdb"]
collections = mydatabase.collection_names(include_system_collections=False)
for collectionName in collections:
print(collectionName)
mydata = mydatabase.collectionName.find({})
for value in mydata:
print(value)
for key,valueOFproject in value:
print(key)
print(value)
Db
1st collection
{
"_id" : "hiphdkTest_HIPHDK_76P1_P00_19WW09Test",
"project" : "hiphdktest",
"config" : "HIPHDK_76P1_P00_19WW09test",
"manual" : {
"tag1" : "fdsfsdfsd",
"No" : "No",
"prqdata1" : "fsdfadfasdfasdfsdfsd",
"admin1" : "dbhiphdk"
}
}
2nd collection
{
"_id" : "hiphdk_HIPHDK_76P1_P00",
"project" : "hiphdk",
"config" : "HIPHDK_76P1_P00",
"manual" : {
"tag1" : "fdsfsdfsd",
"No" : "No",
"prqdata1" : "fsdfadfasdfasdfsdfsd",
"admin1" : "dbhiphdk"
}
}
3rd collection
{
"_id" : "hiphdk_HIPHDK_76P1_P00_19WW09",
"project" : "hiphdk",
"config" : "HIPHDK_76P1_P00_19WW09",
"manual" : {
"tag1" : "fdsfsdfsd",
"No" : "No",
"prqdata1" : "fsdfadfasdfasdfsdfsd",
"admin1" : "dbhiphdk"
}
}
only getting collection names
hiphdk_HIPHDK_76P1_P00_19WW09
hiphdk_HIPHDK_76P1_P00
hiphdkTest_HIPHDK_76P1_P00_19WW09Test
it should print collection names and data of each collection.
The key change is this line mydata = mydatabase[collectionName].find({})
See if this gives you what you need:
from pymongo import MongoClient
from bson.json_util import dumps
client = MongoClient("localhost", 27017, maxPoolSize=50)
mydatabase = client["testdb"]
collections = mydatabase.list_collection_names(include_system_collections=False)
for collectionName in collections:
mydata = mydatabase[collectionName].find({})
for value in mydata:
print(dumps(value))
I must be really slow because I spent a whole day googling and trying to write Python code to simply list the "code" values only so my output will be Service1, Service2, Service2. I have extracted json values before from complex json or dict structure. But now I must have hit a mental block.
This is my json structure.
myjson='''
{
"formatVersion" : "ABC",
"publicationDate" : "2017-10-06",
"offers" : {
"Service1" : {
"code" : "Service1",
"version" : "1a1a1a1a",
"index" : "1c1c1c1c1c1c1"
},
"Service2" : {
"code" : "Service2",
"version" : "2a2a2a2a2",
"index" : "2c2c2c2c2c2"
},
"Service3" : {
"code" : "Service4",
"version" : "3a3a3a3a3a",
"index" : "3c3c3c3c3c3"
}
}
}
'''
#convert above string to json
somejson = json.loads(myjson)
print(somejson["offers"]) # I tried so many variations to no avail.
Or, if you want the "code" stuffs :
>>> [s['code'] for s in somejson['offers'].values()]
['Service1', 'Service2', 'Service4']
somejson["offers"] is a dictionary. It seems you want to print its keys.
In Python 2:
print(somejson["offers"].keys())
In Python 3:
print([x for x in somejson["offers"].keys()])
In Python 3 you must use the list comprehension because in Python 3 keys() is a 'view', not a list.
This should probably do the trick , if you are not certain about the number of Services in the json.
import json
myjson='''
{
"formatVersion" : "ABC",
"publicationDate" : "2017-10-06",
"offers" : {
"Service1" : {
"code" : "Service1",
"version" : "1a1a1a1a",
"index" : "1c1c1c1c1c1c1"
},
"Service2" : {
"code" : "Service2",
"version" : "2a2a2a2a2",
"index" : "2c2c2c2c2c2"
},
"Service3" : {
"code" : "Service4",
"version" : "3a3a3a3a3a",
"index" : "3c3c3c3c3c3"
}
}
}
'''
#convert above string to json
somejson = json.loads(myjson)
#Without knowing the Services:
offers = somejson["offers"]
keys = offers.keys()
for service in keys:
print(somejson["offers"][service]["code"])
I have two collections in the mongo database. At the moment I have an ID document from collection1 in document in collection2. I want to copy some values from Collection1 to nested field (dataFromCollection1) in related documents in Collection2. I'm looking for help because I can not find a solution to pass values from the mongo base fields to variables in python.
Collection1:
{
"_id" : ObjectId("583d498214f89c3f08b10e2d"),
"name" : "Name",
"gender" : "men",
"secondName" : "",
"testData" : [ ],
"numberOf" : NumberInt(0),
"place" : "",
"surname" : "Surname",
"field1" : "eggs",
"field2" : "hamm",
"field3" : "foo",
"field4" : "bar"
}
Collection2:
{
"_id" : ObjectId("58b028e26900ed21d5153a36"),
"collection1" : ObjectId("583d498214f89c3f08b10e2d")
"fieldCol2_1" : "123",
"fieldCol2_2" : "332",
"fieldCol2_3" : "133",
"dataFromCollection1" : {
"name" : " ",
"surname" : " ",
"field1" : " ",
"field2" : " ",
"field3" : " ",
"field4" : " "
}
}
I think you should use aggregate function in pymongo package, in aggregate you can use $lookup to match the key value pairs and project for projecting the required fields this question has already been asked so this pymongo - how to match on lookup? might helps you.
Then you can use the $out function in aggregate to create the new updated collection of your interest or you can also update the existing collection by using the update in pymongo.
Hi I want to perform a nested search using elasticsearch dsl where a document field has nested json data in it so I want specific nested key values from it like -
Below is the document:-
{
"_index" : "data",
"_type" : "users",
"_id" : "15",
"_version" : 1,
"found" : true,
"_source" : {
"data" : {
"Gender" : "M",
"Marks" : "80",
"name" : "Mayank",
"Address" : "India"
},
"last_updated" : "2017-04-09T01:54:33.764573"
}
}
I only want field values which are stored in an array.
fields_want = ['name', 'Marks']
Output should be like -> {"name":"Mayank", "Marks":"80"}
Elasticsearch dsl documentation is pretty hard to understandand for me.
https://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html#
Dsl code:-
client = Elasticsearch()
s = Search(using=client, index="data") \
.query("match", _type="users") \
.query("match", _id=15)
response = s.execute()
for hit in s:
print(hit.data)
From this code I can get the whole json object under data field.
Can somebody guide me here ?
It was solved.
I have used source filter to get nested output.
client = Elasticsearch()
s = Search(using=client, index="data") \
.query("match", _type="users") \
.query("match", _id=15) \
.source(['data.Name', 'data.Marks'])
response = s.execute()
print response
Output -
{u'Name': u'Mayank', u'Marks': u'80'}
I'm trying out json since I have to work with Cisco API and I can't figure out how to loop through the json object.I can get my keys but i can't get the values.
I am using http://www.jsoneditoronline.org/ to me understand json format but this is what I have so far..
json file :
{
"queryResponse" : {
"#rootUrl" : "\/webacs\/data",
"#requestUrl" : "https : \/\/192.168.116.207\/webacs\/api\/v1\/data\/DeviceGroups\/42",
"#responseType" : "getEntity",
"entity" : {
"#url" : "\/webacs\/data\/className\/15",
"#type" : "className",
"#dtoType" : "deviceGroupsDTO_$$_javassist_5196",
"deviceGroupsDTO" : {
"#id" : "15",
"#displayName" : "String value",
"clearedAlarms" : 1,
"criticalAlarms" : 1,
"groupId" : 2,
"groupName" : "String value",
"informationAlarms" : 1,
"majorAlarms" : 1,
"minorAlarms" : 1,
"name" : "String value",
"warningAlarms" : 1
}
}
}
}
My python script :
import json
jsondata = json.load(open('data.json'))
for rows in jsondata['queryResponse']['entity']['deviceGroupsDTO']:
print(rows)
it print's :
name
#id
warningAlarms
#displayName
informationAlarms
clearedAlarms
majorAlarms
groupId
groupName
criticalAlarms
minorAlarms
not sure what i'm doing wrong...
jsondata['queryResponse']['entity']['deviceGroupsDTO'] is a dictionary.
Iterate over items() to get key, value pairs:
for key, value in jsondata['queryResponse']['entity']['deviceGroupsDTO'].items():
print(key, value)
Note that, in case of python2, you would better use iteritems() in place of items().
See also: What is the difference between dict.items() and dict.iteritems()?