Using Azure Cosmos DB with the Python SDK, I'm trying to select a value inside a JSON file structured like this:
{
"id": "40",
"data": [
{
"x": "0.0959",
"y": "-0.1303",
"z": "0.0202"
}
]
}
My query works with getting all three values x, y, z inside data but when I try to select a single value with data.x it returns an empty list. My query looks like this:
Select f.data, f.id from file as f where f.id = "40"
What am I doing wrong?
data field type seems as an Array, so below query has worked for me with your data,
Select f.data[0].x, f.data[0].y, f.data[0].z from file as f where f.id = '40'
Related
how to create document and collection in mongodb to make python code configuration. Get attribute name, datatype, function to be called from mongodb ?
mongodb collection sample example
db.attributes.insertMany([
{ attributes_names: "email", attributes_datype: "string", attributes_isNull="false", attributes_std_function = "email_valid" }
{ attributes_names: "address", attributes_datype: "string", attributes_isNull="false", attributes_std_function = "address_valid" }
]);
Python script and function
def email_valid(df):
df1 = df.withColumn(df.columns[0], regexp_replace(lower(df.columns[0]), "^a-zA-Z0-9#\._\-| ", ""))
extract_expr = expr(
"regexp_extract_all(emails, '(\\\w+([\\\.-]?\\\w+)*#\\[A-Za-z\-\.]+([\\\.-]?\\\w+)*(\\\.\\\w{2,3})+)', 0)")
df2 = df1.withColumn(df.columns[0], extract_expr) \
.select(df.columns[0])
return df2
How to get all the mongodb values in python script and call the function according to attribues.
To create MongoDB collection from a python script :
import pymongo
# connect to your mongodb client
client = pymongo.MongoClient(connection_url)
# connect to the database
db = client[database_name]
# get the collection
mycol = db[collection_name]
from bson import ObjectId
from random_object_id import generate
# create a sample dictionary for the collection data
mydict = { "_id": ObjectId(generate()),
"attributes_names": "email",
"attributes_datype": "string",
"attributes_isNull":"false",
"attributes_std_function" : "email_valid" }
# insert the dictionary into the collection
mycol.insert_one(mydict)
To insert multiple values in the MongoDB, use insert_many() instead of insert_one() and pass the list of dictionary to it. So your list of dictionary will look like this
mydict = [{ "_id": ObjectId(generate()),
"attributes_names": "email",
"attributes_datype": "string",
"attributes_isNull":"false",
"attributes_std_function" : "email_valid" },
{ "_id": ObjectId(generate()),
"attributes_names": "email",
"attributes_datype": "string",
"attributes_isNull":"false",
"attributes_std_function" : "email_valid" }]
To get all the data from MongoDB collection into python script :
data = list()
for x in mycol.find():
data.append(x)
import pandas as pd
data = pd.json_normalize(data)
And then access the data as you access an element of a list of dictionaries:
value = data[0]["attributes_names"]
I have a mongo database including the following collection:
"
"_id": {
"$oid": "12345"
},
"id": "333555",
"token": [
{
"access_token": "ac_33bc",
"expires_in": 3737,
"token_type": "bearer",
"expires_at": {
"$date": "2021-07-02T13:37:28.123Z"
}
}
]
}
In the next python script I'm trying to return and print only the access_token but can't figure out how to do so. I've tried various methods which none of the worked.I've given the "id" as a parameter
def con_mongo():
try:
client = pymongo.MongoClient("mongodb:localhost")
#DB name
db = client["db1"]
#Collection
coll = db["coll1"]
#1st method
x = coll.find({"id":"333555"},{"token":"access_token"})
for data in x:
print(x)
#2nd method
x= coll.find({"id":"333555"})
tok=x.distinct("access_token")
#print(x[0])
for data in tok:
print(data)
except Exception:
logging.info(Exception)
It doesn't work this way, although if I replace (or remove) the "access_token" with simply "token" it works but I get back all the informations included in the field "token" where I only need the value of the "access_token".
Since access_token is an array element, you need to qualify it's name with the name of the array, to properly access its value.
Actually you can first extract the whole document and get the desired value through simple list and dict indexing.
So, assuming you are retrieving many documents with that same id:
x = [doc["token"][0]["access_token"] for doc in coll.find({"id":"333555"})]
The above, comprehensively creates a list with the access_tokens of all the documents matching the given id.
If you just need the first (and maybe only) occurrence of a document with that id, you can use find_one() instead:
x = coll.find_one({"id":"333555"})["token"][0]["access_token"]
# returns ac_33bc
token is a list so you have to reference the list element, e.g.
x = coll.find({"id":"333555"},{"token.access_token"})
for data in x:
print(data.get('token')[0].get('access_token'))
prints:
ac_33bc
I've been experimenting with python for a while and just ran into an issue I can't find an answer to. Just started building a small 'banking' console app and it's "database" is in JSON file. Now I added a command called 'transfer' which should transfer 'funds' from one user_id to another.
JSON file looks like this:
{
"users": [
{
"user_id": "u111111111",
"pin": "2222",
"balance": "50800"
},
{
"user_id": "u123456789",
"pin": "1234",
"balance": "0"
}
]
}
Last thing I tried was something like this and got completely stuck.
user_database = 'accounts.json'
ujf = json.loads(open(user_database, 'r+').read())
for element in ujf['users']:
if (element['user_id'] == str(user_id_one)):
element['balance'] = str(user_one_new_balance)
else:
pass
How do I update the 'balance' value for user_id u111111111 and for user_id u123456789 and save the file, without creating a new file?
First import JSON module:
import json
Then load the JSON file as dictionary:
d = json.load(r'PATH')
Then inside the loop you can assign user['balance'] to whatever you like, e.g. 8 here:
for user in d['users']:
if user['user_id'] in ['u111111111','u123456789']:
user['balance'] = 8
Then dump back:
json.dump(d,(open(r'PATH','w')))
The result I got from SQLite in Python looks like this:
{"John", "Alice"}, {"John", "Bob"}, {"Jogn", "Cook"} ......
I want to convert the result into JSON format like this:
{
"Teacher": "John",
"Students": ["Alice", "Bob", "Cook" .....]
}
I used GROUP_CONCAT to concat all the students' name and the following code:
row_headers = [x[0] for x in cursor.description] #this will extract row headers
result = []
for res in cursor.fetchall():
result.append(dict(zip(row_headers, res)))
I was able to get this result:
{
"Teacher": "John",
"Students": "Alice, Bob, Cook"
}
How can I make the students into array format?
If your version of sqlite has the JSON1 extension enabled, it's easy to do in pure SQL:
SELECT json_object('Teacher', teacher,
'Students', json_group_array(student)) AS result
FROM ex
GROUP BY teacher;
DB Fiddle example
You could just do result["Students"] = result["Students"].split(", ").
I'm currently working with a database and i would like to display its values on a webpage, using highcharts.
Here is what i use to fetch the data in the web app :
#app.route("/data.json")
def data():
connection = sqlite3.connect("/home/pi/database/Main_Database.db")
cursor = connection.cursor()
cursor.execute("SELECT epochTime, data_x from table")
results = cursor.fetchall()
return json.dumps(results)
Then i currently get this value by doing this in my html:
$.getJSON('http://192.168.1.xx/data.json', function (data) {
// Create the chart
$('#container').highcharts('StockChart', {
rangeSelector : {
selected : 1
},
title : {
text : 'title'
},
series : [{
name : 'Value',
data : data,
tooltip: {
valueDecimals: 2
}, .......
This works if i want to display only one data array.
If i want to display more than one array, then it looks like each arrays must be preceded by its name respecting a certain parsing (i checked on the data sample used by highcharts).
Example:
data1:[(epochTime, 200),(epochTime,400)];data2:[(epochTime, 2),(epochTime,4)]
I have some trouble to json.dumps two arrays from two different tables for example. I tried to use this following command : json.dumps({data1:results}).
But the results is still not readable.
Do you have any advice ? Or example/templates of webapp/highcharts using sqlite?
Thanks a lot !
I think this should work:
In the controller:
Fetch 2 results and put them in a dictionary.
#app.route("/data.json")
def data():
connection = sqlite3.connect("/home/pi/database/Main_Database.db")
cursor = connection.cursor()
cursor.execute("SELECT epochTime, data_x from table")
results1 = cursor.fetchall()
cursor.execute("SELECT epochTime, data_x from table2")
results2 = cursor.fetchall()
return json.dumps({'result1': results1,
'result2': results2})
On the page:
$.getJSON('http://192.168.1.xx/data.json', function (data) {
// Create the chart
$('#container').highcharts('StockChart', {
rangeSelector : {
selected : 1
},
title : {
text : 'title'
},
series : [{
name : 'Value1',
data : data.result1,//read result1
tooltip: {
valueDecimals: 2
},
{
name : 'Value2',
data : data.result2,//read result2
tooltip: {
valueDecimals: 2
}, .......