I have the following schema -
{
"_id" : ObjectId("60c3253f19862e6347bc9f4e"),
"farm_id": "Gustavo-chainer",
"first_ts" : ISODate("2021-05-18T09:53:00.000Z"),
"last_ts" : ISODate("2021-05-18T12:53:00.000Z"),
"sensor_data" : [
{
"data" : 76.0,
"sensor": "temperature-sensor",
"start_ts" : ISODate("2021-05-18T09:33:00.000Z"),
"end_ts" : ISODate("2021-05-18T09:53:00.000Z")
},
{
"data" : 74.0,
"sensor": "temperature-sensor",
"start_ts" : ISODate("2021-05-18T12:33:00.000Z"),
"end_ts" : ISODate("2021-05-18T12:53:00.000Z")
}
]
}
where first_ts = minimum of all the values of start_ts present in the sensor_data array and last_ts = maximum of all the values of end_ts present in the sensor_data array.
I want to delete a data point from sensor_data array given the start_ts and end_ts and after deletion, have to update the first_ts and last_ts accordingly.
Example -
Delete data point with "start_ts" : ISODate("2021-05-18T12:33:00.000Z") and "end_ts" : ISODate("2021-05-18T12:53:00.000Z"). After deletion, the document should look like -
{
"_id" : ObjectId("60c3253f19862e6347bc9f4e"),
"first_ts" : ISODate("2021-05-18T09:53:00.000Z"),
"last_ts" : ISODate("2021-05-18T09:53:00.000Z"),
"sensor_data" : [
{
"data" : 76.0,
"sensor": "temperature-sensor"
"start_ts" : ISODate("2021-05-18T09:33:00.000Z"),
"end_ts" : ISODate("2021-05-18T09:53:00.000Z")
}
]
}
I need to write a pymongo query that can do the above task in a single query.
You can try update with aggregation pipeline starting from MongoDB 4.2,
$filter to iterate loop of sensor_data array, check both fields date condition and $not for the opposite condition to exclude matching documents
$min to get minimum start_ts date from sensor_data.start_ts
$max to get maximum end_ts date from sensor_data.end_ts
collection.update(
{
sensor_data: {
$elemMatch: {
start_ts: ISODate("2021-05-18T12:33:00.000Z"),
end_ts: ISODate("2021-05-18T12:53:00.000Z")
}
}
},
[{
$set: {
sensor_data: {
$filter: {
input: "$sensor_data",
cond: {
$not: {
$and: [
{ $eq: ["$$this.start_ts", ISODate("2021-05-18T12:33:00.000Z")] },
{ $eq: ["$$this.end_ts", ISODate("2021-05-18T12:53:00.000Z")] }
]
}
}
}
}
}
},
{
$set: {
first_ts: { $min: "$sensor_data.start_ts" },
last_ts: { $max: "$sensor_data.end_ts" }
}
}],
{ multi: true }
)
Playground
Related
{
"_id" : ObjectId("63920f965d15e98e3d7c450c"),
"first_name" : "mymy",
"last_activity" : 1669278303.4341061,
"username" : null,
"dates" : {
"29.11.2022" : {
},
"30.11.2022" : {
}
},
"user_id" : "1085116517"
}
How can I find all documents with 29.11.2022 contained in date? I tried many things but in all of them it detects the dot letter as something else.
Use $getField in $expr.
db.collection.find({
$expr: {
$eq: [
{},
{
"$getField": {
"field": "29.11.2022",
"input": "$dates"
}
}
]
}
})
Mongo Playground
{
'userid' : '5e6f2f38e8cfcfaf34ee76a6',
'c':[
{'cid':123 ,'flist':['5e6de87050fba047c4c666e1','5e65e475aa1d2a77e1e7d9b3','5e75e5a02dfcda6e321be941']} ,
{'cid':321 ,'flist':['5e92533b0f93cb0f6d813631','5e946afbfd003483a47d412b','5e6de87050fba047c4c666e1']} ,
{'cid':431 ,'flist':['5e65e475aa1d2a77e1e7d9b3','5e946afbfd003483a47d412b','5e75e5a02dfcda6e321be941']}
]
}
userid ='5e6f2f38e8cfcfaf34ee76a6'
fid = '5e6de87050fba047c4c666e1'
db.find({'userid':userid ,'c.flist':{'$eq':fid}} , {'c.$.cid':1} )
i am trying to get all cid that the flist contain fid
i tryed this method but i got only first match without {'c.$.cid':1} i got the whole list
if your intention is to get only cid alone, then below query would work,
db.collection.aggregate([{
'$match': {
'userid': '5e6f2f38e8cfcfaf34ee76a6'
}
},
{
'$unwind': {
'path': '$c'
}
}, {
'$match': {
'c.flist': '5e6de87050fba047c4c666e1'
}
},
{
'$project': {
"c.cid": 1,
"_id": 0
}
}])
would give you below output
{ "c" : { "cid" : "123" } }
{ "c" : { "cid" : "321" } }
I think as per your need, you need to change the structure of collection. And as mention in document of MongoDb projection param,
The $ operator projects the first matching array element from each document in a collection based on some condition from the query statement
you can check that in https://docs.mongodb.com/manual/reference/operator/projection/positional/#project-array-documents.
You need to make cid outside the array I think.
You can use below query
db.collection.aggregate([ { $match: { userid: "5e6f2f38e8cfcfaf34ee76a6", "c.flist": "5e6de87050fba047c4c666e1" } }, { $addFields: { c: { $filter: { input: { $reduce: { input: "$c", initialValue: [], in: { $concatArrays: [ "$$value", [ { cid: "$$this.cid", flist: { $filter: { input: "$$this.flist", as: "item", cond: { $eq: [ "$$item", "5e6de87050fba047c4c666e1" ] } } } } ] ] } } }, as: "item2", cond: { $gt: [ "$$item2.flist", [] ] } } } } } ]).pretty()
to get the following output
{
"_id" : ObjectId("5e95ca8801423e0f9af19b4b"),
"userid" : "5e6f2f38e8cfcfaf34ee76a6",
"c" : [
{
"cid" : 123,
"flist" : [
"5e6de87050fba047c4c666e1"
]
},
{
"cid" : 321,
"flist" : [
"5e6de87050fba047c4c666e1"
]
}
]
}
I have JSON document recorded to MongoDB with structure like so:
[{ "SessionKey": "172e3b6b-509e-4ef3-950c-0c1dc5c83bab",
"Query": {"Date": "2020-03-04"},
"Flights": [
{"LegId":"13235",
"PricingOptions": [
{"Agents": [1963108],
"Price": 61763.64 },
{"Agents": [4035868],
"Price": 62395.83 }]},
{"LegId": "13236",
"PricingOptions": [{
"Agents": [2915951],
"Price": 37188.0}]}
...
The result I'm trying to get is "LegId":"sum_per_flight", in this case -> {'13235': (61763.64+62395.83), '13236': 37188.0} and then get flights with price < N
I've tried to run this pipeline for aggregation step (but it returns list of ALL prices - I don't know how to sum them up properly):
result = collection.aggregate([
{'$match': {'Query.Date': '2020-03-01'}},
{'$group': {'_id': {'Flight':'$Flights.LegId', 'Price':'$Flights.PricingOptions.Price'}}} ])
Also I've tried this pipeline, but it returns 0 for 'total_price_per_flight':
result = collection.aggregate({'$project': {
'Flights.LegId':1,
'total_price_per_flight': {'$sum': '$Flights.PricingOptions.Price'}
}})
You need to use $unwind to flatten Flights array to able iterate individually.
With $reduce operator, we iterate PricingOptions array and sum Price fields (accumulate prices).
The last step we return your documents into original structure. Before that, you may apply "get flights with price < N"
db.collection.aggregate([
{
"$match": {
"Query.Date": "2020-03-04"
}
},
{
$unwind: "$Flights"
},
{
$addFields: {
"Flights.LegId": {
$arrayToObject: [
[
{
k: "$Flights.LegId",
v: {
$reduce: {
input: "$Flights.PricingOptions",
initialValue: 0,
in: {
$add: [
"$$value",
"$$this.Price"
]
}
}
}
}
]
]
}
}
},
{
$group: {
_id: "$_id",
SessionKey: {
$first: "$SessionKey"
},
Query: {
$first: "$Query"
},
Flights: {
$push: "$Flights"
}
}
}
])
MongoPlayground
Thanks for reading my question.
Please execuse any mistakes, i'm working on improving my English.
I have > 4000 records in my MongoDB, this is one of my records :
{
"_id" : ObjectId("5763821ffefb61074041477e"),
"sessionId" : "5138A3B4A5966CE4B2203B8BFC90055F",
"objects" : [
{
"id" : "334449673730",
"point" : 0.5
},
{
"id" : "790373008255",
"point" : 0.5
},
{
"id" : "790373008255",
"point" : 1.0
},
{
"id" : "572453522243",
"point" : 0.5
},
{
"id" : "572453522243",
"point" : 1.0
}
]
}
My result, i want to delete duplicate id but keep the point : 1.0
Result :
{
"_id" : ObjectId("5763821ffefb61074041477e"),
"sessionId" : "5138A3B4A5966CE4B2203B8BFC90055F",
"objects" : [
{
"id" : "334449673730",
"point" : 0.5
},
{
"id" : "790373008255",
"point" : 1.0
},
{
"id" : "572453522243",
"point" : 1.0
}
]
}
I follow this post : How to remove duplicates with a certain condition in mongodb?
its similary with my question but i don't know why result as not as i want :
pipeline = ([
{
"$group": {
"_id": "$id",
"count": { "$sum": 1 },
#"uniqueIds": { "$addToSet": "$_id" },
"Point": { "$max": "$point" }
}
},
{
"$match": {
"count": { "$gte": 1 }
}
}
])
for test_item in collection_forTest.aggregate(pipeline):
print(test_item)
Result :
{'Point': None, 'count': 1, '_id': None}
I can use python code, load all records, check where same id in list, compare if point = 1 and remove same record with point != 1 but i think its slower than aggregation
Can you help me with my problem for all > 4000 records ?
Thanks very much !
MongoDB noob here...
when I do db.students.find().pretty() in the shell I get a long list from my collection...like so..
{
"_id" : 19,
"name" : "Gisela Levin",
"scores" : [
{
"type" : "exam",
"score" : 44.51211101958831
},
{
"type" : "quiz",
"score" : 0.6578497966368002
},
{
"type" : "homework",
"score" : 93.36341655949683
},
{
"type" : "homework",
"score" : 49.43132782777443
}
]
}
now I've got about over 100 of these...I need to run the following on each of them...
lowest_hw_score =
db.students.aggregate(
// Initial document match (uses index, if a suitable one is available)
{ $match: {
_id : 0
}},
// Expand the scores array into a stream of documents
{ $unwind: '$scores' },
// Filter to 'homework' scores
{ $match: {
'scores.type': 'homework'
}},
// Sort in descending order
{ $sort: {
'scores.score': 1
}},
{ $limit: 1}
)
So I can run something like this on each result
for item in lowest_hw_score:
print lowest_hw_score
Right now "lowest_score" works on only one item I to run this on all items in the collection...how do I do this?
> db.students.aggregate(
{ $match : { 'scores.type': 'homework' } },
{ $unwind: "$scores" },
{ $match:{"scores.type":"homework"} },
{ $group: {
_id : "$_id",
maxScore : { $max : "$scores.score"},
minScore: { $min:"$scores.score"}
}
});
You don't really need the first $match, but if "scores.type" is indexed, it means it would be used before unwinding the scores. (I don't believe after the $unwind mongo would be able to use the index.)
Result:
{
"result" : [
{
"_id" : 19,
"maxScore" : 93.36341655949683,
"minScore" : 49.43132782777443
}
],
"ok" : 1
}
Edit: tested and updated in mongo shell