i've tried to use $cond within $match in one of the stages of an aggregation as shown below :
{ "$match" : {
"field1" : {
"$cond" : {
"if" : { "$eq" : [ "$id" , 1206]},
"then" : 0,
"else" : 1545001200
}
},
"field2" : value2 }}
But i got this error :
Error:
Assert: command failed: {
"ok" : 0,
"errmsg" : "unknown operator: $cond",
"code" : 2,
"codeName" : "BadValue"
} : aggregate failed
The mongodb version is 3.4.4.
Any idea about this issue?
You just have to reword the logic a little bit.
{ $match: { $expr: {
$or: [
{ $and: [
{ $eq: [ "$id", 1206 ] },
{ $eq: [ "$field1", 0 ] }
]},
{ $and: [
{ $ne: [ "$id", 1206 ] },
{ $eq: [ "$field1", 1545001200 ] }
]},
],
}}}
Logically, the two statements are equivalent:
Match the document by checking field1 == 0 if id == 1206, otherwise match the document by checking field1 == 1545001200
Match the document if either (id == 1206 and field1 == 0) or (id != 1206 and field1 == 1545001200).
For those coming across this later on down the road:
This won't work for 3.4.4. But in MongoDB 3.6 they introduced the $expr operator that lets you use $cond and other operations within a $match query.
https://docs.mongodb.com/manual/reference/operator/aggregation/match/
For an example see iamfrank's answer.
Also as mentioned in the comments you could do this later down the pipeline. But ideally you'll want to filter out results as early on in the pipeline using $match in order to improve processing times.
Unless important details were left out of the question, I think you are complicating something simple. Filtering during a $match aggregation step is the natural expected thing it should do.
For this particular example, there are only two simple scenarios to match a document. There is no need to use any other operators, just define the two different mutually exclusive queries and put them in an $or logical operator:
{'$match': {'$or': [
{'id': 1206, 'field1': 0},
{'id': {'$ne': 1206}, 'field1': 1545001200},
]}}
Related
Using Ubuntu 21.04, MongoDB Community 4.4.9, pymongo in Python 3.9:
I'm merging data from two collections on one shared key, membershipNumber. membershipNumber is associated with a different user-level identifier, an_user_id, in another collection, and should be unique. However, in many cases, there are n an_user_ids for a single membershipNumber. Right now, this means that I have many duplicate membershipNumbers, causing there to be duplicate documents where everything - apart from an_user_id - is the same in my newly created collection.
In order to circumvent this issue, I want the following to happen:
whenever there are >1 an_user_ids which match a given membershipNumber, I want to create an array that holds ALL an_user_ids that match a given membershipNumber in a newly created collection (using $out)
that way, every membershipNumber in the collection will be unique.
One question re the practicality of this also remains: Will this mean I'll be able to $merge or $insert data which is linked via an_user_id and from a different collection/aggregation onto this newly created collection?
Any help would be hugely appreciated. Thanks!
Working code that I have (which however doesn't prevent duplication):
p = [
{
'$project' : {
'_id' : 0,
'membershipNumber' : 1,
'address' : 1,
'joinDate' : 1,
'membershipType' : 1
}
},
# THE JOIN!!
{
'$lookup': {
'from': "an_users", # the other collection
'localField' : 'membershipNumber',
'foreignField' : 'memref',
'as': "details"
}
},
# retain unmatchable cases
{
'$unwind' : {
'path' : '$details',
'preserveNullAndEmptyArrays' : True
}
},
{
'$project' : {
'_id' : 0,
'membershipNumber' : 1,
'home' : 1,
'joinDate' : 1,
'membershipType' : 1,
'an_user_id' : '$details.user_id',
}
},
{
'$out' : {
'db' : 'mydb',
'coll' : 'new_coll'
}
}
]
members.aggregate(pipeline=p)
And this is what the (unwanted) duplicate data look like in the new collection:
{
"_id": 1,
"membershipNumber": "123456",
"membershipType": "STD",
"home: "Hogwarts",
"joinDate": {
"$date": "2000-01-01T00:00:00.000Z"
},
"an_user_id": "12345"
},
{
"_id": 2,
"membershipNumber": "123456",
"membershipType": "STD",
"home": "Hogwarts"
"joinDate": {
"$date": "2000-01-01T00:00:00.000Z"
},
"an_user_id": "12346"
}
And this is what I'd like it to look like...
{
"_id": 1,
"membershipNumber": "123456",
"membershipType": "STD",
"home": "Hogwarts"
"joinDate": {
"$date": "2000-01-01T00:00:00.000Z"
},
"an_user_id": ["12345", "12346"]
}
Not exactly sure how the $out conditionally comes into play here, but given two collections as follows:
db.foo.insert([
{_id:1, membershipNumber: 1, type: "STD"},
{_id:3, membershipNumber: 5, type: "STD"},
{_id:8, membershipNumber: 8, type: "STD"}
]);
db.foo2.insert([
{_id:1, memref: 1, an_user_id: 1},
{_id:2, memref: 1, an_user_id: 2},
{_id:3, memref: 1, an_user_id: 3},
{_id:4, memref: 5, an_user_id: 5}
// No lookup for memref 8, just to test
]);
Then this pipeline produces the target output. No initial $project is required.
db.foo.aggregate([
// Call the join field "an_user_id" because we are going to OVERWRITE
// it in the next stage. This avoids creating extra fields that we will
// want to $unset later to minimize clutter.
{$lookup: {from: "foo2",
localField: "membershipNumber",
foreignField: "memref",
as: "an_user_id"}}
// Turn the big array of objects into an array of just an_user_id:
,{$addFields: {an_user_id: {$map: {
input: "$an_user_id",
in: "$$this.an_user_id"
}}
}}
]);
I have a collection of accounts and I am trying to find an account in which the targetAmount >= totalAmount + N
{
"_id": {
"$oid": "60d097b761484f6ad65b5305"
},
"targetAmount": 100,
"totalAmount": 0,
"highPriority": false,
"lastTimeUsed": 1624283088
}
Now I just select all accounts, iterate over them and check if the condition is met. But I'm trying to do this all in a query:
amount = 10
tasks = ProviderAccountTaskModel.objects(
__raw__={
'targetAmount': {
'$gte': {'$add': ['totalAmount', amount]}
}
}
).order_by('-highPriority', 'lastTimeUsed')
I have also tried using the $sum, but both options do not work.
Can't it be used when searching, or am I just going the wrong way?
You can use a $where. Just be aware it will be fairly slow (has to execute Javascript code on every record) so combine with indexed queries if you can.
db.getCollection('YourCollectionName').find( { $where: function() { return this.targetAmount > (this.totalAmount + 10) } })
or more compact way of doing it will be
db.getCollection('YourCollectionName').find( { $where: "this.targetAmount > this.totalAmount + 10" })
You have to use aggregation instead of the find command since self-referencing of documents in addition to arithmetic operations won't work on it.
Below is the aggregation command you are looking for. Convert it into motoengine equivalent command.
db.collection.aggregate([
{
"$match": {
"$expr": {
"$gte": [
"$targetAmount",
{
"$sum": [
"$totalAmount",
10
]
},
],
},
},
},
{
"$sort": {
"highPriority": -1,
"lastTimeUsed": 1,
},
},
])
Mongo Playground Sample Execution
I am using Python and MongoEngine to try and query the below Document in MongoDB.
I need a query to efficiently get the Documents only when they contain Embedded Documents 'Keywords' that match the following criteria:
Keywords Filtered where the Property 'SFR' is LTE '100000'
SUM the filtered keywords
Return the parent documents where SUM of the keywords matching the criteria is Greater than '9'
Example structure:
{
"_id" : ObjectId("5eae60e4055ef0e717f06a50"),
"registered_data" : ISODate("2020-05-03T16:12:51.999+0000"),
"UniqueName" : "SomeUniqueNameHere",
"keywords" : [
{
"keyword" : "carport",
"search_volume" : NumberInt(10532),
"sfr" : NumberInt(20127),
"percent_contribution" : 6.47,
"competing_product_count" : NumberInt(997),
"avg_review_count" : NumberInt(143),
"avg_review_score" : 4.05,
"avg_price" : 331.77,
"exact_ppc_bid" : 3.44,
"broad_ppc_bid" : 2.98,
"exact_hsa_bid" : 8.33,
"broad_hsa_bid" : 9.29
},
{
"keyword" : "party tent",
"search_volume" : NumberInt(6944),
"sfr" : NumberInt(35970),
"percent_contribution" : 4.27,
"competing_product_count" : NumberInt(2000),
"avg_review_count" : NumberInt(216),
"avg_review_score" : 3.72,
"avg_price" : 210.16,
"exact_ppc_bid" : 1.13,
"broad_ppc_bid" : 0.55,
"exact_hsa_bid" : 9.66,
"broad_hsa_bid" : 8.29
}
]
}
From the research I have been doing, I believe an Aggregate type query might do what I am attempting.
Unfortunately, being new to MongoDB / MongoEngine I am struggling to figure out how to structure the query and have failed in finding an example similar to what I am attempting to do (RED FLAG RIGHT????).
I did find an example of a aggregate but unsure how to structure my criteria in it, maybe something like this is getting close but does not work.
pipeline = [
{
"$lte": {
"$sum" : {
"keywords" : {
"$lte": {
"keyword": 100000
}
}
}: 9
}
}
]
data = product.objects().aggregate(pipeline)
Any guidance would be greatly appreciated.
Thanks,
Ben
you can try something like this
db.collection.aggregate([
{
$project: { // the first project to filter the keywords array
registered_data: 1,
UniqueName: 1,
keywords: {
$filter: {
input: "$keywords",
as: "item",
cond: {
$lte: [
"$$item.sfr",
100000
]
}
}
}
}
},
{
$project: { // the second project to get the length of the keywords array
registered_data: 1,
UniqueName: 1,
keywords: 1,
keywordsLength: {
$size: "$keywords"
}
}
},
{
$match: { // then do the match
keywordsLength: {
$gte: 9
}
}
}
])
you can test it here Mongo Playground
hope it helps
Note, I used sfr property only from the keywords array for simplicity
I'm writing a code in which i find this kind of database (i'm using pymongo).
How can i attribute these arrays inside the wishlist field to python arrays?
Alternatively, how can i search my database for a value inside an array inside the wishlist field. E.g.: i want to find all IDs that have, say, ["feldon","c15", "sp"] in their wishlists
{
"_id" : "david",
"password" : "azzzzzaa",
"url" : "url3",
"old_url" : "url3",
"new_url" : ["url1", "url2"],
"wishlist" : [
["all is dust", "mm4", "nm"],
["feldon", "c15", "sp"],
["feldon", "c15", "sp"],
["jenara", "shards", "nm"],
["rafiq", "shards", "nm"]
]
}
You can use distinct if elements in your sublist and are exactly in the same order.
db.collection.distinct("_id", {"wishlist": ["feldon", "c15", "sp"]})
If not, you need to use the aggregate method and the $redact operator.
db.collection.aggregate([
{"$redact": {
"$cond": [
{"$setIsSubset": [
[["feldon","c15", "sp"]],
"$wishlist"
]},
"$$KEEP",
"$$PRUNE"
]
}},
{"$group": {
"_id": None,
"ids": {"$push": "$_id"}
}}
])
I am having a document which is structured like this
{
"_id" : ObjectId("564c0cb748f9fa2c8cdeb20f"),
"username" : "blah",
"useremail" : "blah#blahblah.com",
"groupTypeCustomer" : true,
"addedpartners" : [
"562f1a629410d3271ba74f74",
"562f1a6f9410d3271ba74f83"
],
"groupName" : "Mojito",
"groupTypeSupplier" : false,
"groupDescription" : "A group for fashion designers"
}
Now I want to delete one of the values from this 'addedpartners' array and update the document.
I want to just delete 562f1a6f9410d3271ba74f83 from the addedpartners array
This is what I had tried earlier.
db.myCollection.update({'_id':'564c0cb748f9fa2c8cdeb20f'},{'$pull':{'addedpartners':'562f1a6f9410d3271ba74f83'}})
db.myCollection.update(
{ _id: ObjectId(id) },
{ $pull: { 'addedpartners': '562f1a629410d3271ba74f74' } }
);
Try with this
db.myCollection.update({}, {$unset : {"addedpartners.1" : 1 }})
db.myCollection.update({}, {$pull : {"addedpartners" : null}})
No way to delete array directly, i think this is going to work, i haven't tried yet.