Mongo multiple $lookup and $group in aggregate
aggregation-framework, mongodb
Solution
Thanks to the help of @Veeram and a tiny bit of tweaking I ended up with the following working code:
Groups.aggregate([
{ $match: { _parent: groupId } },
{ $sort: { name: 1 } },
{
$lookup: {
from: 'users',
localField: '_id',
foreignField: 'group',
as: 'users'
}
},
{
$unwind: {
path: "$users",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: 'lists',
localField: 'userId',
foreignField: 'users._id',
as: 'users.lists'
}
},
{
$unwind: {
path: "$users.lists",
preserveNullAndEmptyArrays: true
}
},
{
$match: {
$or: [
{ "users.lists": { $exists: false } },
{ "users.lists.supplier_id": supplierId }
]
}
},
{ $sort: { "users.lists.name": 1 } },
{
$project: {
"name": 1,
"type": { $literal: 'group' },
"users._id": 1,
"users.name": { $ifNull: ["$users.profile.company.name", "$users.username"] },
"users.type": { $literal: 'user' },
"users.lists._id": 1,
"users.lists.name": 1,
"users.lists.item_count": 1,
"users.lists.type": { $literal: 'list' }
}
},
{
$group: {
_id: {
_id: "$_id",
name: "$name",
type: "$type",
user_id: "$users._id"
},
user_id: {
$first: "$users._id"
},
name: {
$first: "$users.name"
},
type: {
$first: "$users.type"
},
children: {
$push: "$users.lists"
}
}
},
{ $sort: { name: 1 } },
{
$group: {
_id: "$_id._id",
name: {
$first: "$_id.name"
},
type: {
$first: "$_id.type"
},
children: {
$push: {
_id: "$user_id",
name: "$name",
type: "$type",
children: "$children"
}
}
}
},
{ $sort: { name: 1 } }
]);
Problem
So I am trying to "join" 3 different mongo collections in a query. So what I need is multiple `$lookup` and `$group` statements in a mongo aggregation. My 3 collections look like this: users: (Pseudocode) ``` { _id, username: String, group: <some group._id> } ``` groups: ``` { _id, name: String, _parent: <another group._id>, } ``` lists: (these are "itemlists" which are owned by a user): ``` { _id, name: String, userId: <some user._id> } ``` So what I would like to do is, given some group id or `null` (for groups with no parent - groups on the highest level) - get all `groups` inside of this group, find all `users` in these groups and also their `lists`. So in the end I would need something like this: ``` [ { _id: someGroupId, name: someGroupName, type: "group", users: [ { _id: someUserId, name: someUserName, type: "user", lists: [ ... and the same again for the lists (type: "list") ] }, ... another user ] }, ... another group ] ``` I hope you understand what I mean! What I have now (thanks to Simon Tretter and lots of research) - please don't mind the Meteor agnostic syntax, you get the idea: ``` Groups.aggregate([ { $match: { _parent: groupId } }, { $sort: { name: 1 } }, { $lookup: { from: 'users', localField: '_id', foreignField: 'group', as: 'users' } }, { $unwind: { path: "$users", preserveNullAndEmptyArrays: true } }, { $lookup: { from: 'lists', localField: 'users._id', foreignField: 'userId', as: 'users.lists' } }, { $unwind: { path: "$users.lists", preserveNullAndEmptyArrays: true } }, { $match: { $or: [ { "users.lists": { $exists: false } }, { "users.lists.supplier_id": supplierId } ] } }, { $sort: { "users.lists.name": 1 } }, { $project: { "name": 1, "type": { $literal: 'group' }, "users._id": 1, "users.name": { $ifNull: ["$users.profile.company.name", "$users.username"] }, "users.type": { $literal: 'user' }, "users.lists._id": 1, "users.lists.name": 1, "users.lists.item_count": 1, "users.lists.type": { $literal: 'list' } } }, { $group: { _id: '$users._id', name: { $first: "$users.name" }, type: { $first: "$users.type" }, children: { $push: "$users.lists" } } }, { $sort: { "users.name": 1 } }, // until here I have one document per user, with their lists inside the "children" key - all good! // now I have to group the users inside their groups ... // NOT WORKING: returns completely wrong stuff { $group: { _id: '$_id', name: { $first: "$name" }, type: { $first: "$type" }, children: { $push: "$users" } } }, { $sort: { name: 1 } } ]); ``` I hope someone can get my on the right track ... the best similar Stack Overflow question I could find didn't help me that much.