Concat arrays using PyMongo failed with unknown gr

Question

I have mongodb data like:

{'word': 'good', 'info': [{'tbl_id': 'd1', 'term_freq': 2}, {'tbl_id': 'd2', 'term_freq': 56}, {'tbl_id': 'd3', 'term_freq': 3}]}
{'word': 'spark', 'info': [{'tbl_id': 'd1', 'term_freq': 6}, {'tbl_id': 'd3', 'term_freq': 11}, {'tbl_id': 'd4', 'term_freq': 10}]}
{'word': 'good', 'info': [{'tbl_id': 'd4', 'term_freq': 12}, {'tbl_id': 'd5', 'term_freq': 8}, {'tbl_id': 'd8', 'term_freq': 7}]}
{'word': 'spark', 'info': [{'tbl_id': 'd5', 'term_freq': 6}, {'tbl_id': 'd6', 'term_freq': 11}, {'tbl_id': 'd7', 'term_freq': 10}]}

and I want to use pymongo to process it, the result should be:

{'word': 'good',
 'info': [{'tbl_id': 'd1', 'term_freq': 2}, {'tbl_id': 'd2', 'term_freq': 56}, {'tbl_id': 'd3', 'term_freq': 3},
          {'tbl_id': 'd4', 'term_freq': 12}, {'tbl_id': 'd5', 'term_freq': 8}, {'tbl_id': 'd8', 'term_freq': 7}]}
{'word': 'spark',
 'info': [{'tbl_id': 'd1', 'term_freq': 6}, {'tbl_id': 'd3', 'term_freq': 11}, {'tbl_id': 'd4', 'term_freq': 10},
          {'tbl_id': 'd5', 'term_freq': 6}, {'tbl_id': 'd6', 'term_freq': 11}, {'tbl_id': 'd7', 'term_freq': 10}]}

I use group in pymongo:

a = mycol.aggregate([{"$group": {"_id":"$word", 'infos': {"$concatArrays": 1}}}])
for i in a:
    print(i)

It went wrong: pymongo.errors.OperationFailure: unknown group operator '$concatArrays'. and I use group keyword:

a = mycol.group(key='word',condition=None, initial={'infos': []}, reduce={"$concatArrays": "info"})
for i in a:
    print(i)

It also went wrong:

Traceback (most recent call last):File "F:/programs/SearchEngine/test.py", line 167, in <module> a = mycol.group(key='word',condition=None, initial={'infos': []}, reduce={"$concatArrays": "info"})  File "C:\Users\ll\.virtualenvs\SearchEngine\lib\site-packages\pymongo\collection.py", line 2550, in group  group["$reduce"] = Code(reduce)  File "C:\Users\ll\.virtualenvs\SearchEngine\lib\site-packages\bson\code.py", line 54, in __new__  "instance of %s" % (string_type.__name__))
TypeError: code must be an instance of str

Answer 1

The reason you are getting this error message is because the $concatArrays operator is an expression operator not a $group accumulator.

That being said, you can do this with the following pipeline:

[
    {
        "$group": {
            "_id": "$word",
            "info": {
                "$push": "$info"
            }
        }
    },
    {
        "$project": {
            "_id": 0,
            "word": "$_id",
            "info": {
                "$reduce": {
                    "input": "$info",
                    "initialValue": [

                    ],
                    "in": {
                        "$concatArrays": [
                            "$$value",
                            "$$this"
                        ]
                    }
                }
            }
        }
    }
]

We create a 2d list of info in the $group stage with the $push operator then in the another $project stage you flatten the list using the $reduce and $concatArrays.