When there is a $lookup stage to join a list of large documents, an error Total size of documents in ... matching ... exceeds maximum document size can arrive.

It’s possible to avoid this with $unwind stage right after $lookup. More explanations in the documentation. And then the documents can be regrouped with the required fields.

Order.objects.aggregate(
    {
        '$lookup': {
            'from': 'item',
            'localField': '_id',
            'foreignField': 'order_id',
            'as': 'items'
        }
    },
    {
        "$unwind": "$items"
    },
    {
        "$group": {
            "_id": "$_id",
            "date": {"$last": "$date"},
            "items": {
                "$push": {
                    "name": "$items.name",
                    "price": "$items.price"
                }
            }
        }
    }
)

For example there is a collection device_status which stores the different states for the devices. The task is to find the devices which passed from off to on at least one time.

{ "device" : "device1", "state" : "on", "ts": ISODate("2018-06-07T17:05:29.340+0000") }
{ "device" : "device2", "state" : "off", "ts": ISODate("2018-06-08T17:05:29.340+0000") }
{ "device" : "device3", "state" : "on", "ts": ISODate("2018-06-09T17:05:29.340+0000")}
{ "device" : "device3", "state" : "shutdown", "ts": ISODate("2018-06-09T18:05:29.340+0000")}
{ "device" : "device2", "state" : "load", "ts": ISODate("2018-06-09T19:05:29.340+0000") }
{ "device" : "device2", "state" : "on", "ts": ISODate("2018-06-10T17:05:29.340+0000") }
{ "device" : "device3", "state" : "off", "ts": ISODate("2018-06-11T17:05:29.340+0000") }
{ "device" : "device1", "state" : "idle", "ts": ISODate("2018-06-11T18:05:29.340+0000") }
{ "device" : "device3", "state" : "on", "ts": ISODate("2018-06-12T17:05:29.340+0000") }
...

The first stage is to sort the data by device and date.

Starting a new project is a common task in microservices architecture. To do this it’s better to have a some template. I put my version to the flask-mongoengine-bootstrap repository. The key point are:

• very basic, only flask, flask-mongoengine and structlog in requirements
• configuration through environment variables
• configured logging in JSON format
• marking log records with request_id
• possibility to run development version make dev and tests make test through docker
• a template for Makefile
• examples of model, api route and tests

If you’ll use it, do not forget to change SECRET_KEY.

If you look for a MongoDB-as-a-service and want to try Azure Cosmos DB, be careful, because not all mongo’s functions are supported. For example, there is a problem with aggregation and group by. I didn’t found the list about what else can be different.

Other solid solutions are MongoDB Atlas, it supports Azure from this summer, and mLab.

One of the rules to prevent XSS attack is to sanitize HTML. Here is an example how to do this with Mongoengine custom field and bleach library.

An example of the use of the Q class to build the query.

It’s always better to have a faster tests. For tests which use a database there are two ways to increase the speed:

• a mock library, like mongomock
• in-memory database

There are several possibilities to use MongoDB in Django:

• PyMongo as is
• Django MongoDB Engine
• Mongoengine
• MongoKit

Django MongoDB uses django-nonrel, which is a fork based on Django 1.3.
I don’t like this idea, because now Django 1.5 is ready to out. Beetween Mongoengine and MongoKit, I like more Mongoengine. There are several comparative articles:

• Using MongoDB in your Django app
• What’s the best MongoDB ORM for Python?

So I created a simple CRUD app using Mongoengine. The model definition in Mongoengine looks like in Django.