pymongo: name 'ISODate' is not defined

Question

I have problem when i try to select data in mongodb with pymongo, this is my code :

import pymongo
from pymongo import MongoClient
import sys
from datetime import datetime

try:
    conn=pymongo.MongoClient('10.33.109.228',27017)
    db=conn.mnemosyne
    data_ip=db.session.aggregate({'$match':{'timestamp':{'$gte': ISODate('2016-11-11T00:00:00.000Z'),'$lte': ISODate('2016-11-11T23:59:59.000Z')}}},{'$group':{'_id':'$source_ip'}})
    for f in data_ip:
        print f['_id']

except pymongo.errors.ConnectionFailure, e:
    print "Could not connect to MongoDB: %s" % e

and when i execute it i have some error like this:

Traceback (most recent call last):
  File "test.py", line 9, in <module>
    data_ip=db.session.aggregate({'$match':{'timestamp':{'$gte': ISODate('2016-11-11T00:00:00.000Z'),'$lte': ISODate('2016-11-11T23:59:59.000Z')}}},{'$group':{'_id':'$source_ip'}})
NameError: name 'ISODate' is not defined

I want the result like this:

{ "_id" : "60.18.133.207" }
{ "_id" : "178.254.52.96" }
{ "_id" : "42.229.218.192" }
{ "_id" : "92.82.171.117" }
{ "_id" : "103.208.120.205" }
{ "_id" : "185.153.208.142" }

this is example structure of mydatabase:

> db.session.findOne()

    {
            "_id" : ObjectId("5786398d1f50070f31f27f7c"),
            "protocol" : "epmapper",
            "hpfeed_id" : ObjectId("5786398d1f50070f31f27f7b"),
            "timestamp" : ISODate("2016-07-13T12:52:29.112Z"),
            "source_ip" : "23.251.55.182",
            "source_port" : 2713,
            "destination_port" : 135,
            "identifier" : "d3374f14-48f7-11e6-9e19-0050569163b4",
            "honeypot" : "dionaea"
    }

Please help me to fix the error

Answer 1

ISODate is a function in the Mongo shell, which is a javascript environment, it's not available within Python.

You can use dateutil for converting a string to datetime object in Python,

import dateutil.parser
dateStr = "2016-11-11T00:00:00.000Z"
dateutil.parser.parse(dateStr)  # returns a datetime.datetime(2016, 11, 11, 00, 0, tzinfo=tzutc())

Using PyMongo, if you want to insert datetime in MongoDB you can simply do the following:

import pymongo
import dateutil
dateStr = '2016-11-11T00:00:00.000Z'
myDatetime = dateutil.parser.parse(dateStr)
client = pymongo.MongoClient()
client.db.collection.insert({'date': myDatetime})

Answer 2

ISODate is a JavaScript Date object. To query range of date using PyMongo, you need to use a datetime.datetime instance which mongod will convert to the appropriate BSON type. You don't need any third party library.

Also you shouldn't be using the Aggregation Framework to do this because the _id field is unique within the collection which makes this a perfect job for the distinct() method.

import datetime


start = start = datetime.datetime(2016, 11, 11)
end = datetime(2016, 11, 11, 23, 59, 59)

db.session.distinct('_id', {'timestamp': {'$gte': start, '$lte': end}})

If you really need to use the aggregate() method, your $match stage must look like this:

{'$match': {'timestamp': {'$gte': start, '$lte': end}}}