How to remove column from child collection

Question

I have a collection in MongoDB called CrawlUser. It has a list called CrawlStatuses, which is a list of CrawlStatus objects. CrawlStatus has a property called LastErrorMessage which I want to remove from the collections.

I tried to do the following to remove it but it didn't work... No error message given, but the LastErrorMessage column is still there.

db.CrawlUser.update( {}, { $unset: { "CrawlStatuses.LastErrorMessage": 1 } }, false, true);

Any ideas what I'm doing wrong?

One other related question, if I do the $unset command on a column in a collection that is very large (millions of rows), mongodb uses up all of the ram on the server (as if it's trying to store the entire collection in memory), then the server crashes. Is there a better way to remove columns when you have large collections?

Answer 1

The update with the empty parameter doesn't seem to work. I tried it in the mongo shell and mongoconsole. In the mongoconsole it gave an error about update expecting the first parameter to be an array or an object.

However, you can do the same thing using the $exists find query.

Try:

`db.CrawlUser.update( {CrawlStatuses:{$exists:true}}, { $unset: { "CrawlStatuses.LastErrorMessage": 1 } }, false, true);`

That worked for me.

Keep in mind that based on the docs, $exists doesn't use an index, so it will be slower. I suggest adding a parameter that you can add an index on and query it when doing the $unset.

Answer 2

Looks like you have a couple of issues here.

#1: The $unset command

As far as I can see, this should work just fine. I got the following output on my test:

MongoDB shell version: 1.6.5
connecting to: test
> db.foo.save( { _id : 1, status : { err : 'blah', y : 1 } } )
> db.foo.save( { _id : 2, status : { err : 'blahblah', y : 5 } } )
> db.foo.find()
{ "_id" : 1, "status" : { "err" : "blah", "y" : 1 } }
{ "_id" : 2, "status" : { "err" : "blahblah", "y" : 5 } }
> db.foo.update( { }, { $unset : { "status.err" : 1 } }, false, true )
> db.foo.find()
{ "_id" : 1, "status" : { "y" : 1 } }
{ "_id" : 2, "status" : { "y" : 5 } }

#2: Using up RAM

if I do the $unset command on a column in a collection that is very large, mongodb uses up all of the ram on the server (as if it's trying to store the entire collection in memory)

That's exactly what MongoDB is trying to do. MongoDB uses memory-mapped files. MongoDB will pull all of the data into RAM and let the operating system manage the virtual memory concerns.

So when you do a query with no indexes, you're basically asking MongoDB to walk through every item in the collection. It's basically a giant for loop operating on all of your data, so this is going to require loading everything from disk.

Up to now, this is all normal.

... then the server crashes

This is not normal. I have run this type of update command on hundreds of millions of documents without crashing the server. Are you able to provide any more detail on this problem? Do you have log files?

If so, I would suggest taking your bugs to the Google Groups, so they can help identify the source of the crash. http://groups.google.com/group/mongodb-user