Can rollback still occur on a MongoDB replica set

Question

I have been reading the docs and from my understanding I could see a scenario whereby a rollback could still occur:

• Write goes to primary which confirms that the journal has been written to disk
• Majority of the secondaries confirm the write but do not write to disk
• Power fails on entire cluster
• Primary for some reason does not start back up when power is restored
• A secondary takes the primary role
• The original primary finally starts, rejoins the set as a secondary and rolls back

Is this scenario plausible?

Answer 1

This could be a plausible case for rollback yes, if the power fails between the other members getting the command and writing to disk.

In this case, as you point out, the primary could not start back up and so would, once back up, contain operations that the rest of the set could not validate causing a rollback.

It is also good to note, as a curve ball that if the primary were not to go down then it would return a successful write and the application would be none the wiser that the set has gone down and their {w: majority} wasn't written to disk. This is, of course, an edge case.

Answer 2

based on the docs, my understanding is that if you set j=1 then w > 1 doesn't matter. your app will have the write ack'd only once (and as soon as) the primary has committed the write to its own journal. writes to replicas will happen but don't factor into your write concern.

in light of this, the senario of "can the primary commit to journal, ack the write, and have the cluster go down before the secondaries commit to their journal and then roll back the primary when a secondary comes back up as primary" is more likely (but still of very low likelihood) than the original question implies.

from the docs:

Requiring journaled write concern in a replica set only requires a journal commit of the write
operation to the primary of the set regardless of the level of replica acknowledged write concern.

http://docs.mongodb.org/manual/core/write-concern/

Answer 3

Don't think it will happen in MongoDB 3.2+, as in here, you see:

Changed in version 3.2: With j: true, MongoDB returns only after the requested number of members, including the primary, have written to the journal. Previously j: true write concern in a replica set only requires the primary to write to the journal, regardless of the w: write concern.