Canonical Juju

MongoDB used by Juju has replication bug

Bug #1670891 reported by Menno Finlay-Smits
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
Critical
Michael Hudson-Doyle
Canonical Juju 2.2-beta1
juju-mongodb3.2 (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

IS are seeing frequent MongoDB failures like this:

Mar 7 21:26:41 juju-4da59b22-9710-4e69-840a-be49ee864a97-machine-2 mongod.37017[5180]: [NetworkInterfaceASIO-BGSync-0] Successfully connected to 10.25.2.111:37017
<pjdc> Mar 7 21:26:41 juju-4da59b22-9710-4e69-840a-be49ee864a97-machine-2 mongod.37017[5180]: [rsBackgroundSync] Starting rollback due to OplogStartMissing: our last op time fetched: (term: 279, timestamp: Mar 7 20:20:47:2b). source's GTE: (term: 280, timestamp: Mar 7 20:21:19:1) hashes: (6725952777685747593/-748731992779411949)
Mar 7 21:26:41 juju-4da59b22-9710-4e69-840a-be49ee864a97-machine-2 mongod.37017[5180]: [rsBackgroundSync] Fatal assertion 18750 UnrecoverableRollbackError: need to rollback, but in inconsistent state. minvalid: (term: 280, timestamp: Mar 7 20:21:22:4) > our last optime: (term: 279, timestamp: Mar 7 20:20:47:2b)
Mar 7 21:26:41 juju-4da59b22-9710-4e69-840a-be49ee864a97-machine-2 mongod.37017[5180]: [rsBackgroundSync] #012#012***aborting after fassert() failure

Some research reveals that this is the likely fix:
https://jira.mongodb.org/browse/SERVER-7200

The fix is in MongoDB 3.2.11 but the juju-mongodb package is based around 3.2.9. The current latest 3.2 is 3.2.12 so I suggested we upgrade the package to that.

See original description
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

There are packages for testing in https://launchpad.net/~mwhudson/+archive/ubuntu/devirt/+packages now. Unfortunately they fail to build on s390x (probably for silly reasons) and arm64 (for mysterious reasons) so that needs to be looked at before we can upload them to the archive

Menno Finlay-Smits (menno.smits)
description: updated
Anastasia (anastasia-macmood)
Changed in juju:
status: New → Triaged
milestone: none → 2.2.0
Anastasia (anastasia-macmood)
Changed in juju:
assignee: nobody → Michael Hudson-Doyle (mwhudson)
milestone: 2.2-beta1 → 2.2-alpha2
Michael Hudson-Doyle (mwhudson)
Changed in juju:
status: Triaged → In Progress
Michael Hudson-Doyle (mwhudson)
Changed in juju-mongodb3.2 (Ubuntu):
status: New → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package juju-mongodb3.2 - 3.2.12-0ubuntu1

---------------
juju-mongodb3.2 (3.2.12-0ubuntu1) zesty; urgency=medium

  * New upstream release. (LP: #1670891)
  * Do not use CRC32 instructions on s390x.
  * Correct architecture detection on arm64.

 -- Michael Hudson-Doyle <email address hidden> Wed, 08 Mar 2017 13:50:17 +1300

Changed in juju-mongodb3.2 (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Anastasia (anastasia-macmood) wrote :

SRU bug for Xenial and Yakkety: https://bugs.launchpad.net/ubuntu/+source/juju-mongodb3.2/+bug/1673652

Michael Hudson-Doyle (mwhudson)
no longer affects: juju-mongodb3.2 (Ubuntu Xenial)
no longer affects: juju-mongodb3.2 (Ubuntu Yakkety)
Revision history for this message
Anastasia (anastasia-macmood) wrote :

Since the package is now in Zesty, I am marking this as Fix Committed. SRU bug linked in comment # 3 will be used to track backports.

Changed in juju:
status: In Progress → Fix Committed
Canonical Juju QA Bot (juju-qa-bot)
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.