mongodb driver does not create indices on project_id as intended

Bug #1291666 reported by Eoghan Glynn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
High
Eoghan Glynn

Bug Description

The mongodb driver clearly intends to create indices for both user_id and project_id on the meter and resource collections:

   https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_mongodb.py#L420

However since it uses the same per-collection index name on each iteration of the index-creating for loop, the project_id indices are never created:

> db.meter.getIndexes()
[
 {
  "v" : 1,
  "key" : {
   "_id" : 1
  },
  "ns" : "ceilometer.meter",
  "name" : "_id_"
 },
 {
  "v" : 1,
  "key" : {
   "resource_id" : 1,
   "user_id" : 1,
   "counter_name" : 1,
   "timestamp" : 1,
   "source" : 1
  },
  "ns" : "ceilometer.meter",
  "name" : "meter_idx"
 },
 {
  "v" : 1,
  "key" : {
   "timestamp" : -1
  },
  "ns" : "ceilometer.meter",
  "name" : "timestamp_idx"
 }
]
> db.resource.getIndexes()
[
 {
  "v" : 1,
  "key" : {
   "_id" : 1
  },
  "ns" : "ceilometer.resource",
  "name" : "_id_"
 },
 {
  "v" : 1,
  "key" : {
   "user_id" : 1,
   "source" : 1
  },
  "ns" : "ceilometer.resource",
  "name" : "resource_idx"
 },
 {
  "v" : 1,
  "key" : {
   "last_sample_timestamp" : -1
  },
  "ns" : "ceilometer.resource",
  "sparse" : true,
  "name" : "last_sample_timestamp_idx"
 }
]

Instead, a distinguished index name should be used for the project_id index in each case (e.g. meter_project_id_idx) to avoid the name clash.

Also this new index should be created with background=True to avoid slow start-up on upgrade with large existing collections.

Eoghan Glynn (eglynn)
Changed in ceilometer:
importance: Undecided → High
assignee: nobody → Eoghan Glynn (eglynn)
milestone: none → icehouse-rc1
status: New → In Progress
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/80118

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/80118
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=9ffba6fb700e6830c62a5ce68e4c2753128f75bf
Submitter: Jenkins
Branch: master

commit 9ffba6fb700e6830c62a5ce68e4c2753128f75bf
Author: Eoghan Glynn <email address hidden>
Date: Wed Mar 12 23:36:12 2014 +0000

    Ensure intended indices on project_id are created for mongo

    Fixes bug 1291666

    Previously, despite the clear intent of the code, the mongodb
    driver failed to create indices on project_id for the meter
    and resource collections, due to a index name clash.

    Now, we ensure a different index name is used for the user_id
    and project_id indices on each collection.

    The previously missing index is created in the background to
    avoid blocking other DB operations when upgrading against an
    existing mongo instance with large collections.

    Change-Id: Ia3c6765445f16ab9beafbd7fd9265a045814003f

Changed in ceilometer:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.