Comment 15 for bug 1492742

> We've been thinking about this and discussing possible solutions. One of
> the leading ones right now is that we should create any pools with an
> absolutely minimal number of PGs and, maybe in the update-status hook,
> check if Ceph is claiming too FEW placement groups, and bump them then.

I'm -1 on this idea as I don't think an update-status hook should be
actively changing the pools' pgs as that could cause data shuffle in
the cluster not initiated by a user request. Its difficult enough to
track what Ceph is doing with your data without the charms seemingly
randomly changing pool pg_nums.

It might be better to add an option to the ceph-mon which tells
indicates the minimum number of OSDs to consider when doing pg_num
calculations. This allows the CephBroker to make a more intelligent
choice for the pg_num when combined with a usage percentage for the
pool. This allows for the single-shot deployments to calculate based
on the number of OSDs to expect while still providing an appropriate
number of placement groups for a pool.

I'll pitch up a more concrete proposal to move this forward unless
there are strong objections to my approach.

>
>
> --
> You received this bug notification because you are a member of OpenStack
> Charmers, which is subscribed to glance in Juju Charms Collection.
> https://bugs.launchpad.net/bugs/1492742
>
> Title:
> charm needs a pg_num cap when deploying to avoid creating too many pgs
>
> Status in Charm Helpers:
> Invalid
> Status in cinder-ceph package in Juju Charms Collection:
> Triaged
> Status in glance package in Juju Charms Collection:
> Triaged
> Status in nova-compute package in Juju Charms Collection:
> Triaged
>
> Bug description:
> As PG cannot be decreased, charms should create fewer PGs to be fail-safe.
> http://ceph.com/docs/master/rados/operations/placement-groups/#set-the-number-of-placement-groups
>
> $ sudo ceph status
> cluster 026593f1-93c3-4144-bb72-69c75c7eb3fd
> health HEALTH_WARN
> too many PGs per OSD (337 > max 300)
> monmap e2: 3 mons at {angha=10.230.123.3:6789/0,axex=10.230.123.5:6789/0,duwende=10.230.123.4:6789/0}
> election epoch 8, quorum 0,1,2 angha,duwende,axex
> osdmap e26: 5 osds: 5 up, 5 in
> pgmap v835: 562 pgs, 4 pools, 12270 MB data, 1568 objects
> 36779 MB used, 2286 GB / 2322 GB avail
> 562 active+clean
> client io 26155 kB/s wr, 6 op/s
>
> $ sudo ceph osd dump | grep pg_num
> pool 0 'rbd' replicated size 3 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 flags hashpspool stripe_width 0
> pool 1 'nova' replicated size 3 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 166 pgp_num 166 last_change 13 flags hashpspool stripe_width 0
> pool 2 'glance' replicated size 3 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 166 pgp_num 166 last_change 25 flags hashpspool stripe_width 0
> pool 3 'cinder-ceph' replicated size 3 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 166 pgp_num 166 last_change 20 flags hashpspool stripe_width 0
>
> $ sudo ceph osd pool set glance pg_num 64
> Error EEXIST: specified pg_num 64 <= current 166
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-helpers/+bug/1492742/+subscriptions