Deploying ceph-osd without ceph-cluster-network config set generated "Falling back to public interface" error message in Ceph

Bug #1966515 reported by Sandor Zeestraten
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceph OSD Charm
Triaged
Medium
Unassigned

Bug Description

# Problem

Deploying ceph-osd with bindings for `public` and `cluster` defined results in the following error message for OSDs (for example /var/log/ceph/ceph-osd.0.log)

2022-03-25T18:00:05.060+0000 7f6eecde02c0 -1 Falling back to public interface

The error stems from:
https://github.com/ceph/ceph/blob/ee28fb57e47e9f88813e24bbf4c14496ca299d31/src/common/pick_address.cc#L411-L416

As far as I can tell when testing, unless the `cluster network` is set in /etc/ceph/ceph.conf (via `ceph-cluster-network` config flag), Ceph will fall back to just public interface.

So if you're only using the network space bindings as recommended in the ceph-osd charm documentation, it will not generate the correct ceph configuration file.

# Bundle
relations:
- - ceph-osd:mon
  - ceph-mon:osd
series: focal
applications:
  ceph-mon:
    charm: cs:ceph-mon-73
    num_units: 3
    constraints: tags=ceph-mon
    bindings:
      "": site2-oam
      public: site2-ceph-public
    options:
      monitor-count: 3
      source: cloud:focal-xena
  ceph-osd:
    charm: cs:ceph-osd-513
    num_units: 3
    constraints: tags=ceph-osd
    options:
      osd-devices: "/dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm"
      source: cloud:focal-xena
    bindings:
      "": site2-oam
      public: site2-ceph-public
      cluster: site2-ceph-cluster

# juju spaces
Name Space ID Subnets
alpha 0
site2-os-public 1 10.42.208.0/23
site2-neutron-external 2 10.42.210.0/23
site2-os-data 3 172.17.100.0/23
site2-os-internal 4 172.17.102.0/23
site2-ceph-public 5 172.17.104.0/23
site2-ceph-cluster 6 172.17.106.0/23
site2-oam 7 10.42.200.0/23
site2-provision 8 10.42.196.0/23

# versions
juju 2.9.27
ceph-mon rev 73
ceph-osd rev 513
ceph 16.2.6

Revision history for this message
Luciano Lo Giudice (lmlogiudice) wrote :

Looking at the code, it appears that the only way for the template to render the `cluster_network` parameter that Ceph requires is by supplying a config value for `ceph-cluster-network`. What I think would be needed is to look at Juju spaces and append the cluster networks provided via bindings as well.

Changed in charm-ceph-osd:
importance: Undecided → Medium
Changed in charm-ceph-osd:
status: New → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.