The address for the public interface is not accessible, move its address to the ovs switch it is now connected to.

Wouldn't it be expected to move the address to the bridge? Why does that prevent connectivity to the address?

I've copy pasted Ian's explanation of what he did here. All I've done is made the config optional.
-----
That 'fix the network interface numbering' stuff requires we install this. Basically the init script ordering is:

networking - apply /etc/network/interfaces

openvswitch - start OVS daemons. OVS switches are created per the persistent OVS database. Interfaces are put into those switches per the OVS database.

... at this point in our setup, bond0 goes into br-internal and the address on bond0 becomes unreachable.

openvswitch-fix-interfaces - our custom script per above. We do the same network address moving that we did when we initially put bond0 into br-internal - that is, remove the address from bond0, put it onto br-internal, and bring br-internal up. The address is on the equivalent interface and is now reachable as br-internal actually terminates packets.

PR is here, probably easier to just look at the code.

https://github.com/CiscoSystems/folsom-manifests/pull/65

Folsom fix rebased and merged

If that FOlsom fix is my post-OVS startup script from a couple of months
ago, I wasn't aware it had gone upstream, but I've since learned that it's
flawed and doesn't work reliably.

If that's the case are you happy to leave this out?

I think we have to if that's the only fix we have, to be honest.

OK, well I'm just going to leave this bug as wishlist, and if it becomes urgent then you can bump it or point me at a better fix.

Actually - I've bumped the priority. It's a specific customer issue and requires resolving. The issue is not with the problem, but that the proposed solution doesn't work.

Right, since we're back at the analysis phase, let me lead you through the problem in steps:

- /etc/network/interfaces is applied. bonds are made, interfaces are alive, one of them has the address we care about if you put it in interfaces on the component interface
- openvswitch starts via the startup scripts.
- OVS creates switches and adds ports. In creating switches, the switch interfaces turn up. Unfortunately it doesn't bother to run ifup for those interfaces, so any /etc/network/interfaces config for them is ignored. If you instead elected to put the config on a component interface, then the DNS / gateway config holds true, but the IP address is now on an interface in a bridge, that interface (being in a bridge) never receives packets to the kernel stack - they're all fed to the bridge interface instead - and you can't, therefore, contact any addresses it may have.

The original fix was to run a script after OVS startup to transfer all addressing information from the component interface to the switch interface. Unfortunately, because OVS is asynchronous, the switch interface often hasn't been created at this point and the transfer of information fails.

The right fix would be, I think, either to find a hook out of the OVS management software to call to a script to transfer the interface, or, better, to run ifup on the interface as you would for any other interface so that /etc/network/interfaces information would be taken up. This still has a shortcoming, in that, during the initial puppet run, you need an addressable interface; if that interface is put into a switch then its address becomes inaccessible; problems as above, plus you've cut yourself off from the puppet server. This could be fixed with some very specific puppet to the circumstance in question, but it's hard to see how you could write puppet that makes sense in the context of puppet. Perhaps it would explicitly unconfigure the component interface then configure the bridge per the interfaces file? Whatever, a simple address transfer doesn't really make any sense unless it happens in conjunction with an interfaces file change.

Fixes need to address all the problems listed here. And work on Redhat, which I've not looked at.