Activity log for bug #1880610

Date Who What changed Old value New value Message
2020-05-25 22:23:16 Andrea Ieri bug added bug
2020-05-25 22:23:38 Andrea Ieri bug added subscriber Canonical IS BootStack
2020-05-25 22:23:45 Andrea Ieri bug added subscriber The Canonical Sysadmins
2020-05-25 22:39:40 Andrea Ieri affects charm-designate charm-hacluster
2020-05-25 22:40:09 Andrea Ieri description The current haproxy config adds health checks for all backends. These are however the default TCP checks[0], which succeed simply if a connection to the specified port is possible, and don't go any further in trying to assess if the backend service is actually healthy. In some partial failure scenarios - e.g. extreme load or memory pressure - TCP checks may succeed where more thorough ones would not, causing API traffic to be routed to backends that aren't actually able to process it. HAproxy offers the possibility of reconfiguring the layer 4 checks into layer 7 ones by using the httpchk option[1]. This is still fairly basic, but configuring the http-check option[4] makes the check a bit more accurate, while service-specific checks are available for some backends such as mysql[2] and postgres[3]. Finally, external scripts[5] can be invoked for a fully customized check routine. In order to improve the control plane resilience we should gradually switch to more advanced backend checks. Plain TCP checks should remain the default, but principal charms should provide more accurate check specifications via their relation to hacluster. Depending on the service, a properly configured httpchk may suffice, while in some cases a fully custom script may be more appropriate. [0] https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/ [1] https://www.haproxy.com/documentation/hapee/latest/onepage/#4-option%20httpchk [2] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-option%20mysql-check [3] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-option%20pgsql-check [4] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-http-check%20expect [5] https://www.haproxy.com/documentation/hapee/latest/onepage/#external-check%20command The current haproxy config adds health checks for all backends. These are however the default TCP checks[0], which succeed simply if a connection to the specified port is possible, and don't go any further in trying to assess if the backend service is actually healthy. In some partial failure scenarios - e.g. extreme load or memory pressure - TCP checks may succeed where more thorough ones would not, causing API traffic to be routed to backends that aren't actually able to process it. HAproxy offers the possibility of reconfiguring the layer 4 checks into layer 7 ones by using the httpchk option[1]. This is still fairly basic, but configuring the http-check option[4] makes the check a bit more accurate, while service-specific checks are available for some backends such as mysql[2] and postgres[3]. Finally, external scripts[5] can be invoked for a fully customized check routine. In order to improve the control plane resilience we should gradually switch to more advanced backend checks. Plain TCP checks should remain the default, but principal charms should provide more accurate check specifications via their relation to hacluster. Depending on the service, a properly configured httpchk may suffice, while in some cases a fully custom script may be more appropriate. This is not in contrast with giving more control to pacemaker via openstack resource agents, as described in bug 1880611. [0] https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/ [1] https://www.haproxy.com/documentation/hapee/latest/onepage/#4-option%20httpchk [2] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-option%20mysql-check [3] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-option%20pgsql-check [4] https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-http-check%20expect [5] https://www.haproxy.com/documentation/hapee/latest/onepage/#external-check%20command
2020-09-02 00:05:45 Jose Guedez charm-hacluster: assignee Jose Guedez (jfguedez)
2020-09-08 00:32:51 Jose Guedez bug task added charm-glance
2020-09-08 00:41:08 Jose Guedez charm-glance: assignee Jose Guedez (jfguedez)
2020-09-08 00:41:36 Jose Guedez charm-glance: status New In Progress
2020-09-08 04:47:44 Jose Guedez charm-hacluster: status New Invalid
2020-09-09 02:03:53 Jose Guedez bug added subscriber Canonical Field Medium
2022-09-27 01:28:50 Andrea Ieri charm-hacluster: assignee Jose Guedez (jfguedez)
2022-09-27 01:28:52 Andrea Ieri charm-glance: assignee Jose Guedez (jfguedez)
2022-09-27 01:30:05 Andrea Ieri bug task added charm-gnocchi
2022-09-27 21:10:55 Andrea Ieri tags bseng-436
2022-10-03 14:49:49 Junien Fridrick bug added subscriber Junien Fridrick
2022-10-24 20:52:10 Gabriel Cocenza charm-glance: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-01-18 19:04:12 OpenStack Infra charm-glance: status In Progress Fix Committed
2023-02-15 18:13:31 Gabriel Cocenza bug task added charm-ceilometer
2023-02-15 18:14:53 Gabriel Cocenza bug task added charm-nova-cloud-controller
2023-02-15 18:15:55 Gabriel Cocenza bug task added charm-keystone
2023-02-15 18:16:29 Gabriel Cocenza bug task added charm-neutron-api
2023-02-15 18:17:14 Gabriel Cocenza bug task added charm-openstack-dashboard
2023-02-15 18:17:45 Gabriel Cocenza bug task added charm-cinder
2023-02-15 18:18:19 Gabriel Cocenza bug task added charm-heat
2023-02-15 18:18:47 Gabriel Cocenza bug task added charm-swift-proxy
2023-02-15 18:19:22 Gabriel Cocenza bug task added charm-designate
2023-02-15 18:19:44 Gabriel Cocenza bug task added charm-neutron-gateway
2023-02-15 18:20:03 Gabriel Cocenza bug task added charm-percona-cluster
2023-02-15 18:20:53 Gabriel Cocenza bug task added charm-ironic-api
2023-02-15 20:56:10 OpenStack Infra charm-gnocchi: status New In Progress
2023-02-15 21:00:36 Gabriel Cocenza charm-gnocchi: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-15 22:10:48 OpenStack Infra charm-keystone: status New In Progress
2023-02-15 22:13:51 Gabriel Cocenza charm-keystone: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-16 04:13:01 Nobuto Murata bug added subscriber Nobuto Murata
2023-02-16 15:21:38 Gabriel Cocenza charm-ceilometer: status New Invalid
2023-02-16 21:51:28 Gabriel Cocenza charm-openstack-dashboard: status New Invalid
2023-02-17 00:17:46 OpenStack Infra charm-cinder: status New In Progress
2023-02-17 00:37:20 Gabriel Cocenza charm-cinder: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-17 13:24:03 OpenStack Infra charm-designate: status New In Progress
2023-02-17 13:24:31 Gabriel Cocenza charm-designate: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-17 15:05:33 OpenStack Infra charm-heat: status New In Progress
2023-02-17 20:46:35 Gabriel Cocenza charm-neutron-api: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-17 20:53:52 OpenStack Infra charm-neutron-api: status New In Progress
2023-02-23 19:23:10 Gabriel Cocenza bug task deleted charm-neutron-gateway
2023-02-23 20:04:27 Gabriel Cocenza bug task deleted charm-percona-cluster
2023-02-23 22:40:35 OpenStack Infra charm-ironic-api: status New In Progress
2023-02-24 00:07:53 OpenStack Infra charm-swift-proxy: status New In Progress
2023-02-24 02:35:56 Gabriel Cocenza bug task deleted charm-ceilometer
2023-02-24 02:36:03 Gabriel Cocenza bug task deleted charm-openstack-dashboard
2023-02-24 02:36:17 Gabriel Cocenza charm-heat: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-24 02:36:27 Gabriel Cocenza charm-ironic-api: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-25 15:24:14 OpenStack Infra charm-nova-cloud-controller: status New In Progress
2023-02-27 02:06:31 Gabriel Cocenza charm-nova-cloud-controller: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-27 02:06:41 Gabriel Cocenza charm-swift-proxy: assignee Gabriel Angelo Sgarbi Cocenza (gabrielcocenza)
2023-02-28 09:26:07 OpenStack Infra charm-ironic-api: status In Progress Fix Committed
2023-02-28 09:26:37 OpenStack Infra charm-keystone: status In Progress Fix Committed
2023-02-28 09:27:31 OpenStack Infra charm-designate: status In Progress Fix Committed
2023-02-28 09:37:07 OpenStack Infra charm-swift-proxy: status In Progress Fix Committed
2023-02-28 09:40:04 OpenStack Infra charm-heat: status In Progress Fix Committed
2023-03-13 07:30:51 OpenStack Infra charm-gnocchi: status In Progress Fix Committed
2023-03-13 15:45:05 Felipe Reyes bug task added charm-guide
2023-03-29 08:03:09 OpenStack Infra charm-neutron-api: status In Progress Fix Committed
2023-03-29 15:11:27 OpenStack Infra charm-nova-cloud-controller: status In Progress Fix Committed
2023-03-29 15:14:58 OpenStack Infra charm-cinder: status In Progress Fix Committed
2023-06-16 10:30:18 Alex Kavanagh charm-guide: importance Undecided High
2023-06-16 10:30:18 Alex Kavanagh charm-guide: status New Triaged