swift commands fail due to a "Connection refused" error in a full ha setup

Full Ha setup was not able to execute any swift commands to the following error

HTTPConnectionPool(host='172.29.74.140', port=8080): Max retries exceeded with url: /v1/AUTH_c86100006917462999f3158b65e6b623?format=json (Caused by <class 'socket.error'>: [Errno 111] Connection refused)

On investigation looks like the /etc/haproxy/haproxy.cfg file , did not have this entry on the load-balancers
listen swift_proxy_cluster
  bind <Virtual IP>:8080
  balance source
  option tcplog
  option tcpka
  server controller1 10.0.0.1:8080 check inter 2000 rise 2 fall 5
  server controller2 10.0.0.2:8080 check inter 2000 rise 2 fall 5

I manually added this entry and restarted haproxy, and swift commands worked.