Activity log for bug #2090806

Date Who What changed Old value New value Message
2024-12-02 03:14:50 Matthew Ruffell bug added bug
2024-12-02 03:15:02 Matthew Ruffell nominated for series Ubuntu Plucky
2024-12-02 03:15:02 Matthew Ruffell bug task added openldap (Ubuntu Plucky)
2024-12-02 03:15:02 Matthew Ruffell nominated for series Ubuntu Focal
2024-12-02 03:15:02 Matthew Ruffell bug task added openldap (Ubuntu Focal)
2024-12-02 03:15:02 Matthew Ruffell nominated for series Ubuntu Oracular
2024-12-02 03:15:02 Matthew Ruffell bug task added openldap (Ubuntu Oracular)
2024-12-02 03:15:02 Matthew Ruffell nominated for series Ubuntu Noble
2024-12-02 03:15:02 Matthew Ruffell bug task added openldap (Ubuntu Noble)
2024-12-02 03:15:02 Matthew Ruffell nominated for series Ubuntu Jammy
2024-12-02 03:15:02 Matthew Ruffell bug task added openldap (Ubuntu Jammy)
2024-12-02 03:15:12 Matthew Ruffell openldap (Ubuntu Focal): status New In Progress
2024-12-02 03:15:14 Matthew Ruffell openldap (Ubuntu Jammy): status New In Progress
2024-12-02 03:15:16 Matthew Ruffell openldap (Ubuntu Noble): status New In Progress
2024-12-02 03:15:18 Matthew Ruffell openldap (Ubuntu Oracular): status New In Progress
2024-12-02 03:15:20 Matthew Ruffell openldap (Ubuntu Plucky): status New In Progress
2024-12-02 03:15:23 Matthew Ruffell openldap (Ubuntu Focal): importance Undecided Medium
2024-12-02 03:15:24 Matthew Ruffell openldap (Ubuntu Jammy): importance Undecided Medium
2024-12-02 03:15:26 Matthew Ruffell openldap (Ubuntu Noble): importance Undecided Medium
2024-12-02 03:15:27 Matthew Ruffell openldap (Ubuntu Oracular): importance Undecided Medium
2024-12-02 03:15:29 Matthew Ruffell openldap (Ubuntu Plucky): importance Undecided Medium
2024-12-02 03:15:31 Matthew Ruffell openldap (Ubuntu Focal): assignee Matthew Ruffell (mruffell)
2024-12-02 03:15:33 Matthew Ruffell openldap (Ubuntu Jammy): assignee Matthew Ruffell (mruffell)
2024-12-02 03:15:34 Matthew Ruffell openldap (Ubuntu Noble): assignee Matthew Ruffell (mruffell)
2024-12-02 03:15:37 Matthew Ruffell openldap (Ubuntu Oracular): assignee Matthew Ruffell (mruffell)
2024-12-02 03:16:07 Matthew Ruffell tags sts
2024-12-09 03:11:09 Matthew Ruffell attachment added Debdiff for openldap on oracular https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/2090806/+attachment/5843020/+files/lp2090806_oracular.debdiff
2024-12-09 03:11:47 Matthew Ruffell attachment added Debdiff for openldap on noble https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/2090806/+attachment/5843021/+files/lp2090806_noble.debdiff
2024-12-09 03:12:11 Matthew Ruffell attachment added Debdiff for openldap on jammy https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/2090806/+attachment/5843022/+files/lp2090806_jammy.debdiff
2024-12-09 03:12:43 Matthew Ruffell attachment added Debdiff for openldap on focal https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/2090806/+attachment/5843023/+files/lp2090806_focal.debdiff
2024-12-09 04:21:01 Ubuntu Foundations Team Bug Bot tags sts patch sts
2025-01-13 23:23:33 Sergio Durigan Junior openldap (Ubuntu Plucky): status In Progress Fix Committed
2025-01-13 23:23:35 Sergio Durigan Junior openldap (Ubuntu Plucky): assignee Sergio Durigan Junior (sergiodj)
2025-01-22 22:14:30 Matthew Ruffell openldap (Ubuntu Plucky): status Fix Committed Fix Released
2025-01-23 04:55:07 Matthew Ruffell attachment removed Debdiff for openldap on focal https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/2090806/+attachment/5843023/+files/lp2090806_focal.debdiff
2025-01-23 04:55:39 Matthew Ruffell openldap (Ubuntu Focal): status In Progress Won't Fix
2025-01-24 18:59:07 Andreas Hasenack description [Impact] If you connect to a slapd server using TLS and for some reason, the connection hangs or fails to completely establish, TIMEOUT and NETWORK_TIMEOUT values do not function as they are intended. They get ignored, and the connecting process e.g. ldapsearch just sits there at 100% CPU waiting forever. If slapd is set up in High Availability with servers to failover to, the failover will never occur, due to the connecting processes sitting there at 100% CPU waiting for the connection that will never succeed, leading to outages. You can simulate this easily with: $ ldapsearch -x -H ldaps://example.com $ sudo kill -STOP <slapd pid> $ ldapsearch -x -H ldaps://example.com and the last call will hang forever. There doesn't seem to be any workaround for this, apart from not using SSL. [Testcase] You can get away with just the one VM for this. Set up the ldap server (from server docs): https://ubuntu.com/server/docs/install-and-configure-ldap $ sudo apt install slapd ldap-utils $ sudo dpkg-reconfigure slapd Set DNS to example.com $ ldapsearch -x -LLL -H ldap:/// -b dc=example,dc=com dn dn: dc=example,dc=com $ vim add_content.ldif dn: ou=People,dc=example,dc=com objectClass: organizationalUnit ou: People dn: ou=Groups,dc=example,dc=com objectClass: organizationalUnit ou: Groups dn: cn=miners,ou=Groups,dc=example,dc=com objectClass: posixGroup cn: miners gidNumber: 5000 dn: uid=john,ou=People,dc=example,dc=com objectClass: inetOrgPerson objectClass: posixAccount objectClass: shadowAccount uid: john sn: Doe givenName: John cn: John Doe displayName: John Doe uidNumber: 10000 gidNumber: 5000 userPassword: {CRYPT}x gecos: John Doe loginShell: /bin/bash homeDirectory: /home/john $ ldapadd -x -D cn=admin,dc=example,dc=com -W -f add_content.ldif $ ldapsearch -x -LLL -b dc=example,dc=com '(uid=john)' cn gidNumber dn: uid=john,ou=People,dc=example,dc=com cn: John Doe gidNumber: 5000 $ ldappasswd -x -D cn=admin,dc=example,dc=com -W -S uid=john,ou=people,dc=example,dc=com New password: Re-enter new password: Enter LDAP Password: Configure the server for SSL (from server docs): https://ubuntu.com/server/docs/ldap-and-transport-layer-security-tls $ sudo apt install gnutls-bin ssl-cert $ sudo certtool --generate-privkey --bits 4096 --outfile /etc/ssl/private/mycakey.pem $ sudo vim /etc/ssl/ca.info cn = Example Company ca cert_signing_key expiration_days = 3650 $ sudo certtool --generate-self-signed \ --load-privkey /etc/ssl/private/mycakey.pem \ --template /etc/ssl/ca.info \ --outfile /usr/local/share/ca-certificates/mycacert.crt $ sudo update-ca-certificates $ sudo certtool --generate-privkey \ --bits 2048 \ --outfile /etc/ldap/ldap01_slapd_key.pem $ sudo vim /etc/ssl/ldap01.info organization = Example Company cn = example.com tls_www_server encryption_key signing_key expiration_days = 365 $ sudo certtool --generate-certificate \ --load-privkey /etc/ldap/ldap01_slapd_key.pem \ --load-ca-certificate /etc/ssl/certs/mycacert.pem \ --load-ca-privkey /etc/ssl/private/mycakey.pem \ --template /etc/ssl/ldap01.info \ --outfile /etc/ldap/ldap01_slapd_cert.pem $ sudo chgrp openldap /etc/ldap/ldap01_slapd_key.pem $ sudo chmod 0640 /etc/ldap/ldap01_slapd_key.pem $ sudo vim certinfo.ldif dn: cn=config add: olcTLSCACertificateFile olcTLSCACertificateFile: /etc/ssl/certs/mycacert.pem - add: olcTLSCertificateFile olcTLSCertificateFile: /etc/ldap/ldap01_slapd_cert.pem - add: olcTLSCertificateKeyFile olcTLSCertificateKeyFile: /etc/ldap/ldap01_slapd_key.pem $ sudo ldapmodify -Y EXTERNAL -H ldapi:/// -f certinfo.ldif $ sudo vim /etc/default/slapd Change LAPD_SERVICES="ldap:/// ldapi:///" to SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///" Next add TIMEOUT and NETWORK_TIMEOUT to the config file. $ sudo vim /etc/ldap/ldap.conf NETWORK_TIMEOUT 30 TIMEOUT 30 $ sudo systemctl restart slapd Reproduce the problem: $ ldapsearch -x -H ldaps://example.com # extended LDIF # # LDAPv3 # base <> (default) with scope subtree # filter: (objectclass=*) # requesting: ALL # # search result search: 2 result: 32 No such object # numResponses: 1 $ ps aux | grep slapd openldap 4097 0.0 0.1 1224136 7876 ? Ssl 02:22 0:00 /usr/sbin/slapd -h ldap:/// ldapi:/// ldaps:/// -g openldap -u openldap -F /etc/ldap/slapd.d $ sudo kill -STOP 4097 $ ldapsearch -x -H ldaps://example.com Check top: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4237 ubuntu 20 0 20724 10956 6656 R 100.0 0.3 0:35.70 ldapsearch wait for 30 seconds, see nothing changes. If you install packages from the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf401397-test You should see the timeout work as expected, and the ldapsearch process will give up after 30 seconds. [Where problems could occur] We are changing how connections are tracked within openldap to better accurately say what state we are in, and for the timeouts to work when the connection doesn't make any progress. This will cause TIMEOUT and NETWORK_TIMEOUT to start working as intended, which might surprise some sysadmins who configured it, and likely forgot about it when they never had any timeout issues. If a regression were to occur, it would affect initial connections to openldap servers, or even long running commands that might exceed the TIMEOUT on queries over large ldap databases. [Other info] Upstream issues: https://bugs.openldap.org/show_bug.cgi?id=8047 https://bugs.openldap.org/show_bug.cgi?id=10141 This was fixed in 2.6.9 upstream by: commit d143f7a2dc82fb66e7741b93a1ae9e874ce2ac46 Author: Ondřej Kuzník <ondra@mistotebe.net> Date: Mon, 21 Oct 2024 11:50:11 +0100 Subject: ITS#8047 Fix TLS connection timeout handling Link: https://git.openldap.org/openldap/openldap/-/commit/d143f7a2dc82fb66e7741b93a1ae9e874ce2ac46 This fix is required in all releases. [Impact] If you connect to a slapd server using TLS and for some reason, the connection hangs or fails to completely establish, TIMEOUT and NETWORK_TIMEOUT values do not function as they are intended. They get ignored, and the connecting process e.g. ldapsearch just sits there at 100% CPU waiting forever. If slapd is set up in High Availability with servers to failover to, the failover will never occur, due to the connecting processes sitting there at 100% CPU waiting for the connection that will never succeed, leading to outages. You can simulate this easily with: $ ldapsearch -x -H ldaps://example.com $ sudo kill -STOP <slapd pid> $ ldapsearch -x -H ldaps://example.com and the last call will hang forever. There doesn't seem to be any workaround for this, apart from not using SSL. [Testcase] You can get away with just the one VM for this. Set up the ldap server (from server docs): https://ubuntu.com/server/docs/install-and-configure-ldap $ sudo apt install slapd ldap-utils $ sudo dpkg-reconfigure slapd Set DNS to example.com $ ldapsearch -x -LLL -H ldap:/// -b dc=example,dc=com dn dn: dc=example,dc=com $ vim add_content.ldif dn: ou=People,dc=example,dc=com objectClass: organizationalUnit ou: People dn: ou=Groups,dc=example,dc=com objectClass: organizationalUnit ou: Groups dn: cn=miners,ou=Groups,dc=example,dc=com objectClass: posixGroup cn: miners gidNumber: 5000 dn: uid=john,ou=People,dc=example,dc=com objectClass: inetOrgPerson objectClass: posixAccount objectClass: shadowAccount uid: john sn: Doe givenName: John cn: John Doe displayName: John Doe uidNumber: 10000 gidNumber: 5000 userPassword: {CRYPT}x gecos: John Doe loginShell: /bin/bash homeDirectory: /home/john $ ldapadd -x -D cn=admin,dc=example,dc=com -W -f add_content.ldif $ ldapsearch -x -LLL -b dc=example,dc=com '(uid=john)' cn gidNumber dn: uid=john,ou=People,dc=example,dc=com cn: John Doe gidNumber: 5000 $ ldappasswd -x -D cn=admin,dc=example,dc=com -W -S uid=john,ou=people,dc=example,dc=com New password: Re-enter new password: Enter LDAP Password: Configure the server for SSL (from server docs): https://ubuntu.com/server/docs/ldap-and-transport-layer-security-tls $ sudo apt install gnutls-bin ssl-cert $ sudo certtool --generate-privkey --bits 4096 --outfile /etc/ssl/private/mycakey.pem $ sudo vim /etc/ssl/ca.info cn = Example Company ca cert_signing_key expiration_days = 3650 $ sudo certtool --generate-self-signed \ --load-privkey /etc/ssl/private/mycakey.pem \ --template /etc/ssl/ca.info \ --outfile /usr/local/share/ca-certificates/mycacert.crt $ sudo update-ca-certificates $ sudo certtool --generate-privkey \ --bits 2048 \ --outfile /etc/ldap/ldap01_slapd_key.pem $ sudo vim /etc/ssl/ldap01.info organization = Example Company cn = example.com tls_www_server encryption_key signing_key expiration_days = 365 $ sudo certtool --generate-certificate \ --load-privkey /etc/ldap/ldap01_slapd_key.pem \ --load-ca-certificate /etc/ssl/certs/mycacert.pem \ --load-ca-privkey /etc/ssl/private/mycakey.pem \ --template /etc/ssl/ldap01.info \ --outfile /etc/ldap/ldap01_slapd_cert.pem $ sudo chgrp openldap /etc/ldap/ldap01_slapd_key.pem $ sudo chmod 0640 /etc/ldap/ldap01_slapd_key.pem $ sudo vim certinfo.ldif dn: cn=config add: olcTLSCACertificateFile olcTLSCACertificateFile: /etc/ssl/certs/mycacert.pem - add: olcTLSCertificateFile olcTLSCertificateFile: /etc/ldap/ldap01_slapd_cert.pem - add: olcTLSCertificateKeyFile olcTLSCertificateKeyFile: /etc/ldap/ldap01_slapd_key.pem $ sudo ldapmodify -Y EXTERNAL -H ldapi:/// -f certinfo.ldif $ sudo vim /etc/default/slapd Change LAPD_SERVICES="ldap:/// ldapi:///" to SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///" Next add TIMEOUT and NETWORK_TIMEOUT to the config file. $ sudo vim /etc/ldap/ldap.conf NETWORK_TIMEOUT 30 TIMEOUT 30 $ sudo systemctl restart slapd Reproduce the problem: $ ldapsearch -x -H ldaps://example.com # extended LDIF # # LDAPv3 # base <> (default) with scope subtree # filter: (objectclass=*) # requesting: ALL # # search result search: 2 result: 32 No such object # numResponses: 1 $ ps aux | grep slapd openldap 4097 0.0 0.1 1224136 7876 ? Ssl 02:22 0:00 /usr/sbin/slapd -h ldap:/// ldapi:/// ldaps:/// -g openldap -u openldap -F /etc/ldap/slapd.d $ sudo kill -STOP 4097 $ ldapsearch -x -H ldaps://example.com Check top:     PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND    4237 ubuntu 20 0 20724 10956 6656 R 100.0 0.3 0:35.70 ldapsearch wait for 30 seconds, see nothing changes. If you install packages from proposed, you should see the timeout work as expected, and the ldapsearch process will give up after 30 seconds. [Where problems could occur] We are changing how connections are tracked within openldap to better accurately say what state we are in, and for the timeouts to work when the connection doesn't make any progress. This will cause TIMEOUT and NETWORK_TIMEOUT to start working as intended, which might surprise some sysadmins who configured it, and likely forgot about it when they never had any timeout issues. If a regression were to occur, it would affect initial connections to openldap servers, or even long running commands that might exceed the TIMEOUT on queries over large ldap databases. [Other info] Upstream issues: https://bugs.openldap.org/show_bug.cgi?id=8047 https://bugs.openldap.org/show_bug.cgi?id=10141 This was fixed in 2.6.9 upstream by: commit d143f7a2dc82fb66e7741b93a1ae9e874ce2ac46 Author: Ondřej Kuzník <ondra@mistotebe.net> Date: Mon, 21 Oct 2024 11:50:11 +0100 Subject: ITS#8047 Fix TLS connection timeout handling Link: https://git.openldap.org/openldap/openldap/-/commit/d143f7a2dc82fb66e7741b93a1ae9e874ce2ac46 This fix is required in all releases.
2025-01-24 19:03:53 Andreas Hasenack openldap (Ubuntu Oracular): status In Progress Fix Committed
2025-01-24 19:03:56 Andreas Hasenack bug added subscriber Ubuntu Stable Release Updates Team
2025-01-24 19:04:01 Andreas Hasenack tags patch sts patch sts verification-needed verification-needed-oracular
2025-01-24 19:04:58 Andreas Hasenack openldap (Ubuntu Noble): status In Progress Fix Committed
2025-01-24 19:05:05 Andreas Hasenack tags patch sts verification-needed verification-needed-oracular patch sts verification-needed verification-needed-noble verification-needed-oracular
2025-01-24 19:05:39 Andreas Hasenack openldap (Ubuntu Jammy): status In Progress Fix Committed
2025-01-24 19:05:46 Andreas Hasenack tags patch sts verification-needed verification-needed-noble verification-needed-oracular patch sts verification-needed verification-needed-jammy verification-needed-noble verification-needed-oracular
2025-01-26 02:40:25 Matthew Ruffell tags patch sts verification-needed verification-needed-jammy verification-needed-noble verification-needed-oracular patch sts verification-done-oracular verification-needed verification-needed-jammy verification-needed-noble
2025-01-26 03:13:39 Matthew Ruffell tags patch sts verification-done-oracular verification-needed verification-needed-jammy verification-needed-noble patch sts verification-done-noble verification-done-oracular verification-needed verification-needed-jammy
2025-01-26 04:00:45 Matthew Ruffell tags patch sts verification-done-noble verification-done-oracular verification-needed verification-needed-jammy patch sts verification-done-jammy verification-done-noble verification-done-oracular
2025-01-28 02:08:25 Sergio Durigan Junior tags patch sts verification-done-jammy verification-done-noble verification-done-oracular patch sts verification-done verification-done-jammy verification-done-noble verification-done-oracular
2025-02-20 11:39:31 Andreas Hasenack removed subscriber Ubuntu Stable Release Updates Team
2025-02-20 13:22:27 Launchpad Janitor openldap (Ubuntu Oracular): status Fix Committed Fix Released
2025-02-20 15:35:59 Launchpad Janitor openldap (Ubuntu Noble): status Fix Committed Fix Released
2025-02-20 15:36:25 Launchpad Janitor openldap (Ubuntu Jammy): status Fix Committed Fix Released