TIMEOUT and NETWORK_TIMEOUT don't work when SSL is involved
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
openldap (Ubuntu) | Status tracked in Plucky | |||||
Focal |
Won't Fix
|
Medium
|
Matthew Ruffell | |||
Jammy |
Fix Released
|
Medium
|
Matthew Ruffell | |||
Noble |
Fix Released
|
Medium
|
Matthew Ruffell | |||
Oracular |
Fix Released
|
Medium
|
Matthew Ruffell | |||
Plucky |
Fix Released
|
Medium
|
Sergio Durigan Junior |
Bug Description
[Impact]
If you connect to a slapd server using TLS and for some reason, the connection
hangs or fails to completely establish, TIMEOUT and NETWORK_TIMEOUT values
do not function as they are intended. They get ignored, and the connecting
process e.g. ldapsearch just sits there at 100% CPU waiting forever.
If slapd is set up in High Availability with servers to failover to, the
failover will never occur, due to the connecting processes sitting there at
100% CPU waiting for the connection that will never succeed, leading to outages.
You can simulate this easily with:
$ ldapsearch -x -H ldaps://example.com
$ sudo kill -STOP <slapd pid>
$ ldapsearch -x -H ldaps://example.com
and the last call will hang forever.
There doesn't seem to be any workaround for this, apart from not using SSL.
[Testcase]
You can get away with just the one VM for this.
Set up the ldap server (from server docs):
https:/
$ sudo apt install slapd ldap-utils
$ sudo dpkg-reconfigure slapd
Set DNS to example.com
$ ldapsearch -x -LLL -H ldap:/// -b dc=example,dc=com dn
dn: dc=example,dc=com
$ vim add_content.ldif
dn: ou=People,
objectClass: organizationalUnit
ou: People
dn: ou=Groups,
objectClass: organizationalUnit
ou: Groups
dn: cn=miners,
objectClass: posixGroup
cn: miners
gidNumber: 5000
dn: uid=john,
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: john
sn: Doe
givenName: John
cn: John Doe
displayName: John Doe
uidNumber: 10000
gidNumber: 5000
userPassword: {CRYPT}x
gecos: John Doe
loginShell: /bin/bash
homeDirectory: /home/john
$ ldapadd -x -D cn=admin,
$ ldapsearch -x -LLL -b dc=example,dc=com '(uid=john)' cn gidNumber
dn: uid=john,
cn: John Doe
gidNumber: 5000
$ ldappasswd -x -D cn=admin,
New password:
Re-enter new password:
Enter LDAP Password:
Configure the server for SSL (from server docs):
https:/
$ sudo apt install gnutls-bin ssl-cert
$ sudo certtool --generate-privkey --bits 4096 --outfile /etc/ssl/
$ sudo vim /etc/ssl/ca.info
cn = Example Company
ca
cert_signing_key
expiration_days = 3650
$ sudo certtool --generate-
--load-privkey /etc/ssl/
--template /etc/ssl/ca.info \
--outfile /usr/local/
$ sudo update-
$ sudo certtool --generate-privkey \
--bits 2048 \
--outfile /etc/ldap/
$ sudo vim /etc/ssl/
organization = Example Company
cn = example.com
tls_www_server
encryption_key
signing_key
expiration_days = 365
$ sudo certtool --generate-
--load-privkey /etc/ldap/
--load-
--load-ca-privkey /etc/ssl/
--template /etc/ssl/
--outfile /etc/ldap/
$ sudo chgrp openldap /etc/ldap/
$ sudo chmod 0640 /etc/ldap/
$ sudo vim certinfo.ldif
dn: cn=config
add: olcTLSCACertifi
olcTLSCACertifi
-
add: olcTLSCertifica
olcTLSCertifica
-
add: olcTLSCertifica
olcTLSCertifica
$ sudo ldapmodify -Y EXTERNAL -H ldapi:/// -f certinfo.ldif
$ sudo vim /etc/default/slapd
Change
LAPD_SERVICES=
to
SLAPD_SERVICES=
Next add TIMEOUT and NETWORK_TIMEOUT to the config file.
$ sudo vim /etc/ldap/ldap.conf
NETWORK_TIMEOUT 30
TIMEOUT 30
$ sudo systemctl restart slapd
Reproduce the problem:
$ ldapsearch -x -H ldaps://example.com
# extended LDIF
#
# LDAPv3
# base <> (default) with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#
# search result
search: 2
result: 32 No such object
# numResponses: 1
$ ps aux | grep slapd
openldap 4097 0.0 0.1 1224136 7876 ? Ssl 02:22 0:00 /usr/sbin/slapd -h ldap:/// ldapi:/// ldaps:/// -g openldap -u openldap -F /etc/ldap/slapd.d
$ sudo kill -STOP 4097
$ ldapsearch -x -H ldaps://example.com
Check top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4237 ubuntu 20 0 20724 10956 6656 R 100.0 0.3 0:35.70 ldapsearch
wait for 30 seconds, see nothing changes.
If you install packages from proposed, you should see the timeout work as expected, and the ldapsearch process will give up after 30 seconds.
[Where problems could occur]
We are changing how connections are tracked within openldap to better accurately
say what state we are in, and for the timeouts to work when the connection
doesn't make any progress.
This will cause TIMEOUT and NETWORK_TIMEOUT to start working as intended, which
might surprise some sysadmins who configured it, and likely forgot about it
when they never had any timeout issues.
If a regression were to occur, it would affect initial connections to openldap
servers, or even long running commands that might exceed the TIMEOUT on queries
over large ldap databases.
[Other info]
Upstream issues:
https:/
https:/
This was fixed in 2.6.9 upstream by:
commit d143f7a2dc82fb6
Author: Ondřej Kuzník <email address hidden>
Date: Mon, 21 Oct 2024 11:50:11 +0100
Subject: ITS#8047 Fix TLS connection timeout handling
Link: https:/
This fix is required in all releases.
Changed in openldap (Ubuntu Focal): | |
status: | New → In Progress |
Changed in openldap (Ubuntu Jammy): | |
status: | New → In Progress |
Changed in openldap (Ubuntu Noble): | |
status: | New → In Progress |
Changed in openldap (Ubuntu Oracular): | |
status: | New → In Progress |
Changed in openldap (Ubuntu Plucky): | |
status: | New → In Progress |
Changed in openldap (Ubuntu Focal): | |
importance: | Undecided → Medium |
Changed in openldap (Ubuntu Jammy): | |
importance: | Undecided → Medium |
Changed in openldap (Ubuntu Noble): | |
importance: | Undecided → Medium |
Changed in openldap (Ubuntu Oracular): | |
importance: | Undecided → Medium |
Changed in openldap (Ubuntu Plucky): | |
importance: | Undecided → Medium |
Changed in openldap (Ubuntu Focal): | |
assignee: | nobody → Matthew Ruffell (mruffell) |
Changed in openldap (Ubuntu Jammy): | |
assignee: | nobody → Matthew Ruffell (mruffell) |
Changed in openldap (Ubuntu Noble): | |
assignee: | nobody → Matthew Ruffell (mruffell) |
Changed in openldap (Ubuntu Oracular): | |
assignee: | nobody → Matthew Ruffell (mruffell) |
tags: | added: sts |
tags: | added: patch |
Changed in openldap (Ubuntu Plucky): | |
status: | In Progress → Fix Committed |
assignee: | nobody → Sergio Durigan Junior (sergiodj) |
Changed in openldap (Ubuntu Plucky): | |
status: | Fix Committed → Fix Released |
Changed in openldap (Ubuntu Focal): | |
status: | In Progress → Won't Fix |
tags: | added: verification-done |
Attached is a debdiff for oracular which fixes this issue.