Activity log for bug #1940715

Date Who What changed Old value New value Message
2021-08-20 20:29:18 Dan Streetman bug added bug
2021-08-20 20:29:34 Dan Streetman nominated for series Ubuntu Hirsute
2021-08-20 20:29:34 Dan Streetman bug task added systemd (Ubuntu Hirsute)
2021-08-20 20:29:34 Dan Streetman nominated for series Ubuntu Bionic
2021-08-20 20:29:34 Dan Streetman bug task added systemd (Ubuntu Bionic)
2021-08-20 20:29:34 Dan Streetman nominated for series Ubuntu Impish
2021-08-20 20:29:34 Dan Streetman bug task added systemd (Ubuntu Impish)
2021-08-20 20:29:34 Dan Streetman nominated for series Ubuntu Focal
2021-08-20 20:29:34 Dan Streetman bug task added systemd (Ubuntu Focal)
2021-08-20 20:29:40 Dan Streetman systemd (Ubuntu Bionic): status New In Progress
2021-08-20 20:29:41 Dan Streetman systemd (Ubuntu Focal): status New In Progress
2021-08-20 20:29:43 Dan Streetman systemd (Ubuntu Hirsute): status New In Progress
2021-08-20 20:29:45 Dan Streetman systemd (Ubuntu Bionic): importance Undecided Low
2021-08-20 20:29:47 Dan Streetman systemd (Ubuntu Focal): importance Undecided Low
2021-08-20 20:29:48 Dan Streetman systemd (Ubuntu Hirsute): importance Undecided Low
2021-08-20 20:29:51 Dan Streetman systemd (Ubuntu Bionic): assignee Dan Streetman (ddstreet)
2021-08-20 20:29:52 Dan Streetman systemd (Ubuntu Focal): assignee Dan Streetman (ddstreet)
2021-08-20 20:29:54 Dan Streetman systemd (Ubuntu Hirsute): assignee Dan Streetman (ddstreet)
2021-08-24 12:29:38 Dan Streetman description [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] TBD [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed for all releases this still needs fixing upstream [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] enable debug logging in systemd-resolved, with 'sudo systemctl edit systemd-resolved' and then add: [Service] Environment=SYSTEMD_LOG_LEVEL=debug then save that file and restart systemd-resolved (or reboot). Make sure to flush the cache and reset server features before reproducing: $ sudo resolvectl flush-caches $ sudo resolvectl reset-server-features Lookup 'toomany.ddstreet.org' and check systemd-resolved logs to see if it used TCP fallback or not: ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Reply truncated, retrying via TCP. ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Verified we get a response at feature level TCP from DNS server 10.202.51.1. Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Added positive unauthenticated cache entry for toomany.ddstreet.org IN A 1799s on eth0/INET/10.202.51.1 A correct lookup using larger EDNS0 response size looks like: ... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using feature level UDP+EDNS0 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using DNS server 10.202.51.1 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Announcing packet size 1472 in egress EDNS(0) packet. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Emitting UDP, link MTU is 1500, socket MTU is 0, minimal MTU is 40 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Sending query packet with id 43808 of size 49. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing query... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Received dns UDP packet of size 689, ifindex=131, ttl=0, fragsize=0 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing incoming packet of size 689 on transaction 43808 (rcode=SUCCESS). Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Added positive unauthenticated non-confidential cache entry for toomany.ddstreet.org IN A 1175s on eth0/INET/10.202.51.1 ... [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed for all releases this still needs fixing upstream
2021-08-24 12:30:07 Dan Streetman description [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] enable debug logging in systemd-resolved, with 'sudo systemctl edit systemd-resolved' and then add: [Service] Environment=SYSTEMD_LOG_LEVEL=debug then save that file and restart systemd-resolved (or reboot). Make sure to flush the cache and reset server features before reproducing: $ sudo resolvectl flush-caches $ sudo resolvectl reset-server-features Lookup 'toomany.ddstreet.org' and check systemd-resolved logs to see if it used TCP fallback or not: ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Reply truncated, retrying via TCP. ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Verified we get a response at feature level TCP from DNS server 10.202.51.1. Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Added positive unauthenticated cache entry for toomany.ddstreet.org IN A 1799s on eth0/INET/10.202.51.1 A correct lookup using larger EDNS0 response size looks like: ... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using feature level UDP+EDNS0 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using DNS server 10.202.51.1 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Announcing packet size 1472 in egress EDNS(0) packet. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Emitting UDP, link MTU is 1500, socket MTU is 0, minimal MTU is 40 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Sending query packet with id 43808 of size 49. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing query... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Received dns UDP packet of size 689, ifindex=131, ttl=0, fragsize=0 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing incoming packet of size 689 on transaction 43808 (rcode=SUCCESS). Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Added positive unauthenticated non-confidential cache entry for toomany.ddstreet.org IN A 1175s on eth0/INET/10.202.51.1 ... [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed for all releases this still needs fixing upstream [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] enable debug logging in systemd-resolved, with 'sudo systemctl edit systemd-resolved' and then add: [Service] Environment=SYSTEMD_LOG_LEVEL=debug then save that file and restart systemd-resolved (or reboot). Make sure to flush the cache and reset server features before reproducing: $ sudo resolvectl flush-caches $ sudo resolvectl reset-server-features Lookup 'toomany.ddstreet.org' and check systemd-resolved logs to see if it used TCP fallback or not: ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Reply truncated, retrying via TCP. ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Verified we get a response at feature level TCP from DNS server 10.202.51.1. Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Added positive unauthenticated cache entry for toomany.ddstreet.org IN A 1799s on eth0/INET/10.202.51.1 A correct lookup using larger EDNS0 response size looks like: ... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using feature level UDP+EDNS0 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using DNS server 10.202.51.1 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Announcing packet size 1472 in egress EDNS(0) packet. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Emitting UDP, link MTU is 1500, socket MTU is 0, minimal MTU is 40 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Sending query packet with id 43808 of size 49. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing query... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Received dns UDP packet of size 689, ifindex=131, ttl=0, fragsize=0 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing incoming packet of size 689 on transaction 43808 (rcode=SUCCESS). Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Added positive unauthenticated non-confidential cache entry for toomany.ddstreet.org IN A 1175s on eth0/INET/10.202.51.1 ... [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed for all releases this still needs fixing upstream: https://github.com/systemd/systemd/pull/20528
2021-10-13 11:13:45 Dan Streetman bug watch added https://github.com/systemd/systemd/issues/20993
2021-10-13 11:13:45 Dan Streetman bug task added systemd
2021-10-13 12:14:16 Bug Watch Updater systemd: status Unknown New
2021-12-08 08:44:17 Bug Watch Updater systemd: status New Fix Released
2022-03-10 10:48:17 Launchpad Janitor systemd (Ubuntu): status New Confirmed
2022-03-10 10:48:17 Launchpad Janitor systemd (Ubuntu Impish): status New Confirmed
2022-05-22 18:40:21 Dan Streetman systemd (Ubuntu Hirsute): status In Progress Won't Fix
2022-05-22 18:40:33 Dan Streetman systemd (Ubuntu): status Confirmed Fix Released
2022-05-22 18:50:07 Dan Streetman description [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] enable debug logging in systemd-resolved, with 'sudo systemctl edit systemd-resolved' and then add: [Service] Environment=SYSTEMD_LOG_LEVEL=debug then save that file and restart systemd-resolved (or reboot). Make sure to flush the cache and reset server features before reproducing: $ sudo resolvectl flush-caches $ sudo resolvectl reset-server-features Lookup 'toomany.ddstreet.org' and check systemd-resolved logs to see if it used TCP fallback or not: ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Reply truncated, retrying via TCP. ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Verified we get a response at feature level TCP from DNS server 10.202.51.1. Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Added positive unauthenticated cache entry for toomany.ddstreet.org IN A 1799s on eth0/INET/10.202.51.1 A correct lookup using larger EDNS0 response size looks like: ... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using feature level UDP+EDNS0 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using DNS server 10.202.51.1 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Announcing packet size 1472 in egress EDNS(0) packet. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Emitting UDP, link MTU is 1500, socket MTU is 0, minimal MTU is 40 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Sending query packet with id 43808 of size 49. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing query... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Received dns UDP packet of size 689, ifindex=131, ttl=0, fragsize=0 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing incoming packet of size 689 on transaction 43808 (rcode=SUCCESS). Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Added positive unauthenticated non-confidential cache entry for toomany.ddstreet.org IN A 1175s on eth0/INET/10.202.51.1 ... [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed for all releases this still needs fixing upstream: https://github.com/systemd/systemd/pull/20528 [impact] when talking to upstream nameservers, systemd-resolved limits its advertised max packet size as 512 in its edns0 opt. However, one of the primary benefits of edns0 is to allow using packet sizes larger than 512, which is the pre-edns0 max packet size. this results in systemd-resolved failing to handle responses larger than 512 with udp/edns0, and having to fall back to tcp. This is not optimal (since tcp dns imposes significantly higher overhead) and may even cause failures, if a firewall allows udp dns but blocks tcp dns traffic. [test case] enable debug logging in systemd-resolved, with 'sudo systemctl edit systemd-resolved' and then add: [Service] Environment=SYSTEMD_LOG_LEVEL=debug then save that file and restart systemd-resolved (or reboot). Make sure to flush the cache and reset server features before reproducing: $ sudo resolvectl flush-caches $ sudo resolvectl reset-server-features Lookup 'toomany.ddstreet.org' and check systemd-resolved logs to see if it used TCP fallback or not: ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Reply truncated, retrying via TCP. ... Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Verified we get a response at feature level TCP from DNS server 10.202.51.1. Aug 24 12:17:48 lp1940715-f systemd-resolved[1199]: Added positive unauthenticated cache entry for toomany.ddstreet.org IN A 1799s on eth0/INET/10.202.51.1 A correct lookup using larger EDNS0 response size looks like: ... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using feature level UDP+EDNS0 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Using DNS server 10.202.51.1 for transaction 43808. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Announcing packet size 1472 in egress EDNS(0) packet. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Emitting UDP, link MTU is 1500, socket MTU is 0, minimal MTU is 40 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Sending query packet with id 43808 of size 49. Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing query... Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Received dns UDP packet of size 689, ifindex=131, ttl=0, fragsize=0 Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Processing incoming packet of size 689 on transaction 43808 (rcode=SUCCESS). Aug 24 12:28:13 lp1940715-f systemd-resolved[174]: Added positive unauthenticated non-confidential cache entry for toomany.ddstreet.org IN A 1175s on eth0/INET/10.202.51.1 ... [regression potential] failure to correctly look up dns records, or other problems while performing dns lookups with systemd-resolved [scope] this is needed in impish and earlier this is fixed in jammy at commit 22545a74c3a24395bc6e894fe6a681deb66d434c this is fixed upstream by commit 526fce97afe130f71dba3bd4646196bbb1188b82 which in included in v250, so this is fixed in kinetic and later the upstream PR is: https://github.com/systemd/systemd/pull/20528
2022-07-18 22:59:58 Brian Murray systemd (Ubuntu Impish): status Confirmed Won't Fix
2022-08-18 16:09:50 Dan Streetman systemd (Ubuntu Hirsute): assignee Dan Streetman (ddstreet)
2022-08-18 16:09:52 Dan Streetman systemd (Ubuntu Focal): assignee Dan Streetman (ddstreet)
2022-08-18 16:09:54 Dan Streetman systemd (Ubuntu Bionic): assignee Dan Streetman (ddstreet)
2022-08-18 16:10:09 Dan Streetman systemd (Ubuntu Focal): status In Progress New
2022-08-18 16:10:11 Dan Streetman systemd (Ubuntu Bionic): status In Progress New