This is especially upsetting in an OpenStack and server environment where external DNS is being used to reach/resolve API endpoints and other systems such as database servers. A small outage to DNS becomes a potentially unbounded outage when the SERVFAIL responses are cached indefinitely, requiring manual intervention on each host in addition to fixing the cause of the SERVFAIL.
This is especially upsetting in an OpenStack and server environment where external DNS is being used to reach/resolve API endpoints and other systems such as database servers. A small outage to DNS becomes a potentially unbounded outage when the SERVFAIL responses are cached indefinitely, requiring manual intervention on each host in addition to fixing the cause of the SERVFAIL.