We do not treat SLDs that act like TLDs as TLDs

I've given this some thought in the past, and without a authoritative list of SLD's, It's hard to come up with a viable solution.

A partial solution that includes "co.uk.", "gov.uk." etc etc is likely more dangerous IMO than no list at all, as users may be left believing that all SLDs are covered.

Mozilla publish a list of SLDs - Maybe we use this? I've no idea how complete it is though .. http://publicsuffix.org/

If we do use this list, I think using a parser for the raw list[1], and allowing end users to optionally place the file in /etc/designate/publicsuffixes.dat is probably a good option..

I'm not sure how legal it is, or how acceptable to OpenStack it is, to include this list directly in our source tree as it's licensed under the MPL.

Something like the publicsuffix[2] library on pypi would likely be useful?

[1]: http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1
[2]: https://pypi.python.org/pypi/publicsuffix/

What about putting both lists (TLDs and SLDs) in a database and adding an API call to be able to view and edit them? That would solve the problem of a trying to place a publicsuffixes.data file in our source tree.

In progress @ https://review.openstack.org/#/c/55871/

Reviewed: https://review.openstack.org/55871
Committed: http://github.com/stackforge/designate/commit/3db98ab1fd1596a544f84c4ad0a6314383ed6629
Submitter: Jenkins
Branch: master

commit 3db98ab1fd1596a544f84c4ad0a6314383ed6629
Author: Vinod Mangalpally <email address hidden>
Date: Tue Nov 19 08:53:53 2013 -0600

    Add SLDs that act like TLDs

    This change offers the designate administrator the ability to configure
    2 lists to indicate the TLDs(accepted_tlds_file) and SLDs and others
    that
    effectively act as TLDs.(effective_tlds_file)

    2 sample files are included to show the format required for these files.
    The TLD file is from http://data.iana.org/TLD/tlds-alpha-by-domain.txt
    Commented lines in this file begin with a #. There is one entry per line
    and the entries are in the IDNA format.

    The sample effective TLD file is from http://publicsuffix.org/list/
    This file is in the UTF-8 format. Lines beginning with a // and ! are
    ignored. The wildcard character * (asterisk) may only be used to
    wildcard
    the topmost level in a domain name. All the entries are converted to
    idna format.

    Closes-Bug: #1249396

    Change-Id: I9718489335e24cb24733d4c4fc2a966e490014d3

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/57286

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/57326

Reviewed: https://review.openstack.org/57326
Committed: http://github.com/stackforge/designate/commit/5a6795d7c8640dcb49e58f5cfa70afdbad267e86
Submitter: Jenkins
Branch: stable/havana

commit 5a6795d7c8640dcb49e58f5cfa70afdbad267e86
Author: Vinod Mangalpally <email address hidden>
Date: Wed Nov 20 12:38:06 2013 -0600

    Add SLDs that act like TLDs

    This change is from the Icehouse release. The effective TLD file need
    not be present. When it is not present the behavior would be the same
    as before. When it is present, the domains that are being created are
    checked to see that it is not an effective TLD.

    This change offers the designate administrator the ability to configure
    2 lists to indicate the TLDs(accepted_tlds_file) and SLDs and others
    that effectively act as TLDs.(effective_tlds_file)

    2 sample files are included to show the format required for these files.
    The TLD file is from http://data.iana.org/TLD/tlds-alpha-by-domain.txt
    Commented lines in this file begin with a #. There is one entry per line
    and the entries are in the IDNA format.

    The sample effective TLD file is from http://publicsuffix.org/list/
    This file is in the UTF-8 format. Lines beginning with a // and ! are
    ignored. The wildcard character * (asterisk) may only be used to
    wildcard the topmost level in a domain name. All the entries are
    converted to idna format.

    Change-Id: I3d2f7e2741f4cda0b227e4a5671b141f99052291
    Closes-Bug: #1249396