launchpad should verify gmail or DomainKeys authenticators

i dont feel to comfortable with that idea.
we can easily check the user GPG key on launchpad, but i cant do the same to a gmail account.

2009/1/12 BUGabundo <email address hidden>:
> we can easily check the user GPG key on launchpad, but i cant do the same to a gmail account.

What do you mean?

--
Martin <http://launchpad.net/~mbp/>

This is a nice idea. Are there any specs/docs about those headers?

Yes, see eg http://en.wikipedia.org/wiki/DomainKeys

This would be particularly nice because of the common practice now of setting a bug to incomplete when wanting a response from the reporter, as just happened here. (Though arguably that's a different bug.)

http://people.canonical.com/~bjorn/lp-architecture/in-depth/incomingmail.html has some amount of information on where this would hook in. http://hewgill.com/pydkim/ is a python library that, given a mail text, tells you true/false whether it can be verified.

Branch underway in lp:~mbp/launchpad/dkim

I'm not sure if this affects me or not :) - I'm using a hosted gmail mail account, which is the gmail interface but my own domain.

*If* gmail hosted accounts can do DKIM, then this being fixed will make me happy ;)

On 24 May 2010 19:28, Robert Collins <email address hidden> wrote:
> I'm not sure if this affects me or not :) - I'm using a hosted gmail
> mail account, which is the gmail interface but my own domain.
>
> *If* gmail hosted accounts can do DKIM, then this being fixed will make
> me happy ;)

Apparently they can't yet.
<http://groups.google.com/group/hosted-setup/browse_thread/thread/cfee3830e9c63dcc/5b6e60d06eeb9d9b?pli=1>
   I think they could, but it's just not implemented.

Your mail does have an X-Google-Sender-Auth, but I can't find any
documentation of how you can verify that, and perhaps since DKIM seems
to be becoming standard it's not worth adding a site-specific
mechanism.

--
Martin <http://launchpad.net/~mbp/>

The attached branch <https://code.edge.launchpad.net/~mbp/launchpad/dkim/+merge/26284> now has something that just might work, including some credible tests. The tests have a canned RSA key and they monkeypatch DNS to pretend it's the key for example.canonical.com.

I hit and fixed some bugs in dkim.py: when signing it generates something that's not precisely either a Simple mode or Relaxed mode signature. This patch fixes it a bit. In light of it being just one file perhaps we should just put it in the tree?

Note that gmail.com's dkim signatures are still marked as for testing only.

This kind of application is fundamentally NOT what DKIM was designed to do. It is meant to authenticate the domain, not the user.

On 1 June 2010 10:11, Scott Kitterman <email address hidden> wrote:
> Note that gmail.com's dkim signatures are still marked as for testing
> only.

Specifically it says messages should not be bounced on the basis of
this, and we wouldn't do that. It seems reasonable to assume gmail
will not fail open by suddenly starting to sign messages that didn't
actually come from the named user.

> This kind of application is fundamentally NOT what DKIM was designed to
> do.  It is meant to authenticate the domain, not the user.

RFC4870 says

   The ultimate goal of this framework is to permit
   a signing domain to assert responsibility for a message, thus
   protecting message signer identity and the integrity of the messages
   they convey while retaining the functionality of Internet email as it
   is known today.

...

   It is beyond the scope of this specification to describe what actions
   a verifier system should make, but an authenticated email presents an
   opportunity to a receiving system that unauthenticated email cannot.
   Specifically, an authenticated email creates a predictable identifier
   by which other decisions can reliably be managed, such as trust and
   reputation.

If we back up and look at how Launchpad handles incoming mail: at the
moment a lot of it is just trusted to be from whoever it claims to be
from. In some cases, such as filing a new bug, we judge there is too
much risk that a forged message will cause trouble, so we want a
higher degree of assurance that the message is real. (The message may
be forged either maliciously or because of eg spam or backscatter.)
istm that DKIM will give us an acceptable level of assurance that the
message is authentic.

If a user associates <email address hidden> with their Launchpad account, then
evil.com could send mail falsely claiming to be from Jo. However,
evil.com can also intercept a password reset mail sent to Jo or do
various other attacks, so I don't think this is really a substantial
risk.

There will still be some Launchpad actions such as CoC-signing where
we insist on a GPG key. (Though whether that's actually more secure
is a bit debatable, considering we accept any GPG key uploaded to the
account.)

Thanks for your thoughts and feedback.
--
Martin <http://launchpad.net/~mbp/>

If you allow a dkim signature to mean anything positive (i.e. to allow anything you rely on a GPG key today) you are abusing the protocol and relying on internal implementation details of proprietary web services.

You don't need to quote the RFC at me. I've participated in the IETF DKIM working group since it was started and contributed to the design.

If you think this is OK from a security perspective you really don't understand what DKIM is doing.

Additionally, if you read the RFC, it says explicitly that a broken signature should be treated just like no signature at all, so if you think broken signatures are meaningful, you are misunderstanding the RFC.

You could treat the absence of a valid signature as meaningful from domains that publish ADSP all or discardable policy records as a reason not to trust that the email is valid/authorize and that would be both consistent with the DKIM/ADSP design and a reasonable security practice (since you don't expect inbound mail from mailing lists as was already said).

Thanks for talking this over, Scott. To try to summarize, the typical problem case is this:

example.com allows any of their users to pretend to be any other user

example.com accept smtp submissions from their internal network, and on the way out they add a valid signature. However, they don't check that the user part of the From address corresponds to the actual user who sent the mail, therefore intra-domain or cross-user forgery is possible. If DKIM is deployed as a wrapper around the MTA that doesn't connect to mail submission authentication, and if the submission authentication doesn't prevent spoofing, this is quite possible. We don't know how widespread this is and there is no way to tell from the outside whether a particular sender domain allows this or not.

This would be pretty sloppy on the part of example.com, and I think against the spirit of the DKIM spec, but it may be so common that we can't ignore it.

We could reasonably assume that gmail and yahoo don't allow this but other domains do.

The DKIM spec certainly seems to imply that if the From line is signed by the relevant domain then it can be trusted.

I don't think you can reasonably assume anything about From based on a DKIM signature. If you have some out of band communication with the domain in question and they tell you they limit use of particular From addresses to the one(s) the user is authorized to use, then to the extent you trust that assertion, use of a DKIM signature might be OK.

Even just trusting GMail would make over 40% of our users happy, especially now FireGPG has dropped GMail support. I think it is fair to say that if we receive an email with a valid DKIM signature from <email address hidden> then we can reasonably trust that it came from <email address hidden>.

We are already trusting email providers due to the mechanics of password recovery. This may no longer be true when we become a proper OpenID relying party (at which point we start trusting the OpenID providers), but it is true now.

We probably want a whitelist of domains to trust, or a tick box the user can select against their email address to toggle DKIM trust. Whitelist would be the best start I think.

>Even just trusting GMail would make over 40% of our users happy,
>especially now FireGPG has dropped GMail support. I think it is fair to
>say that if we receive an email with a valid DKIM signature from
><email address hidden> then we can reasonably trust that it came from
><email address hidden>.

What's the basis for this claim?

>We are already trusting email providers due to the mechanics of password
>recovery. This may no longer be true when we become a proper OpenID
>relying party (at which point we start trusting the OpenID providers),
>but it is true now.

Not for actions that require authentication. For those you are trusting a gpg signature.

>We probably want a whitelist of domains to trust, or a tick box the user
>can select against their email address to toggle DKIM trust. Whitelist
>would be the best start I think.

Users are absolutely unqualified to make this decision.

> > Even just trusting GMail would make over 40% of our users happy,
> > especially now FireGPG has dropped GMail support. I think it is
> > fair to say that if we receive an email with a valid DKIM
> > signature from <email address hidden> then we can reasonably trust that it
> > came from <email address hidden>.
>
> What's the basis for this claim?

I can't speak for Stuart, but I would say something like:

- We trust Google to authenticate users.

- We trust that Google only DKIM signs messages from authenticated
  users.

- If the DKIM signature on incoming email from Gmail validates, and
  that the From: header is amongst the list of signed headers, we are
  willing to trust that the From: header is valid, and act on the body
  of the message accordingly.

Additionally, before being able to interact with Bugs from Gmail:

- The user must have confirmed their Gmail email address with
  Launchpad.

- The user therefore trusts Gmail to authenticate him or herself,
  because password resets can be performed by email.

  (On this point it is possible that users are not aware of the trust
  they place in their email provider, and it may be worth reminding
  users of this.)

> > We are already trusting email providers due to the mechanics of
> > password recovery. This may no longer be true when we become a
> > proper OpenID relying party (at which point we start trusting the
> > OpenID providers), but it is true now.
>
> Not for actions that require authentication. For those you are
> trusting a gpg signature.

I don't understand. Password recovery requires neither authentication
nor a gpg signature.

> > We probably want a whitelist of domains to trust, or a tick box
> > the user can select against their email address to toggle DKIM
> > trust. Whitelist would be the best start I think.
>
> Users are absolutely unqualified to make this decision.

Why? Suppose Launchpad gains the ability to trust DKIM signatures from
Gmail, but by default it is switched off. As a Gmail user I might like
to opt in to that for the convenience. The implications could be
explained to me at the point that I choose to opt in.

I am very keen for a feature like this to work, because it would be
really good for a huge proportion of Launchpad's users (including
me). I'm aware that security and convenience are perhaps being traded
off here.

Scott Kitterman wrote on 2010-06-01:
> If you think this is OK from a security perspective you really don't
> understand what DKIM is doing.
>
> Additionally, if you read the RFC, it says explicitly that a broken
> signature should be treated just like no signature at all, so if you
> think broken signatures are meaningful, you are misunderstanding the
> RFC.

I don't think that anyone proposed that broken signatures be treated
as useful in any way.

Scott, it seems you have the expertise. Please help us understand if
there *is* a way we can obtain a level of trust via DKIM sufficient to
allow users to interact with Launchpad via email. If not, why? If only
so that we don't have this same conversation again in 6 months.

From header validity isn't assured by DKIM. What other basis for this assurance do you have?

> From header validity isn't assured by DKIM. What other basis for
> this assurance do you have?

That's seems like enough to scupper this plan.

To try and understand something about DKIM I've been reading the
Wikipedia article: http://en.wikipedia.org/wiki/DKIM

In there, when talking about the DKIM-Signature header, an h tag is
mentioned, which lists the headers that are signed. From what I can
tell, Google includes the From: header in the DKIM signatures they
compose.

All the fact that a DKIM signature covers From can tell you is that it didn't get modified in transit.

On 17 June 2010 21:15, Gavin Panella <email address hidden> wrote:
> I don't think that anyone proposed that broken signatures be treated
> as useful in any way.

I did, in the LEP or email thread, say something about potentially
using failed signatures to reject spam/forgery. I'm not proposing we
do this now or that we do it unless the domain declares that all their
mail is signed. That's probably better done in an anti-spam facility
that stands in front of Launchpad.

This does mean that if a user expects their mail to be DKIM-signed but
we don't verify the signature, we'll just treat it as untrusted, and
reject it if they're trying to something that needs to be strongly
authenticated. I suppose at that point they can contact us and ask
why.

--
Martin

On 17 June 2010 22:19, Gavin Panella <email address hidden> wrote:
>> From header validity isn't assured by DKIM. What other basis for
>> this assurance do you have?

Scott, I want to assume good faith but that statement is so terse that
it is likely to mislead people.

What I think Scott means is that DKIM does not guarantee the From
header is "true". (Note that DKIM requires the From header be
signed.) It can give us good assurance that example.com asserted that
a particular message left example.com with a particular From header,
and that the header was not modified in transit.

If we have a message From example.com and signed by example.com then either:

1- it actually did come from the purported user and we can trust it
2- example.com is an open relay that will sign and forward messages
from anyone who cares to pretend to be in example.com
2a- as a special case, example.com is not a totally open relay but
will allow <email address hidden> to impersonate <email address hidden>

Are there other possibilities?

I'm sure there are domains in class 2, either intentionally or because
they've been compromised or because they're misconfigured. But we
already have to trust the user's email host to some extent and we rely
on them to choose one they feel is sufficiently safe.

It seems like a reasonable assumption that major providers will not be
in class 2. (Or at least not except for hopefully narrow windows of
brokenness.)

One can have a semi-philosophical discussion about whether "From: mbp"
really means from a particular human but for our purposes we assume
that if the user nominates an address for that account, it really is
them.

--
Martin

2a is not rare, so you should not assume it doesn't apply.

So from my reading, with a whitelist of trusted domains we get the same level of trust from DKIM as we do with GPG signed emails. To get on the trusted list, a domain needs to:

 - Sign the From: header
 - Promise that signed emails actually come from the user claimed in the From: header.

If we receive an email with a valid DKIM signature from the trusted domain (confirming the From: header is signed) it means it came from someone with control of that mailbox, the DNS records, or mail server. This means it is trusted as a GPG signed email from Launchpad's perspective (with control of the mailbox, DNS or mail server you can reset the users password, then upload a GPG key which Launchpad will then trust).

If we can verify that yahoo.com, yahoo.co.*, gmail.com, and googlemail.com sign the From: headers, don't allow From: forgery, and we believe they are reasonably competent at running their systems, then that will account for 49.5% of our current preferred email addresses.

If we replace the whitelist with a user selectable toggle, then I think that works too with the user accepting responsibility.

Unless the user understands the limitations of DKIM, they won't understand what they are accepting.

DKIM is substantially less strong cryptographically than gpg, so don't draw the equivalence too strongly. For most purposes, you're probably close enough. If there were a way to affect the contents of the archive or a PPA, then I'd still definitely insist on gpg.

Scott K

See also https://bugs.edge.launchpad.net/ubuntu/+source/pydkim/+bug/587783 for the associated bug in python-dkim and https://code.edge.launchpad.net/~mbp/ubuntu/lucid/pydkim/587783-signing for a fix.

The branch for this should now be in production, but dkim-signed mail sent to Launchpad from whitelisted domains are rejected as not signed. So the next step is to see what was logged on the production machine that processes incoming mail.

I'm just tried sending mail from my gmail account to <email address hidden>, and it's rejected with "Error message: To report bugs by e-mail, you need to sign the message with an OpenPGP key
that is registered in Launchpad." However, the DKIM debug log seems to show it did pass validation.

In my testing this seems to now be working, subject to some conditions:

 * not for mail to <email address hidden> (bug 643219)
 * the From address must be in the signing domain (eg at gmail.com not elsewhere) (bug 643223)

Also

 * dkim whitelist should be in configuration (bug 643224)
 * error messages and user documentation should mention dkim (bug 643225)

and barely relevant

 * bug 387327, dkim-sign outgoing mail

Fixed in stable r12008 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12008>.

I don't think this needs further qa now: things are working subject to the restrictions in <https://bugs.edge.launchpad.net/launchpad-registry/+bug/316272/comments/29> and some of the limitations have been fixed by bug 643219, which is now qa'd.