LocationError raised in build page and distribution arch series binary package page

Erm, I know there is as constraint on LFA so that it shouldn't be possible, but at first glance those oopses suggest that build id=198178 has a binary package with a LibraryFileAlias where lfa.content is None?

Is it possible to check all the files associated with:

>>> bin_pkgs = Build.get(198178).binarypackages
>>> lfa_ids = []
>>> for pkg in bin_pkgs:
... lfa_ids += [file.libraryfile.id for file in pkg.files]

and then check those lfas? (or using the SQL equivalent)

I'll look for other possibilities first thing in the morning.

I know that last cycle some changes were made in the librarian schema/code (I think by Abel), it's worth checking with the Foundations guys here.

Yes, I removed the NOT NULL constraint for LFA.content, see bug 397188. But I have no clue why that value became null in this case... AFAIK, explicitly setting LFA.content to NULL is done only for LFA records related to bug attachments.

And in patch-2207-15-0.sql:

UPDATE LibraryFileAlias SET content=NULL
FROM LibraryFileContent
WHERE LibraryFileAlias.content = LibraryFileContent.id
    AND deleted IS TRUE;

so, could it be that LFA.content.deleted was True?

I can certainly reproduce the exact error locally by:

1. Open https://launchpad.dev/ubuntu/+source/commercialpackage/1.0-1/+build/29

2. bin/iharness with the following:

{{{
In [1]: from canonical.launchpad.database import LibraryFileAlias

In [2]: lfa = LibraryFileAlias.get(70)

In [3]: lfa.content = None

In [4]: import transaction;transaction.commit()

}}}

3. Reload https://launchpad.dev/ubuntu/+source/commercialpackage/1.0-1/+build/29

Right, that's the same error. But I'd like to know _why_ lfa.content is None. If this is to be expected, i.e., if the affected records had lfa.content.deleted == True before the schema change, then the affected code must be fixed; otherwise, I'd like to know at first, what else caused lfa.content to become None.

On Fri, Dec 11, 2009 at 10:13 AM, Abel Deuring
<email address hidden>wrote:

> Right, that's the same error. But I'd like to know _why_ lfa.content is
> None. If this is to be expected, i.e., if the affected records had
> lfa.content.deleted == True before the schema change, then the affected
> code must be fixed; otherwise, I'd like to know at first, what else
> caused lfa.content to become None.
>
>
Right. It is certainly old content - according to:

https://edge.launchpad.net/ubuntu/+source/sysvinit

the release in question is over 3 years old (from dapper). I've grepped
('\.content = None') and like you can't see any other points where the
content is explicitly set to None, so I'll chase up the deleted being set to
True - thanks for the tip Abel!

On Friday 11 December 2009 09:13:43 Abel Deuring wrote:
> Right, that's the same error. But I'd like to know _why_ lfa.content is
> None. If this is to be expected, i.e., if the affected records had
> lfa.content.deleted == True before the schema change, then the affected
> code must be fixed; otherwise, I'd like to know at first, what else
> caused lfa.content to become None.
>

It would have been when we obsoleted old distroseries and then ran the
librarian GC to force a librarian cleanup.

So, given that dapper is a long-term-service and not yet obsoleted (security on the server for 5 years), we're still not sure why but the current theory is that the data had been corrupt for a long time (with LFC.deleted == True), but it was only with the recent patch-2207-15-0.sql that the LFA.content was set to None (with the LFC.deleted column being dropped in the same patch).

According to the column comment (for LFC.deleted, which no longer exists):

COMMENT ON COLUMN LibraryFileContent.deleted IS 'This file has been removed from disk by the librarian garbage collector.'

It would suggest that the binaries themselves were already removed from disk, but the lfc was still around with deleted=True? Now that it has been deleted by the db patch, there is nothing in the librariangc to deal with an lfa without content (it removes lfas that are unreferenced in the DB, or LFCs that aren't referenced by any LFAs).

As it turns out, there are *lots* of LFAs with null content currently in the DB (over 1 million)? We would need to look at a backup of the DB to see if all those LFAs did in fact have LFC.deleted==True.

But back to this actual bug, I'll ensure that we do not present lfas for a build's binary packages that have been deleted (ie. have no content).

Fixed in devel r10047 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/10047>