duplicate orig for "linux" package in hardy

The issue is that pub 277962's linux_2.6.24.orig.tar.gz was somehow replaced with a different one. Archive.getFilesAndSha1s assumes that there is only one hash for a particular file in an archive, and is sensitive to DB ordering when there are multiple.

The best fix is probably to identify and expire all the conflicting files, since none of them should be published any more.

We don't expire old files until the series is obsolete.

Thanks for the analysis William.

We don't normally, no. But these are broken, so why not do it manually and unbreak the DB?

So looking at this a bit more, the assumption that "there is only one hash for a particular file in an archive" is correct - that should always be the case, so what Archive.getFilesAndSha1s() and the package copier is doing is also correct. Sort order has no bearing on this at all.

The problem is that there is a conflicting file *already in the archive*. How that got there is what I am most interested in now.

Kees, do you have any idea how we'd end up with two different linux_2.6.24.orig.tar.gz files?

lpmain_staging=> select distinct LibraryFileContent.sha1 from LibraryFileAlias, LibraryFileContent, SourcePackagePublishingHistory, SourcePackageReleaseFile, SourcePackageRelease
where SourcePackagePublishingHistory.archive = 1and SourcePackageRelease.id = SourcePackagePublishingHistory.sourcepackagerelease
and SourcePackageReleaseFile.sourcepackagerelease = SourcePackageRelease.id
and LibraryFileAlias.id = SourcePackageReleaseFile.libraryfile
and LibraryFileAlias.filename = 'linux_2.6.24.orig.tar.gz'
and LibraryFileContent.id = LibraryFileAlias.content;
                   sha1
------------------------------------------
 b7b63f52551f9e4bf2e5ba902f8385fbefc5d80c
 ccccdc4759fd780a028000a1b7b15dbd9c60363b
(2 rows)

The md5s might be more useful since that's what's in the UI:

               md5
----------------------------------
 e4aad2f8c445505cbbfa92864f5941ab
 f09806748f6809d5f7d2a1859240e1a5

The one that you say is broken is this source:
https://edge.launchpad.net/ubuntu/+source/linux/2.6.24-5.9

This happened again today when trying to publish a security update for hardy-security. I tried both with our unembargo script and on cocoplum:

$ LPCONFIG=production /srv/launchpad.net/codelines/soyuz-production/scripts/ftpmaster-tools/unembargo-package.py -p ubuntu-security -d ubuntu -s hardy-security linux
...
2010-11-29 23:24:13 ERROR linux 2.6.24-28.81 in hardy (linux_2.6.24.orig.tar.gz already exists in destination archive with different contents.)
Confirm this transaction? [yes, no] no

This is preventing publication of an important kernel security update.

Ok, spm cowboyed a very temporary fix for me for the kernel publication (and quickly removed it). It would be nice if this could get resolved before the next kernel update.

http://pastebin.ubuntu.com/516480/ is the patch we've used for the last two incidents.

We can either fix the data or fix the hash check to only respect the latest hash for each filename.

Jamie, any answer to my question in comment #4?

On Tuesday 30 November 2010 10:02:59 you wrote:
> Jamie, any answer to my question in comment #4?

Ah wgrant just reminded me that we are accepting duplicate files after the old
file is deleted. That should not happen.

Julian,

I don't know how 2.6.24-5.9 got a different orig.tar.gz. That seems to have happened some time in the Hardy development cycle (ie, outside of any security team updates) since the release kernel for Hardy is 2.6.24-16.30 and iirc gutsy had 2.6.22. https://launchpad.net/ubuntu/+source/linux/+publishinghistory is giving me timeout errors so I can't really do more than guess atm.

Thanks Jamie. Don't worry about it for now, I'm landing a change that will
prevent anyone from doing it again.

Fixed in stable r12009 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12009>.