Cve:+index timeouts

406->58 queries for the first url, times out cold on qastaging, 5 seconds hot.
506->??? for the second url - I couldn't get it to render. it doesn't on prod either though so strictly an improvement.

OOPS-1904QS55 from staging may shed light on the cause.

hah, I was testing the unfixed code ><

Fixed in stable r12628 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12628>.

ok, still can't get https://qastaging.launchpad.net/bugs/cve/2009-0745/+index to render: OOPS-1904QS73

both referenced CVE's timed out in prod, updating the description with the new OOPS ids

Note that the total SQL time may look low, but IME this is a symptom of late evaluation - query compilation etc times drive this up, and making things eager load, cacheproperty etc will often drive the non-sql time down too.

so, this is a problem page for the same reason bugtask:+index is I think; I need to check but I suspect its reusing the same nested template + per-row view logic - see https://bugs.qastaging.launchpad.net/++profile++show/bugs/cve/2010-4080/+index

OOPS-1909EA1446

notes:
lib/lp/bugs/browser/buglinktarget.py

  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/buglinktarget.py", line 87, in buglinks
    for bug in self.context.bugs:

 - that triggers

SELECT Bug.date_last_message, Bug.date_last_updated, Bug.date_made_private, Bug.datecreated, Bug.description, Bug.duplicateof, Bug.heat, Bug.heat_last_updated, Bug.id, Bug.latest_patch_uploaded, Bug.message_count, Bug.name, Bug.number_of_duplicates, Bug.owner, Bug.private, Bug.security_related, Bug.title, Bug.users_affected_count, Bug.users_unaffected_count, Bug.who_made_private FROM Bug, BugCve WHERE BugCve.cve = 1 AND Bug.id = BugCve.bug ORDER BY Bug.id

which doesn't have eager loading attached; possibly we need that to be a cachedproperty or some such.

  File "/home/robertc/launchpad/lp-branches/working/lib/lp/services/propertycache.py", line 116, in __get__
    value = self.populate(instance)
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py", line 3500, in target_has_milestones
    return len(self._visible_milestones) > 0
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/services/propertycache.py", line 116, in __get__
    value = self.populate(instance)
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py", line 3474, in _visible_milestones
    return self.milestone_source(self.context).visible_milestones
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/services/propertycache.py", line 116, in __get__
    value = self.populate(instance)
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/registry/vocabularies.py", line 1241, in visible_milestones
    milestones = shortlist(milestones_source, longest_expected=40)
  File "/home/robertc/launchpad/lp-branches/working/lib/canonical/launchpad/helpers.py", line 320, in shortlist
    results = list(sequence[:last])

SELECT Milestone.active, Milestone.codename, Milestone.dateexpected, Milestone.distribution, Milestone.distroseries, Milestone.id, Milestone.name, Milestone.product, Milestone.productseries, Milestone.summary FROM Milestone WHERE Milestone
.distribution = 1 AND Milestone.active = True ORDER BY milestone_sort_key(Milestone.dateexpected, Milestone.name) DESC

- 109 milestone lookups on a local test - and this comes from the
lib/lp/bugs/browser/../templates/bugtask-tasks-and-nominations-table-row.pt template looking to see whether to show the milestone widget

  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py", line 3512, in user_can_edit_importance
    return self.context.userCanEditImportance(self.user)
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/model/bugtask.py", line 1229, in userCanEditImportance
    return (self._userIsPillarEditor(user) or
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/bugs/model/bugtask.py", line 1220, in _userIsPillarEditor
    pillar.userCanEdit(user))
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/registry/model/distribution.py", line 1774, in userCanEdit
    return user.inTeam(self.owner) or user.inTeam(admins)
  File "/home/robertc/launchpad/lp-branches/working/lib/lp/registry/...

> so, this is a problem page for the same reason bugtask:+index is I think;
> I need to check but I suspect its reusing the same nested > template + per-row view logic
> - see https://bugs.qastaging.launchpad.net/++profile++show/bugs/cve/2010-4080/+index

You're correct; it uses the bugtasks-nominations-tasks-row template, which triggers multiple milestone lookups via a very peculiar function in the milestone vocabulary.

On Sat, Mar 26, 2011 at 3:08 AM, j.c.sackett <email address hidden> wrote:
>> so, this is a problem page for the same reason bugtask:+index is I think;
>> I need to check but I suspect its reusing the same nested > template + per-row view logic
>> - see https://bugs.qastaging.launchpad.net/++profile++show/bugs/cve/2010-4080/+index
>
> You're correct; it uses the bugtasks-nominations-tasks-row template,
> which triggers multiple milestone lookups via a very peculiar function
> in the milestone vocabulary.

It actually seems to be worse than bugtask:+index - probably because
that has lots of partial optimisations and I suspect cve:+index has
been largely ignored.

So we have two tasks:
 - get cve:+index doing the same eager loading bugtask:+index already does
 - fix the milestone pathology in both places.

The latter has had me headdesking when I have looked at it for a
couple weeks, I suspect the right answer is:
 - strip stuff back to bare bones
 - rewrite the vocab on the underlying layer
 - get milestones in the view page using the underlying layer

I spoke to deryck and kees (nominated as a key user) about this and removing the edit widgets is simple and ok to do.

Fixed in stable r12704 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12704>.