+filebug pages time out when searching for duplicates

what do you think is causing this to happen?

On Jan 26, 2008 3:41 PM, Graham Binns <email address hidden> wrote:

> ** Description changed:
>
> When people are trying to file bugs they are hitting timeouts on the
> - +filebug pages. Specifically referencing this are OOPS-757D35 and OOPS-
> - 757C47.
> + +filebug pages. Specifically referencing this are OOPS-756C4447, OOPS-
> + 757D35 and OOPS-757C47.
>
> --
> +filebug pages time out
> https://bugs.launchpad.net/bugs/186262
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
http://www.topicks.info
http://webgoodness.blogspot.com

I wonder if this is related to the removal of duplicates in BugTarget.similar_bugs that we released a few days ago. We are using [:4*self._MATCHING_BUGS_LIMIT] to limit the number we process, and we break we we reach matching_bugs_limit, so we should not be doing much more work than we were before.

This is directly related to bug #119780. We're issuing a query to find similar bugs which is, as a result of bug #119780, inefficient in and of itself. However, we're issuing it multiple times and so the process of querying isn't able to complete before the timeout is encountered.

thnx

On Jan 27, 2008 10:05 PM, Graham Binns <email address hidden> wrote:

> This is directly related to bug #119780. We're issuing a query to find
> similar bugs which is, as a result of bug #119780, inefficient in and of
> itself. However, we're issuing it multiple times and so the process of
> querying isn't able to complete before the timeout is encountered.
>
> --
> +filebug pages time out
> https://bugs.launchpad.net/bugs/186262
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
http://www.topicks.info
http://webgoodness.blogspot.com

Another user got OOPS-781EA26 in +filebug. In this instance it's a single query that causes the time-out, taking over 30s to execute, but it looks like it's related to the FTI stuff again.

Got OOPS-1018EC130 on edge and OOPS-1018B3331 on lpnet. Tried on staging and it seemed to work okay. Error seems repeatable, i.e. trying with the same text produces another OOPS, so I'm currently unable to file this bug. The text I'm filing with is:

Intrepid: cannot print to Samsung ML 1710 printer - cups-missing-filter

I also encountered a timeout on this page. The summary I entered was "Twisted 8.1 fails to close TLS connections with pyOpenSSL 0.7". I did manage to get to a bug report page after several timeouts. The last OOPS id I got was OOPS-1019B3074.

I've run into this on a regular basis through-out the jaunty cycle. Several critical bugs have been lost due to this. Please make this a high priority, lots of critical crash bugs are missed because of this issue. In particular, more than half on my Firefox crashes ever get transmitted because I always have lots of TABs and normally the crash dumps are too big for apport/malone to handle. Below is the error I got:

Timeout error

Sorry, something just went wrong in Launchpad.
We’ve recorded what happened, and we’ll fix it as soon as possible. Apologies for the inconvenience.
Trying again in a couple of minutes might work.

(Error ID: OOPS-1194D3715)

martin, I've reported your OOPS as bug 357907. This bug was originally about +filebug spending a lot of time doing db queries, so let's keep it focused on that.

The problem with large crash reports is better tracked in bug 357907.

Ah, sorry for high-jacking the bug.

You might want to consider searching for and displaying the duplicates asynchronously and poppnig them into the form when they are found. I don't know if we will be able to remove all the edge cases in this complex search, but if AJAX was used problems with the search don't break the work flow. The entire page will load faster too - if the search is being done explicitly using a slave database, your AJAX requests can be running in parallel on multiple cores on multiple database backends rather than the current serial process.

> You might want to consider searching for and displaying the duplicates
> asynchronously and poppnig them into the form when they are found.

And that's something we wanted to do anyway, as part of the work on improving +filebug.

This is becoming more and more of a problem lately, and given our zero OOPS policy now, we have to do something about making this experience better and fixing any issues that we can.

One workaround for this is to reduce the number of words in the title. The problem occurs because Launchpad does a full text search against a very big set of bug titles, and the more words in your bug title the longer that search will take.

If you reduce the number of words you use in your initial title you will increase the risk of filing a duplicate, but I don't think you need worry about that overmuch. Duplicates are cheap; better to file a dupe than no bug at all.

In order to fix this we need to land the code for asynchronous dupe searches (bug 358510). I'm tempted to mark this as a duplicate of that bug, because that would substantially fix the problem.

Still can't enter my Ubuntu bug...

@amay82 On the first page when filing your bug, try just entering "zzzzzzzzzzzzzzzzzzzzzzzz" for the summary. This should mean that no duplicates are found, and, crucially, not found *quickly* so that the page does not time out. Don't forget to change the summary to something useful on the next page before submitting the report.

The async-dupefinder work should fix this bug. If not, we'll open a new one, since the timeouts will be occurring elsewhere.

More recent examples of this bug OOPS-1438B1945, OOPS-1438A1705, OOPS-1438A1600, OOPS-1438A2555

2009/12/11 Diogo Matsubara <email address hidden>:
> More recent examples of this bug OOPS-1438B1945, OOPS-1438A1705, OOPS-
> 1438A1600, OOPS-1438A2555
>

Right. FTR, all of those happened before the inline work landed. I'm
not guaranteeing that it will fix all the timeouts - far from it; we
need to make the dupe search algorithm / query more efficient to stop
that.

However, I think that once we stop seeing timeouts on +filebug (and if
we're still seeing them after next week we should have a good look at
*why* we're still seeing them) and start seeing them on
+filebug-show-similar (which I'm sure we will) we should track that in
a separate bug. The main problem with timeouts in +filebug was that
the user couldn't subsequently file a bug. With the inline dupe
search, even with a timeout, the bug can still be filed.

I'm agreed with Graham here. The timeouts on the +filebug page should be fixed now. We should open a new bug if/when OOPS start appearing for +filebug-show-similar and track that work separately from this bug. So I'm marking this one Fix Committed.

You are absolutely right, the problem always trouble me when searching for duplicates. thanks for suggesting nice review

- - - Txtimpact- - - -
<a href="http://www.txtimpact.com">Shared Short Code</a>