Bib records that are both deleted and empty show in staff search

I should clarify, this is true for both the web staff client and XUL client.

These bibs do NOT show up in patron OPAC search, only when searching via the staff clients.

I tried this in the web client on a couple of test VMs: one was a clean master install and the other was an upgraded database.

I added a record and didn't add any holdings. I performed a search and was able to find the test record, as expected.

I deleted the record. When I performed the exact same search, I was able to retrieve the record. However, if I performed a different search, I got no results, leading me to believe that the first search was relying on cached data. In my testing, I wasn't able to replicate this bug except in those instances.

I think there was an issue in the early 3.0 days where deleted records were retrieved in staff client searches, but I can't find the bug number.

I may be seeing different behavior because I was adding the records after the new search visibility code was added to the system. It may only be an issue for records that were there before the upgrade. There were other search visibility issues that only occurred on upgraded records, not on ones that were added after the upgrade.

Kathy, if I change the search, to title and author instead of keyword, I still get the deleted records in the results.

You may be right that it has to do with records that existed before the upgrade. In that case, it seems that something is missing from the upgrade instructions. Perhaps the bib vis attr update needs to be run on deleted records, too?

I tried updating the vis_attr_vector on deleted bibs using the steps from the upgrade. It does not help. The deleted and empty bibs still show up in search. I have tried different variations from before to avoid cached results. I also stopped services in between.

We discovered this last week. The deleted records show up in or result set as if they are blank records (see attached screenshots. When you click on one to display, you just get a white screen. In order to see the deleted record, we have to click on a previous record and then page through to the deleted one.

If I do keyword or title search, the deleted records show up in the staff client (both XUL and web client 3.0.2).

If I do a TCN search in the web client, I do not retrieve the record and get a message Bib Record Not Found: {TCN}, A record ID search retrieves the deleted record.

In the XUL client, when I search for a deleted record with the TCN, a dialog box appears telling me that record is deleted and asking show the record anyway? This is preferred behavior since catalogers might have the TCN of a deleted record and need to see the record in case it was deleted in error and should be undeleted. They are unlikely to have the database ID (Our TCNs are based on the OCLC TCN, primarily)

In the XUL client, a database/record ID search retrieves the deleted record.

I am not able to reproduce this issue on a 3.0.7 system. Neither a newly added empty record which is subsequently deleted, nor an existing empty deleted record are showing up in staff searches. I tested in both the xul and web clients.

We are on 3.06 and long deleted records are appearing in both the staff & web clients. Records added & deleted after the upgrade are appearing also. Title, author,keyword & record ID searches all retrieve delete records.

Jason's initial report noted that this is happening with empty deleted bibs. For those who are confirming this bug, is it also only happening with deleted records or is it happening with other records too?

Sorry, my question should have been:

Is it also only happening with *empty* records?

I'm not sure what you mean by *empty* records. Undeleted records with no items can show in the staff client depending on how long the network wants to keep them. We keep them for 6 months from the date added for OCLC records and 1 month for brief records libraries create.
CWMARS has the same experience Pamela describes. The records are visible in both xul and web client. For us, they show in results list and link to cover images and Novelist content. We do not have the blank records Elaine describes.
I also noticed that when a record is deleted this is indicated, in both xul and web, in the record summary area by red highlight or the word "deleted" respectively and in the bib display by a red outlined message. If I undelete a record these indicators go away as intended. If I subsequently delete the record a second time only the record summary area indicates the record is deleted. The bib display does not have the red bordered message.

Forgot to say we are on 3.0.8.

I have only seen deleted records and not empty records (bib records brought into the systems where holdings are not yet attached).

We have far more deleted records than empty records so that might be a factor. Cataloging workflows generally mean empty records get processed within a week or so, although some do get left behind.

Sorry to be unclear. What I meant to say was that Jason originally reported that it was happening to records that were both deleted AND empty (no holdings). I have no concerns about undeleted records showing up in staff client searches.

However, from your recent comments, it sounds like you're all reporting that many different types of deleted records are showing up in a search, not just the empty ones.

Yes, for CWMARS it is both deleted and empty. I tried to delete a record with holdings but am not able to do so. For that the system is working correctly.
As for the delete/undelete/delete I described above, the red bordered message in the bib record display does finally appear when a record is deleted a second time, so there is a delay there.

Empty vols should show up in a staff search, so I would expect them to be in the list. But deleted records shouldn't. And it appears that the deleted records displaying are not recent deletes necessarily

Here's a branch that forces an explicit check on the deleted flag of biblio.record_entry:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp-1773832-hide-deleted-bibs

From the commit message:

There are cases where we avoid checking the deleted flag of bibs as an optimization that avoids a JOIN in the search query. However, for some shapes of real-world data, particularly in upgraded instances, deleted records can slip through. This commit adds an explicit check for the flag to all search queries.

I tested it and it worked well.

I have tested this code and consent to signing off on it with my name,
Dan Pearl, and my email address, <email address hidden> .

I tested the branch, too, and it is working for me, so I added Dan's signoff and pushed to master, rel_3_1, and rel_3_0.

Thanks, everyone!