Use batch methods for multi-row grid actions

Another example:

1. Do a patron search that returns a large number of results (e.g. all patrons where profile = "Users").
2. Select all rows.
3. Add To Bucket.

There will be one open-ils.actor.container.item.create call per row, which can exhaust open-ils.actor drones pretty quickly with a sufficiently large number of rows.

+1 to batch API calls.

A potential stop-gap would be to modify the JS to serialize the API calls instead of blasting them all at once. This can typically be done with only minor JS changes.

Here's a branch to serialize the patron-bucket-add scenario:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1896285-user-bucket-batch-add

Here's one for mark missing:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1896285-batch-mark-missing

Thanks, Bill. Unfortunately, after applying those branches in a test environment, both my test cases still max out available drones and give "no children available" warnings in the logs.

Thanks for testing, Jeff. Hmm, I'm seeing different results.

I just re-tested the patron bucket scenario:

I set up my VM with max-children=5 for open-ils.actor. Loaded up a patron search with 30 results. At this point I have 2 actor drones alive. Added all of the patrons to a bucket and the number of actor drones rose by exactly one, stopping at 3, which is what I would have expected, given that each call waits in the previous call to complete.

Can you set up a similar test and see now many new actor drones are created in this scenario?

Seeing trouble in PINES with this call in the activity log:

2020-12-14 09:24:11 brick04-head gateway: [ACT:102767:osrf-websocket-stdio.c:559:16079558091027671352] [127.0.0.1] [] open-ils.actor open-ils.actor.ou_setting.ancestor_default.batch "188", ["cat.default_copy_status_normal"], "<REDACTED>"

In our case, it caused two separate system outage events in one day.

Fix for the issue described above by Chris Sharp:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1896285-add-item-setting-lookup-loop

The patch for patron bucket add from comment #3 is working for me on a test VM running 3.2.10. I assume it will work on any later release, too. I'll refrain from adding a sign off branch because Chris Sharp mentioned in IRC that he's also looking at these patches.

If anyone wants to add my signoff to that one, I consent to signing off on the branch with my name and email address: Jason Stephenson <email address hidden>.

I have been running all three of the above branches in PINES production for the last day and can vouch for their working - both not affecting anything adversely and definitely seeing fewer system calls resulting in crazy drone explosions. I'm interested in what our OP Jeff Davis thinks about where we are on this before submitting a signoff branch?

Upon re-testing, the patron bucket add patch is working for me too. I must have done something wrong during my initial test. I think we can assume that one is working just fine.

However, the batch mark missing patch still does not work for me. With the patch applied, when I go through the test steps from the original bug report, I still end up with maxed-out actor drones and "no children available" warnings in logs. It takes a few seconds for the drones to max out -- perhaps the issue is with the post-action grid reload, rather than (or in addition to) the actual action of marking the items as missing? I'm attaching a CSV file with my test set of 100 barcodes to be marked as missing.

Thanks, Jeff. I'm looking closer at the Missing use case.

Your instincts were correct, Jeff. The post-action copy loading was running all in parallel. I've pushed another commit to the same mark-missing branch to address this. I also tweaked it so it waits until all of the data has been fetched before it refresh the grid to avoid the grid flickering.

Thanks Bill, that does the trick for me -- with both commits I no longer see a high drone count when marking many items as missing.

Working branch user/jeffdavis/lp1896285-batch-mark-missing has a signoff for the two commits, plus an additional commit adding a bare progress dialog (since the grid can take a little while to reload).

I removed the 3.4 target on this bug because 3.4.6 is going to be a security release, and I don't think this is a security bug.

Here are signoff branches:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/csharp/lp1896285-user-bucket-batch-add-signoff

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/csharp/lp1896285-batch-mark-missing-signoff

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/csharp/lp1896285-add-item-setting-lookup-loop-signoff

Thanks Chris and Jeff.

Code merged to master and 3.6

I had a merge conflict in 3.5 that I was able to fix but unable to test, so I have pushed a separate branch for 3.5 in the hopes someone with 3.5 handy can give it a once over. Specifically, this commit resulted in the conflict:

https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=0a7eef2a73ddf43dd6dfd313980eb56419832415

3.5 Branch:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1896285-batch-grid-actions-serialize-combo-3.5

We've had the 3.5 branch in production for about a week. Since then we haven't had any problems that I can attribute to the user actions/UIs addressed here, although we've had a couple of instances of max drones that I believe are due to other causes. I think the fixes can be committed to rel_3_5.

See also bug 1912834 (limiting the number of parallel requests from the client).

We've had the 3.5 patch running for 2 days without issue. My signoff

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/blake/LP1896285_3_5

3.5 version pushed to rel_3_5.