neutron agent. list API lacks sort and page feature

Bug #1877254 reported by yong sheng gong
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
neutron
New
Medium
Unassigned

Bug Description

We have hundreds of neutron agents deployed, but currently, the agent list API does not support sorting and paging, which is very bad.

Revision history for this message
Slawek Kaplonski (slaweq) wrote :
Download full text (7.5 KiB)

What version are You using? I just checked on RHOSP-16 which is based on stable/train and the result is:

(overcloud) [stack@undercloud-0 ~]$ neutron agent-list --sort-column agent_type
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+--------------------+---------------------------+-------------------+-------+----------------+---------------------------+
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
+--------------------------------------+--------------------+---------------------------+-------------------+-------+----------------+---------------------------+
| 643a494d-ad06-4455-9c6b-6ad35f7b2f5e | DHCP agent | controller-2.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 8a092f5b-f415-4473-ac8e-9ca333b037bc | DHCP agent | controller-0.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 8a384ede-4ccb-4321-b170-2448344b92fd | DHCP agent | controller-1.redhat.local | nova | :-) | True | neutron-dhcp-agent |
| 2ec9c0d6-57d8-4803-9b44-942542e53d6e | L3 agent | controller-0.redhat.local | nova | :-) | True | neutron-l3-agent |
| 758c19ef-0bb3-48bb-a757-90956b830914 | L3 agent | controller-1.redhat.local | nova | :-) | True | neutron-l3-agent |
| 9cec9036-0d0f-432a-a0ea-4420f1e88921 | L3 agent | controller-2.redhat.local | nova | :-) | True | neutron-l3-agent |
| b0d7fb33-607a-4903-9bf3-987743f237a7 | L3 agent | compute-1.redhat.local | nova | :-) | True | neutron-l3-agent |
| bf84115b-fc33-41c6-8734-2cbe1c9a7a5c | L3 agent | compute-0.redhat.local | nova | :-) | True | neutron-l3-agent |
| 386111f1-563c-4a2c-9b41-b292a1aa876e | Metadata agent | compute-0.redhat.local | | :-) | True | neutron-metadata-agent |
| 43d33cd1-c817-4755-8628-e759402acc5b | Metadata agent | controller-0.redhat.local | | :-) | True | neutron-metadata-agent |
| ad765ce7-f2e2-4db3-8ff7-1765e5e5bb93 | Metadata agent | controller-2.redhat.local | | :-) | True | neutron-metadata-agent |
| c832081f-d598-483c-bb57-4f616e631fbb | Metadata agent | controller-1.redhat.local | | :-) | True | neutron-metadata-agent |
| dbd82bdb-43d3-46de-99bc-a8364f260457 | Metadata agent | compute-1.redhat.local | | :-) | True | neutron-metadata-agent |
| 274076b0-56b1-407a-9903-2a6529d6be77 | Open vSwitch agent | compute-1.redhat.local | | :-) | True | neutron-openvswitch-agent |
| 98ed6974-b379-4eb6-8267-d76cd38f9379 | Open vSwitch agent | controller-0.redhat.local | | :-) | True | neutron-openvswitch-...

Read more...

Miguel Lavalle (minsel)
Changed in neutron:
status: New → Incomplete
importance: Undecided → Low
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-lib (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/726497

Revision history for this message
yangkai (yangkaiopenstack) wrote :

Check on stable/train and the result is:

`neutron agent-list --sort-column binary` is sorted by neutronclient

`neutron agent-list --sort-key binnary` will be KO

DEBUG: keystoneauth.session REQ: curl -g -i -X GET http://overcloud.internal.vim1.local:9696/v2.0/agents?sort_key=binnary&sort_dir=asc -H "Accept:
 application/json" -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA256}186f3682c2ce0e17e2e3e74077285d319e5347ef6fca7fb78ccd6554fd21473
e"

DEBUG: neutronclient.v2_0.client Error message: {"NeutronError": {"message": "[u'binnary'] is invalid attribute for sort_keys", "type": "HTTPBadRequest", "detail": ""}}

So we need implement it in neutron-server

Revision history for this message
Miguel Lavalle (minsel) wrote :

I am testing in near master (built it about 6 weeks ago). I can see that the API behavior is inconsistent. If I do:

$ neutron net-list --sort_key name --sort_dir asc

or

$ neutron net-list --sort_key name --sort_dir desc

I get the networks sorted by name correctly, as indicated by the direction argument

If I do:

$ neutron net-list --sort_key name --debug

I get:

DEBUG: keystoneauth.session RESP: [400] Connection: keep-alive Content-Length: 123 Content-Type: application/json Date: Sun, 10 May 2020 17:42:41 GMT X-Openstack-Request-Id: req-3aa8373d-1d00-4a89-b5a5-a14782aa3a46
DEBUG: keystoneauth.session RESP BODY: {"NeutronError": {"type": "HTTPBadRequest", "message": "The number of sort_keys and sort_dirs must be same", "detail": ""}}
DEBUG: keystoneauth.session GET call to network for http://192.168.33.12:9696/v2.0/networks?sort_key=name used request id req-3aa8373d-1d00-4a89-b5a5-a14782aa3a46

Now, in the case of:

$ neutron agent-list --sort_key binary --sort_dir asc

$ neutron agent-list --sort_key binary --sort_dir desc

$ neutron agent-list --sort_key binary

$ neutron agent-list --sort_key binnary

$ neutron agent-list --sort_key binnary --sort_dir asc

I get the exam same list all the time. So the API is just ignoring the sort arguments.

This bug is related to https://bugs.launchpad.net/neutron/+bug/1749820, which was marked as closed. So let's keep this one and continue the conversation here

Changed in neutron:
status: Incomplete → Triaged
Revision history for this message
Miguel Lavalle (minsel) wrote :
Changed in neutron:
importance: Low → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/726724

Revision history for this message
Slawek Kaplonski (slaweq) wrote :

I marked this bug as low-hanging-fruit bug as IMO it is something which should be easy to fix. If someone wants to work on it, feel free to join #openstack-neutron channel @freenode and ask about any help You need (You can ping e.g. slaweq or mlavalle there).
Also I think that https://github.com/openstack/neutron/blob/master/neutron/db/agents_db.py#L298 is probably place to start looking for a fix as it probably should accepts also sorts, limits, marker and page_reverse parameters.

tags: added: api low-hanging-fruit
Revision history for this message
Slawek Kaplonski (slaweq) wrote :

Related patch is proposed https://review.opendev.org/#/c/726724/

Changed in neutron:
assignee: nobody → vinay harsha mitta (vinay7)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-lib (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/738471

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/738475

Changed in neutron:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-lib (master)

Reviewed: https://review.opendev.org/738471
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=5b035972e567e1426657218c52b0ebc06860d3ef
Submitter: Zuul
Branch: master

commit 5b035972e567e1426657218c52b0ebc06860d3ef
Author: vinay_m <email address hidden>
Date: Mon Jun 29 21:22:54 2020 +0530

    Neutron agent-list API added with sort_key keyword

    Defining a API Extention for enabling sorting in agent
    -list attributes.

    Added "is_sort_key" keyword for attributes of agent.
    Neutron API can use "sort_key" and "sort_dir" for sorting on resulted
    list.

    Change-Id: I3155a4f1de664e4ff8e32ccc4314927ea462048c
    Related-Bug: #1877254

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by vinay harsha mitta (<email address hidden>) on branch: master
Review: https://review.opendev.org/738475

Revision history for this message
Abhishek Mahajan (mahajan-abhishek) wrote :

Related fix proposed to branch: master
https://review.opendev.org/#/c/743953/

Revision history for this message
Slawek Kaplonski (slaweq) wrote : auto-abandon-script

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: vinay harsha mitta (vinay7) → nobody
status: In Progress → New
tags: added: timeout-abandon
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/743953
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Changed in neutron:
status: New → In Progress
Changed in neutron:
assignee: nobody → Heather Lemon (hypothetical-lemon)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/791044

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron-lib (master)

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/726497
Reason: This patch has not been attended in 1 year, if needed, you can restore it again.

Revision history for this message
Heather Lemon (hypothetical-lemon) wrote :

I am currently still working on this. There are failing zuul tests that need fixed first.
https://review.opendev.org/c/openstack/neutron/+/791044

Revision history for this message
Slawek Kaplonski (slaweq) wrote : auto-abandon-script

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: Heather Lemon (hypothetical-lemon) → nobody
status: In Progress → New
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/743953
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Slawek Kaplonski (slaweq) wrote : auto-abandon-script

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/791044
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/726724
Reason: Depends-on patch was abandoned. Please feel free to restore this patch and the neutron-lib one.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.