Deleted user still can delete volumes in Horizon

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

Out of curiosity, are you using caching in any of the following areas:

Horizon/Django
Keystonemiddleware (for any of the services)?

It could be possible the token response has been cached. If that is the case, even if the user is deleted, if a service does not check with keystone, it is impossible to know if the user/data/etc has been deleted.

In the past the behavior I just described was highlighted in the documentation but that information may have disappeared in a number of documentation/guide updates.

Let us know and we can work to further dig into the issue/duplicate/add documentation (depending on what is needed).

Thanks!

Horizon (openstack_auth module) stores a token in Django session store.
That's the reason that a user can access services even after the user id deleted.
The token stored in Django session store is used to access back-end services.

A user can be deleted via both CLI and Horizon.
(a) If a user is deleted via horizon, horizon can clear a session and the deleted user cannot access back-end services immediately after deleting the user.
(b) If a user is deleted via CLI (or other ways that horizon is not involved), there is no way that horizon can know it, so the deleted user can continue to access services.

Horizon has a setting SESSION_TIMEOUT (which defaults to 3600) [1]. If the mininum time of the keystonemiddleware cache time and the horizon session timeout passes, the deleted user no longer can access services. As of now, keystonemiddleware cache time is 300 sec and horizon's SESSION_TIMEOUT is 3600 sec by default, so the situation reported here continues for 300 sec after the user is deleted.

I think it is reasonable to keep the current behavior from the performance perspective.
As Morgan commented, we can improve our documentation to highlight this behavior clearly.

[1] https://docs.openstack.org/horizon/latest/configuration/settings.html#session-timeout

Arthur: Do Morgan and Akihiro's comments explain the behavior you observed? If so, I think we can switch this bug to public and work on better documenting the token cache control options, to make it clear to deployers and operators how to strike a balance between increased security and increased performance.

Hello.
Morgan and Akihiro, thank you for your answers.

In this deployment I use Memcached to cache Keystone's Fernet tokens (default configuration). Also I configured default memcached option in Horizon: django.core.cache.backends.memcached.MemcachedCache (as it is suggested in the Stein's installation guide).

It would be great, if you could add some information about caching in docs.

Thank you for pointing the SESSION_TIMEOUT option. I was looking through Horizon options to mitigate this problem and thought about using it.

So is the default keystonemiddleware cache expiration time in such deployment equals to 300 sec? Although I can look up token's expiration time, issuing "openstack token issue" command.

If this is expected behavior, then I don't have any further questions)
Thank you.

Hi Jeremy.
Yes, their comments helped me understand this situation.

Thanks, I'm marking our security advisory task "won't fix" and lifting the private embargo, treating this as a class D report indicating a need for documentation improvements: https://security.openstack.org/vmt-process.html#incident-report-taxonomy

> Thank you for pointing the SESSION_TIMEOUT option. I was looking through Horizon options to mitigate this problem and thought about using it.

For more detail, there are two options involved in horizon.
- SESSION_TIMEOUT
- SESSION_REFRESH
If SESSION_REFRESH is set to True (current default), a shorter SESSION_TIMEOUT would not matter for most cases.

> So is the default keystonemiddleware cache expiration time in such deployment equals to 300 sec? Although I can look up token's expiration time, issuing "openstack token issue" command.

The default value is defined here [1].
You can test the timeout of keystonemiddlware cache using curl.
[2] is an example in my devstack environment. L.39-49 retrieves a token before the user is deleted and tests it works. L.81 confirms the token is still valid just after the user is deleted. I confirmed the curl command failed with auth error a couple of minuites later (though the paste does not cover it). You can try the similar.

[1] https://opendev.org/openstack/keystonemiddleware/src/branch/master/keystonemiddleware/auth_token/_opts.py#L107-L112
[2] http://paste.openstack.org/show/777693/

I am adding keystone to the affected projects as I failed to find a good pointer on this behavior in the keystone documentation. If there is a good point in keystone already, it would be appreciated. I will refer to it in the horizon doc.

Akihiro, thank you for detailed explanation.

The same behaviour should be reproducible via command line client.

This issue is inherent if you use tokens, which are valid for some time, and services only validate the token itself, but not (via keystone), if the token has become invalid.

Added Keystonemiddleware and documentation tags. Marked as "medium" importance as it requires documentation changes but is not critical/RC/otherwise impacting. Clear communication of expected behavior is important and should be found in Horizon and Keystonemiddleware's documentation.

I am marking invalid for Keystone itself as keystone will invalidate it's internal cache (barring cases such as in-memory [not production quality] dict-base cache).

Changing horizon priority to Medium (as the priority in keystone is medium and there is no reason to use higher priority in horizon).

I am clearing my assignee in horizon as I don't see a progress in the keystone side.