Why does an automatic switch of tool occur?

Let's say I selected the tool "Text edit". If I click on a non-text(-editable) object, why does the selected tool automatically switches to the default tool "Modify Object"?

This is very annoying in cases where there are lots of (potentially interlaced) texts/groups and you miss the one you aimed, you now have to steps to do (re-select the Text Edit tool + click again but on the correct text object) to recover the situation.