config change of 'origin' fails to remove old PPAs

After

$ juju config landscape-client origin=ppa:landscape/19.01

On a random unit, I see this:

root@juju-f20ba5-15-lxd-1:/etc/apt/sources.list.d# ls -l landscape*.list
total 24
-rw-r--r-- 1 root root 134 May 31 15:17 landscape-ubuntu-17_03-bionic.list
-rw-r--r-- 1 root root 134 May 31 15:17 landscape-ubuntu-19_01-bionic.list
root@juju-f20ba5-15-lxd-1:/etc/apt/sources.list.d#

The charm should remove the old PPA.