Add ability to prevent image updates for a given image

Yes, not being able to control the image versions creates a situation where reproduction of failed systems becomes impossible or very difficult.

SimpleStreams contains multiple versions per product. MAAS however only works with the latest version. The version is only modeled for products that have been downloaded, they are not modeled in the table used for selection.

We could add something so MAAS doesn't update a product. However I have a couple of concerns with that.
1. The version streams use is based on the date, not product version.
2. The MAAS stream keeps a max of 5 versions per product. Older versions are removed. So if you freeze a product and then do another MAAS deployment you most likely won't be able to get the same version.
3. MAAS doesn't have a way to download older versions even if they are in the stream.
4. We would need a way to make it clear a product is frozen in both the UI and API.

I've thought for awhile that MAAS needs an overhaul of its image import code for awhile. MAAS should allow users to have multiple versions of a product downloaded and be able to select which version to deploy at deployment time. We've also had requests from multiple users to move images out of Postgres.

I think we'd have to get the overhaul on the roadmap so we can fully discuss what the requirements are and spec out what changes need to be made. In the meantime using a custom image or creating a mirror of the stream are the only supported ways to do this.

Side note: Minor version changes to CentOS are expected to be fully backwards compatible with previous versions. CentOS 7.7 to 7.8 should be equivalent to going from Ubuntu 18.04.1 to 18.04.2. If that isn't the case I would file a bug with CentOS.

@ltrager I appreciate the insightful response, and I agree, the image management needs an overhaul. For us, our infiniband drivers had upstream support for the kernel used in centos7.7 but not 7.8. Because of the image auto-sync we got caught red handed in the middle of this. MLNX offers the drivers for centos7.8 from their upstream repo, but they just aren't compatible with the kernel. While the fix wasn't too bad (we now have to build and send our mlnx-ofed built for the specific kernel that we will be running on), I do feel like we are exposed to this sort of situation without some sort of image lock in place. Possibly it isn't centos7.7 -> 7.8 that broke us, but rather a mlnx-ofed vs kernel version. Either way, I feel that if we had the capability to lock our image we would have been able to avoid this happening in the live environment.

MAAS should allow users to have multiple versions of a product downloaded and be able to select which version to deploy at deployment time. - Agreed

It seems dead on that this should sync up with the add-image work being done in juju right now. Currently, juju only supports add-image when using a public cloud. I have been pushing for the add-image functionality to be extended to support user uploaded/custom images in MAAS.

Possibly this work can help lay the foundation for `juju add-image` support for the MAAS provider.

I agree with James above, this is a very common enterprise request.
My previous employers have specified various workflows to prevent minor version bug issues as he outlined above.
Some will run exactly zero updates until snapshotting a virtual machine, some will disable all automatic updates and require an early week scheduled time to run updates, and some will just not run newer versions than they test.

Not that I necessarily agree with all of those methods, but I'd like to emphasis that this is not some weird edge case but version locking is very much standard procedure in many enterprise environments, and should absolutely be a supported option.

Auto updating images have also bitten us on the Ubuntu commissioning OS side of things as well:
https://bugs.launchpad.net/maas/+bug/1965587
https://bugs.launchpad.net/maas/+bug/1966343
Were both caused by *something* being updated in Focal that broke all CentOS deployments, so while I agree that minor updates shouldn't cause issues like this, they clearly do even for Ubuntu.
Administrators should be empowered to test newer versions as they become available and select the latest working version, file bug reports when things break, and then choose to update to the next working version.