backport fix for "OpenSSL 3 cannot decrypt data encrypted with OpenSSL 1.1 with blowfish in OFB or CFB modes" to Jammy

If I read upstream correctly, a fix for this regression was put in master over a year ago (May 2022). Any update to when this will be rolled out on Jammy/current LTS?

I plan to prepare a batch with several fixes late August/September.

I've created a PPA for Jammy that incorporates the fix mentionned. The details are available at https://launchpad.net/~adrien-n/+archive/ubuntu/openssl-jammy-sru . Could you test it and confirm your issue is solved?

Yes, libssl3 3.0.2-0ubuntu1.11~ppa2 appears to fix the Blowfish incompatibility (at least for the original case of connecting to Tinc running on old distributions of Ubuntu).

Test steps:
disabled the OPENSSL_MODULES workaround I had in place on a my Jammy node, and confirmed that Tinc was unable to connect to the remote Tinc node, and generated "Bogus data received from" syslog error messages (as expected due to this bug).

enabled the adrien-n/openssl-jammy-sru PPA, and installed the ...ubuntu1.11~ppa2 versions of openssl and libssl3.

restarted the Tinc service... and saw that the Jammy node was once again able to connect to the remote node (without the OPENSSL_MODULES workaround).

Let me know if I can provide any other information.

(Do you have an ETA for publishing this package to Jammy?)

Thanks.

Thanks a lot for taking the time to test and provide feedback.

I'll continue with the SRU process which should take a few more weeks (I'd say between two and four but that's a very rough guess).

Hello,

ubuntu-sponsors is subscribed to this bug but I couldn't find anything actionable. I'm unsubscribing ubuntu-sponsors; feel free to subscribe it again if there's anything that needs sponsoring. Thanks.

Ah, I noticed that this is part of a big SRU that's being completed on bug #2033422. Just leaving a comment here for the record.

I'm very much against uploading this fix as an SRU. While it is a shame that we have interop problems with the rest of the world, fixing this in Jammy right now means we *will* break production for anyone that stores Blowfish-encrypted data.

Doing `apt upgrade` on a production server is supposed to be a fairly routine affair, whereas running into the interop issue means someone somewhere did a dist-upgrade, changed some configuration, wrote new code, or installed a new system, all of which are actions where the user *should* expect possible breakage.

On Thu, Oct 26, 2023 at 09:54:51AM -0000, Simon Chopin wrote:
> I'm very much against uploading this fix as an SRU. While it is a shame
> that we have interop problems with the rest of the world, fixing this in
> Jammy right now means we *will* break production for anyone that stores
> Blowfish-encrypted data.

Simon, just to clarify: the original issue I ran into was not
"interop problems with the rest of the world" exactly, but rather
interop between Tinc nodes running on different versions of
Ubuntu (i.e. a Tinc node running Jammy can't connect to existing
nodes running on older releases). (Though obviously interop will
fail with the rest of the world, too.)

I am not particularly involved in the OpenSSL community, but have
been trying to keep an eye on this specific issue since I ran
into it, and as far as I have noticed/found on the internet there
haven't been any mentions about this OpenSSL bug other than in
the two LP bugs related to this Tinc issue and the related
Github-for-tinc bug ( https://github.com/gsliepen/tinc/issues/414
).

So while I certainly agree in general with need for "apt upgrade"
to be a very safe operation, in this particular case I am not
sure how to weigh the chance that someone might possibly have a
custom app using Blowfish to encrypt local data (while at the
same time not already having in place mechanisms to deal with
this compatibility issue for pre-and-post Jammy Ubuntu) with the
publicly-raised-by-at-least-a-few-people problem of Tinc
incompatibility between nodes that this causes....

                                                Nathan

On Tue, Oct 31, 2023 at 01:13:11PM -0000, Adrien Nader wrote:
> ** Description changed:
>
> === SRU information ===
> [Meta]
> - This bug is part of a series of four bugs for a single SRU.
> + This bug is part of a series of three bugs for a single SRU.
> The "central" bug with the global information and debdiff is http://pad.lv/2033422
>
> [Impact]
> Decryption for Blowfish with OFB and CFB modes fails due to using a key shorter than expected by default.

Adrien, am I correct that this particular bug (1990216) is actually no longer
included at all in the LP:#2033422 SRU ?

                                                        Nathan

Hi Nathan,

Sorry, I didn't have enough time to comment here before a few days of vacation.

This one is indeed not in the SRU at the moment. The description edit itself did not make much sense.

I first discussed this topic with Simon but then also with Steve Langasek, with others attending the same meeting. The general agreement is that bugs when setting up something for the first time are far less severe than bugs that appear afterwards. One major issue here is everything that exists but that we don't know of, including custom software or scripts.

As far as I'm concerned, I evaluate this roughly like you did but I cannot do something that the SRU team is against (I also trust them and their experience, even when my feeling is different).

Lastly, 22.04 was released more than 18 months ago and 24.04 is around the corner; 18 months is a long delay to introduce breaking changes and by now people probably expect very few changes to 22.04.

As far as I understand, tinc could fairly easily work around this issue by explicitely setting the key size before doing operations. This is the safest approach. It might even be faster than waiting for the SRU and corresponding phased updates.

On Wed, Nov 01, 2023 at 02:39:27PM -0000, Adrien Nader wrote:
> This one is indeed not in the SRU at the moment. The description edit
> itself did not make much sense.

(Okay, that's what I thought. For what it's worth, I noticed afterwards
that the description for LP: #2033422 still has the "four bugs" version
of that sentence.)

> As far as I understand, tinc could fairly easily work around this issue
> by explicitely setting the key size before doing operations. This is the
> safest approach. It might even be faster than waiting for the SRU and
> corresponding phased updates.

Actually this isn't really easy on the Tinc side (at least for
Ubuntu end users).

In particular, the Tinc package actually hasn't changed at all in
several Ubuntu releases, so the big surprise here was that Tinc
1.0.36 on Jammy fails to connect in a network where Tinc 1.0.36
on Focal worked fine (even after making the configuration
adjustments that were expected due to the openssl 1.1 -> 3
transition).

Tinc does not provide any user-level control on these parameters,
and I don't think it even sets the parameters explicitly
internally in the code; rather, it relies on the default settings
for the algorithm, which is what was broken in that brief range
of OpenSSL v3 upstream releases. (Presumably if it already set
the parameters explicitly then it wouldn't have been affected by
this bug.)

In any case, solving this from the Tinc side would involve
(someone) making a Ubuntu-specific customization to the Tinc
source code -- which seems a lot more fraught than simply using a
fixed libssl3 library...

One thought that occurred to me is the idea of a semi-official
PPA build with this Blowfish patch included in it.
"Semi-official" in that you (or someone involved in the OpenSSL
Ubuntu packaging work) would publish it, building on top of the
official Ubuntu package, and thus it would track any other fixes
that were published to the main OpenSSL Jammy package -- but
since it's in a PPA it would be opt-in for those who need it.

This would be certainly be preferable to me over my current
situation using an "orphan" version of the legacy.so library
file, but I don't know how much effort it would be to provide
that on your end...

                                                Nathan