FIPS 140-3 mode on RHEL 9 and RSA validation of <2048 keys

[Petr Menšík]

Forgot to add the bug link.

- openssl: https://bugzilla.redhat.com/show_bug.cgi?id=2077884
- bind: https://bugzilla.redhat.com/show_bug.cgi?id=2077906

On 4/25/22 11:39, Petr Menšík wrote:
> Hello,
>
> I have sent already a notification about SHA-1 not validated in default
> configuration. However that was not end of the story.
>
> A new and even more severe issue has arisen. Our crypto team is
> responsible for preparing RHEL 9 for FIPS 140-3 certification. They said
> there is legal obligation to stop using all RSA signatures with keys
> shorter than 2048 bits. I expect many of you already know especially
> Zone Signing Keys often use 1280 or even 1024b keys without issues.
>
> Current RHEL 9 validates such signatures well, but there is a bug [1]
> filled to change that and start enforcing longer keys usage only.
> Because it would be enforced at crypto library level (openssl), common
> DNSSEC validation software would start failing. On quite a lot of
> domains. And failing to resolve such names.
>
> I am looking for the best way out of this. Once the above happens
> without any changes to used DNS(SEC) software, the only way to keep
> working DNS servers working would be either turning off FIPS more or
> DNSSEC validation. I know the result seems quite ridiculous, because all
> this has been done in order to increase the security. The only
> alternative without changes would be disabling all RSA altogether and
> providing a long list of ECDSA trust anchors for TLD domains, where at
> least ECDSA would offer protection.
>
> I think the only good way would be starting considering shorter keys as
> insecure in FIPS mode. Anything above those domains would be without
> DNSSEC protection, but at least single root key could be used. Do you
> think it would be safe once the DS digest is checked on the key, that
> key is then can be marked as insecure instead of bogus. Just as if the
> algorithm were unsupported, but after checking the length of key.
> Because the crypto library would refuse any operation, including
> signature validation, on the given key. I think the library would not
> even allow loading that key.
>
> I don't think described behavior is possible in any supported version of
> BIND9. I expect it would require not only small and self-contained change.
> Would you know any blockers, which would prevent behavior described in
> the above paragraph? Is it possible to consider all keys with short keys
> to become unsupported, but keys with long enough keys to be still 
> validated as usual?
>
> Note: some companies and agencies have to use only FIPS 140-3 approved
> algorithms. Enabling FIPS mode in RHEL 9 is not optional for them. Fixing
> the problem by disabling FIPS mode is not possible for everyone.
>
> Any comments or suggestions welcome.
>
> Best Regards,
> Petr Menšík
>
-- 
Petr Menšík
Software Engineer
Red Hat, http://www.redhat.com/
email: pemensik at redhat.com
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB