Jon Green, Sr. Director, Security Architecture and Federal CTO, Aruba Networks/HPE
Developers often don't know where the crypto is happening. They may forget to complete certain code segments, relay on 3rd party and open source and rely on multiple frameworks. Or even if they do, they may not want to dive in because of the amount of work required to make things work correctly, particularly from a FIPS perspective..
What about the FIPS code review done by the lab? Almost certainly not, as they are typically looking at the application code - just the low level crypto and RNG functions. Even with the old German scheme for EAL4 deeper code review, still miss issues (like the above TODO code that went through EAL2 and EAL4 review). Testing misses these nuances as well.
Security audits of your code are very fruitful, but very expensive. He's seen success with bug bounty programs, even if the code is closed.
He's also seen problems with FIPS deployments that are leveraging "FIPS inside" where they leverage another module, like OpenSSL, but forgot to turn on FIPS mode and forgot to update the applications so they would not try to use non-FIPS algorithms.
Another problematic approach - the dev follows all the steps to deploy CenOS in FIPS mode by following the RedHat documentation... except that documentation only applies to RedHat and *not* CentOS. Yes, it's the same source code , but validations are not transitive. A RedHat validation only applies to RedHat deployments.
To get this right, identify services that really need to be FIPS validated and focus your efforts there..
Flakes Welcome - "Hi, I'd like a cake, please, and could you put a 'happy birthday' plaque on it?" "No, no, I mean one of those plastic things on a spike - you know, a 'h...