If a fatal flaw afflicts a critical cryptographic function used by no one, what are open-source developers to do? Until recently, such a predicament might have been regarded as a mere philosophical thought experiment, but no more.
An advisory published Thursday warns that a "FIPS module" of the widely used OpenSSL library contained a "fatal bug" in its implementation of Dual EC_DRBG. Credible doubts about the trustworthiness of the deterministic random bit generator surfaced almost immediately after National Security Agency officials shepherded it through an international standards body in 2006. In September, those fears were rekindled when The New York Times reported the algorithm may contain an NSA-engineered backdoor that makes it easier for government spies to decode encrypted communications.
The fatal Dual EC_DRBG bug resides in the FIPS Object Module v2.0, an optional OpenSSL library used to build crypto apps that are certified by the US government's Federal Information Processing Standards. When using the module's implementation of Dual EC_DRBG, the application crashes and can't be recovered. That's an amazing discovery for an application that had to undergo countless hours of testing to be certified by the government of the world's most powerful country. The silver lining seems to be that there's evidence no one has ever actually used Dual EC_DRBG in release versions of the OpenSSL module (though that in turn raises the question of why RSA's BSAFE crypto tool used the RNG by default).