This article discusses how to convert an existing traditional PEM-encoded encrypted private key into the compatible PKCS#8 format for use with Secure Server - FIPS. (To convert an existing PEM-encoded PKCS#8 format encrypted private key, refer to Converting a PEM-Encoded PKCS#8 Format Encrypted Private Key to PKCS#8 Format.)
General Information
When operating in a FIPS-approved mode, PKI key/certificates must be between 1024- bits and 4096-bits, inclusive.
The supported cipher combinations allowed for SSL negotiation are limited to:
SSLv3/TLSv1 - RSA Key Exchange, RSA Authentication, 256 bit AES encryption, and SHA1 HMAC
SSLv3/TLSv1 - RSA Key Exchange, RSA Authentication, 168 bit 3DES encryption, and SHA1 HMAC
SSLv3/TLSv1 - RSA Key Exchange, RSA Authentication, 128 bit AES encryption, and SHA1 HMAC
Each of the above combinations uses RSA key exchange; therefore, RSA based key/certificates must be used.
In FIPS Mode, the private key must use the PKCS#8 format and PKCS#12 compatible encryption of the private key, which allows the use of the necessary strong encryption algorithm of 3DES encryption and SHA1 hashing.
Note that traditional PEM encoded encrypted private key files will typically start with the line:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
Procedure
These instructions assume you have downloaded and installed the Windows binary distribution of OpenSSL. Refer to Using OpenSSL for the general instructions
The private key you want to convert must already be an RSA private key and be between 1024 and 4096 bits in length, inclusive. It is only possible to convert the storage format for the private key. Changing the type of key and its length is not possible and requires generation of a new private key.
Convert the existing traditional PEM encoded encrypted private key to an unencrypted PEM format.
C:\Openssl\bin\openssl.exe rsa -in <Traditional PEM Key Filename> -out <Unencrypted Key Filename>
Where:
<Traditional PEM Key Filename> is the input filename of the incompatible traditional format PEM encoded private key.
<Unencrypted Key Filename> is the output filename of the unencrypted private key in PEM format
For example, type:
>C:\Openssl\bin\openssl.exe rsa -in my_key.key -out my_unencrypted_key.pem
Convert the unencrypted key to a compatible pkcs8 format
>C:\Openssl\bin\openssl.exe pkcs8 -v1 PBE-SHA1-3DES -topk8 -in <Unencrypted Key Filename> -out <Encrypted Key Filename>
Where:
<Unencrypted Key Filename> is the input filename of the previously generated unencrypted private key.
<Encrypted Key Filename> is the output filename of the encrypted private key
For example, type:
>C:\Openssl\bin\openssl.exe pkcs8 -v1 PBE-SHA1-3DES -topk8 -in my_unencrypted_key.pem -out my_encrypted_key.key
Delete the unencrypted private key.