Chapter 9: Using TLS and FIPS in EAServer

Enabling FIPS

You can enable or disable FIPS from either:

EAServer Manager or the standalone Security Manager—“Enabling FIPS mode from EAServer Manager and Security Manager”, or
jagtool, which is a Java command line management toolkit used in EAServer. jagtool provides a command to enable FIPS. See “FIPS-related jagtool commands”.

If FIPS mode is enabled, EAServer logs the message FIPS 140-2 mode enabled to the console. If the mode is not set, no message is logged.

Enabling FIPS has the following effect on EAServer:

Permits TLS protocol only by the SSL/TLS runtime engine.
Permits the use of cipher suites and security characteristics listed in Table 9-2.
Accepts X.509 certificates signed using a SHA1WithRsa algorithm. Certificates signed with any other algorithm are not accepted and generate an error.
Other cryptographic functionality that normally employ a non-FIPS approved algorithm now fail. For example, a PKCS #12 certificate containing a private key shrouded (signed) with a pbeWithSHA1And40bitRc4 algorithm fails to import, since RC4 is not a FIPS 140-2-approved algorithm. The private key and public keys must be shrouded using pbeWithSHA1And3KeyTripleDescbc.

Enabling FIPS mode from EAServer Manager and Security Manager

You can enable or disable FIPS on EAServer from EAServer Manager. Or use the standalone Security Manager to enable or disable FIPS in client-side applications, such as PowerBuilder, stand-alone Java, C++, CORBA, Web server redirectors, and so on.

Expand the EAServer Manager (or Security Manager) icon, highlight the Cryptographic Modules folder, and select the FIPS mode icon. A dialog box displays Enabled or Disabled and allows you to change the setting.

Enabling FIPS from EAServer Manager or Security Manager

Select the Certificates folder.
Select the Cryptographic Modules folder. Enter the PIN that allows you to connect to the EAServer’s PKCS #11 token. The default value is “sybase”.
Right-click the FIPS mode icon and select Properties. Click the Enable FIPS mode check box to enable FIPS.
Restart each server or Web server (for redirector plug-ins) for which you want to enable FIPS. If there are multiple EAServers, you must restart each one to enable FIPS. The same is true if you disable FIPS.

Disabling TLS support

To disable TLS support, and only support SSL, set the environment variable JAGSSL to true in the serverstart.bat file before you start EAServer. You can also set this environment variable in an EAServer client installation:

set JAGSSL=true