PLANNED MAINTENANCE

Hello, DataStax Community!

We want to make you aware of a few operational updates which will be carried out on the site. We are working hard to streamline the login process to integrate with other DataStax resources. As such, you will soon be prompted to update your password. Please note that your username will remain the same.

As we work to improve your user experience, please be aware that login to the DataStax Community will be unavailable for a few hours on:

  • Wednesday, July 15 16:00 PDT | 19:00 EDT | 20:00 BRT
  • Thursday, July 16 00:00 BST | 01:00 CEST | 04:30 IST | 07:00 CST | 09:00 AEST

For more info, check out the FAQ page. Thank you for being a valued member of our community.


question

michael.guissine_30999 avatar image
michael.guissine_30999 asked ·

DSE 6.7.x fails to start on Centos 7.7 (latest)

DSE (6.7.4) fails to start after CentOS was upgraded from 7.6 to 7.7 (latest). We tried upgrading DSE to 6.7.6 but still seeing the same issue, downgrading CentOS back to 7.6 resolves the issue.

This is the error we are seeing in system.log file

ERROR [main] 2019-10-25 20:37:34,728  DseDaemon.java:550 - Unable to start DSE server.                                              
java.lang.RuntimeException: Failed to setup secure pipeline                                                                         
        at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:435)                                   
        at org.apache.cassandra.transport.Server$SecureInitializer.<init>(Server.java:493)                                          
        at org.apache.cassandra.transport.Server.start(Server.java:144)                                                             
        at java.util.ArrayList.forEach(ArrayList.java:1257)                                                                         
        at org.apache.cassandra.service.NativeTransportService.start(NativeTransportService.java:131)                               
        at org.apache.cassandra.service.CassandraDaemon.startNativeTransport(CassandraDaemon.java:795)                              
        at com.datastax.bdp.server.DseDaemon.startNativeTransport(DseDaemon.java:580)                                               
        at org.apache.cassandra.service.CassandraDaemon.lambda$start$8(CassandraDaemon.java:589)                                    
        at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:601)                                             
        at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:545)                                                              
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:751)                                          
        at com.datastax.bdp.DseModule.main(DseModule.java:96)                                                                       
Caused by: java.io.IOException: Error creating the initializing the SSL Context                                                     
        at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:207)                                           
        at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:431)                                   
        ... 11 common frames omitted                                                                                                
Caused by: java.security.NoSuchAlgorithmException: DseClientReloadableTrustManager KeyManagerFactory not available                  
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)                                                           
        at javax.net.ssl.KeyManagerFactory.getInstance(KeyManagerFactory.java:137)                                                  
        at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:183)                                           
        ... 12 common frames omitted                                                                                                
ERROR [main] 2019-10-25 20:37:34,730  CassandraDaemon.java:862 - Exception encountered during startup                               
java.lang.RuntimeException: java.lang.RuntimeException: Failed to setup secure pipeline                                             
        at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:553)                                                              
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:751)                                          
        at com.datastax.bdp.DseModule.main(DseModule.java:96)                                                                       
Caused by: java.lang.RuntimeException: Failed to setup secure pipeline                                                              
        at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:435)                                   
        at org.apache.cassandra.transport.Server$SecureInitializer.<init>(Server.java:493)                                          
        at org.apache.cassandra.transport.Server.start(Server.java:144)                                                             
        at java.util.ArrayList.forEach(ArrayList.java:1257)                                                                         
        at org.apache.cassandra.service.NativeTransportService.start(NativeTransportService.java:131)                               
        at org.apache.cassandra.service.CassandraDaemon.startNativeTransport(CassandraDaemon.java:795)                              
        at com.datastax.bdp.server.DseDaemon.startNativeTransport(DseDaemon.java:580)                                               
        at org.apache.cassandra.service.CassandraDaemon.lambda$start$8(CassandraDaemon.java:589)                                    
        at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:601)                                             
        at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:545)                                                              
        ... 2 common frames omitted                                                                                                 
Caused by: java.io.IOException: Error creating the initializing the SSL Context                                                     
        at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:207)                                           
        at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:431)                                   
        ... 11 common frames omitted                                                                                                
Caused by: java.security.NoSuchAlgorithmException: DseClientReloadableTrustManager KeyManagerFactory not available                  
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)                                                           
        at javax.net.ssl.KeyManagerFactory.getInstance(KeyManagerFactory.java:137)                                                  
        at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:183)                                           
        ... 12 common frames omitted                                                                                        


dsedse 6.7.4
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Erick Ramirez avatar image
Erick Ramirez answered ·

@michael.guissine_30999 specifically for this exception:

Caused by: java.security.NoSuchAlgorithmException: DseClientReloadableTrustManager KeyManagerFactory not available

Do you have the steps to reproduce it? We haven't had this reported before and it appears as though one of the CentOS 7.7 security patches breaks JCE.

FWIW DseClientReloadableTrustManager is just an extension of X509ExtendedTrustManager and the NoSuchAlgorithmException gets thrown when it tries to load the trustStore.

We need to have a look at your cluster's configuration so we can try and replicate the problem. The best way to get the information we require is for you (or someone in your organisation) to log a DataStax Support ticket and attach a Diagnostics Report from Opscenter. Let me know the ticket details and I will work with the Support engineers internally. Cheers!

Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Lewisr650 avatar image
Lewisr650 answered ·

DSE is not currently supported on CentOS 7.7. You can verify supported platforms here:

https://docs.datastax.com/en/landing_page/doc/landing_page/supportedPlatforms.html

8 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks Lewisr650! Any word on when DSE will support Centos 7.7? Centos 7.7 includes numerous critical security updates which we would like to install as soon as possible.

0 Likes 0 · ·

@Lewisr650 CentOS 7.7 was released on 2019-08-06. DSE 7.6 still does not work on it. Is there a good reason for that? Seems overly long to me.

0 Likes 0 · ·

CentOS 7.7 is not currently something we test with based on customer requests. You can submit a a request for CentOS 7.7 to be supported, but then out engineering and testing processes have to go through their cycles. Is there something specific in CentOS 7.7 you are trying to take advantage of?

0 Likes 0 · ·

Yes, security updates. Which are likely what broke DSE in CentOS 7.7.


I'm not sure I understand your logic. I would understand if you did not yet support CentOS 8 because it's a fairly new major release. But if you support CentOS 7.x, then you need to support the latest version because everyone who is on CentOS 7.6 for example, will get 7.7 when they do a simple `yum update`

0 Likes 0 · ·

@michael.guissine_30999, @mscrivo_94376 The evaluation of CentOS 7.7 was completed but there were some issues unrelated to DSE that we uncovered.

We have received reports that upgrading from CentOS 7.6 to 7.7 has some performance degradation irrespective of the software/application/database that is running on the server.

0 Likes 0 · ·
Erick Ramirez avatar image Erick Ramirez ♦♦ Erick Ramirez ♦♦ ·

@michael.guissine_30999 & @mscrivo_94376 I forgot to mention that I maintain this FAQ which lists new OS releases being tested. Cheers!

0 Likes 0 · ·

Furthermore, users who have installed CentOS 7.7 from the start (not upgraded from 7.6) do not run into this performance problem. You will need to conduct your own testing to see how this issue affects you. In the meantime, we will see how this is resolved in RHEL/CentOS. Cheers!

0 Likes 0 · ·
mscrivo_94376 avatar image mscrivo_94376 Erick Ramirez ♦♦ ·

Thanks @Erick Ramirez .. that's helpful information. We'll do some experiments on our end and then open a ticket if we can't figure it out.

0 Likes 0 · ·