Why can't I connect to Cassandra in Docker?

Question

question

sandhyarme asked Nov 20, '20 sandhyarme published Dec 4, '20

Why can't I connect to Cassandra in Docker?

Hi Team,

We have created cassandra db docker container.

docker exec -it cassandra-node bash

With in db container "cqlsh" cmd resulted in below error.

Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

Content of /etc/hosts

root@61b9a810dbcc:/# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost 61b9a810dbcc

Please suggest us on troubleshooting this error.

[UPDATE] We have created only one container i.e, server node.

Created container with 4 gb ram

docker run -it -d --name cassandra-node -p 9042:9042 -m 4000M cassandra

Able to connect to container.

docker exec -it cassandra-node bash

started cassandra

cd /opt/cassandra/
cassandra

checked status nodetool status gave below error.

ERROR [main] 2020-11-21 17:49:12,250 CassandraDaemon.java:803 - Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use (Bind failed)
java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.8.0_275]
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387) ~[na:1.8.0_275]
        at java.net.ServerSocket.bind(ServerSocket.java:390) ~[na:1.8.0_275]
        at java.net.ServerSocket.<init>(ServerSocket.java:252) ~[na:1.8.0_275]
        at javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231) ~[na:1.8.0_275]
        at org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:42) ~[apache-cassandra-3.11.9.jar:3.11.9]
        at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) ~[na:1.8.0_275]
        at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335) ~[na:1.8.0_275]
        at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254) ~[na:1.8.0_275]
        at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) ~[na:1.8.0_275]
        at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:237) ~[na:1.8.0_275]
        at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213) ~[na:1.8.0_275]
        at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173) ~[na:1.8.0_275]
        at org.apache.cassandra.utils.JMXServerUtils$JmxRegistry.<init>(JMXServerUtils.java:295) ~[apache-cassandra-3.11.9.jar:3.11.9]
        at org.apache.cassandra.utils.JMXServerUtils.createJMXServer(JMXServerUtils.java:82) ~[apache-cassandra-3.11.9.jar:3.11.9]
        at org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:159) [apache-cassandra-3.11.9.jar:3.11.9]

root@47668132a2e6:/# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
cassand+ 1 4.3 30.7 2573056 1202856 pts/0 Ssl+ 17:44 1:37 /opt/java/openjdk/bin/java -Xloggc:/opt/cassandra/logs/gc.log -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -XX:
root 357 3.6 0.0 0 0 ? Z 17:47 1:14 [java] <defunct>
root 571 1.5 0.0 7240 576 pts/1 Ss 18:21 0:00 bash
root 578 0.0 0.0 8888 272 pts/1 R+ 18:21 0:00 ps -aux

docker

error123.png (37.5 KiB)

1

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

sandhyarme commented · Nov 22, 2020 at 11:30 AM

We have created only one container i.e, server node.

Created container with 4 gb ram

docker run -it -d --name cassandra-node -p 9042:9042 -m 4000M cassandra

Able to connect to container.

docker exec -it cassandra-node bash

started cassandra

cd /opt/cassandra/

cassandra

----------------------------------------------------------------------------------------------------

checked status

./nodetool status gave below error.

Error.zip

---------------------------------------------------------------------------------

/opt/cassandra/bin/cqlsh

Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

--------------------------------------------------------------------------------------

0 ·

error.zip (113.2 KiB)

DataStax Enterprise is powered by the best distribution of Apache Cassandra ™

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Privacy Policy Terms of Use

Answer 1 · 2020-11-20T16:24:53Z

Aleks Volochnev answered Nov 20, '20 sandhyarme commented Nov 23, '20

Hi @sandhyarme! Could you please provide more information?

First, what is the image used to run the container?
Second, what runs inside the container? You can get it with docker exec CONTAINER ps aux.
Third, how exactly do you launch the container?

1 Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

sandhyarme commented · Nov 21, 2020 at 06:15 PM

[Update posted in original question]

0 ·

Answer 2 · 2020-11-20T22:42:18Z

Erick Ramirez answered Nov 20, '20 sandhyarme published Dec 4, '20

In addition to Aleks' response, how is Cassandra configured in your setup?

Cassandra is listening for CQL client connections on rpc_address. If it's set to a value other than localhost, you need to specify the IP address when connecting with cqlsh.

Also note that if the Cassandra instance isn't running, you won't be able to connect to it. But it's really difficult to respond to your question since you've provided very limited information. Cheers!

[UPDATE] Based on the output you provided, this exception is reported because another process is already running that is already using port 7199:

ERROR [main] 2020-11-21 17:49:12,250 CassandraDaemon.java:803 - Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use (Bind failed)

You can't start another Cassandra process when another is already running.

6 Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

sandhyarme commented · Nov 23, 2020 at 12:24 PM

Error123.PNG

Restarted container, ps -ef shows cassandra running.

but why unable to access via cqlsh and nodetool status doesn't list server node ?

Attached screenshot.

0 ·

error123.png (37.5 KiB)

Erick Ramirez ♦♦ sandhyarme commented · Nov 24, 2020 at 12:10 AM

As the error suggests, it is probably still bootstrapping which means you won't be able to connect to it yet. Just because you restarted the container doesn't mean Cassandra is operational -- it is a database and needs to go through normal initialisation steps just like any other database.

You need to review the Cassandra system.log to see what is going on. Cheers!

0 ·

sandhyarme Erick Ramirez ♦♦ commented · Nov 24, 2020 at 08:39 AM

Error234.PNG log.txt

Attached system log/container log..

Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out

Have added below lines .. in /etc/security/limits.conf

- memlock unlimited

* - nofile 100000

* - nproc 32768

* - as unlimited

these are warnings no error as such in log.. nodetool status is not listing server node thats the big question..

0 ·

error234.png (172.2 KiB)

log.txt (69.1 KiB)

Show more comments

question