question

Duck avatar image
Duck asked Duck commented

Is Cassandra a good fit for a logging use case?

Hi,

We've been working a project for a while. We are using MongoDB as a primary database for our project. We've been searching solution for logging for a while. For that purpose, we've found Cassandra. Befure using Cassandra, we've to know answers for some questions.

About Project

It's an online game project which aims to have 25 million players in next 5 years. The type of the game is MMORPG(Massive Multiplayer Online Role-Playing Game).

What We Need

As I mentioned it above, we are currently using MongoDB as a primary database but we do not think MongoDB fits our use purpose for logging.

  1. We're planning to log almost everything except application logs(big text files, console logs) and metrics(CPU usage, memory, dedicated server status etc.). We also collect logs with timestamps, basically TSD(Time Series Data). Is Cassandra work well for that purpose?
  2. Since we are planning to have millions of player/user, we need to store a lot of logs. Is there any limitation for that?
  3. We love to analyze and see metrics from past. Therefore, we must store logs as long as possible, we don't like retentions or TTL for logs at all. After logs size grow up, can we add more memory(SSD, HDD) to database to handle more logs over time? Or add more storage dedicated servers(nodes)?

Example Schemes

(It's just an example to show you what kind of things we're planning to log.)

Scheme 1

timestamp: (Millisecond)
user-id: (UUID)
product-id: 1
amount: 5
price: 1500
seller: (UUID)

Scheme 2

timestamp: (Millisecond)
user-id: (UUID)
to: (UUID)
channel: message_channel_1
message: "It's a test message for logging. I can see this message whenever I want

Feel free to ask me anything. Thanks in advance!

use cases
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Duck commented

The quick answer is yes, Cassandra is a good fit for your use case.

To respond to your specific questions:

  1. Yes, Cassandra will work well.
  2. No, it is virtually limitless. You can store as many as 2^128 players/users.
  3. Cassandra scales linearly. As the data size on each node grows above 500GB, you need to add more nodes to keep the node density to less than 1TB. Also, all operations in Cassandra are online including upgrades -- it's an always-on database. Therefore adding nodes does not require an outage and data will automatically rebalance among existing nodes in the cluster.

Cheers!

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Duck avatar image Duck commented ·

Thanks for the information! Your answer is solution for this thread but I'd like to ask some related questions. I don't want to separate threads for that.

After a lot of searching, I found that I should work with TWCS.

  1. Should I use TWCS for that prupose? I'll use timestamp(millisecond) for all logs but they'll stay as long as possible(might be forever.).
  2. Since we're just starting, we might have small data. Even we have few data, should we separate keyspaces by time using TWCS? In the future, can we change time range?
0 Likes 0 ·