question

David Jones-Gilardi avatar image
David Jones-Gilardi asked David Jones-Gilardi answered

What is the recommended number of columns indexed per table using DSE Search?

Per the following question regarding DSE Search:

For example, I can limit the number of columns to say less than 10, but would be good to have 10-15 tables indexed. Should I worry about the increased indexes in my cluster?

Obviously part of the sizing question here is what is in these indexes, how much RAM, # replicas, etc... but without knowing that and just looking at the overall number of indexes are there any general guidelines that we can give on the number of columns indexed per table or overall per cluster?

searchperformancecapacity
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

David Jones-Gilardi avatar image
David Jones-Gilardi answered

After some conversations and a little research I'm going to take a crack at answering this myself.

First thing, start here -> https://docs.datastax.com/en/dse-planning/doc/planning/capacityPlanningSearch.html. Using Search indexes will use more resources and need proper planning in a production environment. There are many variables to consider.


  • From there, as a general guideline, you should generally keep a dozen or less columns indexed per table.
  • By default, only index the columns you NEED as compared to indexing all columns in a table.
  • Start with just a single table, index, test, and ensure things look good before moving on to the next. Try to avoid indexing ALL of your tables before any testing.
  • Have metrics in place to monitor if something goes wrong.
    • Things like
      • iostat
      • filter cache evictions (the Solr filter cache)
      • how often you're flushing segments
      • GC logs
  • Store your Solr data on a separate SSD from your C* data (This is in the planning doc above)


This is by no means exhaustive, but a start.

Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.