question

reshab200 avatar image
reshab200 asked Erick Ramirez answered

Why does performance degrade when multiple threads run in parallel?

I have a function that queries multiple partition asynchronously and combine the result it works well when one thread is executing this function, but performance degrades for all threads when multiple threads parallelly execute this function, I am not able to understand why is this happening. Below is my function.


   @Override
public List<ConversationDetail> getConversationDetailByCreateDateAndCmId(Timestamp startDateTimestamp, Timestamp endDateTimestamp, int cmId) {


 LocalDate startDate = startDateTimestamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate();
 LocalDate endDate = endDateTimestamp.toInstant().atZone(ZoneId.systemDefault()).toLocalDate();


        List<ConversationDetail> results = new ArrayList<>();
        List<ResultSetFuture> futures = new ArrayList<>();


        try {
            for (LocalDate date = startDate.plusDays(1); date.isBefore(endDate); date = date.plusDays(1)) {
                ResultSetFuture resultSetFuture = session.executeAsync(findByCreateDayAndCmId.bind(date, cmId).setReadTimeoutMillis(60000));
                futures.add(resultSetFuture);
            }
            futures.add(session.executeAsync(findByCreateDayAndCmIdAndTimestampGreaterThanEqual.bind(startDate, cmId, startDateTimestamp).setReadTimeoutMillis(60000)));
            futures.add(session.executeAsync(findByCreateDayAndCmIdAndTimestampLessThanEqual.bind(endDate, cmId, endDateTimestamp).setReadTimeoutMillis(60000)));
           for (ResultSetFuture future : futures) {
                try {
                    ResultSet rows = future.get();
                    Iterator<ConversationDetail> it = mapper.map(rows).iterator();
                    while (it.hasNext()){
                        results.add(it.next());
                    }
                }catch (Exception e){
                    System.out.println("Exception1 " + e.getMessage());
                }
            }
        }catch (Exception e){
            System.out.println("Exception2 "+e.getMessage());
        }
        return results;
    }


java driver
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered reshab200 commented

With the limited information you provided, my guess is that your queries are overloading the cluster.

You need to review the underlying queries you are running, particularly if you are not retrieving data by partition key. If you are using ALLOW FILTERING without filtering on a single partition key, it is very expensive and would explain why the performance eventually tanks. Cheers!

Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.