question

rkagan_146583 avatar image
rkagan_146583 asked Erick Ramirez edited

Reducing number of long running queries with timeAllowed - does it resolve the problem?

Hello:

I wonder - we have the following software:


Search 6.0.1
Cassandra 3.11.3.5113
DSE 5.1.13


Some of our queries are non-performant. Is setting timeAllowed will help us to eliminate this problem? We deployed the code with such a setting - however we see no effect, nor warnings from DSESolr.


Here's more info on that:

https://docs.datastax.com/en/dse/5.1/dse-dev/datastax_enterprise/search/limitQueriesByTime.html

search
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

maedhroz avatar image
maedhroz answered rkagan_146583 commented

OSS Solr's TimeLimitingCollector, which is used by DSE Search, suffers from two problems. The first is that for many queries that are likely to take an inordinately long time, like pivot facet queries, it has essentially no effect, because it is evaluated during the Lucene collection phase. The second is that even there, it carries with it a significant performance penalty for queries that won't hit the time limit in the first place.

Having said that, we are currently working on a timeAllowed mechanism that addresses both these issues, and that will land in an upcoming 6.0.x patch release.

2 comments Share
10 |1000

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

rkagan_146583 avatar image rkagan_146583 commented ·

@maedhroz - Would you please refer Jira tickets for aforementioned 'upcoming 6.0.x patch release' that deals with Lucene's TimeLimitingCollector class and 'timeAllowed' parameter? I would greatly appreciate that.

0 Likes 0 ·
maedhroz avatar image maedhroz rkagan_146583 commented ·

That issue, which is obviously private at the moment, will be DSP-19781. It will likely be very similar to the unresolved OSS issue SOLR-6831.

0 Likes 0 ·