Elasticsearch problems related to sort and track_total_hits.
We have to acknowledge that sort
and track_total_hits
are expensive operations in most cases, and we should not perform them as a default on every ES query.
- We should make these functions explicit and non default in
MetadataPagination
objects. And changing all clients that we control (GUI, API examples) respectively. - We should apply indexsorting for
[upload_date, entry_id]
for all new indices (and we can enable this when migrating the existing indicies to the new cluster). Theupload_date
is the default sort in the UI and the most relavant. Combining it withentry_id
would also give a good default forpage_after_value
pagination, which requires some sort. - I do not understand, why sorting has a perf impact on searches with only a few total hits. From what the docs say (sort only happens after search), it should not have an impact, but it does.