Improved searchbar
As brought up by Oasis users (e.g. @alfu) in the project meeting, we should add support for a fully free-text search. This means that you can type in any text in the search bar, and get the best matches without having to have exact matches or having to target a specific field in the metainfo.
My suggestion would be to have a generic text mapping in our metadata section that would be reserved for this purpose. This would generalize across all entries, and filling it could be even done through the ES annotation, with some intelligent default (all string fields in a schema...?). This field could also be used to test some better suggestion mechanisms that could work with filters.
This addition has a big impact on the search bar logic: we need to fully refactor how suggestions are prioritized and shown, and make sure that the UX does not suffer. Also, we need a mechanism for deciding which types of queries are available in the apps: free-text search should be initially disabled in the most basic apps as it will create confusion due to not returning any results when dealing with computational and not-reprocessed data.
Tasks:
-
Add new metainfo that stores the free-text data(e.g. metadata.keywords
) -
Setup the ES config with an annotation. Should be annotated with text
, queried withmatch
. Default analyzer should be used for now. -
Add logic for populating the free-text field: all string quantities in custom schemas should be considered. Every unique string (whitespace in beginning and end removed) field should be added as a new entry into the free-text search. By using unique values we can avoid populating terms that are used multiple times e.g. in repeating sections. -
The apps should be able to define whether they want to enable free-text queries. It is impossible to automate this decision reliably, because an app may work with any kind of data which may or may not have the keywords
field. This functionality can be added in a generic way, which can later be used to control/add new kinds of query syntaxes for the search bar. -
Sorting relevance logic should be changed: -
By default sorting happens by what is set in the app -
When a free-text query is executed, sorting should be changed to relevance automatically -
If a user explicitly sets sorting, this is used until a new free-text query is created
-
-
Improve search bar UX: -
The suggestions type should be shown, and the priority of the suggestions should be logical -
Executed queries should be stored into localStorage
so that we can show suggestions from history -
Items in search history can be deleted -
Autoselect text when navigating suggestions (helps with e.g. modifying a query in history) -
Add a better explanation of the available search queries
-
-
Add tests that check that correct string quantities are added to the new quantity -
Add search tests that check that the API does the correct queries on this field -
Extend search bar tests
In the future, this feature could also be extended using some LLM model: the metadata.keywords
field would be translated into a latent space vector of an LLM and queries would be performed against it.