Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 219
    • Issues 219
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 32
    • Merge requests 32
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-labnomad-lab
  • nomad-FAIRnomad-FAIR
  • Merge requests
  • !444

Improved the suggestion feature

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Lauri Himanen requested to merge suggestions into v1.0.0 Nov 01, 2021
  • Overview 6
  • Commits 7
  • Pipelines 4
  • Changes 13

In the GUI we offer suggestions for certain index fields. These are implemented using the ES Completion suggester feature which is a special feature built exactly for this kind of task.

Previously we were defining a new ES field with the completion type in order to create these suggestions. This way the indexing of suggestion values comes pretty much for free and the index size is not affected too much. This means, however, that only one suggestion value was supported per quantity, which makes it impossible to make autocompletion suggestions if the user types e.g. a word in the middle of a keyword or string: e.g. typing salt would not create a suggestion for rock salt. Technically this is because the Completion suggester is based on a special graph data structure that can only start suggesting values from the beginning of a string.

To overcome this limitation, the default approach is to augment the suggestion values by adding several suggestion values at index time (examples of this discussion here, here and here). This merge request implements this mechanism by storing suggestions under <quantity name>__suggestion when the values need to be tokenized.

The suggestion mechanism is selected by specifying a predefined string option (suggestion="simple": the old behaviour, suggestion="default": tokenization using whitespaces and underscores, suggestion="formula": tokenization using formula fragments, suggestion=<function>: custom tokenization function). The old mechanism of storing a single suggestion string under a field is used when suggestion="simple".

The downside of using a new index attribute instead of a field for each suggestion is that the index size becomes bigger and that the source documents look quite busy with all this new suggestion data. By default, the suggestion values are completely excluded from the source documents in all metadata searches. Depending on our experience with the index size, we can think about limiting suggestions, using a separate index for suggestions (really nasty), or experimenting with other suggestion mechanisms (here is a really good article about all reasonable options).

Edited Nov 03, 2021 by Lauri Himanen
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: suggestions