Skip to content

2D statistic plot

We already allow users to enable statistics "widgets" per search quantity. This should be extended to include combinations of numeric values in order to show 2D scatter plots. This would be great for the solar cell data where you could plot e.g. the solar cell efficiency vs some other property.

1. General widget component

We need a generic widget component that wraps all the widgets including the scatter plot.

  • Needs to be resizable.
  • Needs to be closeable.
  • Needs to be draggable.
  • Needs customizable actions bar at the top. E.g. the plot actions will be shown there.
  • Needs to have a title + possible description.

2. Scatter plot component

The first task would be to test out the plot implementation without thinking too much about how we get the data. The rough specifications would include:

  • Button to show/hide the edit dialog (see below).
  • Show the quantity names as x/y axis labels with a tooltip.
  • Additionally, the ability to choose quantities associated with the marker color. These quantities do not necessarily need to be numerical, but also text-type for the color. If a string type quantity is picked as a color, a marker label could be added showing the value of that quantity. Filtering the data by clicking and removing labels would be added value. If a numerical field is chosen, a color scale bar should be shown instead of the labels.
  • Should resize to fit parent.
  • Reacts to changes in the "global" units.
  • Shows Placeholder when loading
  • Shows NoData when no results are available. This breaks e.g. the region focus selection, simply show an empty plot instead.
  • The plot should be able to react to changes. E.g. when a user enables additional filters, the plot data will change quite often.
  • Pan, zoom, view reset.
  • Ability to click individual calculations, leading the user to that entry. Will be done later
  • Tooltip when hovering on individual calculations to show the plotted values and entry id Will be done later
  • Should be able to handle large amounts of data, something in the 10K range.
  • Visuals should be customized to match the look and feel of our UI. Mostly this can be achieved by using the theme font and colors. MUI components should be used whenever possible, e.g. Tooltip, Select etc.
  • Ability to select the maximum number of data points to retrieve and plot. A dropdown with a few presets should do. E.g. 100, 1K, 10K, 100K, all. Should maybe default to 10K?
  • The component should show some notifications about the data: The number of retrieved samples, and some indication of whether all available data was loaded or not. Will be done later

We should try two parallel implementations: one using Plotly (@mnakh) and one using Vega d3.js (@himanel1). This will give us an idea about the pros/cons of these two approaches and we can then decide if we keep one of them or both of them.

3. Scatter plot edit component.

We need a separate edit dialog for the scatter plot. It will be shown when first adding the plot and whenever the user wants to edit the widget.

  • The ability to freely choose the x and y axes. This could e.g. be e.g. two dropdowns - one for the x-axis and one for the y-axis - with some text search possibility + autocompletion. All numeric quantities would be shown as options.
  • The ability to choose a third quantity for coloring the points. This could e.g. be a dropdown. All quantities with a scalar value (can be numeric or string) should be available.
  • Ability to choose the unit for the x, y, and coloring axes. Will be done later
  • Ability to propagate changes between the edit dialog and the plot itself.

4. Data fetching

Initially, we will simply use some dummy data (dummy_data.json) to do the component implementation, but in the final version, all of the data will be served by ES. We need to see if using simple ES hits is performant enough or if ES provides some special tooling for these graphs. The way Kibana does this should be used as inspiration. The requirements for the data fetching are:

  • Should be able to handle large amounts of data, something in the 10K range. The delay to get this data can be slightly bigger, as these queries will be executed in parallel to the queries that populate other parts of the UI.
  • The fetched data should be pseudo-random. This is the only way we get a representative sample.
  • SearchContext needs the ability to display different sets of results with different pagination.

5. Other changes

This new feature will also affect other parts of the UI:

  • The dashboard needs to be collapsible.
  • We need a URL serialization schema for these kinds of widgets. Url serialization probably not enough. Needs to be serialized in JSON.
  • We need a place in the search UI where these more general widgets can be added. The initial idea is to have buttons for different widgets above the dashboard.
  • We need to start using the term "widget" in our codebase and in the UI. Currently, we say that we are "anchoring a statistic", but the new terminology should revolve around "displaying a widget".
  • The initial dashboard for each app is loaded using configs from nomad.yaml.
  • The dashboard config needs JSON export/import functionality.
  • Download data from the chart as CSV with a button. Maybe this dash component helps.Will be done later
Edited by Lauri Himanen