Graph-style API

tltr: We might want an API that returns all necessary information "full-picture" for a given section ref.

Assume a client (e.g. our GUI) looks at a particular archive section (e.g. via the entry/<id>/archive/query API. To get the full picture, the client still needs to look at the global metainfo, potentially used custom schemas, potentially resolve references, get basic entry metadata, and more.

With our GUI as an example, this leads to additional requests, duplication of code in JS, unnecessary GUI artifacts in the image, etc. The GUI is too complex as it is and we already have loads of troubles testing it. We should move more of this responsibility into the API where it is easier to test and maintain. We should do this at the expanse of client side caching and more server load. This is somewhat opposite to the GUI "store" concept.

This aligns with other efforts. E.g., we need to create upload "bundles" that localize all context information (schemas, etc.) about an upload. Here would aim for a single-request full-picture API for sections instead of uploads.

I could imagine an api that takes a reference-style url server.de/nomad/api/v1/uploads/raw/my.archive.yaml/archive#/path/to/section and returns:

{
    "upload_id": "...",
    "entry_id": "...",
    "path": "/path/to/section",
    "link": "https://server.de/nomad/api/v1/uploads/<id>/entry/<id>/archive#/path/to/section",
    "data": {
        "entry": {
            "is_author": true,
            "is_reviewer": true,
            "can_edit": true 
        },
        "archive": {
            "m_ref_archives": {
                // e.g. other archives that contain the necessary schema
                // this should include "global schemas" somehow
                // ...
            },
            "metadata": {
                // includes authors, ids, references, existing sections/quantities, etc.
                // we could also add client friendly urls to stuff like the reference records
                ...
            },
            "path": {
                "m_def": "/local/path/to/definition",
                "m_reference": "../uploads/<id>/entry/<id>/archive#/path",
                "m_link": "https://server.de/nomad/api/v1/uploads/<id>/entry/<id>/archive#/path",
                "to": {
                    "m_def": "/local/path/to/definition",
                    "m_reference": "../uploads/<id>/entry/<id>/archive#/path/to",
                    "m_link": "https://server.de/nomad/api/v1/uploads/<id>/entry/<id>/archive#/path/to",
                    "section": {
                        ...
                    }
                }
            }
        }
    }
}

This is very similar to what you can already do with the archive API, but

• the necessary "required" record is created from the path /path/to/section automatically
• we add m_reference, m_link so that the client does not need to create these itself
• we add necessary schemas via m_ref_archives in the same way we would add referenced data, makes it easy for the client to resolve
• we add derived user specific entry metadata (can_edit, etc.) so that the client does not have to make these decisions

You might think this is more and more a graphql style API. I would like to think, it is more like an API with the one predefined graph that you always need. I guess simple feature-toggle parameters (include_schema, include_references, etc.) should be enough.