Tabular parser - using column and row modes together only works for children repeated sections

Some more headache to refine the tabular parser:

  • The three following raw files should generate an archive inheriting from MyMixedMode class where four subsections are filled by Tabular parser.

    I only get the first two filled, specifically they are repeated sections, for which I need to state in the "data" raw file their m_def value

  • If I uncomment the TableData inheritance into the root section MyMixedMode, because I want to parse "my_quantity_6" there, the parser gives an error. Somehow when the parser goes back up to the root section, the annotations of the "data_file" quantity are not correctly read and this cause a failure in parsing. Note: if I don't want to parse stuff into the root section, everything is fine and good.

definitions:
  name: 'Tabular Parser examples'
  sections:
    MyColumnMode:
      base_sections:
        - nomad.parsing.tabular.TableData
      m_annotations:
        eln:
      quantities:
        data_file:
          type: str
          m_annotations:
            tabular_parser:
              comment: '#'
              mode: column
              path_to_data_file: '#data/data_file'  # only needed for MyMixedMode
            browser:
              adaptor: RawFileAdaptor
            eln:
              component: FileEditQuantity
        my_quantity_1:
          type: np.float64
          shape: ['*']  # in column mode, quantities are arrays
          m_annotations:
            tabular:
              name: Sheet1/Header 1
      sub_sections:
        MyColumnModeSubsection:
          section: # Important! you must define sub_section as "section" type
            quantities:
              my_quantity_2:
                type: str
                shape: ['*']  # in column mode, quantities are arrays
                m_annotations:
                  tabular:
                    name: Sheet1/Header 2
              my_quantity_3:
                type: str
                shape: ['*']  # in column mode, quantities are arrays
                m_annotations:
                  tabular:
                    name: Sheet1/Header 3
    MyRowMode:
      base_sections:
        - nomad.parsing.tabular.TableData
      m_annotations:
        eln:
      quantities:
        data_file:
          type: str
          m_annotations:
            tabular_parser:
              # path_to_data_file: '#data/data_file'  # only needed for MyMixedMode
              comment: '#'
              mode: row
              target_sub_section:
                - MyRowModeSubsection
            browser:
              adaptor: RawFileAdaptor
            eln:
              component: FileEditQuantity
      sub_sections:
        MyRowModeSubsection:
          repeats: true  # Note: you need to make this section repeated
          section: # Important! you must define sub_section as "section" type
            more:
              label_quantity: my_quantity_4  # It shows this name in the list of instanced sections
            quantities:
              my_quantity_4:
                type: str
                m_annotations:
                  tabular:
                    name: Sheet1/Header 4
    MyRowMode2:
      base_sections:
        - nomad.parsing.tabular.TableData
      m_annotations:
        eln:
      quantities:
        data_file:
          type: str
          m_annotations:
            tabular_parser:
              #path_to_data_file: '#data/data_file'  # only needed for MyMixedMode
              comment: '#'
              mode: row
              target_sub_section:
                - MyRowModeSubsection2
            browser:
              adaptor: RawFileAdaptor
            eln:
              component: FileEditQuantity
      sub_sections:
        MyRowModeSubsection2:
          repeats: true  # Note: you need to make this section repeated
          section: # Important! you must define sub_section as "section" type
            more:
              label_quantity: my_quantity_6  # It shows this name in the list of instanced sections
            quantities:
              my_quantity_6:
                type: str
                m_annotations:
                  tabular:
                    name: Sheet1/Header 6
    MyRowMode3:
      base_sections:
        - nomad.parsing.tabular.TableData
        - nomad.datamodel.data.EntryData
      m_annotations:
        eln:
      quantities:
        data_file:
          type: str
          m_annotations:
            tabular_parser:
              path_to_data_file: '#data/data_file'  # only needed for MyMixedMode
              comment: '#'
              mode: row
              target_sub_section:
                - MyRowModeSubsection2
            browser:
              adaptor: RawFileAdaptor
            eln:
              component: FileEditQuantity
      sub_sections:
        MyRowModeSubsection2:
          repeats: true  # Note: you need to make this section repeated
          section: # Important! you must define sub_section as "section" type
            more:
              label_quantity: my_quantity_5  # It shows this name in the list of instanced sections
            quantities:
              my_quantity_5:
                type: str
                m_annotations:
                  tabular:
                    name: Sheet1/Header 5
    MyMixedMode:
      base_sections:
       # - nomad.parsing.tabular.TableData
        - nomad.datamodel.data.EntryData
      m_annotations:
        eln:
      quantities:
        data_file:
          type: str
          description: "A reference to an uploaded .xlsx"
          m_annotations:
            tabular_parser:
              # sep: ','  # needed for csv files
              comment: '#'
              mode: column
            browser:
              adaptor: RawFileAdaptor
            eln:
              component: FileEditQuantity
        # my_quantity_6:
        #   type: str
        #   shape: ['*']
        #   m_annotations:
        #     tabular:
        #       name: Sheet1/Header 6
      sub_sections:
        MySubsection1:
          repeats: true
          section: '#/MyRowMode' # Important! you must define sub_section as "section" type
        MySubsection2:
          repeats: true
          section: # Important! you must define sub_section as "section" type
            base_section: '#/MyColumnMode'
        MySubsection3:
          section: '#/MyRowMode2'
        MySubsection4:
          section:
            base_section: '#/MyRowMode3'
data:
  m_def: ../upload/raw/tabular-parser.schema.archive.yaml#MyMixedMode
  data_file: datafile.xlsx
  MySubsection1:
    - m_def: ../upload/raw/tabular-parser.schema.archive.yaml#MyRowMode
  MySubsection2:
    - m_def: ../upload/raw/tabular-parser.schema.archive.yaml#MyColumnMode

datafile.xlsx

@amgo already has the files to reproduce this error

Assignee Loading
Time tracking Loading