Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 216
    • Issues 216
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 27
    • Merge requests 27
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-labnomad-lab
  • nomad-FAIRnomad-FAIR
  • Issues
  • #484
Closed
Open
Issue created Jan 21, 2021 by Markus Scheidgen@mscheidgOwner19 of 21 checklist items completed19/21 checklist items

Problems in new parsers

These are the top 20 exception from the last reprocessing of aims, exciting, abinit, crystal.

In addition to these, there are many calculations that produce incomplete results. This is most easily to spot, if you look at the system metadata. Many calculations have no formula. This indicates they have no system. This indicates the parser (even through running successfully) did not match much. Probably these calculation require slightly more relaxed regexps due to code version differences.

  • 16879, parsers/fhi-aims, xx9izr2yRputru0JleV7-Q:t-ZPO5zE2Jw9DkP0m8fGQgR6vC-7
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
    self.parse_method()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1302, in parse_method
    sec_method.x_fhi_aims_controlInOut_number_of_spin_channels = self.out_parser.get_number_of_spin_channels()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 710, in get_number_of_spin_channels
    return array_size.get('Number of spin channels', 1)
AttributeError: 'NoneType' object has no attribute 'get'
  • 5891, in normalizer, qhx0VdcgS3inMmKECuN3KA:zw5o6RH9ZBAyDlsc4CXhWml09CLc
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 558, in normalizing
    normalizer(self._parser_results).normalize(logger=logger)
  File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 286, in normalize
    self.fill(context)
  File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 188, in fill
    struct.normalize(context)
  File "/app/nomad/normalizing/encyclopedia/material.py", line 739, in normalize
    self.periodicity(ideal, prim_atoms)
  File "/app/nomad/normalizing/encyclopedia/material.py", line 577, in periodicity
    raise ValueError("Could not detect the periodic dimensions in a 1D system.")
ValueError: Could not detect the periodic dimensions in a 1D system.
  • 3084, parsers/lammps, DDZKox_nQ_CLTPPaGSr0_w:zzStifnyE7SwZuV3SGagFAfCtekQ
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 787, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 766, in parse
    sec_md.with_trajectory = self.traj_parser.with_trajectory()
  File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 199, in with_trajectory
    return self.get('atoms_info') is not None
  File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
    val = self.results.get(key, None)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
    self.parse(self._key, **self._kwargs)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse
    if self.file_mmap is None:
  File "/app/nomad/parsing/file_parser/text_parser.py", line 345, in file_mmap
    with open(self.mainfile) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
  • 2711, parsers/fhi-aims, o-8XYG0JS5OJU59uchgA7w:zjS-6QqvVUrBNsQgaQGNEHRcipV1
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1473, in parse
    sec_parallel_task_assignement.x_fhi_aims_parallel_task_host = task_hosts[i]
  File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
    return super().__setattr__(name, value)
  File "/app/nomad/metainfo/metainfo.py", line 2221, in __set__
    (value, type(value), self, self.type))
TypeError: The value ['Task' '0' 'of' '512' '(0,0,0,0,0,0)' 'R1B-M0-N00-J00'] with type <class 'numpy.ndarray'> for quantity fhiaimsparser.metainfo.fhi_aims.x_fhi_aims_section_parallel_task_assignement.x_fhi_aims_parallel_task_host:Quantity is not of type <class 'str'>
  • 2271, parsers/fhi-aims, zb79w_T9Q_qbstffWBW46g:JHXpfS4E6piQw8ohphNqL8VglHyr
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
    self.parse_method()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1342, in parse_method
    sec_xc_func.XC_functional_weight = weight(hybrid_coeff)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 766, in <lambda>
    'name': 'GGA_X_PBE', 'weight': lambda x: 0.75 if x is None else 1.0 - x},
TypeError: unsupported operand type(s) for -: 'float' and 'str'
  • 1105, parsers/exciting, pvRdA8EBQ--7UQdVou5s-w:xrP0AfYb8-JsPChh0t0OthNP7K1X
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse
    self.parse_configurations()
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2053, in parse_configurations
    self.parse_file(f, sec_scc)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1332, in parse_file
    parser_function(section)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1171, in _parse_eigenvalues
    sec_eigenvalues.eigenvalues_values = get_data('eigenvalues')
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1164, in get_data
    res = np.hstack([v[key] for v in data])
  File "<__array_function__ internals>", line 6, in hstack
  File "/usr/local/lib/python3.7/site-packages/numpy/core/shape_base.py", line 346, in hstack
    return _nx.concatenate(arrs, 1)
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 2 and the array at index 18 has size 1
  • 1080, in normalizer, uCSlNMEWQfOS5TMvsQXz1A:ziSEudHjaTYIe3Jzj-MKRZn4VkDt
Traceback (most recent call last):
  File "/app/nomad/normalizing/dos.py", line 149, in normalize
    n_atoms=number_of_atoms
  File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 25, in calculate
    self.indices, self.bins = self._calculate_bytes(raw_energies, raw_dos, grid)
  File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 88, in _calculate_bytes
    energy, dos = np.transpose([(e,d) for e,d in zip(energy, dos) if (e >= grid_array[0][0] and e <= grid_array[-1][0])])
ValueError: not enough values to unpack (expected 2, got 0)
  • 768, parsers/fhi-aims, iZGJdsGSRm6DYERVpo74fA:0iEfV-_AQA6wlw6mneY5ol8ia1vj
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
    self.parse_method()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1259, in parse_method
    for key, val in self.control_parser.items():
  File "/app/nomad/parsing/file_parser/text_parser.py", line 365, in items
    yield key, self.get(key)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
    val = self.results.get(key, None)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
    self.parse(self._key, **self._kwargs)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse
    if self.file_mmap is None:
  File "/app/nomad/parsing/file_parser/text_parser.py", line 345, in file_mmap
    with open(self.mainfile) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
  • 341, parsers/fhi-aims, wKiS4s-NS9Ky5o1OtYKI2w:2IIcVSSBI5ktFmjUyfMISbWYaOiI
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
    self.parse_method()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1315, in parse_method
    self.parse_topology()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1424, in parse_topology
    species = self.out_parser['control_inout']['species']
TypeError: 'NoneType' object is not subscriptable
  • 278, parsers/fhi-aims, qSCiRCcYSaifBVn00EFOQA:yHb3bEu31Sb24V5eRXcOJ63XX3bk
Traceback (most recent call last):
  File "/app/nomad/parsing/file_parser/text_parser.py", line 406, in _parse_quantities
    value_processed = quantities[i].to_data(values)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in to_data
    val_out = [process(val) for val in val_in]
  File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in <listcomp>
    val_out = [process(val) for val in val_in]
  File "/app/nomad/parsing/file_parser/text_parser.py", line 158, in process
    val = self.str_operation(val)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 480, in str_to_energy_components
    unit = {'Ha': 'hartree', 'eV': 'eV'}.get(vi[1], None)
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1459, in parse
    value = self.out_parser.get(key)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
    val = self.results.get(key, None)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
    self.parse(self._key, **self._kwargs)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 524, in parse
    self._parse_quantity(quantity)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 463, in _parse_quantity
    value.append(sub_parser.parse())
  File "/app/nomad/parsing/file_parser/text_parser.py", line 516, in parse
    self._parse_quantities(quantities_findall)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 419, in _parse_quantities
    self.logger.warn('Error setting value', data=dict(quantity=quantities[i].name))
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1982, in warn
    warning(msg, *args, **kwargs)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1977, in warning
    root.warning(msg, *args, **kwargs)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1390, in warning
    self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'data'
  • 213, parsers/crystal, gddy5rHgRceN2sZZ6yTnrA:zbuT8J3-vcIknvze4gbDSWQDPEfP
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/crystalparser/crystal_parser.py", line 566, in parse
    dist, minor = distribution.split(" : ", 1)
AttributeError: 'NoneType' object has no attribute 'split'
  • 199, parsers/phonopy, z369XxTOT5S3db4Q2vrbqg:xeYjLT2hhwWskUCXdeqVg3bkND6h
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 498, in process_phonon
    raise ValueError("No method information available in referenced calculation.")
ValueError: No method information available in referenced calculation.
  • 171, parsers/exciting, vDg3jiguSsau7B2R9XjxMQ:o9pLQY2RAWuFd58rYJVgrzvO9SuO
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse
    self.parse_method()
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1802, in parse_method
    xc_functional_names = self.info_parser.get_xc_functional_name()
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1002, in get_xc_functional_name
    xc_functional = self.get('initialization').get('x_exciting_xc_functional', None)
AttributeError: 'NoneType' object has no attribute 'get'
  • 160, parsers/phonopy, win2cjTvRF6t_KjDmfVlZw:IxR6Cf1HLzo8zeedPFUHzO0t74BN
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 185, in parse
    phonopy_obj = self.phonopy_obj
  File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 50, in phonopy_obj
    self._build_phonopy_object_fhi_aims()
  File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 77, in _build_phonopy_object_fhi_aims
    cell_obj, supercell_matrix, displacement, sym)
  File "/usr/local/lib/python3.7/site-packages/phonopyparser/fhiaims_io.py", line 137, in read_forces_aims
    raise Exception("No phonon calculations found")
Exception: No phonon calculations found
  • 144, in normalizer, uCSlNMEWQfOS5TMvsQXz1A:xp4iSuT8IXqePP_KNuQaR0pOeC9z
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 558, in normalizing
    normalizer(self._parser_results).normalize(logger=logger)
  File "/app/nomad/normalizing/band_structure.py", line 63, in normalize
    self.add_band_gaps(band, energy_reference)
  File "/app/nomad/normalizing/band_structure.py", line 181, in add_band_gaps
    energies = np.concatenate(energies, axis=2)
  File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 9 and the array at index 6 has size 27
  • 126, parsers/fhi-aims, jSSVtYWfTZacc7HjWPSqHQ:a8r8KkvKXWams50UhzMGCxY0IGqH
TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1477, in parse
    self.parse_configurations()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1190, in parse_configurations
    parse_section(section)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1147, in parse_section
    forces_raw[-len(forces):], 'eV/angstrom')
  File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
    return super().__setattr__(name, value)
  File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__
    obj.m_set(self, value)
  File "/app/nomad/metainfo/metainfo.py", line 956, in m_set
    value = self.__to_np(quantity_def, value)
  File "/app/nomad/metainfo/metainfo.py", line 920, in __to_np
    value = value.astype(quantity_def.type)
  File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
    value = func(*args, **kwargs)
ValueError: setting an array element with a sequence.
  • 107, parsers/exciting, pzD88LYnTty-c_Xg07dqwg:wRtykAr2p9qUNTGDT-fZabQozNB4
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse
    self.parse_configurations()
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2061, in parse_configurations
    sec_scc.x_exciting_geometry_optimization_method = optimization_step.get('method')
AttributeError: 'NoneType' object has no attribute 'x_exciting_geometry_optimization_method'
  • 101, parsers/fhi-aims, ZFNMJDbJQ9aUq3o-aMq7lw:n43gSkg2cIhEM60wbmipsKQo1rig
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1477, in parse
    self.parse_configurations()
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1190, in parse_configurations
    parse_section(section)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1163, in parse_section
    parse_dos(section)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 941, in parse_dos
    energies, dos = read_projected_dos(proj_dos_files)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 897, in read_projected_dos
    energies = pint.Quantity(data[0], 'eV')
TypeError: 'NoneType' object is not subscriptable
  • 61, parsers/exciting, ff8C1I9LTuiNVu5fyPVxnw:xHhm3RgE-DzKTJ8X1qYc2xm01Ld6
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2132, in parse
    self.parse_xs()
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1631, in parse_xs
    self.parse_file('input.xml', sec_method)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1332, in parse_file
    parser_function(section)
  File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1368, in _parse_input_xs
    sec_method.x_exciting_xs_screening_rgkmax = rgkmax
  File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
    return super().__setattr__(name, value)
  File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__
    obj.m_set(self, value)
  File "/app/nomad/metainfo/metainfo.py", line 956, in m_set
    value = self.__to_np(quantity_def, value)
  File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np
    (quantity_def, value))
nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_screening_rgkmax:Quantity has not a unit, but value 3.5 bohr has.
  • 15, parsers/crystal, llZrblH4THCuJvVjEBXBFA:VdOUqF-aHTwlDSRcuoubmNBPFxHo
Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/crystalparser/crystal_parser.py", line 618, in parse
    atomic_numbers,
UnboundLocalError: local variable 'atomic_numbers' referenced before assignment
  • 13, parsers/fhi-aims, w8VbMKDwSb27iSM2oCE4Kg:kEp6pGbAJfjwn5r1PuvciCi1IIPd
Traceback (most recent call last):
  File "/app/nomad/parsing/file_parser/text_parser.py", line 406, in _parse_quantities
    value_processed = quantities[i].to_data(values)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in to_data
    val_out = [process(val) for val in val_in]
  File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in <listcomp>
    val_out = [process(val) for val in val_in]
  File "/app/nomad/parsing/file_parser/text_parser.py", line 158, in process
    val = self.str_operation(val)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 467, in str_to_eigenvalues
    eig, occ = eigs[1:3]
ValueError: not enough values to unpack (expected 2, got 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/nomad/processing/data.py", line 453, in parsing
    self._parser_results, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
    parser.parse(filepath, archive, logger)
  File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1459, in parse
    value = self.out_parser.get(key)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
    val = self.results.get(key, None)
  File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
    self.parse(self._key, **self._kwargs)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 524, in parse
    self._parse_quantity(quantity)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 463, in _parse_quantity
    value.append(sub_parser.parse())
  File "/app/nomad/parsing/file_parser/text_parser.py", line 516, in parse
    self._parse_quantities(quantities_findall)
  File "/app/nomad/parsing/file_parser/text_parser.py", line 419, in _parse_quantities
    self.logger.warn('Error setting value', data=dict(quantity=quantities[i].name))
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1982, in warn
    warning(msg, *args, **kwargs)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1977, in warning
    root.warning(msg, *args, **kwargs)
  File "/usr/local/lib/python3.7/logging/__init__.py", line 1390, in warning
    self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'data'

Old issues description: These are exceptions thrown by parsers during reprocessing all non staging exciting and abinit code runs. I only checked the first one, but it reproduces with nomad parse ....

exception_hash.keyword: Descending # calc_id upload_id mainfile parser exception
s7J4tcDHcGPdLgK2ZnnzqhE56hd3 1105 xrP0AfYb8-JsPChh0t0OthNP7K1X pvRdA8EBQ--7UQdVou5s-w volume-1.5/0.965489384605630/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1176, in _parse_eigenvalues sec_eigenvalues.eigenvalues_values = get_data('eigenvalues') File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1169, in get_data res = np.hstack([v[key] for v in data]) File "<array_function internals>", line 6, in hstack File "/usr/local/lib/python3.7/site-packages/numpy/core/shape_base.py", line 346, in hstack return _nx.concatenate(arrs, 1) File "<array_function internals>", line 6, in concatenate ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 2 and the array at index 18 has size 1
hwe7MyzCotT17K4FWIlLLnLyII3_ 171 yEqUiNBD7iu3P4YZmzgvpj2CbFV6 QLFt5odgS1S1LQ_MDbZgpQ silver-tddft/silver-loss/6/INFOXS.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2124, in parse self.parse_method() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1807, in parse_method xc_functional_names = self.info_parser.get_xc_functional_name() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1002, in get_xc_functional_name xc_functional = self.get('initialization').get('x_exciting_xc_functional', None) AttributeError: 'NoneType' object has no attribute 'get'
MEQ6y79UyB-M8DAu-NR4SBHzEJIq 107 wRtykAr2p9qUNTGDT-fZabQozNB4 pzD88LYnTty-c_Xg07dqwg home1/UTE/ETERRNAL/DATA/FINISHED/CGeSn/000016/GG/L_GAMMA/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2066, in parse_configurations sec_scc.x_exciting_geometry_optimization_method = optimization_step.get('method') AttributeError: 'NoneType' object has no attribute 'x_exciting_geometry_optimization_method'
HXW2tu3a-mtJmMCuvzjUTwTvISba 62 xHhm3RgE-DzKTJ8X1qYc2xm01Ld6 ff8C1I9LTuiNVu5fyPVxnw AB@Graphene/azobenzene/cis/BSE/rpa/INFO_QMT001.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2130, in parse self.parse_xs() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1636, in parse_xs self.parse_file('input.xml', sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1373, in _parse_input_xs sec_method.x_exciting_xs_screening_rgkmax = rgkmax File "/app/nomad/metainfo/metainfo.py", line 849, in setattr return super().setattr(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in set obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np (quantity_def, value)) nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_screening_rgkmax:Quantity has not a unit, but value 3.5 bohr has.
3kmtj-uvUs_AtxPSd_mgAoWmUY5i 14 LuH6KeBENcz3_mwE8HOwvYKJyqLw 0NR0E6gqQ2yVicJCMEP4SA c-Si-OCTSF/12x12x12-BSE-PBE/INFO_QMT001.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2130, in parse self.parse_xs() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1636, in parse_xs self.parse_file('input.xml', sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1360, in _parse_input_xs sec_method.x_exciting_xs_rgkmax = rgkmax File "/app/nomad/metainfo/metainfo.py", line 849, in setattr return super().setattr(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in set obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np (quantity_def, value)) nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_rgkmax:Quantity has not a unit, but value 8.0 bohr has.
2khWqBfkrqi65UqODhX0_U-Dp5qs 6 fwG3jU4_PBV1uWdAzFtXotFs293m dkZ_BRDqSA2DeGsH3ItFyw conv_ecut/Si.out0006 parsers/abinit Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 983, in parse self.parse_dataset(dataset) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 847, in parse_dataset sec_method = self.parse_method(dataset) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 759, in parse_method sec_method.number_of_spin_channels = int(self.out_parser.get_input_var('nsppol', nd, 1)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
h04dOeanRAxBsfNPaVUnydXG8kSu 5 mVpv7QL-IkSnnbPdZB0CxrPjhtwG ML2qGfRqSnmxDLvPKkrLMQ LiH-pbc/130/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2051, in parse_configurations sec_scc = parse_configuration(self.info_parser.get('groundstate')) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2042, in parse_configuration sec_system = self.parse_system(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1995, in parse_system setattr(sec_system, name, val) File "/app/nomad/metainfo/metainfo.py", line 849, in setattr return super().setattr(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in set obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 934, in __to_np value = quantity_def.type.type(value) ValueError: invalid literal for int() with base 10: '********'
cfgruvgumKUmo3jBj8wCYAPfsg7e 2 Ukv9vOpmfindZocfvucn3eYOFFZq bBwLURrsQzyT-avIDAsNhA silicon-gw/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse self.parse_gw() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1712, in parse_gw self.parse_file(f, sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1665, in _parse_input_gw 'gw/selfenergy/npol', 0) File "/app/nomad/metainfo/metainfo.py", line 849, in setattr return super().setattr(name, value) File "/app/nomad/metainfo/metainfo.py", line 2221, in set (value, type(value), self, self.type)) TypeError: The value 2 with type <class 'numpy.int64'> for quantity nomad.datamodel.metainfo.common_dft.Method.gw_self_energy_c_number_of_poles:Quantity is not of type <class 'int'>
QptEjMdzQgru7wm-iUM0sYRQktG0 1 9UM_CpSzaystbWPZoghgjg3dZmC8 CgtgzCMXTUyY0tjfBYXAnA TiO2/anatese/K/nomad/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1106, in _parse_dos if self.dos_parser.get('totaldos', None) is None: File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 505, in parse if not self.total_dos: File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 471, in total_dos self._total_dos = self.root.findall('./%s/%s' % (self._totaldos_key, self._diagram_key)) File "/app/nomad/parsing/file_parser/xml_parser.py", line 52, in root self._file_handler = ElementTree.parse(self.mainfile).getroot() File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse tree.parse(source, parser) File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 598, in parse self._root = parser._parse_whole(source) File "", line None xml.etree.ElementTree.ParseError: no element found: line 111468, column 0
g4xYYjGHqxmbahc5Xk0jQrB2BvLv 1 B7YX4kB9f1PSwYbeJeUcExy-UKZw cdAK5_-gQ4GLFbZrgBRpQQ conv/workdir/ngridk_08-rgkmax_05/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1162, in _parse_eigenvalues if self.eigval_parser.get('eigenvalues_occupancies', None) is None: File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse if self.file_mmap is None: File "/app/nomad/parsing/file_parser/text_parser.py", line 348, in file_mmap offset=self._file_offset) ValueError: cannot mmap an empty file
iwirABQpTlKQWqHEg7jS-2l-oA5w 1 qQbcfYsilb40jkYEN5hoW9PUdUrY mcyQ9xEwQu2uNrkEtKNobg volumes-rgkmax13/volume-2.1/1.01315940382018/INFO.OUT parsers/exciting Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2124, in parse self.parse_method() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1813, in parse_method correlation = self.input_xml_parser.get('libxc/correlation', None) File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/app/nomad/parsing/file_parser/xml_parser.py", line 76, in parse if not self.root: File "/app/nomad/parsing/file_parser/xml_parser.py", line 52, in root self._file_handler = ElementTree.parse(self.mainfile).getroot() File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse tree.parse(source, parser) File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 598, in parse self._root = parser._parse_whole(source) File "", line None xml.etree.ElementTree.ParseError: no element found: line 1, column 0

In addition, there are plenty of ABINIT calc that do not have a section_system, but no parser errors: https://nomad-lab.eu/prod/rae/reprocess/gui/search?visualization=system&dft.code_name=ABINIT&dft.system=unavailable

I am not sure if this is all, because the numbers do not add up. There are roughly ~2500 exciting runs with missing formulas/systems, but only ~2000 exceptions. We will see after bug fixes.

Edited Jan 26, 2021 by Alvin Noe Ladines
Assignee
Assign to
Time tracking