nomad-FAIR issueshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues2021-03-17T08:15:08Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/500Problems with the new VASP parser2021-03-17T08:15:08ZMarkus ScheidgenProblems with the new VASP parserThis time its really about the new VASP parser.
There are still two important issue types:
- [x] Some of the materials project data (27k entries) the run XML is invalid. A few closing tags missing. In the old parser, we somehow ignored ...This time its really about the new VASP parser.
There are still two important issue types:
- [x] Some of the materials project data (27k entries) the run XML is invalid. A few closing tags missing. In the old parser, we somehow ignored the XML parser errors and could still process the files. Try to mimic the same. Example: https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/uiNy_zS0S7Kx-Ru-rAlSNA/phFMspmymFkaFA7n6XFrLFx5jeuI/logs
- [x] For a lot of entries (32k) the parse creates a system with site coordinates, but the atom labels are missing. Example: https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/nHo8kktjSviBVsqMt0BLuw/wHrB0Pk-p0pypeE0pzNQaWf1RcN8/archive/section_workflow/calculation_result_ref/single_configuration_calculation_to_system_ref/atom_positions
Otherwise, these are the results from the reprocessing with your last fixes:
- [ ] **480**, parsers/vasp, [v_XEJQ1ATCmRv6xutSXbzg:mNrUgVAN0brYiQDt8lw_ksRBarRP](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/v_XEJQ1ATCmRv6xutSXbzg/mNrUgVAN0brYiQDt8lw_ksRBarRP/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1101, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 885, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 639, in atom_info
self._atom_info['n_atoms'] = sum(ions)
TypeError: unsupported operand type(s) for +: 'int' and 'str'
```
- [ ] **34**, parsers/vasp, [wS4nTynCRVinVLoXPp283w:xru7dr5W31rwqfDUroffjJLLPbww](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/wS4nTynCRVinVLoXPp283w/xru7dr5W31rwqfDUroffjJLLPbww/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1103, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1072, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1037, in parse_dos
dos, fields = self.parser.get_partial_dos(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 756, in get_partial_dos
n_atoms = self.atom_info['n_atoms']
KeyError: 'n_atoms'
```
- [ ] **1**, parsers/vasp, [N6rQDIFJT36PituBeUrqEA:JNHBGOZn-kSVR29iavzBvae19G_C](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/N6rQDIFJT36PituBeUrqEA/JNHBGOZn-kSVR29iavzBvae19G_C/logs)
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
next(loop)
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 303, in create_loop
poll_timeout = fire_timers(propagate=propagate) if scheduled else 1
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 145, in fire_timers
entry()
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/timer.py", line 68, in __call__
return self.fun(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/timer.py", line 130, in _reschedules
return fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/kombu/connection.py", line 313, in heartbeat_check
return self.transport.heartbeat_check(self.connection, rate=rate)
File "/usr/local/lib/python3.7/site-packages/kombu/transport/pyamqp.py", line 149, in heartbeat_check
return connection.heartbeat_tick(rate=rate)
File "/usr/local/lib/python3.7/site-packages/amqp/connection.py", line 744, in heartbeat_tick
self.send_heartbeat()
File "/usr/local/lib/python3.7/site-packages/amqp/connection.py", line 695, in send_heartbeat
self.frame_writer(8, 0, None, None, None)
File "/usr/local/lib/python3.7/site-packages/amqp/method_framing.py", line 189, in write_frame
write(view[:offset])
File "/usr/local/lib/python3.7/site-packages/amqp/transport.py", line 305, in write
self._write(s)
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1101, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 885, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 639, in atom_info
self._atom_info['n_atoms'] = sum(ions)
TypeError: unsupported operand type(s) for +: 'int' and 'str'
```
- [ ] **1**, parsers/vasp, [LX65oD_mSI-zJYP4ttIChg:kJV30wnielD5VE6_tW3eVlK5_Dhq](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/LX65oD_mSI-zJYP4ttIChg/kJV30wnielD5VE6_tW3eVlK5_Dhq/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1101, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 885, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 416, in atom_info
array_info[fields[i]].append(val[i])
TypeError: 'NoneType' object is not subscriptable
```
**Old issue:**
- [x] **9372**, *in normalizer*, [8Y5OzjCbQKCAPH9OSJftfA:_W-3vpvD5mfmH0VpDIqRbxItHwZw](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/8Y5OzjCbQKCAPH9OSJftfA/_W-3vpvD5mfmH0VpDIqRbxItHwZw/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 301, in symmetry_analysis
space_group_number = symm.get_space_group_number()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 114, in get_space_group_number
dataset = self.get_symmetry_dataset()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 566, in get_symmetry_dataset
'Spglib error when finding symmetry dataset.')
matid.exceptions.CellNormalizationError: Spglib error when finding symmetry dataset.
```
- [x] **562**, parsers/vasp, [6jR5WJQvSVSrflHsGFx4JA:c5MNoko2F5JRyd21ng1M4R4N2CrV](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/6jR5WJQvSVSrflHsGFx4JA/c5MNoko2F5JRyd21ng1M4R4N2CrV/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1086, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 870, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 628, in atom_info
self._atom_info['n_atoms'] = sum(ions)
TypeError: unsupported operand type(s) for +: 'int' and 'str'
```
- [x] **125**, parsers/vasp, [6Jsud_OzS06cVDUunSLKtQ:idG3yalNgMALlY5L2mUN_-mhPylM](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/6Jsud_OzS06cVDUunSLKtQ/idG3yalNgMALlY5L2mUN_-mhPylM/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1086, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 870, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 411, in atom_info
array_info[fields[i]].append(val[i])
TypeError: 'NoneType' object is not subscriptable
```
- [x] **105**, *in normalizer*, [1lW8mBSbTii8wIBpaQaRtg:WTvXT5LHquxyhxQ5lFbDih3t6y2b](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/1lW8mBSbTii8wIBpaQaRtg/WTvXT5LHquxyhxQ5lFbDih3t6y2b/logs)
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
next(loop)
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 301, in create_loop
item()
File "/usr/local/lib/python3.7/site-packages/vine/promises.py", line 170, in __call__
return self.throw()
File "/usr/local/lib/python3.7/site-packages/vine/promises.py", line 167, in __call__
retval = fun(*final_args, **final_kwargs)
File "/usr/local/lib/python3.7/site-packages/kombu/transport/base.py", line 218, in _read
raise RecoverableConnectionError('Socket was disconnected')
amqp.exceptions.RecoverableConnectionError: Socket was disconnected
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 301, in symmetry_analysis
space_group_number = symm.get_space_group_number()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 114, in get_space_group_number
dataset = self.get_symmetry_dataset()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 566, in get_symmetry_dataset
'Spglib error when finding symmetry dataset.')
matid.exceptions.CellNormalizationError: Spglib error when finding symmetry dataset.
```
- [x] **86**, parsers/vasp, [6Jsud_OzS06cVDUunSLKtQ:nYUkIOLtmqvxVDZdSh6KwePYl0eF](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/6Jsud_OzS06cVDUunSLKtQ/nYUkIOLtmqvxVDZdSh6KwePYl0eF/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1088, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1054, in parse_configurations
parse_eigenvalues(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 954, in parse_eigenvalues
eigenvalues = self.parser.get_eigenvalues(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 443, in get_eigenvalues
n_kpts = len(self.kpoints_info.get('k_mesh_points', []))
TypeError: object of type 'NoneType' has no len()
```
- [ ] **35**, *in normalizer*, [-VXrWVKjQIOHNVoU8Ft40Q:tsBC1IaYWFKfPbXlHB_ERQlx1LBh](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/-VXrWVKjQIOHNVoU8Ft40Q/tsBC1IaYWFKfPbXlHB_ERQlx1LBh/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 195, in normalize_system
atoms.set_positions(1e10 * atom_positions.magnitude)
File "/usr/local/lib/python3.7/site-packages/ase/atoms.py", line 662, in set_positions
self.set_array('positions', newpositions, shape=(3,))
File "/usr/local/lib/python3.7/site-packages/ase/atoms.py", line 475, in set_array
(name, a.shape, b.shape))
ValueError: Array "positions" has wrong shape (0,) != (0, 3).
```
- [x] **34**, *in normalizer*, [yPkSfxx8RVSOcTQNKwhQJw:nslOUAQgpEUrfbMrtuQPU4YftT-z](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/yPkSfxx8RVSOcTQNKwhQJw/nslOUAQgpEUrfbMrtuQPU4YftT-z/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 303, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 205, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 740, in normalize
prim_atoms = symmetry_analyzer.get_primitive_system()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 272, in get_primitive_system
conv_sys = self.get_conventional_system()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 305, in get_conventional_system
spglib_conv_sys = self._get_spglib_conventional_system()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 582, in _get_spglib_conventional_system
dataset = self.get_symmetry_dataset()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 566, in get_symmetry_dataset
'Spglib error when finding symmetry dataset.')
matid.exceptions.CellNormalizationError: Spglib error when finding symmetry dataset.
```
- [x] **24**, *in normalizer*, [5IvJz3uWTeu7Sgf197BSnw:YrIvxg5rzU9_y0HZIPP5hWXjvBIA](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/5IvJz3uWTeu7Sgf197BSnw/YrIvxg5rzU9_y0HZIPP5hWXjvBIA/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 303, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 205, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 522, in normalize
spg_number = symmetry_analyzer.get_space_group_number()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 114, in get_space_group_number
dataset = self.get_symmetry_dataset()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 566, in get_symmetry_dataset
'Spglib error when finding symmetry dataset.')
matid.exceptions.CellNormalizationError: Spglib error when finding symmetry dataset.
```
- [x] **18**, parsers/vasp, [plbG1LdbQ2aYp6eiB7ol8g:PjVZ8C35JZM5w8G_rlLaIhC0weTh](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/plbG1LdbQ2aYp6eiB7ol8g/PjVZ8C35JZM5w8G_rlLaIhC0weTh/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1086, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 847, in parse_method
incar_parameters = self.parser.get_incar()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 539, in get_incar
self.parser.mainfile).strip('OUTCAR'))
File "/usr/local/lib/python3.7/posixpath.py", line 146, in basename
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
```
- [x] **9**, *in normalizer*, [5IvJz3uWTeu7Sgf197BSnw:oBh5MiHjYXmDT8SkDoRN49tDdCW_](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/5IvJz3uWTeu7Sgf197BSnw/oBh5MiHjYXmDT8SkDoRN49tDdCW_/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 259, in system_type_analysis
cls = classifier.classify(atoms)
File "/usr/local/lib/python3.7/site-packages/matid/classification/classifier.py", line 185, in classify
system.wrap()
File "/usr/local/lib/python3.7/site-packages/ase/atoms.py", line 1890, in wrap
self.positions[:] = self.get_positions(wrap=True, **wrap_kw)
File "/usr/local/lib/python3.7/site-packages/ase/atoms.py", line 678, in get_positions
return wrap_positions(self.positions, self.cell, **wrap_kw)
File "/usr/local/lib/python3.7/site-packages/ase/geometry/geometry.py", line 81, in wrap_positions
np.asarray(positions).T).T - shift
File "<__array_function__ internals>", line 6, in solve
File "/usr/local/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 393, in solve
r = gufunc(a, b, signature=signature, extobj=extobj)
File "/usr/local/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 88, in _raise_linalgerror_singular
raise LinAlgError("Singular matrix")
numpy.linalg.LinAlgError: Singular matrix
```
- [x] **9**, *in normalizer*, [28T0hCzfQuCZ50gH-QeKAw:lP2LexSZzBoxD8OMTkF-YM0AN-KG](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/28T0hCzfQuCZ50gH-QeKAw/lP2LexSZzBoxD8OMTkF-YM0AN-KG/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 303, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 205, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 760, in normalize
self.lattice_parameters(ideal, std_atoms, ideal.periodicity)
File "/app/nomad/normalizing/encyclopedia/material.py", line 573, in lattice_parameters
a = np.linalg.norm(cell[periodic_indices[0], :]) * 1e-10
IndexError: index 0 is out of bounds for axis 0 with size 0
```
- [x] **5**, parsers/vasp, [plbG1LdbQ2aYp6eiB7ol8g:EYiQm81mGpHxIShi9y8Ub_F5c-VI](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/plbG1LdbQ2aYp6eiB7ol8g/EYiQm81mGpHxIShi9y8Ub_F5c-VI/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1088, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1057, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1001, in parse_dos
energies, values, integrated, e_fermi = self.parser.get_total_dos(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 724, in get_total_dos
n_dos = int(line.split()[2])
ValueError: invalid literal for int() with base 10: '5.08315306'
```
- [x] **5**, *in normalizer*, [4SFPVdb5Sj6Kn4U_cwWohg:f_qztS81ukPrLObA41RnC5i7JVZu](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/4SFPVdb5Sj6Kn4U_cwWohg/f_qztS81ukPrLObA41RnC5i7JVZu/logs)
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
next(loop)
File "/usr/local/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 364, in create_loop
cb(*cbargs)
File "/usr/local/lib/python3.7/site-packages/kombu/transport/base.py", line 238, in on_readable
reader(loop)
File "/usr/local/lib/python3.7/site-packages/kombu/transport/base.py", line 218, in _read
raise RecoverableConnectionError('Socket was disconnected')
amqp.exceptions.RecoverableConnectionError: Socket was disconnected
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 301, in symmetry_analysis
space_group_number = symm.get_space_group_number()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 114, in get_space_group_number
dataset = self.get_symmetry_dataset()
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 566, in get_symmetry_dataset
'Spglib error when finding symmetry dataset.')
matid.exceptions.CellNormalizationError: Spglib error when finding symmetry dataset.
```
- [x] **5**, *in normalizer*, [iGLjoQSPRc6d9IPJMGpxvg:ogRag81kOdbI2LCn0HFNsBk_sPp-](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/iGLjoQSPRc6d9IPJMGpxvg/ogRag81kOdbI2LCn0HFNsBk_sPp-/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 259, in system_type_analysis
cls = classifier.classify(atoms)
File "/usr/local/lib/python3.7/site-packages/matid/classification/classifier.py", line 212, in classify
return_factors=True
File "/usr/local/lib/python3.7/site-packages/matid/geometry.py", line 676, in get_displacement_tensor
D, D_len, factors = find_mic(flattened_disp, cell, pbc, max_distance)
File "/usr/local/lib/python3.7/site-packages/matid/geometry.py", line 725, in find_mic
Dr = np.dot(D, np.linalg.inv(cell))
File "<__array_function__ internals>", line 6, in inv
File "/usr/local/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 545, in inv
ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)
File "/usr/local/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 88, in _raise_linalgerror_singular
raise LinAlgError("Singular matrix")
numpy.linalg.LinAlgError: Singular matrix
```
- [x] **4**, *in normalizer*, [mBb0_3cyTR6eJoN4qX-bGg:zVHAXa8GKnEZTBsLibSwd3iiNeJu](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/mBb0_3cyTR6eJoN4qX-bGg/zVHAXa8GKnEZTBsLibSwd3iiNeJu/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 259, in system_type_analysis
cls = classifier.classify(atoms)
File "/usr/local/lib/python3.7/site-packages/matid/classification/classifier.py", line 212, in classify
return_factors=True
File "/usr/local/lib/python3.7/site-packages/matid/geometry.py", line 676, in get_displacement_tensor
D, D_len, factors = find_mic(flattened_disp, cell, pbc, max_distance)
File "/usr/local/lib/python3.7/site-packages/matid/geometry.py", line 750, in find_mic
tvecs = np.dot(tvec_factors, cell)
File "<__array_function__ internals>", line 6, in dot
ValueError: On entry to DGEMM parameter number 4 had an illegal value
```
- [x] **4**, parsers/vasp, [VshkgA9fRVaBUM1A99_pTA:-CjT6qeDr5dhnp50fV3uGVp-RnMJ](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/VshkgA9fRVaBUM1A99_pTA/-CjT6qeDr5dhnp50fV3uGVp-RnMJ/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1088, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1049, in parse_configurations
sec_system = parse_system(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 906, in parse_system
structure = self.parser.get_structure(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 658, in get_structure
positions = pint.Quantity(np.dot(positions, cell), 'angstrom')
File "<__array_function__ internals>", line 6, in dot
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'
```
- [x] **55**, *in normalizer*, [2jFT-SBtT4mnxQg06pgJOA:SzG3VquLpYI9AqsHd-_qXGZdcYBN](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/2jFT-SBtT4mnxQg06pgJOA/SzG3VquLpYI9AqsHd-_qXGZdcYBN/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/base.py", line 433, in wrapper
func(self, *args, **kwargs)
File "/app/nomad/processing/data.py", line 662, in archiving
archive_size = self.write_archive(self._parser_results)
File "/app/nomad/processing/data.py", line 703, in write_archive
raise e
File "/app/nomad/processing/data.py", line 696, in write_archive
return self.upload_files.write_archive(self.calc_id, archive.m_to_dict())
File "/app/nomad/files.py", line 354, in write_archive
raise e
File "/app/nomad/files.py", line 348, in write_archive
write_archive(archive_file_object.os_path, 1, data=[(calc_id, data)])
File "/app/nomad/archive.py", line 540, in write_archive
writer.add(uuid, entry)
File "/app/nomad/archive.py", line 203, in __exit__
raise exc_val
File "/app/nomad/archive.py", line 540, in write_archive
writer.add(uuid, entry)
File "/app/nomad/archive.py", line 238, in add
packed = self._toc_packer.pack(data)
File "/app/nomad/archive.py", line 103, in pack
result = super().pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 848, in pack
self._pack(obj)
File "/app/nomad/archive.py", line 119, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 839, in _pack
nest_limit - 1)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 941, in _pack_map_pairs
self._pack(v, nest_limit - 1)
File "/app/nomad/archive.py", line 142, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 835, in _pack
self._pack(obj[i], nest_limit - 1)
File "/app/nomad/archive.py", line 119, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 839, in _pack
nest_limit - 1)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 941, in _pack_map_pairs
self._pack(v, nest_limit - 1)
File "/app/nomad/archive.py", line 142, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 835, in _pack
self._pack(obj[i], nest_limit - 1)
File "/app/nomad/archive.py", line 116, in _pack
pack_result = self._buffer.write(packb(obj))
File "/app/nomad/archive.py", line 55, in packb
return __packer.pack(o)
File "msgpack/_packer.pyx", line 284, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 290, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 287, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 234, in msgpack._packer.Packer._pack
File "msgpack/_packer.pyx", line 263, in msgpack._packer.Packer._pack
File "msgpack/_packer.pyx", line 281, in msgpack._packer.Packer._pack
TypeError: can't serialize array([[ 0. , 0. , 0. ],
[ 0.16666667, 0. , 0. ],
[ 0.33333333, 0. , 0. ],
[ 0.5 , 0. , 0. ],
[ 0. , 0.16666667, -0. ],
[ 0.16666667, 0.16666667, 0. ],
[ 0.33333333, 0.16666667, 0. ],
[ 0.5 , 0.16666667, 0. ],
[-0.33333333, 0.16666667, 0. ],
[-0.16666667, 0.16666667, -0. ],
[ 0. , 0.33333333, -0. ],
[ 0.16666667, 0.33333333, 0. ],
[ 0.33333333, 0.33333333, -0. ],
[ 0.5 , 0.33333333, 0. ],
[-0.33333333, 0.33333333, -0. ],
[-0.16666667, 0.33333333, -0. ],
[ 0. , 0.5 , -0. ],
[ 0.16666667, 0.5 , 0. ],
[ 0.33333333, 0.5 , -0. ],
[ 0.5 , 0.5 , 0. ],
[ 0. , 0. , 0.16666667],
[ 0.16666667, 0. , 0.16666667],
[ 0.33333333, 0. , 0.16666667],
[ 0.5 , 0. , 0.16666667],
[-0.33333333, 0. , 0.16666667],
[-0.16666667, -0. , 0.16666667],
[ 0. , 0.16666667, 0.16666667],
[ 0.16666667, 0.16666667, 0.16666667],
[ 0.33333333, 0.16666667, 0.16666667],
[ 0.5 , 0.16666667, 0.16666667],
[-0.33333333, 0.16666667, 0.16666667],
[-0.16666667, 0.16666667, 0.16666667],
[ 0. , 0.33333333, 0.16666667],
[ 0.16666667, 0.33333333, 0.16666667],
[ 0.33333333, 0.33333333, 0.16666667],
[ 0.5 , 0.33333333, 0.16666667],
[-0.33333333, 0.33333333, 0.16666667],
[-0.16666667, 0.33333333, 0.16666667],
[ 0. , 0.5 , 0.16666667],
[ 0.16666667, 0.5 , 0.16666667],
[ 0.33333333, 0.5 , 0.16666667],
[ 0.5 , 0.5 , 0.16666667],
[-0.33333333, 0.5 , 0.16666667],
[-0.16666667, 0.5 , 0.16666667],
[ 0. , -0.33333333, 0.16666667],
[ 0.16666667, -0.33333333, 0.16666667],
[ 0.33333333, -0.33333333, 0.16666667],
[ 0.5 , -0.33333333, 0.16666667],
[-0.33333333, -0.33333333, 0.16666667],
[-0.16666667, -0.33333333, 0.16666667],
[ 0. , -0.16666667, 0.16666667],
[ 0.16666667, -0.16666667, 0.16666667],
[ 0.33333333, -0.16666667, 0.16666667],
[ 0.5 , -0.16666667, 0.16666667],
[-0.33333333, -0.16666667, 0.16666667],
[-0.16666667, -0.16666667, 0.16666667],
[ 0. , 0. , 0.33333333],
[ 0.16666667, 0. , 0.33333333],
[ 0.33333333, 0. , 0.33333333],
[ 0.5 , 0. , 0.33333333],
[-0.33333333, 0. , 0.33333333],
[-0.16666667, -0. , 0.33333333],
[ 0. , 0.16666667, 0.33333333],
[ 0.16666667, 0.16666667, 0.33333333],
[ 0.33333333, 0.16666667, 0.33333333],
[ 0.5 , 0.16666667, 0.33333333],
[-0.33333333, 0.16666667, 0.33333333],
[-0.16666667, 0.16666667, 0.33333333],
[ 0. , 0.33333333, 0.33333333],
[ 0.16666667, 0.33333333, 0.33333333],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.5 , 0.33333333, 0.33333333],
[-0.33333333, 0.33333333, 0.33333333],
[-0.16666667, 0.33333333, 0.33333333],
[ 0. , 0.5 , 0.33333333],
[ 0.16666667, 0.5 , 0.33333333],
[ 0.33333333, 0.5 , 0.33333333],
[ 0.5 , 0.5 , 0.33333333],
[-0.33333333, 0.5 , 0.33333333],
[-0.16666667, 0.5 , 0.33333333],
[ 0. , -0.33333333, 0.33333333],
[ 0.16666667, -0.33333333, 0.33333333],
[ 0.33333333, -0.33333333, 0.33333333],
[ 0.5 , -0.33333333, 0.33333333],
[-0.33333333, -0.33333333, 0.33333333],
[-0.16666667, -0.33333333, 0.33333333],
[ 0. , -0.16666667, 0.33333333],
[ 0.16666667, -0.16666667, 0.33333333],
[ 0.33333333, -0.16666667, 0.33333333],
[ 0.5 , -0.16666667, 0.33333333],
[-0.33333333, -0.16666667, 0.33333333],
[-0.16666667, -0.16666667, 0.33333333],
[ 0. , 0. , 0.5 ],
[ 0.16666667, 0. , 0.5 ],
[ 0.33333333, 0. , 0.5 ],
[ 0.5 , 0. , 0.5 ],
[ 0. , 0.16666667, 0.5 ],
[ 0.16666667, 0.16666667, 0.5 ],
[ 0.33333333, 0.16666667, 0.5 ],
[ 0.5 , 0.16666667, 0.5 ],
[-0.33333333, 0.16666667, 0.5 ],
[-0.16666667, 0.16666667, 0.5 ],
[ 0. , 0.33333333, 0.5 ],
[ 0.16666667, 0.33333333, 0.5 ],
[ 0.33333333, 0.33333333, 0.5 ],
[ 0.5 , 0.33333333, 0.5 ],
[-0.33333333, 0.33333333, 0.5 ],
[-0.16666667, 0.33333333, 0.5 ],
[ 0. , 0.5 , 0.5 ],
[ 0.16666667, 0.5 , 0.5 ],
[ 0.33333333, 0.5 , 0.5 ],
[ 0.5 , 0.5 , 0.5 ]])
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 423, in on_fail
self.write_archive(self._parser_results)
File "/app/nomad/processing/data.py", line 703, in write_archive
raise e
File "/app/nomad/processing/data.py", line 696, in write_archive
return self.upload_files.write_archive(self.calc_id, archive.m_to_dict())
File "/app/nomad/files.py", line 354, in write_archive
raise e
File "/app/nomad/files.py", line 348, in write_archive
write_archive(archive_file_object.os_path, 1, data=[(calc_id, data)])
File "/app/nomad/archive.py", line 540, in write_archive
writer.add(uuid, entry)
File "/app/nomad/archive.py", line 203, in __exit__
raise exc_val
File "/app/nomad/archive.py", line 540, in write_archive
writer.add(uuid, entry)
File "/app/nomad/archive.py", line 238, in add
packed = self._toc_packer.pack(data)
File "/app/nomad/archive.py", line 103, in pack
result = super().pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 848, in pack
self._pack(obj)
File "/app/nomad/archive.py", line 119, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 839, in _pack
nest_limit - 1)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 941, in _pack_map_pairs
self._pack(v, nest_limit - 1)
File "/app/nomad/archive.py", line 142, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 835, in _pack
self._pack(obj[i], nest_limit - 1)
File "/app/nomad/archive.py", line 119, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 839, in _pack
nest_limit - 1)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 941, in _pack_map_pairs
self._pack(v, nest_limit - 1)
File "/app/nomad/archive.py", line 142, in _pack
pack_result = super()._pack(obj, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/msgpack/fallback.py", line 835, in _pack
self._pack(obj[i], nest_limit - 1)
File "/app/nomad/archive.py", line 116, in _pack
pack_result = self._buffer.write(packb(obj))
File "/app/nomad/archive.py", line 55, in packb
return __packer.pack(o)
File "msgpack/_packer.pyx", line 284, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 290, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 287, in msgpack._packer.Packer.pack
File "msgpack/_packer.pyx", line 234, in msgpack._packer.Packer._pack
File "msgpack/_packer.pyx", line 263, in msgpack._packer.Packer._pack
File "msgpack/_packer.pyx", line 281, in msgpack._packer.Packer._pack
TypeError: can't serialize array([[ 0. , 0. , 0. ],
[ 0.16666667, 0. , 0. ],
[ 0.33333333, 0. , 0. ],
[ 0.5 , 0. , 0. ],
[ 0. , 0.16666667, -0. ],
[ 0.16666667, 0.16666667, 0. ],
[ 0.33333333, 0.16666667, 0. ],
[ 0.5 , 0.16666667, 0. ],
[-0.33333333, 0.16666667, 0. ],
[-0.16666667, 0.16666667, -0. ],
[ 0. , 0.33333333, -0. ],
[ 0.16666667, 0.33333333, 0. ],
[ 0.33333333, 0.33333333, -0. ],
[ 0.5 , 0.33333333, 0. ],
[-0.33333333, 0.33333333, -0. ],
[-0.16666667, 0.33333333, -0. ],
[ 0. , 0.5 , -0. ],
[ 0.16666667, 0.5 , 0. ],
[ 0.33333333, 0.5 , -0. ],
[ 0.5 , 0.5 , 0. ],
[ 0. , 0. , 0.16666667],
[ 0.16666667, 0. , 0.16666667],
[ 0.33333333, 0. , 0.16666667],
[ 0.5 , 0. , 0.16666667],
[-0.33333333, 0. , 0.16666667],
[-0.16666667, -0. , 0.16666667],
[ 0. , 0.16666667, 0.16666667],
[ 0.16666667, 0.16666667, 0.16666667],
[ 0.33333333, 0.16666667, 0.16666667],
[ 0.5 , 0.16666667, 0.16666667],
[-0.33333333, 0.16666667, 0.16666667],
[-0.16666667, 0.16666667, 0.16666667],
[ 0. , 0.33333333, 0.16666667],
[ 0.16666667, 0.33333333, 0.16666667],
[ 0.33333333, 0.33333333, 0.16666667],
[ 0.5 , 0.33333333, 0.16666667],
[-0.33333333, 0.33333333, 0.16666667],
[-0.16666667, 0.33333333, 0.16666667],
[ 0. , 0.5 , 0.16666667],
[ 0.16666667, 0.5 , 0.16666667],
[ 0.33333333, 0.5 , 0.16666667],
[ 0.5 , 0.5 , 0.16666667],
[-0.33333333, 0.5 , 0.16666667],
[-0.16666667, 0.5 , 0.16666667],
[ 0. , -0.33333333, 0.16666667],
[ 0.16666667, -0.33333333, 0.16666667],
[ 0.33333333, -0.33333333, 0.16666667],
[ 0.5 , -0.33333333, 0.16666667],
[-0.33333333, -0.33333333, 0.16666667],
[-0.16666667, -0.33333333, 0.16666667],
[ 0. , -0.16666667, 0.16666667],
[ 0.16666667, -0.16666667, 0.16666667],
[ 0.33333333, -0.16666667, 0.16666667],
[ 0.5 , -0.16666667, 0.16666667],
[-0.33333333, -0.16666667, 0.16666667],
[-0.16666667, -0.16666667, 0.16666667],
[ 0. , 0. , 0.33333333],
[ 0.16666667, 0. , 0.33333333],
[ 0.33333333, 0. , 0.33333333],
[ 0.5 , 0. , 0.33333333],
[-0.33333333, 0. , 0.33333333],
[-0.16666667, -0. , 0.33333333],
[ 0. , 0.16666667, 0.33333333],
[ 0.16666667, 0.16666667, 0.33333333],
[ 0.33333333, 0.16666667, 0.33333333],
[ 0.5 , 0.16666667, 0.33333333],
[-0.33333333, 0.16666667, 0.33333333],
[-0.16666667, 0.16666667, 0.33333333],
[ 0. , 0.33333333, 0.33333333],
[ 0.16666667, 0.33333333, 0.33333333],
[ 0.33333333, 0.33333333, 0.33333333],
[ 0.5 , 0.33333333, 0.33333333],
[-0.33333333, 0.33333333, 0.33333333],
[-0.16666667, 0.33333333, 0.33333333],
[ 0. , 0.5 , 0.33333333],
[ 0.16666667, 0.5 , 0.33333333],
[ 0.33333333, 0.5 , 0.33333333],
[ 0.5 , 0.5 , 0.33333333],
[-0.33333333, 0.5 , 0.33333333],
[-0.16666667, 0.5 , 0.33333333],
[ 0. , -0.33333333, 0.33333333],
[ 0.16666667, -0.33333333, 0.33333333],
[ 0.33333333, -0.33333333, 0.33333333],
[ 0.5 , -0.33333333, 0.33333333],
[-0.33333333, -0.33333333, 0.33333333],
[-0.16666667, -0.33333333, 0.33333333],
[ 0. , -0.16666667, 0.33333333],
[ 0.16666667, -0.16666667, 0.33333333],
[ 0.33333333, -0.16666667, 0.33333333],
[ 0.5 , -0.16666667, 0.33333333],
[-0.33333333, -0.16666667, 0.33333333],
[-0.16666667, -0.16666667, 0.33333333],
[ 0. , 0. , 0.5 ],
[ 0.16666667, 0. , 0.5 ],
[ 0.33333333, 0. , 0.5 ],
[ 0.5 , 0. , 0.5 ],
[ 0. , 0.16666667, 0.5 ],
[ 0.16666667, 0.16666667, 0.5 ],
[ 0.33333333, 0.16666667, 0.5 ],
[ 0.5 , 0.16666667, 0.5 ],
[-0.33333333, 0.16666667, 0.5 ],
[-0.16666667, 0.16666667, 0.5 ],
[ 0. , 0.33333333, 0.5 ],
[ 0.16666667, 0.33333333, 0.5 ],
[ 0.33333333, 0.33333333, 0.5 ],
[ 0.5 , 0.33333333, 0.5 ],
[-0.33333333, 0.33333333, 0.5 ],
[-0.16666667, 0.33333333, 0.5 ],
[ 0. , 0.5 , 0.5 ],
[ 0.16666667, 0.5 , 0.5 ],
[ 0.33333333, 0.5 , 0.5 ],
[ 0.5 , 0.5 , 0.5 ]])
```
**The old issue:**
@ladinesa ideally you would prioritise this (at least the first 5).
- [x] **1252**, parsers/vasp, [yJ_Jq-c_SzKd7OmSJChQ3A:tLuoZg0AlBPDXXyR0b5kox_Ncim-](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/yJ_Jq-c_SzKd7OmSJChQ3A/tLuoZg0AlBPDXXyR0b5kox_Ncim-/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1076, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 860, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 625, in atom_info
assert len(ions) == len(species)
AssertionError
```
- [x] **935**, parsers/vasp, [yJ_Jq-c_SzKd7OmSJChQ3A:kBmXMBOGS90-cRB6f2O6fYGiA1Wl](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/yJ_Jq-c_SzKd7OmSJChQ3A/kBmXMBOGS90-cRB6f2O6fYGiA1Wl/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1078, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1039, in parse_configurations
sec_system = parse_system(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 896, in parse_system
structure = self.parser.get_structure(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 656, in get_structure
positions = pint.Quantity(np.dot(positions, cell), 'angstrom')
File "<__array_function__ internals>", line 6, in dot
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'
```
- [x] **588**, parsers/vasp, [xwA0ep9zRgSypxxkCokJ1w:rT8lZLyJCz5ZvXe3S2LrQTTCrgO8](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/xwA0ep9zRgSypxxkCokJ1w/rT8lZLyJCz5ZvXe3S2LrQTTCrgO8/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1078, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1047, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1014, in parse_dos
sec_dos.dos_values_lm = pint.Quantity(dos, '1/eV').to('1/joule').magnitude
UnboundLocalError: local variable 'sec_dos' referenced before assignment
```
- [x] **483**, parsers/vasp, [yJ_Jq-c_SzKd7OmSJChQ3A:wrEVx9FRKpR8JV_fFXi0mJpO8MKE](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/yJ_Jq-c_SzKd7OmSJChQ3A/wrEVx9FRKpR8JV_fFXi0mJpO8MKE/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1076, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 860, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 626, in atom_info
self._atom_info['n_atoms'] = sum(ions)
TypeError: unsupported operand type(s) for +: 'int' and 'str'
```
- [x] **73**, parsers/vasp, [vez_1F7xRxyLBw-aw6dWPw:ynnk4mILRzZGDjsnA-bsV4oscC2X](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/vez_1F7xRxyLBw-aw6dWPw/ynnk4mILRzZGDjsnA-bsV4oscC2X/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1078, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1044, in parse_configurations
parse_eigenvalues(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 944, in parse_eigenvalues
eigenvalues = self.parser.get_eigenvalues(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 692, in get_eigenvalues
eigenvalues = np.reshape(eigenvalues, (n_eigs, self.ispin, n_kpts, self.n_bands, 3))
File "<__array_function__ internals>", line 6, in reshape
File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 299, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 55, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 44, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: cannot reshape array of size 12204 into shape (0,1,228,18,3)
```
- [x] **64**, parsers/vasp, [kuWccrUZQFyoy5TCrabBsA:0v83TNUp3MjSsF6wBWDZdquATszi](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/kuWccrUZQFyoy5TCrabBsA/0v83TNUp3MjSsF6wBWDZdquATszi/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1078, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1021, in parse_configurations
for n_scf in range(self.parser.get_n_scf(n)):
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 416, in get_n_scf
return len(self._calculation_parsers[n_calc].get('scstep/'))
TypeError: object of type 'NoneType' has no len()
```
- [x] **17**, parsers/vasp, [kuWccrUZQFyoy5TCrabBsA:0aFQOX4UZ9ze2oyG-qiXPmdN-0cA](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/kuWccrUZQFyoy5TCrabBsA/0aFQOX4UZ9ze2oyG-qiXPmdN-0cA/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1076, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 860, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 411, in atom_info
array_info[fields[i]].append(val[i])
TypeError: 'NoneType' object is not subscriptable
```
- [x] **8**, parsers/vasp, [kuWccrUZQFyoy5TCrabBsA:1YEsYLq1CQjR1VQPyynpL2f-BGhI](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/kuWccrUZQFyoy5TCrabBsA/1YEsYLq1CQjR1VQPyynpL2f-BGhI/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1076, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 860, in parse_method
atomtypes = self.parser.atom_info.get('atomtypes', {})
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 401, in atom_info
e.get('name', None) for e in self.parser.get('atominfo/array/')]
TypeError: 'NoneType' object is not iterable
```
- [x] **1**, parsers/vasp, [mn3D9gABQd6FBNVvwnoH0A:9qDsS4sG-2e6oPjYrMLsh8qCYUL-](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/mn3D9gABQd6FBNVvwnoH0A/9qDsS4sG-2e6oPjYrMLsh8qCYUL-/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1078, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1047, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1012, in parse_dos
dos, fields = self.parser.get_partial_dos(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 770, in get_partial_dos
dos = np.reshape(dos, (n_lm, self.ispin, n_atoms, n_dos))
File "<__array_function__ internals>", line 6, in reshape
File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 299, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/usr/local/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
return bound(*args, **kwds)
ValueError: cannot reshape array of size 1203 into shape (601,2,4,301)
```
The origin issue:
Performance is fine. I am through the first third of the AFLOW calcs (2,5 million). There are errors, but it seems mostly the same cause:
- [x] **10684**, parsers/vasp, [MQUEA7eSRxaNAxRtWP1ciw:UK41Lt_FMJ-q1PwzVo7gQx4pExGT](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/MQUEA7eSRxaNAxRtWP1ciw/UK41Lt_FMJ-q1PwzVo7gQx4pExGT/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1056, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1025, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 975, in parse_dos
energies, values, integrated, e_fermi = self.parser.get_total_dos(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 460, in get_total_dos
dos = np.array([e.text.split() for e in dos], dtype=float)
ValueError: could not convert string to float: '********'
```
- [x] **1967**, parsers/vasp, [LKZi3yswQ9u2Bau6wleiEw:yYyBxunitnqYWsgC3TYfHzQ_a8KL](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/LKZi3yswQ9u2Bau6wleiEw/yYyBxunitnqYWsgC3TYfHzQ_a8KL/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1056, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1022, in parse_configurations
parse_eigenvalues(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 933, in parse_eigenvalues
eigenvalues = self.parser.get_eigenvalues(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 448, in get_eigenvalues
eigenvalues = np.array([e.text.split() for e in eigenvalues], dtype=float)
ValueError: could not convert string to float: '********'
```
- [x] **1001**, parsers/vasp, [LNy7pWuzQ96uYCTkDGq4MQ:u2-u5CeXo4CHfMS90y4cMnsEEaWd](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/LNy7pWuzQ96uYCTkDGq4MQ/u2-u5CeXo4CHfMS90y4cMnsEEaWd/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1056, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1025, in parse_configurations
parse_dos(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 996, in parse_dos
dos, fields = self.parser.get_partial_dos(n_calc)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 485, in get_partial_dos
dos = np.array([e.text.split() for e in dos], dtype=float)
ValueError: could not convert string to float: '********'
```
- [x] **8**, *in normalizer*, [H7uVs_JbThiWMCyb0lYhhA:busgM6i7Bg-v8F2Bx72ZwqftQLyO](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/H7uVs_JbThiWMCyb0lYhhA/busgM6i7Bg-v8F2Bx72ZwqftQLyO/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 303, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 205, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 512, in normalize
symmetry_analyzer = self.get_symmetry_analyzer(repr_atoms)
File "/app/nomad/normalizing/encyclopedia/material.py", line 482, in get_symmetry_analyzer
raise ValueError("Could not detect the periodic dimensions in a 2D system.")
ValueError: Could not detect the periodic dimensions in a 2D system.
```
- [x] **7**, parsers/vasp, [MDgZ4Hx_SlqflIAB-Zh4BQ:GOjXhYSa3RoH5_FJtEYdqK7kUTVM](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/MDgZ4Hx_SlqflIAB-Zh4BQ/GOjXhYSa3RoH5_FJtEYdqK7kUTVM/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1056, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1011, in parse_configurations
sec_scc.atom_forces = pint.Quantity(forces, 'eV/angstrom')
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 950, in __to_np
value = value.astype(quantity_def.type)
File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
value = func(*args, **kwargs)
ValueError: could not convert string to float: '****************'
```
- [x] **4**, parsers/vasp, [FjiYy5-ETRCrs5ktzJA55w:RHIPXUiH563rwAzEoOueSfW_c5Fs](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/FjiYy5-ETRCrs5ktzJA55w/RHIPXUiH563rwAzEoOueSfW_c5Fs/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1056, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 1022, in parse_configurations
parse_eigenvalues(n)
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 945, in parse_eigenvalues
occs[i] >= 0.5)]) for i in range(len(eigs))]
File "/usr/local/lib/python3.7/site-packages/vaspparser/vasp_parser.py", line 945, in <listcomp>
occs[i] >= 0.5)]) for i in range(len(eigs))]
ValueError: max() arg is an empty sequence
```
- [x] **2**, *in normalizer*, [F0mY7hOAR8iXpkl4lOdnng:WkAte4KiVai15-sFchLn4b-OBSL7](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/F0mY7hOAR8iXpkl4lOdnng/WkAte4KiVai15-sFchLn4b-OBSL7/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 303, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 205, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 392, in normalize
wyckoff_sets = symmetry_analyzer.get_wyckoff_sets_conventional(return_parameters=True)
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 508, in get_wyckoff_sets_conventional
return_parameters=return_parameters,
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 1406, in _get_wyckoff_sets
.format(wset.wyckoff_letter, wset.space_group, wset.element, wset.indices)
ValueError: Could not resolve the free Wyckoff parameters for Wyckoff letter 'b' in space group 178. Problem in determining variables for element 'Hf' at indices '[18, 19, 20, 21, 22, 23]'.
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/499On Chrome materia's orthographiccontrols prevents all mousemove events.2021-02-16T15:13:58ZMarkus ScheidgenOn Chrome materia's orthographiccontrols prevents all mousemove events.This only seems to happen in Chrome. On all NOMAD pages, even directly after loading the page, `orthographiccontrols.js` adds a mousemove event listener to the DOM (you can verify via the chrome dev-tools Elements/Event-Listeners) that p...This only seems to happen in Chrome. On all NOMAD pages, even directly after loading the page, `orthographiccontrols.js` adds a mousemove event listener to the DOM (you can verify via the chrome dev-tools Elements/Event-Listeners) that prevents users from selecting text (e.g. for copy and paste).v0.10.0Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/498Problems in the OLD VASP parser2021-03-17T08:15:04ZMarkus ScheidgenProblems in the OLD VASP parser**I did a mistake and this was done for the OLD VASP parser!!!**
I reprocesses all AFLOW calculations. Overall it run pretty smoothly. Performance was good. There have been a few exceptions. @ladinesa can you have a look, even though th...**I did a mistake and this was done for the OLD VASP parser!!!**
I reprocesses all AFLOW calculations. Overall it run pretty smoothly. Performance was good. There have been a few exceptions. @ladinesa can you have a look, even though they are not coming from the parser it self. Most of them are probably more data problems that parser problems and the normalisers are just lacking some exception handling that you could add. The first one, might be an indication that the parser produced bad DOS data.
I continue to re-process the rest of VASP. Which is more diverse and will show us more errors probably.
- [x] **19340**, *in normalizer*, [ziG2UwD3QZu8vJOM4yBi4Q:xLsKSrKokXU-qmQOCEOqMA0Gbnb6](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/ziG2UwD3QZu8vJOM4yBi4Q/xLsKSrKokXU-qmQOCEOqMA0Gbnb6/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/dos.py", line 149, in normalize
n_atoms=number_of_atoms
File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 22, in calculate
raw_energies, raw_dos = self._integrate_to_bins(energy, dos)
File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 59, in _integrate_to_bins
xstart = round(int(xs[0] / (self.stepsize * 1.)) * self.stepsize, 8) # define the limits that fit with the predefined stepsize
ValueError: cannot convert float NaN to integer
```
- [x] **37**, *in normalizer*, [joJG3dz3SrK30Yn4R_zktA:x4ZzvLi_jkjWN52HFH6D28HjJcTz](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/joJG3dz3SrK30Yn4R_zktA/x4ZzvLi_jkjWN52HFH6D28HjJcTz/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 286, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 188, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 392, in normalize
wyckoff_sets = symmetry_analyzer.get_wyckoff_sets_conventional(return_parameters=True)
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 508, in get_wyckoff_sets_conventional
return_parameters=return_parameters,
File "/usr/local/lib/python3.7/site-packages/matid/symmetry/symmetryanalyzer.py", line 1406, in _get_wyckoff_sets
.format(wset.wyckoff_letter, wset.space_group, wset.element, wset.indices)
ValueError: Could not resolve the free Wyckoff parameters for Wyckoff letter 'b' in space group 178. Problem in determining variables for element 'Cl' at indices '[0, 1, 2, 3, 4, 5]'.
```
- [x] **18**, *in normalizer*, [pNlJJXqsR_yEZrNTDZJZbw:jw-0Rm72RDVmqIYMSuGA1b1k067j](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/pNlJJXqsR_yEZrNTDZJZbw/jw-0Rm72RDVmqIYMSuGA1b1k067j/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 572, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 286, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 188, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 512, in normalize
symmetry_analyzer = self.get_symmetry_analyzer(repr_atoms)
File "/app/nomad/normalizing/encyclopedia/material.py", line 482, in get_symmetry_analyzer
raise ValueError("Could not detect the periodic dimensions in a 2D system.")
ValueError: Could not detect the periodic dimensions in a 2D system.
```
- [ ] **3**, *in normalizer*, [dcDrKssEQVCwXF3aAEqaVA:fzdq5wrhbyVHDje05JlI_Lim-Rcj](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/dcDrKssEQVCwXF3aAEqaVA/fzdq5wrhbyVHDje05JlI_Lim-Rcj/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/system.py", line 259, in system_type_analysis
cls = classifier.classify(atoms)
File "/usr/local/lib/python3.7/site-packages/matid/classification/classifier.py", line 318, in classify
dist_matrix_radii_pbc
File "/usr/local/lib/python3.7/site-packages/matid/classification/classifier.py", line 463, in cross_validate_region
dist_matrix_radii_pbc,
File "/usr/local/lib/python3.7/site-packages/matid/classification/periodicfinder.py", line 134, in get_region
bond_threshold
File "/usr/local/lib/python3.7/site-packages/matid/classification/periodicfinder.py", line 586, in _find_proto_cell
offset = proto_cell.get_positions()[seed_group_index]
IndexError: index 1 is out of bounds for axis 0 with size 1
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/495Overview page issues2021-02-23T08:22:43ZMarkus ScheidgenOverview page issues- [x] javascript error: https://nomad-lab.eu/prod/rae/beta/gui/entry/id/F0rCHPUqSXuUfjGHTQF2Rg/1DJ1jBfnztScXgBfLbB0cOWR_aSa: This was caused by completely missing SCCs.
- [x] Handling completely missing section_system and section_method,...- [x] javascript error: https://nomad-lab.eu/prod/rae/beta/gui/entry/id/F0rCHPUqSXuUfjGHTQF2Rg/1DJ1jBfnztScXgBfLbB0cOWR_aSa: This was caused by completely missing SCCs.
- [x] Handling completely missing section_system and section_method, e.g. in the case of MD: https://nomad-lab.eu/prod/rae/beta/gui/entry/id/DDZKox_nQ_CLTPPaGSr0_w/Aiz98ZThjE3h1al6Z03uJ7gXvrX3
- [x] Material type "unavailable" is not italic: https://nomad-lab.eu/prod/rae/beta/gui/entry/id/10k9kKRNTiWFsaK56Ft3og/-4oK7xt-I9sbkL7684cpd4v5TrS_
- [x] Omit the "Processing information" heading. It more miscellaneous; its fine to just drop the title
- [x] Make the material and api button "outlined", we do not use the other button styles elsewhere _Now material button is inside the material card, as text-variant (like discussed, and like other buttons that are inside cards). The API button is now at the very bottom right of the left column as discussed. It uses the outlined-variant, like other buttons that are on the grey background (it looks very weird if using the text-variant)_
- [x] The "api access" button, should just be "api"
- [x] Add a padding to the grey BS "no data" box. You don't have to align it correctly with the dos "chart", but we can cheat and try to give the appearance that it just covers the area inside the axis.
- [x] The legend with the line labels of the geometry optimisation is over the end of the lines very often. Either we have a smart placement, or put it somewhere top/bottom center/left. Having it over the "more interesting" right end of the lines is annoying. Or you put it additionally under the line? _Now placed top left as discussed, has to be over the lines as it looks super weird if under._
- [x] Can you open a bug report about the "too" short geometry optimisations that are filtered out. They should not end up as workflows in the archive. _As discussed, it is actually not possible to fix a step limit for too short optimizations. The viewer will now show the geometry optimization card if there are 1 or more steps._
- [x] Add links to the coresponding archive sections in the archive browser for plots and structures.
- [x] Improved Plot margin behaviour by adding a new fixedMargins flag (enabled by default). With this flag on the margins will be set once automatically on startup (with user controllable padding) and will not be recomputed again until the data changes.
Leave it open, there might be more soon.v0.10.0Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/492Missing section system and section scc without parser error2021-03-17T08:15:07ZMarkus ScheidgenMissing section system and section scc without parser errorAt the end you find example calculations for each parser, where there are no parser errors, but still neither a section system nor section scc in the archive. At least with aims and exciting the numbers are lower with the old parsers.
C...At the end you find example calculations for each parser, where there are no parser errors, but still neither a section system nor section scc in the archive. At least with aims and exciting the numbers are lower with the old parsers.
Compared to the old parsers the differences in number roughly are
|code|missing systems|missing SCCs|
|-|-|-|
|aims|4.000|4.000|
|exciting|71|1.1000|
|abinit|1.200|3|
@ladinesa Please have a look at examples for:
- [x] aims
- [x] exciting
- [x] abinit
```json
{
"took": 107,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 24461,
"max_score": 0.0,
"hits": []
},
"aggregations": {
"code-name": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "FHI-aims",
"doc_count": 21550,
"uploads": {
"doc_count_error_upper_bound": 25,
"sum_other_doc_count": 1373,
"buckets": [
{
"key": "x3XJQjRoSpWBcbt64-v_Wg",
"doc_count": 16592,
"example": {
"hits": {
"total": 16592,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "38o6zX9Nf8qJ0tXBN_ivHEv8-Dld",
"_score": 1.0,
"_source": {
"upload_id": "x3XJQjRoSpWBcbt64-v_Wg",
"mainfile": "to_Nomad/Matrix_embedded_2nm/Finite_differences/config_1273_p/control.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "38o6zX9Nf8qJ0tXBN_ivHEv8-Dld"
}
}
]
}
}
},
{
"key": "xKlQgmAtRD2zMMU-OFK27w",
"doc_count": 2702,
"example": {
"hits": {
"total": 2702,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "-swwMQilYhOVhxeO49gRLNHk2g9h",
"_score": 1.0,
"_source": {
"upload_id": "xKlQgmAtRD2zMMU-OFK27w",
"mainfile": "green_kubo_2/backups/backup.02762/calculations/errors/aims.190421.scalapack.mpi.x.80s-263927,co4426.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "-swwMQilYhOVhxeO49gRLNHk2g9h"
}
}
]
}
}
},
{
"key": "UIb4mzHbRS2pmIO1NjDZ0w",
"doc_count": 240,
"example": {
"hits": {
"total": 240,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "-smvxAYj3C12UcS9DxqjgZU1392a",
"_score": 1.0,
"_source": {
"upload_id": "UIb4mzHbRS2pmIO1NjDZ0w",
"mainfile": "ConvergenceParameters/TCNE_standing/system/PRECON_3.0/MIX_0.40/n_10/output.aims",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "-smvxAYj3C12UcS9DxqjgZU1392a"
}
}
]
}
}
},
{
"key": "UOm6k1H7SM6B60sSt9m1ow",
"doc_count": 182,
"example": {
"hits": {
"total": 182,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "u_vMwaOHo8qGV_x3MOsciq0L5P81",
"_score": 1.0,
"_source": {
"upload_id": "UOm6k1H7SM6B60sSt9m1ow",
"mainfile": "bulk_references/C/C_graphite/140708_Graphite_interlayer/HSE+MBD/lattice_c_6.860/aims.out.old",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "u_vMwaOHo8qGV_x3MOsciq0L5P81"
}
}
]
}
}
},
{
"key": "E00BJDb2TKeyioTrsePgpg",
"doc_count": 117,
"example": {
"hits": {
"total": 117,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "0mHnrwozwOPeb1-gnqacCzqwlLY3",
"_score": 1.0,
"_source": {
"upload_id": "E00BJDb2TKeyioTrsePgpg",
"mainfile": "Structure_Search/Substrates/(4, 2, -3, 3)/aims.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "0mHnrwozwOPeb1-gnqacCzqwlLY3"
}
}
]
}
}
},
{
"key": "UmaA5mM7SPS5fu7w8ZFwLg",
"doc_count": 104,
"example": {
"hits": {
"total": 104,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "0CmBql1OUmO2wPfXYc19Y2iSAsvU",
"_score": 1.0,
"_source": {
"upload_id": "UmaA5mM7SPS5fu7w8ZFwLg",
"mainfile": "mnt/lxfs2/scratch/mazheika/for_sergey/part2/NiO/cluster/Ni19O14/0.0/charge+10/NiO-really.out-0.82-0.01",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "0CmBql1OUmO2wPfXYc19Y2iSAsvU"
}
}
]
}
}
},
{
"key": "YXUIZpw5RJyV3LAsFI2MmQ",
"doc_count": 68,
"example": {
"hits": {
"total": 68,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "Ww15MpWMgMwVA0eT1h6Mcag2nPEi",
"_score": 1.0,
"_source": {
"upload_id": "YXUIZpw5RJyV3LAsFI2MmQ",
"mainfile": "SrTiO3/110/Ne2-tests4.5/0.25/PBE-light+tight-rho2-kerker.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "Ww15MpWMgMwVA0eT1h6Mcag2nPEi"
}
}
]
}
}
},
{
"key": "hSVvPAypRVmhmJB6Nuhnbw",
"doc_count": 62,
"example": {
"hits": {
"total": 62,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "yRErQhwxLuKyauIXf-_-O0YXuXrO",
"_score": 1.0,
"_source": {
"upload_id": "hSVvPAypRVmhmJB6Nuhnbw",
"mainfile": "58_Ce/cebulk/real-space/fcc/fm/k_04/tier1/pbe0/restart_2.3/2.65/Ce.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "yRErQhwxLuKyauIXf-_-O0YXuXrO"
}
}
]
}
}
},
{
"key": "6cZnVKTLRIq27IATPtQNmQ",
"doc_count": 56,
"example": {
"hits": {
"total": 56,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "3I0Kmn7Hh5LYqtqh1Nz1akaRTumv",
"_score": 1.0,
"_source": {
"upload_id": "6cZnVKTLRIq27IATPtQNmQ",
"mainfile": "scgw0/HCN/tier_2.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "3I0Kmn7Hh5LYqtqh1Nz1akaRTumv"
}
}
]
}
}
},
{
"key": "X2LLO0gKRvm75OMSihIZHA",
"doc_count": 54,
"example": {
"hits": {
"total": 54,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "685nNqKQlrVwFNCIlO16BdmWtbWR",
"_score": 1.0,
"_source": {
"upload_id": "X2LLO0gKRvm75OMSihIZHA",
"mainfile": "Mg2Ox/O7/str78/pbe0.out",
"dft": {
"code_name": "FHI-aims"
},
"calc_id": "685nNqKQlrVwFNCIlO16BdmWtbWR"
}
}
]
}
}
}
]
}
},
{
"key": "exciting",
"doc_count": 64,
"uploads": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "TVT2lO_wSj-MQ_2rzG1XiA",
"doc_count": 63,
"example": {
"hits": {
"total": 63,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "4Pqp6hHDmjuq7DxhdTLq6yj2T0DQ",
"_score": 1.0,
"_source": {
"upload_id": "TVT2lO_wSj-MQ_2rzG1XiA",
"mainfile": "1_2/Q0106_0506_0000_MOD005_DISP09/.INFOXS_par002.OUT",
"dft": {
"code_name": "exciting"
},
"calc_id": "4Pqp6hHDmjuq7DxhdTLq6yj2T0DQ"
}
}
]
}
}
},
{
"key": "ff8C1I9LTuiNVu5fyPVxnw",
"doc_count": 1,
"example": {
"hits": {
"total": 1,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "7xVdVq0MpZe3QKP64Ylg8tZDSLuR",
"_score": 1.0,
"_source": {
"upload_id": "ff8C1I9LTuiNVu5fyPVxnw",
"mainfile": "AB@Graphene/interface/trans/BSE/ip/calc/INFOXS.OUT",
"dft": {
"code_name": "exciting"
},
"calc_id": "7xVdVq0MpZe3QKP64Ylg8tZDSLuR"
}
}
]
}
}
}
]
}
},
{
"key": "ABINIT",
"doc_count": 5,
"uploads": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "dkZ_BRDqSA2DeGsH3ItFyw",
"doc_count": 4,
"example": {
"hits": {
"total": 4,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "Qp4sqtZIPs166omVKIJ4ahQgY4Ky",
"_score": 1.0,
"_source": {
"upload_id": "dkZ_BRDqSA2DeGsH3ItFyw",
"mainfile": "conv_ecut/Si.out0002",
"dft": {
"code_name": "ABINIT"
},
"calc_id": "Qp4sqtZIPs166omVKIJ4ahQgY4Ky"
}
}
]
}
}
},
{
"key": "OKYOLQhlTUqF01c8fvoG2w",
"doc_count": 1,
"example": {
"hits": {
"total": 1,
"max_score": 1.0,
"hits": [
{
"_index": "fairdi_nomad_reprocess_v0_8",
"_type": "doc",
"_id": "Bh8qTWHtlTV_UpsD0_9gZmRKfOym",
"_score": 1.0,
"_source": {
"upload_id": "OKYOLQhlTUqF01c8fvoG2w",
"mainfile": "conv_kpt/Si.out0001",
"dft": {
"code_name": "ABINIT"
},
"calc_id": "Bh8qTWHtlTV_UpsD0_9gZmRKfOym"
}
}
]
}
}
}
]
}
},
]
},
"errors": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "no \"representative\" section system found",
"doc_count": 24459
}
]
}
}
}
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/491More problems in new parsers2021-03-03T08:45:13ZMarkus ScheidgenMore problems in new parsers@ladinesa These are aims/exciting/abinit errors in the last reprocess run.
- [ ] **16**, parsers/fhi-aims, [hb5jvpgBSACVtVLJOmFhCw:rIXI0B5ryHlZ9XgBJPZRLy-Bzu5V](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/hb5jvpgBSACVtVLJOmFhCw...@ladinesa These are aims/exciting/abinit errors in the last reprocess run.
- [ ] **16**, parsers/fhi-aims, [hb5jvpgBSACVtVLJOmFhCw:rIXI0B5ryHlZ9XgBJPZRLy-Bzu5V](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/hb5jvpgBSACVtVLJOmFhCw/rIXI0B5ryHlZ9XgBJPZRLy-Bzu5V/logs)
```
TypeError: float() argument must be a string or a number, not 'list'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1535, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1236, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1202, in parse_section
parse_scf(scf_iteration)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1051, in parse_scf
sec_eigenvalues.eigenvalues_values = eigenvalues[1]
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 950, in __to_np
value = value.astype(quantity_def.type)
File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
value = func(*args, **kwargs)
ValueError: setting an array element with a sequence.
```
- [ ] **11**, parsers/exciting, [kT-zz5bGQe263rp0SnrRsw:DpUeWFRyH_DW4hdzP8qc0iEJxshl](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/kT-zz5bGQe263rp0SnrRsw/DpUeWFRyH_DW4hdzP8qc0iEJxshl/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2106, in parse_configurations
sec_scc = parse_configuration(structure_optimization)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2068, in parse_configuration
sec_system = self.parse_system(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1987, in parse_system
for n in range(len(atoms)):
TypeError: object of type 'numpy.float64' has no len()
```
- [ ] **9**, parsers/exciting, [QLFt5odgS1S1LQ_MDbZgpQ:SxI6MECg2gDj1cQkqasDqrq7UXeh](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/QLFt5odgS1S1LQ_MDbZgpQ/SxI6MECg2gDj1cQkqasDqrq7UXeh/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2106, in parse_configurations
sec_scc = parse_configuration(structure_optimization)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2068, in parse_configuration
sec_system = self.parse_system(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1988, in parse_system
atom_labels.extend([species[n].split('.')[0]] * len(atoms[n]))
TypeError: object of type 'numpy.float64' has no len()
```
- [ ] **8**, parsers/abinit, [k3obHR1SQ8606OLrErgApQ:wMDaL3GXazbuzsjT4Z61mVN4c7de](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/k3obHR1SQ8606OLrErgApQ/wMDaL3GXazbuzsjT4Z61mVN4c7de/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 985, in parse
self.parse_var()
File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 943, in parse_var
if val[i] is None:
IndexError: list index out of range
```
- [ ] **5**, parsers/exciting, [ML2qGfRqSnmxDLvPKkrLMQ:mVpv7QL-IkSnnbPdZB0CxrPjhtwG](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/ML2qGfRqSnmxDLvPKkrLMQ/mVpv7QL-IkSnnbPdZB0CxrPjhtwG/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2081, in parse_configurations
sec_scc = parse_configuration(self.info_parser.get('groundstate'))
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2068, in parse_configuration
sec_system = self.parse_system(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2023, in parse_system
setattr(sec_system, name, val)
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 964, in __to_np
value = quantity_def.type.type(value)
ValueError: invalid literal for int() with base 10: '********'
```
- [ ] **4**, parsers/fhi-aims, [ozK5RJx9QB-sE9ZLPy7vzw:zdSDn56Iywa88cKUQceCktWwAj57](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/ozK5RJx9QB-sE9ZLPy7vzw/zdSDn56Iywa88cKUQceCktWwAj57/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1535, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1236, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1220, in parse_section
parse_gw(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1089, in parse_gw
sec_calc_to_calc_refs = sec_run.m_create(CalculationToCalculationRefs)
File "/app/nomad/metainfo/metainfo.py", line 1129, in m_create
raise TypeError('There is no sub section to hold a %s in %s.' % (section_def, self.m_def))
TypeError: There is no sub section to hold a nomad.datamodel.metainfo.common_dft.CalculationToCalculationRefs:Section in nomad.datamodel.metainfo.common_dft.Run:Section.
```
- [ ] **4**, parsers/fhi-aims, [LZ4sNtvUT4GlbcY46Mt9Bw:wqS75Y9uIHloNTn2rmLpFVk0UblH](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/LZ4sNtvUT4GlbcY46Mt9Bw/wqS75Y9uIHloNTn2rmLpFVk0UblH/logs)
```
TypeError: float() argument must be a string or a number, not 'list'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1535, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1236, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1176, in parse_section
sec_eigenvalues.eigenvalues_values = eigenvalues[1]
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 950, in __to_np
value = value.astype(quantity_def.type)
File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
value = func(*args, **kwargs)
ValueError: setting an array element with a sequence.
```
- [ ] **2**, parsers/exciting, [bBwLURrsQzyT-avIDAsNhA:Ukv9vOpmfindZocfvucn3eYOFFZq](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/bBwLURrsQzyT-avIDAsNhA/Ukv9vOpmfindZocfvucn3eYOFFZq/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2165, in parse
self.parse_gw()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1721, in parse_gw
self.parse_file(f, sec_method)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1341, in parse_file
parser_function(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1674, in _parse_input_gw
'gw/selfenergy/npol', 0)
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2251, in __set__
(value, type(value), self, self.type))
TypeError: The value 2 with type <class 'numpy.int64'> for quantity nomad.datamodel.metainfo.common_dft.Method.gw_self_energy_c_number_of_poles:Quantity is not of type <class 'int'>
```
- [ ] **2**, parsers/fhi-aims, [XUXzuLLNQKyuDGiXwTqc_A:J6HB4O_h6bimMfNfVUiJnb1wjMC2](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/XUXzuLLNQKyuDGiXwTqc_A/J6HB4O_h6bimMfNfVUiJnb1wjMC2/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1533, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1266, in parse_method
species = self.control_parser.get('species')
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 513, in parse
self._parse_quantities(quantities_findall)
File "/app/nomad/parsing/file_parser/text_parser.py", line 370, in _parse_quantities
matches = re.findall(re_findall, self.file_mmap)
File "/usr/local/lib/python3.7/re.py", line 225, in findall
return _compile(pattern, flags).findall(string)
File "/usr/local/lib/python3.7/site-packages/billiard/pool.py", line 229, in soft_timeout_sighandler
raise SoftTimeLimitExceeded()
billiard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()
```
- [ ] **2**, parsers/fhi-aims, [RfpFzoO2T4mTVhblG3gTbg:o21LEcL5U4nQwacbpdqlCk_lPnFt](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/RfpFzoO2T4mTVhblG3gTbg/o21LEcL5U4nQwacbpdqlCk_lPnFt/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1514, in parse
value = self.out_parser.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 521, in parse
self._parse_quantity(quantity)
File "/app/nomad/parsing/file_parser/text_parser.py", line 460, in _parse_quantity
value.append(sub_parser.parse())
File "/app/nomad/parsing/file_parser/text_parser.py", line 513, in parse
self._parse_quantities(quantities_findall)
File "/app/nomad/parsing/file_parser/text_parser.py", line 393, in _parse_quantities
values.append(' '.join([m.decode() for m in non_empty_match]))
File "/app/nomad/parsing/file_parser/text_parser.py", line 393, in <listcomp>
values.append(' '.join([m.decode() for m in non_empty_match]))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 2665: invalid start byte
```
- [ ] **2**, parsers/exciting, [sB1iGmp5Sq2U4CH2y47HRQ:CJtR-92xmAbUTF0FS09_f5sIRfeQ](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/sB1iGmp5Sq2U4CH2y47HRQ/CJtR-92xmAbUTF0FS09_f5sIRfeQ/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2081, in parse_configurations
sec_scc = parse_configuration(self.info_parser.get('groundstate'))
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2068, in parse_configuration
sec_system = self.parse_system(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1981, in parse_system
lattice_vectors *= self.input_xml_parser.get('structure/crystal/scale', 1.0)
numpy.core._exceptions.UFuncTypeError: Cannot cast ufunc 'multiply' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
```
- [ ] **1**, parsers/exciting, [CgtgzCMXTUyY0tjfBYXAnA:9UM_CpSzaystbWPZoghgjg3dZmC8](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/CgtgzCMXTUyY0tjfBYXAnA/9UM_CpSzaystbWPZoghgjg3dZmC8/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2088, in parse_configurations
self.parse_file(f, sec_scc)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1341, in parse_file
parser_function(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1106, in _parse_dos
if self.dos_parser.get('totaldos', None) is None:
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 505, in parse
if not self.total_dos:
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 471, in total_dos
self._total_dos = self.root.findall('./%s/%s' % (self._totaldos_key, self._diagram_key))
File "/app/nomad/parsing/file_parser/xml_parser.py", line 52, in root
self._file_handler = ElementTree.parse(self.mainfile).getroot()
File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
tree.parse(source, parser)
File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 598, in parse
self._root = parser._parse_whole(source)
File "<string>", line None
xml.etree.ElementTree.ParseError: no element found: line 111468, column 0
```
- [ ] **1**, parsers/exciting, [1eMMOR07QTOUNDXy_7VlkQ:y1Vqyvq_7u35Luwr6jJRdKS29LeN](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/1eMMOR07QTOUNDXy_7VlkQ/y1Vqyvq_7u35Luwr6jJRdKS29LeN/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2167, in parse
self.parse_xs()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1653, in parse_xs
if xstype.lower() == 'bse':
AttributeError: 'NoneType' object has no attribute 'lower'
```
- [ ] **1**, parsers/exciting, [cdAK5_-gQ4GLFbZrgBRpQQ:B7YX4kB9f1PSwYbeJeUcExy-UKZw](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/cdAK5_-gQ4GLFbZrgBRpQQ/B7YX4kB9f1PSwYbeJeUcExy-UKZw/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2163, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2088, in parse_configurations
self.parse_file(f, sec_scc)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1341, in parse_file
parser_function(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1162, in _parse_eigenvalues
if self.eigval_parser.get('eigenvalues_occupancies', None) is None:
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 498, in parse
if self.file_mmap is None:
File "/app/nomad/parsing/file_parser/text_parser.py", line 343, in file_mmap
offset=self._file_offset)
ValueError: cannot mmap an empty file
```
- [ ] **1**, parsers/fhi-aims, [RfpFzoO2T4mTVhblG3gTbg:ot1m7t1rXb7eEkIhr1felzfCNKYc](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/RfpFzoO2T4mTVhblG3gTbg/ot1m7t1rXb7eEkIhr1felzfCNKYc/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1514, in parse
value = self.out_parser.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 513, in parse
self._parse_quantities(quantities_findall)
File "/app/nomad/parsing/file_parser/text_parser.py", line 393, in _parse_quantities
values.append(' '.join([m.decode() for m in non_empty_match]))
File "/app/nomad/parsing/file_parser/text_parser.py", line 393, in <listcomp>
values.append(' '.join([m.decode() for m in non_empty_match]))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 542: invalid start byte
```
- [ ] **1**, parsers/fhi-aims, [JA6KMv6XQSC2Hx4PLglCHA:PhTx-akA_BFskGrQSO-gY0lEyVLZ](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/JA6KMv6XQSC2Hx4PLglCHA/PhTx-akA_BFskGrQSO-gY0lEyVLZ/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1533, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1330, in parse_method
parse_basis_set(species)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1295, in parse_basis_set
val[0], dtype=float))
ValueError: could not convert string to float: '#'
```
- [ ] **1**, parsers/fhi-aims, [jSSVtYWfTZacc7HjWPSqHQ:vzV7EoLAe3LEBffjjLLa5i_z6W-i](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/jSSVtYWfTZacc7HjWPSqHQ/vzV7EoLAe3LEBffjjLLa5i_z6W-i/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1535, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1242, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1202, in parse_section
parse_scf(scf_iteration)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1012, in parse_scf
sec_scf = sec_scc.m_create(ScfIteration)
File "/app/nomad/metainfo/metainfo.py", line 1144, in m_create
sub_section = section_cls(**kwargs)
File "/app/nomad/metainfo/metainfo.py", line 726, in __init__
self.m_def = cls.m_def
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/usr/local/lib/python3.7/site-packages/billiard/pool.py", line 229, in soft_timeout_sighandler
raise SoftTimeLimitExceeded()
billiard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()
```
- [ ] **1**, parsers/fhi-aims, [XUXzuLLNQKyuDGiXwTqc_A:fA6FNTfenBhW5xldBbbB9J_6YC1n](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/XUXzuLLNQKyuDGiXwTqc_A/fA6FNTfenBhW5xldBbbB9J_6YC1n/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1514, in parse
value = self.out_parser.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 81, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 521, in parse
self._parse_quantity(quantity)
File "/app/nomad/parsing/file_parser/text_parser.py", line 446, in _parse_quantity
for res in quantity.re_pattern.finditer(self.file_mmap):
File "/usr/local/lib/python3.7/site-packages/billiard/pool.py", line 229, in soft_timeout_sighandler
raise SoftTimeLimitExceeded()
billiard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()
```
- [ ] **1**, parsers/fhi-aims, [wJ7g5Tj1S5G3ooINgLvtTg:uvKF8Cgb4grSTW74NKsyseip12Ij](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/wJ7g5Tj1S5G3ooINgLvtTg/uvKF8Cgb4grSTW74NKsyseip12Ij/logs)
```
TypeError: float() argument must be a string or a number, not 'list'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1535, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1239, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1202, in parse_section
parse_scf(scf_iteration)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1051, in parse_scf
sec_eigenvalues.eigenvalues_values = eigenvalues[1]
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 950, in __to_np
value = value.astype(quantity_def.type)
File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
value = func(*args, **kwargs)
ValueError: setting an array element with a sequence.
```
- [ ] **1**, parsers/fhi-aims, [N-bjfU-gQOejz93MXF-rsA:N0WJHJONigdpP527QTuMuHi50OgT](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/N-bjfU-gQOejz93MXF-rsA/N0WJHJONigdpP527QTuMuHi50OgT/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1533, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1401, in parse_method
xc_parameters.setdefault('$\\omega$ in m^-1', hse_omega.to('1/m').magnitude)
AttributeError: 'float' object has no attribute 'to'
```
- [ ] **1**, parsers/fhi-aims, [EbYKP9bsRAy9faXyzpdZMA:4WICKJiugfWhFzud8Ywe7YBL9JRg](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/EbYKP9bsRAy9faXyzpdZMA/4WICKJiugfWhFzud8Ywe7YBL9JRg/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 460, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1533, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1370, in parse_method
self.parse_topology()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1482, in parse_topology
parse_atom_type(specie)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1434, in parse_atom_type
sec_atom_species.x_fhi_aims_controlInOut_species_cut_pot_scale = val[0][2]
File "/app/nomad/metainfo/metainfo.py", line 879, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2141, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 986, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 964, in __to_np
value = quantity_def.type.type(value)
ValueError: could not convert string to float: 'Task'
```v0.10.0https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/488Adapt encyclopedia normalizer to workflows2021-02-11T06:11:59ZMarkus ScheidgenAdapt encyclopedia normalizer to workflowsThe encyclopedia normalizer depends on framesequence and sampling method to determine a calculation type. @ladinesa started to remove the use of these old metainfo definitions. @himanel1 should adapt the encyclopedia normalizer. For now ...The encyclopedia normalizer depends on framesequence and sampling method to determine a calculation type. @ladinesa started to remove the use of these old metainfo definitions. @himanel1 should adapt the encyclopedia normalizer. For now it should look for workflow information and use this. Only if no workflows are present, I should revert to the old behaviour. Once the migration to workflows is complete this should also be removed.
Currently (branch new-vasp-parser) this breaks the CI/CD because `tests/data/api/enc_private_material.zip::private/vasprun.xml` is parsed by the new vasp parser and no framesequence is generated anymore -> calc_type=unavailable -> no enc entry -> broken assertion in `tests/app/flask/test_api_encyclopedia.py::TestEncyclopedia::test_material`
Btw, the test case is terrible as it depends on multiple parsers, test data, etc. If one of these deps changes, its really hard to figure out, why this fails. Ideally this could be broken down into multiple test cases at some point (e.g. when implementing a new enc API based on flask api).v0.10.0Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/486Potential performance problem with new parser framework and large files2021-01-25T07:43:13ZMarkus ScheidgenPotential performance problem with new parser framework and large filesThis plots some parser runs of the new and the old FHI-aims parser (compared to mainfile size).
![image](/uploads/0b7efb2571a9670ad2f96f5eb293cfd9/image.png)
As you see, the general performance is comparable. Maybe the new has a little...This plots some parser runs of the new and the old FHI-aims parser (compared to mainfile size).
![image](/uploads/0b7efb2571a9670ad2f96f5eb293cfd9/image.png)
As you see, the general performance is comparable. Maybe the new has a little steeper time-to-input relation, but thats not too bad. The green oval is a little weird. Maybe this is because I disabled the parser instance reuse.
The problematic one is the red oval. For very large files, we get these extra long run times. Don't let you fool by the logarithmic scale. This is quite drastic and raises the average exec time from 100s to 1.000s. Ten fold. Instead of 1.000.000 calcs, I only get 100.000 calcs overnight. It's kind of a deal breaker.
This does not happen with the old parser, see also:
![image](/uploads/5b8ff202c1a534c70d1085fc567da5fa/image.png)
@ladinesa Could this be memory limitations due to the memory input mapping thing? Would there be a work around?
@himanel1 Not sure if this is related to #481v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/485Re-use of parser instances causes unpredictable behaviour2021-02-10T08:42:06ZMarkus ScheidgenRe-use of parser instances causes unpredictable behaviourThe exceptions/problems reported in #475 and #477 only appear when parser instances are reused. There seems to be something missing in resetting the parsers before running.The exceptions/problems reported in #475 and #477 only appear when parser instances are reused. There seems to be something missing in resetting the parsers before running.v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/484Problems in new parsers2021-03-17T08:15:07ZMarkus ScheidgenProblems in new parsersThese are the top 20 exception from the last reprocessing of aims, exciting, abinit, crystal.
In addition to these, there are many calculations that produce incomplete results. This is most easily to spot, if you look at the system meta...These are the top 20 exception from the last reprocessing of aims, exciting, abinit, crystal.
In addition to these, there are many calculations that produce incomplete results. This is most easily to spot, if you look at the system metadata. Many calculations have no formula. This indicates they have no system. This indicates the parser (even through running successfully) did not match much. Probably these calculation require slightly more relaxed regexps due to code version differences.
- [x] **16879**, parsers/fhi-aims, [xx9izr2yRputru0JleV7-Q:t-ZPO5zE2Jw9DkP0m8fGQgR6vC-7](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/xx9izr2yRputru0JleV7-Q/t-ZPO5zE2Jw9DkP0m8fGQgR6vC-7/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1302, in parse_method
sec_method.x_fhi_aims_controlInOut_number_of_spin_channels = self.out_parser.get_number_of_spin_channels()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 710, in get_number_of_spin_channels
return array_size.get('Number of spin channels', 1)
AttributeError: 'NoneType' object has no attribute 'get'
```
- [x] **5891**, *in normalizer*, [qhx0VdcgS3inMmKECuN3KA:zw5o6RH9ZBAyDlsc4CXhWml09CLc](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/qhx0VdcgS3inMmKECuN3KA/zw5o6RH9ZBAyDlsc4CXhWml09CLc/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 558, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 286, in normalize
self.fill(context)
File "/app/nomad/normalizing/encyclopedia/encyclopedia.py", line 188, in fill
struct.normalize(context)
File "/app/nomad/normalizing/encyclopedia/material.py", line 739, in normalize
self.periodicity(ideal, prim_atoms)
File "/app/nomad/normalizing/encyclopedia/material.py", line 577, in periodicity
raise ValueError("Could not detect the periodic dimensions in a 1D system.")
ValueError: Could not detect the periodic dimensions in a 1D system.
```
- [x] **3084**, parsers/lammps, [DDZKox_nQ_CLTPPaGSr0_w:zzStifnyE7SwZuV3SGagFAfCtekQ](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/DDZKox_nQ_CLTPPaGSr0_w/zzStifnyE7SwZuV3SGagFAfCtekQ/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 787, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 766, in parse
sec_md.with_trajectory = self.traj_parser.with_trajectory()
File "/usr/local/lib/python3.7/site-packages/lammpsparser/lammps_parser.py", line 199, in with_trajectory
return self.get('atoms_info') is not None
File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse
if self.file_mmap is None:
File "/app/nomad/parsing/file_parser/text_parser.py", line 345, in file_mmap
with open(self.mainfile) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
```
- [x] **2711**, parsers/fhi-aims, [o-8XYG0JS5OJU59uchgA7w:zjS-6QqvVUrBNsQgaQGNEHRcipV1](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/o-8XYG0JS5OJU59uchgA7w/zjS-6QqvVUrBNsQgaQGNEHRcipV1/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1473, in parse
sec_parallel_task_assignement.x_fhi_aims_parallel_task_host = task_hosts[i]
File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2221, in __set__
(value, type(value), self, self.type))
TypeError: The value ['Task' '0' 'of' '512' '(0,0,0,0,0,0)' 'R1B-M0-N00-J00'] with type <class 'numpy.ndarray'> for quantity fhiaimsparser.metainfo.fhi_aims.x_fhi_aims_section_parallel_task_assignement.x_fhi_aims_parallel_task_host:Quantity is not of type <class 'str'>
```
- [x] **2271**, parsers/fhi-aims, [zb79w_T9Q_qbstffWBW46g:JHXpfS4E6piQw8ohphNqL8VglHyr](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/zb79w_T9Q_qbstffWBW46g/JHXpfS4E6piQw8ohphNqL8VglHyr/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1342, in parse_method
sec_xc_func.XC_functional_weight = weight(hybrid_coeff)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 766, in <lambda>
'name': 'GGA_X_PBE', 'weight': lambda x: 0.75 if x is None else 1.0 - x},
TypeError: unsupported operand type(s) for -: 'float' and 'str'
```
- [x] **1105**, parsers/exciting, [pvRdA8EBQ--7UQdVou5s-w:xrP0AfYb8-JsPChh0t0OthNP7K1X](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/pvRdA8EBQ--7UQdVou5s-w/xrP0AfYb8-JsPChh0t0OthNP7K1X/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2053, in parse_configurations
self.parse_file(f, sec_scc)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1332, in parse_file
parser_function(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1171, in _parse_eigenvalues
sec_eigenvalues.eigenvalues_values = get_data('eigenvalues')
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1164, in get_data
res = np.hstack([v[key] for v in data])
File "<__array_function__ internals>", line 6, in hstack
File "/usr/local/lib/python3.7/site-packages/numpy/core/shape_base.py", line 346, in hstack
return _nx.concatenate(arrs, 1)
File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 2 and the array at index 18 has size 1
```
- [x] **1080**, *in normalizer*, [uCSlNMEWQfOS5TMvsQXz1A:ziSEudHjaTYIe3Jzj-MKRZn4VkDt](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/uCSlNMEWQfOS5TMvsQXz1A/ziSEudHjaTYIe3Jzj-MKRZn4VkDt/logs)
```
Traceback (most recent call last):
File "/app/nomad/normalizing/dos.py", line 149, in normalize
n_atoms=number_of_atoms
File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 25, in calculate
self.indices, self.bins = self._calculate_bytes(raw_energies, raw_dos, grid)
File "/usr/local/lib/python3.7/site-packages/nomad_dos_fingerprints/DOSfingerprint.py", line 88, in _calculate_bytes
energy, dos = np.transpose([(e,d) for e,d in zip(energy, dos) if (e >= grid_array[0][0] and e <= grid_array[-1][0])])
ValueError: not enough values to unpack (expected 2, got 0)
```
- [x] **768**, parsers/fhi-aims, [iZGJdsGSRm6DYERVpo74fA:0iEfV-_AQA6wlw6mneY5ol8ia1vj](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/iZGJdsGSRm6DYERVpo74fA/0iEfV-_AQA6wlw6mneY5ol8ia1vj/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1259, in parse_method
for key, val in self.control_parser.items():
File "/app/nomad/parsing/file_parser/text_parser.py", line 365, in items
yield key, self.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse
if self.file_mmap is None:
File "/app/nomad/parsing/file_parser/text_parser.py", line 345, in file_mmap
with open(self.mainfile) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
```
- [x] **341**, parsers/fhi-aims, [wKiS4s-NS9Ky5o1OtYKI2w:2IIcVSSBI5ktFmjUyfMISbWYaOiI](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/wKiS4s-NS9Ky5o1OtYKI2w/2IIcVSSBI5ktFmjUyfMISbWYaOiI/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1475, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1315, in parse_method
self.parse_topology()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1424, in parse_topology
species = self.out_parser['control_inout']['species']
TypeError: 'NoneType' object is not subscriptable
```
- [x] **278**, parsers/fhi-aims, [qSCiRCcYSaifBVn00EFOQA:yHb3bEu31Sb24V5eRXcOJ63XX3bk](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/qSCiRCcYSaifBVn00EFOQA/yHb3bEu31Sb24V5eRXcOJ63XX3bk/logs)
```
Traceback (most recent call last):
File "/app/nomad/parsing/file_parser/text_parser.py", line 406, in _parse_quantities
value_processed = quantities[i].to_data(values)
File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in to_data
val_out = [process(val) for val in val_in]
File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in <listcomp>
val_out = [process(val) for val in val_in]
File "/app/nomad/parsing/file_parser/text_parser.py", line 158, in process
val = self.str_operation(val)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 480, in str_to_energy_components
unit = {'Ha': 'hartree', 'eV': 'eV'}.get(vi[1], None)
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1459, in parse
value = self.out_parser.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 524, in parse
self._parse_quantity(quantity)
File "/app/nomad/parsing/file_parser/text_parser.py", line 463, in _parse_quantity
value.append(sub_parser.parse())
File "/app/nomad/parsing/file_parser/text_parser.py", line 516, in parse
self._parse_quantities(quantities_findall)
File "/app/nomad/parsing/file_parser/text_parser.py", line 419, in _parse_quantities
self.logger.warn('Error setting value', data=dict(quantity=quantities[i].name))
File "/usr/local/lib/python3.7/logging/__init__.py", line 1982, in warn
warning(msg, *args, **kwargs)
File "/usr/local/lib/python3.7/logging/__init__.py", line 1977, in warning
root.warning(msg, *args, **kwargs)
File "/usr/local/lib/python3.7/logging/__init__.py", line 1390, in warning
self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'data'
```
- [x] **213**, parsers/crystal, [gddy5rHgRceN2sZZ6yTnrA:zbuT8J3-vcIknvze4gbDSWQDPEfP](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/gddy5rHgRceN2sZZ6yTnrA/zbuT8J3-vcIknvze4gbDSWQDPEfP/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/crystalparser/crystal_parser.py", line 566, in parse
dist, minor = distribution.split(" : ", 1)
AttributeError: 'NoneType' object has no attribute 'split'
```
- [x] **199**, parsers/phonopy, [z369XxTOT5S3db4Q2vrbqg:xeYjLT2hhwWskUCXdeqVg3bkND6h](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/z369XxTOT5S3db4Q2vrbqg/xeYjLT2hhwWskUCXdeqVg3bkND6h/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 498, in process_phonon
raise ValueError("No method information available in referenced calculation.")
ValueError: No method information available in referenced calculation.
```
- [x] **171**, parsers/exciting, [vDg3jiguSsau7B2R9XjxMQ:o9pLQY2RAWuFd58rYJVgrzvO9SuO](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/vDg3jiguSsau7B2R9XjxMQ/o9pLQY2RAWuFd58rYJVgrzvO9SuO/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1802, in parse_method
xc_functional_names = self.info_parser.get_xc_functional_name()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1002, in get_xc_functional_name
xc_functional = self.get('initialization').get('x_exciting_xc_functional', None)
AttributeError: 'NoneType' object has no attribute 'get'
```
- [x] **160**, parsers/phonopy, [win2cjTvRF6t_KjDmfVlZw:IxR6Cf1HLzo8zeedPFUHzO0t74BN](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/win2cjTvRF6t_KjDmfVlZw/IxR6Cf1HLzo8zeedPFUHzO0t74BN/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 185, in parse
phonopy_obj = self.phonopy_obj
File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 50, in phonopy_obj
self._build_phonopy_object_fhi_aims()
File "/usr/local/lib/python3.7/site-packages/phonopyparser/phonopy_parser.py", line 77, in _build_phonopy_object_fhi_aims
cell_obj, supercell_matrix, displacement, sym)
File "/usr/local/lib/python3.7/site-packages/phonopyparser/fhiaims_io.py", line 137, in read_forces_aims
raise Exception("No phonon calculations found")
Exception: No phonon calculations found
```
- [x] **144**, *in normalizer*, [uCSlNMEWQfOS5TMvsQXz1A:xp4iSuT8IXqePP_KNuQaR0pOeC9z](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/uCSlNMEWQfOS5TMvsQXz1A/xp4iSuT8IXqePP_KNuQaR0pOeC9z/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 558, in normalizing
normalizer(self._parser_results).normalize(logger=logger)
File "/app/nomad/normalizing/band_structure.py", line 63, in normalize
self.add_band_gaps(band, energy_reference)
File "/app/nomad/normalizing/band_structure.py", line 181, in add_band_gaps
energies = np.concatenate(energies, axis=2)
File "<__array_function__ internals>", line 6, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 9 and the array at index 6 has size 27
```
- [ ] **126**, parsers/fhi-aims, [jSSVtYWfTZacc7HjWPSqHQ:a8r8KkvKXWams50UhzMGCxY0IGqH](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/jSSVtYWfTZacc7HjWPSqHQ/a8r8KkvKXWams50UhzMGCxY0IGqH/logs)
```
TypeError: only size-1 arrays can be converted to Python scalars
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1477, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1190, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1147, in parse_section
forces_raw[-len(forces):], 'eV/angstrom')
File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 956, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 920, in __to_np
value = value.astype(quantity_def.type)
File "/usr/local/lib/python3.7/site-packages/pint/quantity.py", line 1654, in _numpy_method_wrap
value = func(*args, **kwargs)
ValueError: setting an array element with a sequence.
```
- [x] **107**, parsers/exciting, [pzD88LYnTty-c_Xg07dqwg:wRtykAr2p9qUNTGDT-fZabQozNB4](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/pzD88LYnTty-c_Xg07dqwg/wRtykAr2p9qUNTGDT-fZabQozNB4/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2061, in parse_configurations
sec_scc.x_exciting_geometry_optimization_method = optimization_step.get('method')
AttributeError: 'NoneType' object has no attribute 'x_exciting_geometry_optimization_method'
```
- [x] **101**, parsers/fhi-aims, [ZFNMJDbJQ9aUq3o-aMq7lw:n43gSkg2cIhEM60wbmipsKQo1rig](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/ZFNMJDbJQ9aUq3o-aMq7lw/n43gSkg2cIhEM60wbmipsKQo1rig/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1477, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1190, in parse_configurations
parse_section(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1163, in parse_section
parse_dos(section)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 941, in parse_dos
energies, dos = read_projected_dos(proj_dos_files)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 897, in read_projected_dos
energies = pint.Quantity(data[0], 'eV')
TypeError: 'NoneType' object is not subscriptable
```
- [x] **61**, parsers/exciting, [ff8C1I9LTuiNVu5fyPVxnw:xHhm3RgE-DzKTJ8X1qYc2xm01Ld6](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/ff8C1I9LTuiNVu5fyPVxnw/xHhm3RgE-DzKTJ8X1qYc2xm01Ld6/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2153, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2132, in parse
self.parse_xs()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1631, in parse_xs
self.parse_file('input.xml', sec_method)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1332, in parse_file
parser_function(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1368, in _parse_input_xs
sec_method.x_exciting_xs_screening_rgkmax = rgkmax
File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 956, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np
(quantity_def, value))
nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_screening_rgkmax:Quantity has not a unit, but value 3.5 bohr has.
```
- [ ] **15**, parsers/crystal, [llZrblH4THCuJvVjEBXBFA:VdOUqF-aHTwlDSRcuoubmNBPFxHo](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/llZrblH4THCuJvVjEBXBFA/VdOUqF-aHTwlDSRcuoubmNBPFxHo/logs)
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/crystalparser/crystal_parser.py", line 618, in parse
atomic_numbers,
UnboundLocalError: local variable 'atomic_numbers' referenced before assignment
```
- [x] **13**, parsers/fhi-aims, [w8VbMKDwSb27iSM2oCE4Kg:kEp6pGbAJfjwn5r1PuvciCi1IIPd](https://nomad-lab.eu/prod/rae/reprocess/gui/entry/id/w8VbMKDwSb27iSM2oCE4Kg/kEp6pGbAJfjwn5r1PuvciCi1IIPd/logs)
```
Traceback (most recent call last):
File "/app/nomad/parsing/file_parser/text_parser.py", line 406, in _parse_quantities
value_processed = quantities[i].to_data(values)
File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in to_data
val_out = [process(val) for val in val_in]
File "/app/nomad/parsing/file_parser/text_parser.py", line 220, in <listcomp>
val_out = [process(val) for val in val_in]
File "/app/nomad/parsing/file_parser/text_parser.py", line 158, in process
val = self.str_operation(val)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 467, in str_to_eigenvalues
eig, occ = eigs[1:3]
ValueError: not enough values to unpack (expected 2, got 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 453, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1499, in parse
parser.parse(filepath, archive, logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1459, in parse
value = self.out_parser.get(key)
File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get
val = self.results.get(key, None)
File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results
self.parse(self._key, **self._kwargs)
File "/app/nomad/parsing/file_parser/text_parser.py", line 524, in parse
self._parse_quantity(quantity)
File "/app/nomad/parsing/file_parser/text_parser.py", line 463, in _parse_quantity
value.append(sub_parser.parse())
File "/app/nomad/parsing/file_parser/text_parser.py", line 516, in parse
self._parse_quantities(quantities_findall)
File "/app/nomad/parsing/file_parser/text_parser.py", line 419, in _parse_quantities
self.logger.warn('Error setting value', data=dict(quantity=quantities[i].name))
File "/usr/local/lib/python3.7/logging/__init__.py", line 1982, in warn
warning(msg, *args, **kwargs)
File "/usr/local/lib/python3.7/logging/__init__.py", line 1977, in warning
root.warning(msg, *args, **kwargs)
File "/usr/local/lib/python3.7/logging/__init__.py", line 1390, in warning
self._log(WARNING, msg, args, **kwargs)
TypeError: _log() got an unexpected keyword argument 'data'
```
**Old issues description:**
These are exceptions thrown by parsers during reprocessing all non staging exciting and abinit code runs. I only checked the first one, but it reproduces with `nomad parse ...`.
|exception_hash.keyword: Descending|# |calc_id |upload_id |mainfile |parser |exception |
|----------------------------------|----|----------------------------|----------------------|-----------------------------------------------------------------|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|s7J4tcDHcGPdLgK2ZnnzqhE56hd3 |1105|xrP0AfYb8-JsPChh0t0OthNP7K1X|pvRdA8EBQ--7UQdVou5s-w|volume-1.5/0.965489384605630/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1176, in _parse_eigenvalues sec_eigenvalues.eigenvalues_values = get_data('eigenvalues') File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1169, in get_data res = np.hstack([v[key] for v in data]) File "<__array_function__ internals>", line 6, in hstack File "/usr/local/lib/python3.7/site-packages/numpy/core/shape_base.py", line 346, in hstack return _nx.concatenate(arrs, 1) File "<__array_function__ internals>", line 6, in concatenate ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 2 and the array at index 18 has size 1 |
|hwe7MyzCotT17K4FWIlLLnLyII3_ |171 |yEqUiNBD7iu3P4YZmzgvpj2CbFV6|QLFt5odgS1S1LQ_MDbZgpQ|silver-tddft/silver-loss/6/INFOXS.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2124, in parse self.parse_method() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1807, in parse_method xc_functional_names = self.info_parser.get_xc_functional_name() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1002, in get_xc_functional_name xc_functional = self.get('initialization').get('x_exciting_xc_functional', None) AttributeError: 'NoneType' object has no attribute 'get' |
|MEQ6y79UyB-M8DAu-NR4SBHzEJIq |107 |wRtykAr2p9qUNTGDT-fZabQozNB4|pzD88LYnTty-c_Xg07dqwg|home1/UTE/ETERRNAL/DATA/FINISHED/CGeSn/000016/GG/L_GAMMA/INFO.OUT|parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2066, in parse_configurations sec_scc.x_exciting_geometry_optimization_method = optimization_step.get('method') AttributeError: 'NoneType' object has no attribute 'x_exciting_geometry_optimization_method' |
|HXW2tu3a-mtJmMCuvzjUTwTvISba |62 |xHhm3RgE-DzKTJ8X1qYc2xm01Ld6|ff8C1I9LTuiNVu5fyPVxnw|AB@Graphene/azobenzene/cis/BSE/rpa/INFO_QMT001.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2130, in parse self.parse_xs() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1636, in parse_xs self.parse_file('input.xml', sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1373, in _parse_input_xs sec_method.x_exciting_xs_screening_rgkmax = rgkmax File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__ return super().__setattr__(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__ obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np (quantity_def, value)) nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_screening_rgkmax:Quantity has not a unit, but value 3.5 bohr has. |
|3kmtj-uvUs_AtxPSd_mgAoWmUY5i |14 |LuH6KeBENcz3_mwE8HOwvYKJyqLw|0NR0E6gqQ2yVicJCMEP4SA|c-Si-OCTSF/12x12x12-BSE-PBE/INFO_QMT001.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2130, in parse self.parse_xs() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1636, in parse_xs self.parse_file('input.xml', sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1360, in _parse_input_xs sec_method.x_exciting_xs_rgkmax = rgkmax File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__ return super().__setattr__(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__ obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 917, in __to_np (quantity_def, value)) nomad.metainfo.metainfo.MetainfoError: The quantity excitingparser.metainfo.exciting.section_method.x_exciting_xs_rgkmax:Quantity has not a unit, but value 8.0 bohr has. |
|2khWqBfkrqi65UqODhX0_U-Dp5qs |6 |fwG3jU4_PBV1uWdAzFtXotFs293m|dkZ_BRDqSA2DeGsH3ItFyw|conv_ecut/Si.out0006 |parsers/abinit |Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 983, in parse self.parse_dataset(dataset) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 847, in parse_dataset sec_method = self.parse_method(dataset) File "/usr/local/lib/python3.7/site-packages/abinitparser/abinit_parser.py", line 759, in parse_method sec_method.number_of_spin_channels = int(self.out_parser.get_input_var('nsppol', nd, 1)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' |
|h04dOeanRAxBsfNPaVUnydXG8kSu |5 |mVpv7QL-IkSnnbPdZB0CxrPjhtwG|ML2qGfRqSnmxDLvPKkrLMQ|LiH-pbc/130/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2051, in parse_configurations sec_scc = parse_configuration(self.info_parser.get('groundstate')) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2042, in parse_configuration sec_system = self.parse_system(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1995, in parse_system setattr(sec_system, name, val) File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__ return super().__setattr__(name, value) File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__ obj.m_set(self, value) File "/app/nomad/metainfo/metainfo.py", line 956, in m_set value = self.__to_np(quantity_def, value) File "/app/nomad/metainfo/metainfo.py", line 934, in __to_np value = quantity_def.type.type(value) ValueError: invalid literal for int() with base 10: '********' |
|cfgruvgumKUmo3jBj8wCYAPfsg7e |2 |Ukv9vOpmfindZocfvucn3eYOFFZq|bBwLURrsQzyT-avIDAsNhA|silicon-gw/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2128, in parse self.parse_gw() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1712, in parse_gw self.parse_file(f, sec_method) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1665, in _parse_input_gw 'gw/selfenergy/npol', 0) File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__ return super().__setattr__(name, value) File "/app/nomad/metainfo/metainfo.py", line 2221, in __set__ (value, type(value), self, self.type)) TypeError: The value 2 with type <class 'numpy.int64'> for quantity nomad.datamodel.metainfo.common_dft.Method.gw_self_energy_c_number_of_poles:Quantity is not of type <class 'int'> |
|QptEjMdzQgru7wm-iUM0sYRQktG0 |1 |9UM_CpSzaystbWPZoghgjg3dZmC8|CgtgzCMXTUyY0tjfBYXAnA|TiO2/anatese/K/nomad/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1106, in _parse_dos if self.dos_parser.get('totaldos', None) is None: File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 505, in parse if not self.total_dos: File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 471, in total_dos self._total_dos = self.root.findall('./%s/%s' % (self._totaldos_key, self._diagram_key)) File "/app/nomad/parsing/file_parser/xml_parser.py", line 52, in root self._file_handler = ElementTree.parse(self.mainfile).getroot() File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse tree.parse(source, parser) File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 598, in parse self._root = parser._parse_whole(source) File "<string>", line None xml.etree.ElementTree.ParseError: no element found: line 111468, column 0|
|g4xYYjGHqxmbahc5Xk0jQrB2BvLv |1 |B7YX4kB9f1PSwYbeJeUcExy-UKZw|cdAK5_-gQ4GLFbZrgBRpQQ|conv/workdir/ngridk_08-rgkmax_05/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse self.parse_configurations() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2058, in parse_configurations self.parse_file(f, sec_scc) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1337, in parse_file parser_function(section) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1162, in _parse_eigenvalues if self.eigval_parser.get('eigenvalues_occupancies', None) is None: File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/app/nomad/parsing/file_parser/text_parser.py", line 501, in parse if self.file_mmap is None: File "/app/nomad/parsing/file_parser/text_parser.py", line 348, in file_mmap offset=self._file_offset) ValueError: cannot mmap an empty file |
|iwirABQpTlKQWqHEg7jS-2l-oA5w |1 |qQbcfYsilb40jkYEN5hoW9PUdUrY|mcyQ9xEwQu2uNrkEtKNobg|volumes-rgkmax13/volume-2.1/1.01315940382018/INFO.OUT |parsers/exciting|Traceback (most recent call last): File "/app/nomad/processing/data.py", line 453, in parsing self._parser_results, logger=logger) File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2124, in parse self.parse_method() File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1813, in parse_method correlation = self.input_xml_parser.get('libxc/correlation', None) File "/app/nomad/parsing/file_parser/file_parser.py", line 78, in get val = self.results.get(key, None) File "/app/nomad/parsing/file_parser/file_parser.py", line 47, in results self.parse(self._key, **self._kwargs) File "/app/nomad/parsing/file_parser/xml_parser.py", line 76, in parse if not self.root: File "/app/nomad/parsing/file_parser/xml_parser.py", line 52, in root self._file_handler = ElementTree.parse(self.mainfile).getroot() File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse tree.parse(source, parser) File "/usr/local/lib/python3.7/xml/etree/ElementTree.py", line 598, in parse self._root = parser._parse_whole(source) File "<string>", line None xml.etree.ElementTree.ParseError: no element found: line 1, column 0 |
In addition, there are plenty of ABINIT calc that do not have a `section_system`, but no parser errors: https://nomad-lab.eu/prod/rae/reprocess/gui/search?visualization=system&dft.code_name=ABINIT&dft.system=unavailable
I am not sure if this is all, because the numbers do not add up. There are roughly ~2500 exciting runs with missing formulas/systems, but only ~2000 exceptions. We will see after bug fixes.v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/482New VASP parser throws exception on one of the encyclopedia test data2021-01-26T12:48:48ZMarkus ScheidgenNew VASP parser throws exception on one of the encyclopedia test dataThis happens on `vasp_si_bands_only/OUTCAR` in `tests/data/api/enc_public.zip`
```
Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/processing/data.py", line 444, in parsing
self._parser...This happens on `vasp_si_bands_only/OUTCAR` in `tests/data/api/enc_public.zip`
```
Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/processing/data.py", line 444, in parsing
self._parser_results, logger=logger)
File "/Users/markus/Documents/Projects/nomad-fairdi/dependencies/parsers/vasp/vaspparser/vasp_parser.py", line 974, in parse
self.parse_configurations()
File "/Users/markus/Documents/Projects/nomad-fairdi/dependencies/parsers/vasp/vaspparser/vasp_parser.py", line 936, in parse_configurations
parse_eigenvalues(n)
File "/Users/markus/Documents/Projects/nomad-fairdi/dependencies/parsers/vasp/vaspparser/vasp_parser.py", line 847, in parse_eigenvalues
eigenvalues = self.parser.get_eigenvalues(n_calc)
File "/Users/markus/Documents/Projects/nomad-fairdi/dependencies/parsers/vasp/vaspparser/vasp_parser.py", line 540, in get_eigenvalues
eigenvalues = np.reshape(eigenvalues, (self.ispin, n_kpts, self.n_bands, 3))
File "<__array_function__ internals>", line 6, in reshape
File "/Users/markus/Documents/Projects/nomad-fairdi/.pyenv/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 299, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/Users/markus/Documents/Projects/nomad-fairdi/.pyenv/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 55, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "/Users/markus/Documents/Projects/nomad-fairdi/.pyenv/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 44, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: cannot reshape array of size 115200 into shape (1,200,32,3)
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/481Crystal parser very slow on some calcs2021-01-25T08:46:42ZMarkus ScheidgenCrystal parser very slow on some calcsWhile the new crystal parser performs well on most calculations, some are processed extremely slow. This example calc_id=`g4uv0JfJhVDonAYjtYnb6Z1udzEs` took over 2000s to process (the mainfile is only 200kb). This only happens on a few e...While the new crystal parser performs well on most calculations, some are processed extremely slow. This example calc_id=`g4uv0JfJhVDonAYjtYnb6Z1udzEs` took over 2000s to process (the mainfile is only 200kb). This only happens on a few entries (~200 of 4000).
I remember @himanel1 complaining about slow runs on the dev cluster. Maybe that has the same cause.v0.10.0Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/478Different PBC in new vs old crystal parser2021-01-21T06:19:57ZMarkus ScheidgenDifferent PBC in new vs old crystal parserThe new crystal parsers assigns `(true, true, true)` to many (or all) calcs' periodic boundary conditions, where the old parser was assigning `(false, false, false)`. As a result many calcs become bulk or without system type that have be...The new crystal parsers assigns `(true, true, true)` to many (or all) calcs' periodic boundary conditions, where the old parser was assigning `(false, false, false)`. As a result many calcs become bulk or without system type that have been molecules on the old system.
You can compare the production (https://nomad-lab.eu/prod/rae/gui/search?visualization=system&dft.code_name=Crystal) with the reprocessing NOMAD installation (https://nomad-lab.eu/prod/rae/reprocess/gui/search?dft.code_name=Crystal&visualization=system)
An example calc where you see the different PBC: 6GPEE6tsj4sVMHkjuhboMyi5f_9u
@himanel1 could you investigate, it the new behaviour is indeed intended/correct?v0.10.0Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/477Wrong dimensionality of atom positions vs number of atoms in new ABINIT parser2021-01-26T12:51:18ZMarkus ScheidgenWrong dimensionality of atom positions vs number of atoms in new ABINIT parserThis happens on almost all ABINIT calculations. For example: aFuxFHVCb5d_tQblAGDoCncE4o-s
If you look at the archive, the atom_positions matrix is one vector to short to what you would expect by the length of atom_labels, species, numbe...This happens on almost all ABINIT calculations. For example: aFuxFHVCb5d_tQblAGDoCncE4o-s
If you look at the archive, the atom_positions matrix is one vector to short to what you would expect by the length of atom_labels, species, number_of_atoms, etc.
As a result the system normalisation fails, e.g. there is not classification on system and symmetry.v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/476Exception in new FHI-aims parser2021-01-26T12:47:59ZMarkus ScheidgenException in new FHI-aims parserI hope this one is not random. It is also type/conversion related. Also see #475.
This happened for ~2000 calcs out of 100.000 calcs.
Example calc: yOVx7KJIL_8KdRVP_TOdyqYINPJ2
```
Traceback (most recent call last):
File "/app/nomad...I hope this one is not random. It is also type/conversion related. Also see #475.
This happened for ~2000 calcs out of 100.000 calcs.
Example calc: yOVx7KJIL_8KdRVP_TOdyqYINPJ2
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 444, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1479, in parse
self.parse_method()
File "/usr/local/lib/python3.7/site-packages/fhiaimsparser/fhiaims_parser.py", line 1283, in parse_method
sec_method.x_fhi_aims_controlIn_relativistic_threshold = val[2]
File "/app/nomad/metainfo/metainfo.py", line 849, in __setattr__
return super().__setattr__(name, value)
File "/app/nomad/metainfo/metainfo.py", line 2111, in __set__
obj.m_set(self, value)
File "/app/nomad/metainfo/metainfo.py", line 956, in m_set
value = self.__to_np(quantity_def, value)
File "/app/nomad/metainfo/metainfo.py", line 934, in __to_np
value = quantity_def.type.type(value)
ValueError: could not convert string to float: 'n'
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/475Exception in new exciting parser, cannot get system data2021-03-17T08:15:07ZMarkus ScheidgenException in new exciting parser, cannot get system dataThis happens to a large amount of exciting calculations (>4000). It's probably due to a specific exciting version.
One example `calc_id=9Y-5JMQpBip1lF5IrAd78XpxA6DU`:
Trace:
```
Traceback (most recent call last):
File "/app/nomad/pro...This happens to a large amount of exciting calculations (>4000). It's probably due to a specific exciting version.
One example `calc_id=9Y-5JMQpBip1lF5IrAd78XpxA6DU`:
Trace:
```
Traceback (most recent call last):
File "/app/nomad/processing/data.py", line 437, in parsing
self._parser_results, logger=logger)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2126, in parse
self.parse_configurations()
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2051, in parse_configurations
sec_scc = parse_configuration(self.info_parser.get('groundstate'))
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 2042, in parse_configuration
sec_system = self.parse_system(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 1955, in parse_system
positions = self.info_parser.get_atom_positions(section)
File "/usr/local/lib/python3.7/site-packages/excitingparser/exciting_parser.py", line 966, in get_atom_positions
positions = np.dot(positions, cell.magnitude)
File "<__array_function__ internals>", line 6, in dot
TypeError: Cannot cast array data from dtype('float64') to dtype('<U32') according to the rule 'safe'
```v0.10.0Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/450Optimade issues2021-03-02T11:38:01ZMarkus ScheidgenOptimade issues```
pip install optimade; optimade-validator -vv http://nomad-lab.eu/...
```
- sort is not working at all```
pip install optimade; optimade-validator -vv http://nomad-lab.eu/...
```
- sort is not working at allv0.10.0Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/430Fix the mail issues2021-01-11T14:46:09ZMarkus ScheidgenFix the mail issuesThere are several problems with the mail "system" and no real tests for it.
- its spam for many people
- google (as a recipient) says
'''
<some.user@googlemail.com>: host gmail-smtp-in.l.google.com[64.233.166.27]
said: 550-5.7.1 [13...There are several problems with the mail "system" and no real tests for it.
- its spam for many people
- google (as a recipient) says
'''
<some.user@googlemail.com>: host gmail-smtp-in.l.google.com[64.233.166.27]
said: 550-5.7.1 [130.xxx.xxx.xxx 11] Our system has detected that this
message is 550-5.7.1 not RFC 5322 compliant: 550-5.7.1 'From' header is
missing. 550-5.7.1 To reduce the amount of spam sent to Gmail, this message
has been 550-5.7.1 blocked. Please visit 550-5.7.1
https://support.google.com/mail/?p=RfcMessageNonCompliant 550 5.7.1 and
review RFC 5322 specifications for more information. xxxxxxxxxxxxxxx.272 -
gsmtp (in reply to end of DATA command)
'''
- the email addresses are not consistent
- do we still need to send emails?v0.10.0https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/496Allow admin user to show/download all data2021-03-02T11:38:01ZMarkus ScheidgenAllow admin user to show/download all dataThis will help debugging parsers (e.g. with errors on embargo entries) and the oasis.This will help debugging parsers (e.g. with errors on embargo entries) and the oasis.v0.10.0Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/490Dict/JSON typed metainfo quantities2021-02-03T21:29:07ZMarkus ScheidgenDict/JSON typed metainfo quantitiesv0.10.0Markus ScheidgenMarkus Scheidgen