Testing plugin schema is not directly accessible
TLDR: Current way of testing a plugin schema never fails the PyTest. Information about this, and how to work around should be readily available.
When testing the schema of a plugin, one should use normalize_all
just like in the nomad-schema-plugin-example repo. But any errors that normalize_all
might encounter are already handled by the structlogger. As a result, running this test would always pass, no matter what.
I think, it's important to share this information with plugin developers, perhaps through a remark on the plugin example repos.
One can work around this issue by using a fixture to process the logs and find if there are keywords matching ['ERROR'] etc. Something like this is available here. If a plugin developer wants to use this, they have to add this fixture in the tests/conftest.py
and use it as an argument to the test function.
However, one can not run these tests directly as they will encounter ImportErrors
because python-logstash
, which is required to run this fixture, is not packaged as a dependency of the nomad-lab. So if the tester installs nomad-lab using pip, they won't be able to run the tests unless they also install python-logstash
.
Can we consider to add python-logstash
as a dependency for the nomad-lab? I understand this is not ideal, maybe there are some better ideas.