diff --git a/examples/data/rdm_tutorial/archives/.gitkeep b/examples/data/rdm_tutorial/archives/.gitkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/examples/data/rdm_tutorial/tutorial.ipynb b/examples/data/rdm_tutorial/tutorial.ipynb index 2e0ffc4f5c87229b3ecb33f19bddb6e08698331f..fdb91445bd7a5cd8f271bd5a8db6332589b56eba 100644 --- a/examples/data/rdm_tutorial/tutorial.ipynb +++ b/examples/data/rdm_tutorial/tutorial.ipynb @@ -37,7 +37,7 @@ "source": [ "# Introduction\n", "\n", - "This notebook will teach you how you can build tailored research data management (RDM) solutions using NOMAD. It uses existing thermally activated delayed fluorescent (TADF) molecule data to teach you how we can use the NOMAD to turn research data into datasets that are FAIR: Findable, Accessible, Interoperable and Reusable. . The end-result can be distributed as a [*NOMAD plugin*](https://nomad-lab.eu/prod/v1/staging/docs/howto/plugins/plugins.html): a self-contained Python package that can be installed on a [*NOMAD Oasis*](https://nomad-lab.eu/prod/v1/staging/docs/howto/oasis/install.html).\n", + "This notebook will teach you how you can build tailored research data management (RDM) solutions using NOMAD. It uses existing thermally activated delayed fluorescent (TADF) molecule data to teach you how we can use the NOMAD to turn research data into datasets that are FAIR: Findable, Accessible, Interoperable and Reusable. The end-result can be distributed as a [*NOMAD plugin*](https://nomad-lab.eu/prod/v1/staging/docs/howto/plugins/plugins.html): a self-contained Python package that can be installed on a [*NOMAD Oasis*](https://nomad-lab.eu/prod/v1/staging/docs/howto/oasis/install.html).\n", "\n", "The notebook is divided into the following sections:\n", "\n", @@ -48,7 +48,7 @@ " 5. **App**: Creating a customized search experience\n", " 6. **Plugin**: Wrapping everything into a plugin\n", " \n", - "This tutorial requires some basic familiar with Python, and the most typical data science libraries, such as `numpy` and `pandas`.\n" + "This tutorial requires basic familiarity with Python, and it helps if you have experience with typical data science libraries, such as `numpy` and `pandas`.\n" ] }, { @@ -5868,6 +5868,7 @@ ], "layout": { "autosize": true, + "height": 360, "legend": { "title": { "text": "specifier" @@ -6701,8 +6702,8 @@ 1 ], "range": [ - 267.7345908607864, - 971.2654091392136 + 264.1923937360179, + 974.807606263982 ], "title": { "text": "peak_emission_wavelength_value" @@ -6813,11 +6814,19 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 1, "metadata": { "tags": [] }, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Schema is deprecated, use plugins. ()\n" + ] + } + ], "source": [ "from nomad.metainfo import Quantity\n", "from nomad.datamodel.data import Schema\n", @@ -6861,7 +6870,7 @@ "metadata": {}, "source": [ "The important things to notice here are:\n", - " - We have defined a new class `FluorescentMolecule` that inherits from `Schema`\n", + " - We have defined a new class `TADFMolecule` that inherits from `Schema`\n", " - We are using the `Quantity` class to describe concrete pieces of data such as numbers or strings\n", " - We have added descriptions to the schema itself and to the stored attributes\n", " - We have added units\n", @@ -6872,7 +6881,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 2, "metadata": { "tags": [] }, @@ -6896,7 +6905,7 @@ "<IPython.core.display.JSON object>" ] }, - "execution_count": 27, + "execution_count": 2, "metadata": { "application/json": { "expanded": false, @@ -6936,7 +6945,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 3, "metadata": { "tags": [] }, @@ -6989,7 +6998,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 4, "metadata": { "tags": [] }, @@ -7017,7 +7026,7 @@ }, { "cell_type": "code", - "execution_count": 30, + "execution_count": 5, "metadata": { "tags": [] }, @@ -7084,7 +7093,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 6, "metadata": { "tags": [] }, @@ -7095,7 +7104,7 @@ "text": [ "Collecting rdkit\n", " Downloading rdkit-2023.9.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.9 MB)\n", - "\u001b[2K \u001b[90mâ”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”\u001b[0m \u001b[32m34.9/34.9 MB\u001b[0m \u001b[31m47.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n", + "\u001b[2K \u001b[90mâ”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”\u001b[0m \u001b[32m34.9/34.9 MB\u001b[0m \u001b[31m29.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n", "\u001b[?25hRequirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from rdkit) (1.22.4)\n", "Requirement already satisfied: Pillow in /opt/conda/lib/python3.10/site-packages (from rdkit) (9.4.0)\n", "Installing collected packages: rdkit\n", @@ -7116,7 +7125,7 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 7, "metadata": { "tags": [] }, @@ -7147,7 +7156,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 8, "metadata": { "tags": [] }, @@ -7178,7 +7187,7 @@ "<IPython.core.display.JSON object>" ] }, - "execution_count": 33, + "execution_count": 8, "metadata": { "application/json": { "expanded": false, @@ -7220,14 +7229,14 @@ "source": [ "### Populating the `results` section\n", "\n", - "In addition to the completely custom `archive.data` subsection, there is also a pre-defined `archive.results` subsection. This is a section that is found in all entries, and it can be used to provide data that is cross-schema compatible. It contains things like formulas, compositions, methodologies, and a subset of relevant physical properties. NOMAD contains many built-in tooling that can pick up on data contained in this section to make queries and visualizations.\n", + "In addition to the completely custom `archive.data` subsection, there is also a pre-defined `archive.results` subsection. This is a section that is found in all entries, and it can be used to provide data that is cross-schema compatible. It contains things like formulas, compositions, methodologies, and a subset of relevant physical properties. NOMAD contains built-in tooling that can pick up on data contained in this section to make queries and visualizations.\n", "\n", "We should use our `normalize` function to populate this part of the archive. By populating this section, we make sure that people can find our data e.g. by querying `results.material.elements:all: ['C', 'H', 'O', 'N']` and we can also store the full 3D structure of the molecule in `results.material.topology` to visualize it automatically. Let's populate `results` by extending our `normalize` function:" ] }, { "cell_type": "code", - "execution_count": 62, + "execution_count": 9, "metadata": { "tags": [] }, @@ -7299,7 +7308,7 @@ }, { "cell_type": "code", - "execution_count": 35, + "execution_count": 10, "metadata": { "tags": [] }, @@ -7308,10 +7317,8 @@ "name": "stderr", "output_type": "stream", "text": [ - "/opt/conda/lib/python3.10/site-packages/nomad/atomutils.py:548: RuntimeWarning:\n", - "\n", - "invalid value encountered in double_scalars\n", - "\n" + "/opt/conda/lib/python3.10/site-packages/nomad/atomutils.py:548: RuntimeWarning: invalid value encountered in double_scalars\n", + " angles[i] = np.dot(cell[j], cell[k]) / (lengths[j] * lengths[k])\n" ] }, { @@ -7639,7 +7646,7 @@ "<IPython.core.display.JSON object>" ] }, - "execution_count": 35, + "execution_count": 10, "metadata": { "application/json": { "expanded": false, @@ -7673,7 +7680,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 11, "metadata": { "tags": [] }, @@ -7751,7 +7758,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 12, "metadata": { "tags": [] }, @@ -7799,7 +7806,7 @@ "\n", "\n", "\n", - "Here you can select the \"Custom schema option\" and type in \"TADFMolecule\" to get a suggestion that correspond to our schema. Select the shown suggestion, give a filename and proceed by pressing \"Create\". After this you will be forwarded to a screen that allows you to edit certain fields from the browser directly." + "Here you can select the \"Custom schema\" option and type in \"TADFMolecules\" to get a suggestion that corresponds to our *schema package* (schema packages can contain multiple schemas and base sections). After selecting this package, you can further select the \"TADFMolecule\" schema. Select it, give a filename and proceed by pressing \"Create\". After this you will be forwarded to a screen that allows you to edit certain fields from the browser directly." ] }, { @@ -7822,7 +7829,7 @@ }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 13, "metadata": { "tags": [] }, @@ -7864,7 +7871,7 @@ }, { "cell_type": "code", - "execution_count": 47, + "execution_count": 14, "metadata": { "tags": [] }, @@ -7926,7 +7933,7 @@ }, { "cell_type": "code", - "execution_count": 48, + "execution_count": 15, "metadata": { "tags": [] }, @@ -7945,7 +7952,7 @@ "<IPython.core.display.JSON object>" ] }, - "execution_count": 48, + "execution_count": 15, "metadata": { "application/json": { "expanded": false, @@ -7973,7 +7980,7 @@ }, { "cell_type": "code", - "execution_count": 49, + "execution_count": 16, "metadata": { "tags": [] }, @@ -8188,269 +8195,269 @@ ], "positions": [ [ - -6.42268645207434e-10, - 2.585856551474502e-10, - -3.179837295826005e-10 + -6.814696206160203e-10, + 1.5339613482155443e-10, + 3.0945049200911305e-10 ], [ - -7.350194434614037e-10, - 1.9372909983603436e-10, - -2.3618769222984095e-10 + -7.604062062952854e-10, + 1.2930774891066646e-10, + 1.9679450047823717e-10 ], [ - -6.942732987540638e-10, - 1.1518730860158562e-10, - -1.2770883279403429e-10 + -7.048048596347e-10, + 8.115241485901814e-11, + 7.767036294898098e-11 ], [ - -5.566083671869699e-10, - 1.0485216150237981e-10, - -1.052979613287706e-10 + -5.667785173451515e-10, + 5.852018673543404e-11, + 7.694743169822082e-11 ], [ - -4.612118264181994e-10, - 1.6883824556309519e-10, - -1.8577033689660822e-10 + -4.850342691594443e-10, + 8.178998452723457e-11, + 1.8855025608136338e-10 ], [ - -5.048215688175231e-10, - 2.469952690942593e-10, - -2.938921030235413e-10 + -5.434502695807768e-10, + 1.299793224706655e-10, + 3.0672355934618925e-10 ], [ - -3.316247138261994e-10, - 1.4350857305551207e-10, - -1.436488814363313e-10 + -3.5224960349401285e-10, + 5.150974722629348e-11, + 1.6351061639609565e-10 ], [ - -3.463774142696162e-10, - 6.554753132591187e-11, - -3.925518502989815e-11 + -3.5112713495716793e-10, + 1.2223862812798571e-11, + 3.8495503895972046e-11 ], [ - -2.689228078271913e-10, - 4.577248954020827e-13, - 5.33577219524511e-11 + -2.627452887254602e-10, + -3.4966900848627846e-11, + -5.512649715199801e-11 ], [ - -1.2729125107536498e-10, - 1.7427207186064554e-12, - 6.023497853573447e-11 + -1.2467507511225188e-10, + -5.141663397864354e-11, + -3.0479241829250434e-11 ], [ - -5.026801228694541e-11, - 7.395523233002485e-11, - -3.1419969241712994e-11 + -5.43755819063466e-11, + -1.5760476767276282e-10, + -8.776773444775391e-11 ], [ - 8.949516461967217e-11, - 7.413061306147575e-11, - -2.3156109228378153e-11 + 8.244761816827424e-11, + -1.733611577116786e-10, + -6.324678729633925e-11 ], [ - 1.542458570995994e-10, - 2.130626474286267e-12, - 7.694843451258885e-11 + 1.4999092286170733e-10, + -8.3440365510834e-11, + 1.926018749151556e-11 ], [ - 3.006612501623222e-10, - -4.927786597636994e-13, - 8.940192621397468e-11 + 2.93703900441862e-10, + -9.591753936053172e-11, + 4.75446521237426e-11 ], [ - 3.6304096751834603e-10, - 3.051648252474827e-11, - 2.1179228130908006e-10 + 3.4753192310284765e-10, + -2.1744650084680956e-10, + 9.407396189942876e-11 ], [ - 5.02333278703636e-10, - 2.8902795329419803e-11, - 2.2723636185848172e-10 + 4.840143651294465e-10, + -2.3306652523875315e-10, + 1.2182736029447062e-10 ], [ - 5.76938668545351e-10, - -5.0912826201750135e-12, - 1.1489418261661813e-10 + 5.647733512769404e-10, + -1.2177414102058032e-10, + 1.0102200356960969e-10 ], [ - 7.127517509232121e-10, - -1.2215037151995892e-11, - 1.0983007929101458e-10 + 6.991566120468404e-10, + -1.170361772703141e-10, + 1.2201455077211776e-10 ], [ - 7.4587188962594e-10, - -4.8608520957298526e-11, - -1.712013803708097e-11 + 7.399140213459271e-10, + 8.857340197749392e-12, + 8.995291409559919e-11 ], [ - 8.756424840064202e-10, - -6.715638919941985e-11, - -6.429783211545464e-11 + 8.710108566203223e-10, + 5.541080678881482e-11, + 9.748009932207535e-11 ], [ - 8.886773993345985e-10, - -1.047239697310715e-10, - -1.983928063111535e-10 + 8.923974230093576e-10, + 1.8844467464662417e-10, + 5.996718974915281e-11 ], [ - 7.764243650114555e-10, - -1.2238721272448714e-10, - -2.7969771555937503e-10 + 7.867570097424908e-10, + 2.693445555065859e-10, + 1.7323370862120943e-11 ], [ - 6.469654075022013e-10, - -1.0296579147373022e-10, - -2.2937486698626645e-10 + 6.557680520763339e-10, + 2.1966310153958118e-10, + 1.0773282429655897e-11 ], [ - 6.315620236219028e-10, - -6.511245901148359e-11, - -9.465600900879553e-11 + 6.320041166920742e-10, + 8.601379304025466e-11, + 4.8084068086379175e-11 ], [ - 5.215817714028972e-10, - -3.65951981916699e-11, - -8.560004310780443e-12 + 5.179500122054107e-10, + 8.093429990468459e-13, + 5.542585257206444e-11 ], [ - 3.8126055887021443e-10, - -3.4183778945008835e-11, - -2.1559909590851648e-11 + 3.8026117791329646e-10, + 1.4001906242945373e-11, + 2.8286687956076562e-11 ], [ - 7.107680369032411e-11, - -6.71362001509032e-11, - 1.6403206901188615e-10 + 7.239122371820574e-11, + 1.8378618276004298e-11, + 7.351872206526169e-11 ], [ - -6.454608713564217e-11, - -6.855772588177993e-11, - 1.5786150446494163e-10 + -6.009231345281886e-11, + 3.5633858218644775e-11, + 5.0586980685559806e-11 ], [ - -3.6162243537681673e-10, - -6.751807516959547e-11, - 1.3918091081804076e-10 + -3.403099097096614e-10, + -6.074901912450157e-11, + -1.7245100183639911e-10 ], [ - -3.4218515536673814e-10, - -1.4829625971138762e-10, - 2.4922402428849546e-10 + -3.0408719787154167e-10, + -1.0165117510325326e-10, + -2.990818809812584e-10 ], [ - -4.552416706201952e-10, - -2.012021800124313e-10, - 3.1347889988292706e-10 + -4.0537226351177007e-10, + -1.162262192727109e-10, + -3.9520071304881506e-10 ], [ - -5.858070431376567e-10, - -1.7329355204479024e-10, - 2.677782938813699e-10 + -5.404278943804877e-10, + -8.851789389468708e-11, + -3.648621774388778e-10 ], [ - -6.070341366589824e-10, - -9.17649307049666e-11, - 1.566214941165019e-10 + -5.779815718460854e-10, + -4.4197705457654896e-11, + -2.38047758483311e-10 ], [ - -4.938566712133077e-10, - -4.178361541681597e-11, - 9.63052485229122e-11 + -4.761449221258896e-10, + -3.1155059418135576e-11, + -1.4645373173719e-10 ], [ - -4.791490965473355e-10, - 4.0602183408213045e-11, - -1.3841353344061406e-11 + -4.776087550979876e-10, + 1.351617917700087e-11, + -1.5319269705563714e-11 ], [ - -6.769496642043926e-10, - 3.1895982432791653e-10, - -4.015528613228136e-10 + -7.274243289262127e-10, + 1.9083860309970143e-10, + 4.006376451018059e-10 ], [ - -8.41293665616658e-10, - 2.0447450428251083e-10, - -2.5720645232647085e-10 + -8.674541835873787e-10, + 1.4841700780088273e-10, + 2.0172204291739224e-10 ], [ - -7.670558567880349e-10, - 6.526790614565691e-11, - -6.480986453900074e-11 + -7.669120197526692e-10, + 6.281281085800999e-11, + -9.203240912419018e-12 ], [ - -4.3252704931838244e-10, - 2.972603133911057e-10, - -3.572300307255887e-10 + -4.816191678845424e-10, + 1.4849087463079216e-10, + 3.9390223050895773e-10 ], [ - -9.642683569860645e-11, - 1.3178996688436396e-10, - -1.1074454389254683e-10 + -1.0521576703730502e-10, + -2.31550921669807e-10, + -1.485860478570573e-10 ], [ - 1.4625693711371275e-10, - 1.3240860803061425e-10, - -9.534114136271845e-11 + 1.3520791980735835e-10, + -2.56521084398323e-10, + -1.0935295658102853e-10 ], [ - 3.024981868021949e-10, - 5.832275874540241e-11, - 2.980537555641191e-10 + 2.8187310250722185e-10, + -3.027872207465806e-10, + 1.1089697879488628e-10 ], [ - 5.500399941739528e-10, - 5.3198899397638166e-11, - 3.2141303196932636e-10 + 5.250064289500067e-10, + -3.2656559612793254e-10, + 1.5804431346851456e-10 ], [ - 9.615479784265078e-10, - -5.304150401383344e-11, - 1.4291456656694622e-13 + 9.517082636935286e-10, + -8.68480508471003e-12, + 1.3081419916141e-10 ], [ - 9.880189058716205e-10, - -1.2043004617568528e-10, - -2.399234919497102e-10 + 9.930874675672618e-10, + 2.2948061346575633e-10, + 6.41797213426903e-11 ], [ - 7.895023930855341e-10, - -1.516710863237489e-10, - -3.8368058608890416e-10 + 8.062568381145381e-10, + 3.725263006137671e-10, + -1.1216126730240736e-11 ], [ - 5.604736357630387e-10, - -1.1701135207447337e-10, - -2.9349239722143945e-10 + 5.744573231016062e-10, + 2.834980926639528e-10, + -2.2451225508111783e-11 ], [ - 3.3602933493358666e-10, - -5.95863415049648e-11, - -1.170857783630566e-10 + 3.420802181449816e-10, + 1.0886772734583234e-10, + -8.603153377599315e-12 ], [ - 1.1222010269950657e-10, - -1.2733810992108964e-10, - 2.4472350951090566e-10 + 1.1532875058250963e-10, + 9.217143137930445e-11, + 1.4088389989047605e-10 ], [ - -2.43092336553776e-10, - -1.714517573457708e-10, - 2.8653737216905885e-10 + -2.0100344860447588e-10, + -1.2021609100621008e-10, + -3.267025530339569e-10 ], [ - -4.417318452743234e-10, - -2.651897942450711e-10, - 4.0050002915836807e-10 + -3.79195075451379e-10, + -1.4856011757095024e-10, + -4.958266255587795e-10 ], [ - -6.708449216908677e-10, - -2.1626207739812938e-10, - 3.203643686050868e-10 + -6.159405453629103e-10, + -1.0064321296382544e-10, + -4.423215014756119e-10 ], [ - -7.070652891498447e-10, - -7.029797736651068e-11, - 1.2146913842421787e-10 + -6.811731073907394e-10, + -2.1000209556077262e-11, + -2.152298862639582e-10 ] ], "species": [ @@ -8567,7 +8574,7 @@ "<IPython.core.display.JSON object>" ] }, - "execution_count": 49, + "execution_count": 16, "metadata": { "application/json": { "expanded": false, @@ -8598,7 +8605,7 @@ }, { "cell_type": "code", - "execution_count": 61, + "execution_count": 18, "metadata": { "tags": [] }, @@ -8659,7 +8666,7 @@ }, { "cell_type": "code", - "execution_count": 45, + "execution_count": 20, "metadata": { "tags": [] }, @@ -8758,7 +8765,7 @@ " autorange=False,\n", " nbins=30,\n", " scale='1/4',\n", - " x=Axis(quantity=f'data.delayed_lifetime#{schema}', unit='µs'),\n", + " quantity=f'data.delayed_lifetime#{schema}',\n", " ),\n", " WidgetHistogram(\n", " type='histogram',\n", @@ -8766,10 +8773,7 @@ " autorange=False,\n", " nbins=30,\n", " scale='linear',\n", - " x=Axis(\n", - " quantity=f'data.singlet_triplet_energy_splitting#{schema}',\n", - " unit='eV',\n", - " ),\n", + " quantity=f'data.singlet_triplet_energy_splitting#{schema}',\n", " ),\n", " ]\n", " ),\n", @@ -8786,6 +8790,7 @@ "\n", " - `filters`: By default, apps will not load any additional schemas to avoid overcrowding the search bar. Only the \"standardized\" set of information found in `archive.metadata` and `archive.results` is available by default. To load our schema into the app, we define in `filters.include` a pattern that matches all information in our schema.\n", " - `filter_menus`: This controls which menus will be visible on the left. Currently, there is only a fixed set of available menus, but we are working on a more flexible customization for them as well.\n", + " - `filters_locked`: Used to pre-filter the results that this app will show. In our case we say we only want entries that originate from the `TADFMolecule` schema.\n", " - `columns`: This controls which columns will be shown in the results table.\n", " - `dashboard`: Controls which widgets to show by default above the search results.\n", "\n",