Merge branch 'fix-mpes-example' into 'develop'

Use zenodo url without redirect See merge request !157

Merge branch 'fix-mpes-example' into 'develop'
a9d52fba · Florian Dobener · 745b7945 · c2d6ba0a · a9d52fba · a9d52fba
Commit a9d52fba authored 1 year ago by Florian Dobener
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -153,5 +153,5 @@ test mpes-jupyter:
      - docker/mpes/example/MoTe2.h5
  script:
    - pip install pytest nbmake
-    - wget -nc https://zenodo.org/record/7573825/files/MoTe2.h5?download=1 -O docker/mpes/example/MoTe2.h5 || true
+    - wget -nc https://zenodo.org/records/7573825/files/MoTe2.h5?download=1 -O docker/mpes/example/MoTe2.h5 || true
    - pytest --nbmake --nbmake-timeout=3600 docker/mpes/example/E1*
--- a/docker/mpes/example/E1 Convert to NeXus.ipynb
+++ b/docker/mpes/example/E1 Convert to NeXus.ipynb
@@ -31,11 +31,13 @@
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
-        "tags": ["skip-execution"]
-    },
+    "tags": [
+     "skip-execution"
+    ]
+   },
   "outputs": [],
   "source": [
-    "! curl -o MoTe2.h5 \"https://zenodo.org/record/7573825/files/MoTe2.h5?download=1\""
+    "! curl -o MoTe2.h5 \"https://zenodo.org/records/7573825/files/MoTe2.h5?download=1\""
   ]
  },
  {

 %% Cell type:markdown id: tags:

 # Converting Multidimensional Photoemission Spectroscopy (MPES) data into the NeXus format

 %% Cell type:markdown id: tags:

 This example shows how to convert x-array based h5 measurement files as generated by the [preprocessing output](https://www.nature.com/articles/s41597-020-00769-8) of the software used at the Fritz-Haber Institute into the [MPES NeXus format](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes).
 For an example on how to generate such a h5 measurement file please refer to the [binning example](./E2%20Binning%20of%20WSe2.ipynb).

 %% Cell type:markdown id: tags:

 ## Download the data
 Since the provided datafile is comparably large (~200MB) it is not directly provided with the example.
 You can [download](https://zenodo.org/record/7573825/files/MoTe2.h5?download=1) it from zenodo. Place the file in the directory of this notebook afterwards. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the file with curl.

 %% Cell type:code id: tags:skip-execution

 ``` python
-! curl -o MoTe2.h5 "https://zenodo.org/record/7573825/files/MoTe2.h5?download=1"
+! curl -o MoTe2.h5 "https://zenodo.org/records/7573825/files/MoTe2.h5?download=1"
 ```

 %% Cell type:markdown id: tags:

 ## Convert the file to NeXus
 To convert the available files to the NeXus format we use the convert function readily supplied by pynxtools.
 It uses the downloaded measurement file, a json config file and optionally an electronic lab notebook (ELN) yaml file that is used in the [binning of WSe2 example](./E2%20Binning%20of%20WSe2.ipynb).
 The json config file maps specific metadata from the h5 measurement file to the nxs file, i.e. a pressure reading which automatically gets collected during measurement.
 The ELN is a file which supplies additional metadata which is written into the NeXus file.
 This is data which is not collected automatically, such as the person conducting the experiment.
 It can be written manually or generated, e.g. by the NOMAD ELN functionality.

 The convert command may also be executed in the command line with the command `dataconverter`:
 ```
 dataconverter \
    --reader mpes \
    --nxdl NXmpes \
    --input-file MoTe2.h5 \
    --input-file config_file.json \
    --output MoTe2.mpes.nxs
 ```

 %% Cell type:code id: tags:

 ``` python
 from pynxtools.dataconverter.convert import convert
 ```

 %% Cell type:markdown id: tags:

 The input parameters are defined as follows:

 **reader**: The specific reader which gets called inside pynxtools. This is supplied in the pynxtools python code. If you create a specific reader for your measurement file it gets selecetd here. If you use the binning procedure from FHI to generate a xarray h5 file you should use the reader called `mpes`.

 **nxdl**: The specific nxdl file to be used. For MPES this should always be `NXmpes` or one of its subdefinitions of the form `NXmpes_<name>`.

 **output**: The output filename of the NeXus file.

 %% Cell type:code id: tags:

 ``` python
 convert(input_file=["MoTe2.h5", "config_file.json"],
        reader='mpes',
        nxdl='NXmpes',
        output='MoTe2.mpes.nxs')
 ```

 %% Cell type:markdown id: tags:

 ## View the data with H5Web
 H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above. For an analysis on NeXus data files please refer to [analysis example](./E3%20pyARPES%20analysis.ipynb).

 You can also view this data with the H5Viewer or other tools from your local filesystem.

 %% Cell type:code id: tags:

 ``` python
 from jupyterlab_h5web import H5Web
 ```

 %% Cell type:code id: tags:

 ``` python
 H5Web('MoTe2.mpes.nxs')
 ```

--- a/docker/mpes/example/E2 Binning of WSe2.ipynb
+++ b/docker/mpes/example/E2 Binning of WSe2.ipynb
@@ -63,7 +63,7 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "! curl -o WSe2.zip \"https://zenodo.org/record/6369728/files/WSe2.zip\""
+    "! curl -o WSe2.zip \"https://zenodo.org/records/6369728/files/WSe2.zip\""
   ]
  },
  {

 %% Cell type:markdown id: tags:

 # Binning raw Multidimensional Photoemission Spectroscopy (MPES) data and converting it into the NeXus format

 %% Cell type:markdown id: tags:

 This example shows how to generate xarray based h5 files from WSe2 trARPES measurement data as detailed in this [paper](https://www.nature.com/articles/s41597-020-00769-8) and how to generate a file in the standardised [MPES NeXus format](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) from it.
 Due to the size of the example file (~6GB) you need at least 40 GB of memory on your computer you're executing this example on. If you just want to have a look on how to convert a pre-binned xarray based h5 file into the NeXus format you may have a look at the simpler [Convert to NeXus example](./E1%20Convert%20to%20NeXus.ipynb), which has lower hardware requirements.

 %% Cell type:markdown id: tags:

 ## Download RAW data (trARPES data of WSe2)

 %% Cell type:markdown id: tags:

 Here, we just set the main file folder for holding the measurement data.

 %% Cell type:code id: tags:

 ``` python
 import os
 FDIR = f'{os.getcwd()}/Scan049_1'
 ECAL = f'{os.getcwd()}/energycal_2019_01_08'
 ```

 %% Cell type:markdown id: tags:

 Since the provided measurement files are rather large (~6GB), they are not directly provided with the example.
 You can [download](https://zenodo.org/record/6369728/files/WSe2.zip) it from zenodo. This may take some time. Place the file in the directory of this notebook afterwards. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the file with curl.

 %% Cell type:code id: tags:

 ``` python
-! curl -o WSe2.zip "https://zenodo.org/record/6369728/files/WSe2.zip"
+! curl -o WSe2.zip "https://zenodo.org/records/6369728/files/WSe2.zip"
 ```

 %% Cell type:markdown id: tags:

 Now we extract the measurement files.

 %% Cell type:code id: tags:

 ``` python
 ! unzip WSe2.zip
 ```

 %% Cell type:markdown id: tags:

 ## Binning of measurement data

 %% Cell type:markdown id: tags:

 First we import the necessary packages. For a manual on how to install this dependencies refer to the provided [INSTALL.md](./INSTALL.md) file. If you're running a pre-built docker container or working with the NORTH tools, these dependencies are already available for you.

 %% Cell type:code id: tags:

 ``` python
 from mpes import base as base, fprocessing as fp, analysis as aly
 import matplotlib.pyplot as plt
 import numpy as np
 import os
 from dask import compute
 import datetime as dt
 import h5py
 ```

 %% Cell type:markdown id: tags:

 ### Initial data binning for distortion correction

 %% Cell type:code id: tags:

 ``` python
 parp = fp.parallelHDF5Processor(folder=FDIR)
 parp.gather(identifier=r'/*.h5', file_sorting=True)
 len(parp.files)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Bin a small range of of files to create a momentum map for distortion correction
 parp.files = parp.files[0:50]
 axes = ['X', 'Y', 't']
 # Important to keep the whole detector area for the initial binning!
 bins = [512, 512, 300]
 ranges = [(0, 2048), (0, 2048), (64000, 68000)]
 parp.parallelBinning(axes=axes, nbins=bins, ranges=ranges, scheduler='threads', ret=False)
 ```

 %% Cell type:markdown id: tags:

 ### Determine correction landmarks

 %% Cell type:code id: tags:

 ``` python
 # Select an energy slice at the valence band maximum, containing the 6 K-points and the Gamma point as distinct features
 mc = aly.MomentumCorrector(parp.combinedresult['binned'])
 mc.selectSlice2D(slice(165, 175), 2)
 # Extract these high-symmetry points
 mc.featureExtract(mc.slice, sigma=5, fwhm=10, sigma_radius=3)
 mc.view(points=mc.features, annotated=True)
 ```

 %% Cell type:markdown id: tags:

 ### Calculate thin plate spline symmetry correction

 %% Cell type:code id: tags:

 ``` python
 # Calculate a non-linear coordinate transformation based on thin plate splines that restores 6-fold symmetry
 mc.splineWarpEstimate(image=mc.slice, landmarks=mc.pouter_ord, include_center=True,
                      iterative=False, interp_order=2, update=True)
 mc.view(image=mc.slice_transformed, annotated=True, points={'feats':mc.ptargs}, backend='bokeh', crosshair=True, radii=[75,110,150], crosshair_thickness=0.2)
 ```

 %% Cell type:markdown id: tags:

 ### Image registration

 %% Cell type:code id: tags:

 ``` python
 # Apply a coordinate translation to move the image into the center of the detector
 mc.coordinateTransform(type='translation', xtrans=70., ytrans=70., keep=True)
 plt.imshow(mc.slice_transformed, origin='lower', cmap='terrain_r')
 plt.axvline(x=256)
 plt.axhline(y=256)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Rotate the image into a high-symmetry direction
 mc.coordinateTransform( type='rotation', angle=-5, center=(256., 256.), keep=True)
 plt.imshow(mc.slice_transformed, origin='lower', cmap='terrain_r')
 plt.axvline(x=256)
 plt.axhline(y=256)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Display the final deformation field
 subs = 20
 plt.scatter(mc.cdeform_field[::subs,::subs].ravel(), mc.rdeform_field[::subs,::subs].ravel(), c='b')
 ```

 %% Cell type:markdown id: tags:

 ### Momentum calibration

 %% Cell type:code id: tags:

 ``` python
 # Pick one high-symmetry point
 point_b = [252.,255.]
 # Pick the BZ center
 point_a = [308.,346.]
 # give the distance of the two in inverse Angstrom
 distance = np.pi*4/3/3.297
 # Momentum calibration assuming equal scaling along both x and y directions (equiscale=True)
 # Requirements : pixel coordinates of and the momentum space distance between two symmetry points,
 # plus the momentum coordinates
 # of one of the two points
 ext = mc.calibrate(mc.slice_transformed,
                   point_from=point_a,
                   point_to=point_b,
                   dist=distance,
                   equiscale=True,
                   ret=['extent'])
 ```

 %% Cell type:code id: tags:

 ``` python
 # Display corrected image in calibrated coordinates
 mc.view(image=mc.slice_transformed, imkwds=ext)
 plt.xlabel('$k_x$', fontsize=15)
 plt.ylabel('$k_y$', fontsize=15)
 ```

 %% Cell type:markdown id: tags:

 ### Energy calibration

 %% Cell type:code id: tags:

 ``` python
 # Bin traces for energy calibration
 axes = ['t']
 bins = [1000]
 ranges = [(63000, 80000)]
 traces, tof = fp.extractEDC(folder=ECAL,
                            axes=axes, bins=bins, ranges=ranges)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Applied bias voltages (negated, in order to achieve negative binding energies, E-E_F)
 voltages = np.arange(-12.2, -23.2, -1)
 ec = aly.EnergyCalibrator(biases=voltages, traces=traces, tof=tof)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Normalize traces to maximum
 ec.normalize(smooth=True, span=7, order=1)
 ec.view(traces=ec.traces_normed, xaxis=ec.tof, backend='bokeh')
 ```

 %% Cell type:code id: tags:

 ``` python
 # Define a TOF feature range, and translate it for each of the traces according to their shift in bias voltage
 rg = [(65000, 65200)]
 ec.addFeatures(traces=ec.traces_normed, refid=0, ranges=rg[0], infer_others=True, mode='append')
 ec.featranges
 ```

 %% Cell type:code id: tags:

 ``` python
 # Extract the first peak from each feature range
 ec.featureExtract(traces=ec.traces_normed, ranges=ec.featranges)
 ec.view(traces=ec.traces_normed, peaks=ec.peaks, backend='bokeh')
 ```

 %% Cell type:markdown id: tags:

 ### Calculate energy calibration

 %% Cell type:code id: tags:

 ``` python
 # calculate the energy calibration (3rd order polynom). Eref corresponds to the binding energy (E-E_F) of the selected feature in the refid trace.
 refid=5
 Eref=-1.3
 axs = ec.calibrate(ret='all', Eref=Eref, t=ec.tof, refid=refid)
 ec.view(traces=ec.traces_normed, xaxis=ec.calibration['axis'], backend='bokeh')
 ```

 %% Cell type:markdown id: tags:

 ### Quality of calibration

 %% Cell type:code id: tags:

 ``` python
 # inspect the quality of the energy calibration
 for i in range(0,len(voltages)):
    plt.plot(ec.calibration['axis']-(voltages[i]-voltages[refid]), ec.traces_normed[i])
 plt.xlim([-15,5])
 ```

 %% Cell type:markdown id: tags:

 ### Inspect calibration function

 %% Cell type:code id: tags:

 ``` python
 # energy calibration function vs. TOF
 ec.view(traces=ec.calibration['axis'][None,:], xaxis=ec.tof, backend='matplotlib', show_legend=False)
 plt.scatter(ec.peaks[:,0], ec.biases-ec.biases[refid]+Eref, s=50, c='k')
 plt.xlabel('Time-of-flight', fontsize=15)
 plt.ylabel('Energy (eV)', fontsize=15)
 plt.ylim([-8,6])
 plt.xlim([63400,69800])
 ```

 %% Cell type:markdown id: tags:

 ### Dataframe processor

 %% Cell type:code id: tags:

 ``` python
 # create the dask data frame processor
 dfp = fp.dataframeProcessor(datafolder=FDIR)
 dfp.read(source='folder', ftype='h5', timeStamps=True)
 ```

 %% Cell type:markdown id: tags:

 ### Apply energy calibration

 %% Cell type:code id: tags:

 ``` python
 # apply the energy calibration
 dfp.appendEAxis(E0=ec.calibration['E0'], a=ec.calibration['coeffs'])
 dfp.edf.head(8)
 ```

 %% Cell type:markdown id: tags:

 ### Apply distortion correction

 %% Cell type:code id: tags:

 ``` python
 # apply the distortion correction
 dfp.applyKCorrection(type='tps_matrix',
                     rdeform_field = mc.rdeform_field,
                     cdeform_field = mc.cdeform_field,
                     X='X', Y='Y', newX='Xm', newY='Ym')
 dfp.edf.head(8)
 ```

 %% Cell type:markdown id: tags:

 ### Apply momentum calibration

 %% Cell type:code id: tags:

 ``` python
 # apply the momentum calibration
 dfp.appendKAxis(point_b[0], point_b[1], X='Xm', Y='Ym', rstart=parp.binranges[0][0],
                cstart=parp.binranges[1][0],
                rstep=parp.binsteps[0],
                cstep=parp.binsteps[1],
                fc=mc.calibration['coeffs'][0],
                fr=mc.calibration['coeffs'][1])
 dfp.edf.head(8)
 ```

 %% Cell type:markdown id: tags:

 ### Apply pump-probe delay axis conversion

 %% Cell type:code id: tags:

 ``` python
 # calculate the pump-probe delay from the ADC coordinates
 ADCRange = (650, 6900)
 timeRange = (-100, 200)
 dfp.edf['delay'] = timeRange[0] + (dfp.edf['ADC']-ADCRange[0]) *\
    (timeRange[1] - timeRange[0])/(ADCRange[1]-ADCRange[0])
 dfp.edf.head(8)
 ```

 %% Cell type:markdown id: tags:

 ### Bin 4D data in transformed grid

 %% Cell type:code id: tags:

 ``` python
 # process the 4-dimensional binning
 axes = ['kx', 'ky', 'E', 'delay']
 bins = [50, 50, 100, 21]
 ranges = [(-2, 2), (-2, 2), (-3, 2), (-110, 190)]
 # jittering of energy and ADC should best be done on the bin size of the hardware,
 # not the rebinned bin size. This requires reverse-calculating the jitter amplitudes
 # from the bin sizes.
 TOFrange=[64500,67000]
 e_t_conversion = (base.tof2evpoly(ec.calibration['coeffs'],
                                  ec.calibration['E0'],
                                  TOFrange[0])
                  - base.tof2evpoly(ec.calibration['coeffs'],
                                    ec.calibration['E0'], TOFrange[1])
                 ) / (TOFrange[1] - TOFrange[0])
 d_adc_conversion = (timeRange[1] - timeRange[0]) / (ADCRange[1] - ADCRange[0])
 jitter_amplitude = [0.5,
                    0.5,
                    1*bins[2]/abs(ranges[2][1]-ranges[2][0])*e_t_conversion,
                    1*bins[3]/abs(ranges[3][1]-ranges[3][0])*d_adc_conversion]
 dfp.distributedBinning(axes=axes,
                       nbins=bins,
                       ranges=ranges,
                       scheduler='threads',
                       ret=False,
                       jittered=True,
                       jitter_amplitude=jitter_amplitude)
 ```

 %% Cell type:markdown id: tags:

 ### Run the following cell to store metadata from EPICS archive only if outside the FHI network
 This adds additional metadata to the xarray. This data may also be provided through additional ELN entries through a NOMAD instance or with a handwritten file directly to the mpes parser.

 %% Cell type:code id: tags:

 ``` python
 metadata = {"file": {}}
 metadata['file']["KTOF:Lens:Extr:I"] = -0.12877
 metadata['file']["KTOF:Lens:UDLD:V"] = 399.99905
 metadata['file']["KTOF:Lens:Sample:V"] = 17.19976
 metadata['file']["KTOF:Apertures:m1.RBV"] = 3.729931
 metadata['file']["KTOF:Apertures:m2.RBV"] = -5.200078
 metadata['file']["KTOF:Apertures:m3.RBV"] = -11.000425

 # Sample motor positions
 metadata['file']['trARPES:Carving:TRX.RBV'] = 7.1900000000000004
 metadata['file']['trARPES:Carving:TRY.RBV'] = -6.1700200225439552
 metadata['file']['trARPES:Carving:TRZ.RBV'] = 33.4501953125
 metadata['file']['trARPES:Carving:THT.RBV'] = 423.30500940561586
 metadata['file']['trARPES:Carving:PHI.RBV'] = 0.99931647456264949
 metadata['file']['trARPES:Carving:OMG.RBV'] = 11.002500171914066
 ```

 %% Cell type:markdown id: tags:

 ### Generate xarray
 Remember to remove the optional argument, metadata_dict, from the gather_metadata() function if the previous cell was not run.
 The missing archive metadata warnings are not critical for this example and can thus be ignored.

 %% Cell type:code id: tags:

 ``` python
 import copy
 res_xarray = dfp.gather_metadata(metadata_dict=copy.deepcopy(metadata), ec=ec, mc=mc)
 ```

 %% Cell type:markdown id: tags:

 ## Create a NeXus file from a xarray
 This conversion basically follows the same procedure as in the [convert to NeXus example](./E1%20Convert%20to%20Nexus.ipynb). Please refer to this notebook for details on the convert function. Here, we are using the objects keywords of `convert` to pass the generated xarray directly, instead of loading a h5 datafile.

 %% Cell type:code id: tags:

 ``` python
 from pynxtools.dataconverter.convert import convert
 ```

 %% Cell type:code id: tags:

 ``` python
 convert(input_file=["config_file.json", "WSe2_eln.yaml"],
        objects=res_xarray,
        reader='mpes',
        nxdl='NXmpes',
        output='WSe2.mpes.nxs')
 ```

 %% Cell type:markdown id: tags:

 ## View the data with H5Web
 H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above. For an analysis on NeXus data files please refer to [analysis example](./E3%20pyARPES%20analysis.ipynb).

 You can also view this data with the H5Viewer or other tools from your local filesystem.

 %% Cell type:code id: tags:

 ``` python
 from jupyterlab_h5web import H5Web
 ```

 %% Cell type:code id: tags:

 ``` python
 H5Web('WSe2.mpes.nxs')
 ```

--- a/docker/mpes/example/E3 pyARPES analysis.ipynb
+++ b/docker/mpes/example/E3 pyARPES analysis.ipynb
@@ -62,8 +62,8 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "! curl -o \"{FDIR}/TiTe2_0deg.nxs\" \"https://zenodo.org/record/5541490/files/TiTe2_0deg.nxs?download=1\"\n",
-    "! curl -o \"{FDIR}/TiTe2_60deg.nxs\" \"https://zenodo.org/record/5541490/files/TiTe2_60deg.nxs?download=1\""
+    "! curl -o \"{FDIR}/TiTe2_0deg.nxs\" \"https://zenodo.org/records/5541490/files/TiTe2_0deg.nxs?download=1\"\n",
+    "! curl -o \"{FDIR}/TiTe2_60deg.nxs\" \"https://zenodo.org/records/5541490/files/TiTe2_60deg.nxs?download=1\""
   ]
  },
  {
@@ -181,7 +181,7 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "! curl -o \"{FDIR}/MoTe2_dyn.nxs\" \"https://zenodo.org/record/4632481/files/201905_MoTe2.nxs?download=1\""
+    "! curl -o \"{FDIR}/MoTe2_dyn.nxs\" \"https://zenodo.org/records/4632481/files/201905_MoTe2.nxs?download=1\""
   ]
  },
  {
@@ -236,7 +236,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.10.4 (main, Jul 28 2022, 10:22:15) [Clang 13.1.6 (clang-1316.0.21.2.5)]"
+   "version": "3.10.4"
  },
  "vscode": {
   "interpreter": {

 %% Cell type:markdown id:17a6a4f9-8c8a-49f3-b267-ee38d68d4a50 tags:

 # Analysing Multidimensional Photoemission Spectroscopy (MPES) data with pyARPES
 This example shows how to analyse data in the NeXus format with the [pyARPES](https://github.com/chstan/arpes) python package. You'll find details on how to generate such NeXus files in the [convert](./E1%20Convert%20to%20NeXus.ipynb) or [binning](./E2%20Binning%20of%20WSe2.ipynb) example.

 %% Cell type:code id:a8fb79b7-7379-4309-9298-dc16fba67c70 tags:

 ``` python
 from arpes.plotting.qt_tool import qt_tool
 from arpes.plotting.basic_tools import path_tool
 from arpes.io import example_data, load_data
 from arpes.endstations.plugin.nexus import NeXusEndstation
 import xarray as xr
 import numpy as np
 from pathlib import Path
 import h5py as h5
 ```

 %% Cell type:markdown id:a98808fd-8d13-4cd9-98a0-ab55c8c40f15 tags:

 ## Download the data
 First we set the file directory and create it if not already present.

 %% Cell type:code id:e9658e8f-21e3-445e-b04f-c8657f707fb6 tags:

 ``` python
 FDIR = "TiTe2"
 ! mkdir -p '{FDIR}'
 ```

 %% Cell type:markdown id:14597c45-b958-434c-b0ed-f31ad68378bc tags:

 Since the provided datafile is comparably large (~200MB) it is not directly provided with the example.
 You can download the data for [zero](https://zenodo.org/record/5541490/files/TiTe2_0deg.nxs?download=1) and [60 degree](https://zenodo.org/record/5541490/files/TiTe2_60deg.nxs?download=1) from zenodo. Place the files in the FDIR directory specified in the cell above. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the files with curl.

 %% Cell type:code id:58315f02-ae42-4318-abdc-e102e1753290 tags:

 ``` python
-! curl -o "{FDIR}/TiTe2_0deg.nxs" "https://zenodo.org/record/5541490/files/TiTe2_0deg.nxs?download=1"
-! curl -o "{FDIR}/TiTe2_60deg.nxs" "https://zenodo.org/record/5541490/files/TiTe2_60deg.nxs?download=1"
+! curl -o "{FDIR}/TiTe2_0deg.nxs" "https://zenodo.org/records/5541490/files/TiTe2_0deg.nxs?download=1"
+! curl -o "{FDIR}/TiTe2_60deg.nxs" "https://zenodo.org/records/5541490/files/TiTe2_60deg.nxs?download=1"
 ```

 %% Cell type:markdown id:ea6ea584-fe25-4af1-ae72-6ae7d7f34830 tags:

 ## Load data into a xarray
 Extract the NeXus file into an xArray. In the future, mode advanced loading APIs will be developed, transferring all useful metadata.

 %% Cell type:code id:3b259695-1f73-4bd1-a1bc-4b5abd0432a6 tags:

 ``` python
 h5_TiTe2_0deg = h5.File(f'{FDIR}/TiTe2_0deg.nxs', 'r' )
 meas_TiTe2_0deg = xr.DataArray(
    h5_TiTe2_0deg['entry/data/Photoemission intensity'][:],
    coords={'BE': h5_TiTe2_0deg['entry/data/calculated_Energy'][:],
            'kx': h5_TiTe2_0deg['entry/data/calculated_kx'][:],
            'ky': h5_TiTe2_0deg['entry/data/calculated_ky'][:]},
 )

 h5_TiTe2_60deg = h5.File(f'{FDIR}/TiTe2_60deg.nxs', 'r' )
 meas_TiTe2_60deg = xr.DataArray(
    h5_TiTe2_60deg['entry/data/Photoemission intensity'][:],
    coords={'BE': h5_TiTe2_60deg['entry/data/calculated_Energy'][:],
            'kx': h5_TiTe2_60deg['entry/data/calculated_kx'][:],
            'ky': h5_TiTe2_60deg['entry/data/calculated_ky'][:]},
 )
 ```

 %% Cell type:markdown id:1d7de2d8-025f-4ead-9342-3c40f530b7fe tags:

 ## Visualize 0º data
 You can use the "Axes" tab to transpose the volume to obtain the optimal panel arrangement.

 %% Cell type:code id:a9436bce-4f5d-4f68-9f8b-c0f5ee1ef3d3 tags:

 ``` python
 qt_tool(meas_TiTe2_0deg)
 ```

 %% Cell type:markdown id:88310069-fb9a-48c4-a490-072118176937 tags:

 ## Visualize 60º data

 %% Cell type:code id:a01b408e-1caf-400c-a72f-0b21d7085916 tags:

 ``` python
 qt_tool(meas_TiTe2_60deg)
 ```

 %% Cell type:markdown id:7769f64f-1efd-4bd6-a905-a7cfcd9aea20 tags:

 ## View the difference between the angles
 Noticed the asymmetry in the electron pockts at the Fermi surface? Let's make it shine!
 We calculate the difference between the two angles and visualize it.
 Unfortunately, there are only linear colorscales available in pyARPES.
 The resulting data resembles the main observation in this [paper](https://arxiv.org/pdf/2107.07158.pdf).

 %% Cell type:code id:9aa56079-be66-4575-91c9-7bc77213893f tags:

 ``` python
 qt_tool(meas_TiTe2_0deg - meas_TiTe2_60deg)
 ```

 %% Cell type:markdown id:6310460a-c58f-49bc-8d64-cfcf87f2fc51 tags:

 ## 4D visualization

 %% Cell type:markdown id:c2e62727-32c9-432e-9d25-9d35b4347811 tags:

 We can also test 4D data. The procedure is the same: [download](https://zenodo.org/record/4632481/files/201905_MoTe2.nxs?download=1) the data, convert it to xarray and show it with `qt_tool`.

 %% Cell type:code id:4bf60894-22d3-4034-a808-311f285278bc tags:

 ``` python
-! curl -o "{FDIR}/MoTe2_dyn.nxs" "https://zenodo.org/record/4632481/files/201905_MoTe2.nxs?download=1"
+! curl -o "{FDIR}/MoTe2_dyn.nxs" "https://zenodo.org/records/4632481/files/201905_MoTe2.nxs?download=1"
 ```

 %% Cell type:code id:dda4bc9a-9729-4bd3-9456-a174427e7a3c tags:

 ``` python
 h5_MoTe2_dyn = h5.File(f'{FDIR}/MoTe2_dyn.nxs', 'r')
 meas_MoTe2_dyn = xr.DataArray(
    h5_MoTe2_dyn['entry/data/Photoemission intensity'][:],
    coords={'tpp': np.squeeze(h5_MoTe2_dyn['entry/data/calculated_Tpp'][:]),
            'BE': np.squeeze(h5_MoTe2_dyn['entry/data/calculated_Energy'][:]),
            'kx': np.squeeze(h5_MoTe2_dyn['entry/data/calculated_kx'][:]),
            'ky': np.squeeze(h5_MoTe2_dyn['entry/data/calculated_ky'][:])},
 )
 ```

 %% Cell type:markdown id:e44915a3-e13f-4bb9-a6d5-3af34c113d09 tags:

 At the Fermi energy, you will see an electron pocket appearing close to time zero and disappearing immediately after wards. This is the signature of a dynamical Lifshitz transition, a change in the topology of the Fermi surface. To learn more, read https://www.science.org/doi/10.1126/sciadv.abd9275'

 %% Cell type:code id:cb0b2144-7ae8-401e-a351-6416098c408a tags:

 ``` python
 qt_tool(meas_MoTe2_dyn)
 ```

--- a/docker/mpes/example/downloads.archive.yaml
+++ b/docker/mpes/example/downloads.archive.yaml
@@ -5,9 +5,9 @@ data:
    - TiTe2/TiTe2_60deg.nxs
    - mpes.archive.json
  downloads:
-    - url: https://zenodo.org/record/7573825/files/MoTe2.h5?download=1
+    - url: https://zenodo.org/records/7573825/files/MoTe2.h5?download=1
      output: MoTe2.h5
-    - url: https://zenodo.org/record/5541490/files/TiTe2_0deg.nxs?download=1
+    - url: https://zenodo.org/records/5541490/files/TiTe2_0deg.nxs?download=1
      output: TiTe2/TiTe2_0deg.nxs
-    - url: https://zenodo.org/record/5541490/files/TiTe2_60deg.nxs?download=1
+    - url: https://zenodo.org/records/5541490/files/TiTe2_60deg.nxs?download=1
      output: TiTe2/TiTe2_60deg.nxs
\ No newline at end of file