diff --git a/Elab.ipynb b/1-eLabFTW.ipynb similarity index 100% rename from Elab.ipynb rename to 1-eLabFTW.ipynb diff --git a/analysis.ipynb b/2-data_analysis.ipynb similarity index 100% rename from analysis.ipynb rename to 2-data_analysis.ipynb diff --git a/3-git-instructions.md b/3-git-instructions.md index 4468c41667e1018a02c874add26a35cb5c09f881..fa85d725766daa31380e7da82e84dc5b929b9f4b 100644 --- a/3-git-instructions.md +++ b/3-git-instructions.md @@ -6,12 +6,13 @@ There's a LOT more to git but for our purposes now this is enough. ## Steps ### Creating a git repo -* In the jupyter environment open a Terminal from the New dropdown menu -* In the terminal create a new folder with `mkdir name-of-project`. This folder will be our git repository and we will keep track of the history of everything we put inside. -* Copy the analysis notebook, the binder folder and the docker file inside with the command `cp <what you want to copy> name-of-project` -* From the jupyter file browser, create a file called `README.md` in this folder and add some information about the project. This will be displayed when you upload to github. -* In the terminal change directories into the folder with `cd name-of-project`. -* Initialize a git repository with `git init` +* In the jupyter environment open a Terminal from the "New" dropdown menu +* The folder "repository" will be our git repository and we will keep track of the history of everything we put inside. +* Copy the analysis notebook inside with the command `cp 2-data_analysis.ipynb repository/analysis.ipynb` +* From the jupyter file browser, create a file called `README.md` in this folder and add some information about the project. This will be displayed when you upload (push) your repository to github. If you prefer, there's also vim and nano available on the command line. +* In the terminal change directories into the folder with `cd repository`. +* Take a brief look at the files already in the repository. We will use them for the next parts of the tutorial. +* Initialize a git repository with `git init`. * Initialize your credentials: ``` $ git config --global user.email "YOUR EMAIL" @@ -20,11 +21,12 @@ $ git config --global user.name "YOUR NAME" ### Committing files to history -**Important: you should clear all output of notebooks and never commit large data files to git history. Git is made for keeping track of text files (under the hood Jupyter notebooks are still text files)** +**Important: you should clear all output of notebooks and never commit large data files to git history. Git is made for keeping track of text files (under the hood Jupyter notebooks are still very large text files)** +**Important 2: everything you "commit" will be forever available in git history (unless you go through painstaking steps to delete it). Git is made so that you have the lowest chance possible to lose work. So it is VERY important not to add passwords, secrets or tokens to files in history!** * Check the status of files with `git status` -* Add all files to the staging area to be committed to history with `git add .`. If you don't want to add all files you should instead o f `.` use file paths -* Create a commit with `git commit -m "some commit message like git commit"` +* Add all files to the staging area to be committed to history with `git add .`. If you don't want to add all files you should instead of `.` use file paths. +* Create a commit with `git commit -m "some commit message to explain what changed"`. * Your files in their current state are now committed to history. If you make changes you can always return to this state. ### Publishing to Github @@ -33,4 +35,4 @@ $ git config --global user.name "YOUR NAME" * Go back to the terminal and type `cat ~/.ssh/id_rsa.pub`. Copy the output to the key field on Github, give it a name and add the key. * Create a new repository on github. You will see instructions there. Add the repository as a remote with `git remote add origin git@github.com...`. * Push your repository up to github with `git push -u origin master`. Refresh the github page to see your repo is now on github. -* Whenever you create commits locally and push, all those versions will be available on github. +* Whenever you create commits locally and push, all those versions will be available on github, and to whomever decides to "clone" the project to their computer. diff --git a/4-mybinder-instructions.md b/4-mybinder-instructions.md index b42424346ce30d2c7c94d2347f7acfbbec85afc1..32e6d7cf1a8d3e17af4ae8babe440368c3bc7068 100644 --- a/4-mybinder-instructions.md +++ b/4-mybinder-instructions.md @@ -1,4 +1,5 @@ -* Ensure you have copied the binder folder into your repository and that it is on Github. +# Running your notebook on mybinder +* Ensure you have the binder folder is in your repository and that it is on Github. * if not, copy it into the folder, add, commit and push. * Go to <https://mybinder.org/> and add the link to your repository in the box. copy the markdown link to the badge and copy it into your README.md file. * add, commit and push. Now you should be able to click the link and build and run the environment. diff --git a/5-docker-instructions.md b/5-docker-instructions.md index c2c2b4574a12c3ae547167c719237805f2e9689c..4d425849b0e7f1c1e241d7cb4e4554db3a31a964 100644 --- a/5-docker-instructions.md +++ b/5-docker-instructions.md @@ -1,9 +1,11 @@ -* check out the docker file -* check out the github actions workflow -* make sure it is available on your github repo. If not, add, commit, push. -* create an account on dockerhub. Go to account settings and create a security access token. Copy it to a safe place, it will only be shown once. +# How to build your project into a docker image and publish on docker hub with Github actions +* check out the docker file in the repository folder +* check out the github actions workflow. You should see it on your github repo. In the jupyter environment you will not see it because it is hidden, but it is there. + * Note: To see it in the jupyter environment go to the command line, change directory into the `repository` folder and do `ls -la`. You now see a `.github` directory, which is hidden (all directories and files starting with . are hidden on linux). Check the contents of the file with `cat .github/workflows/build_docker_image.yaml`. You can edit the file with `nano` or `vim`. +* create an account on [dockerhub](https://hub.docker.com/). Go to account settings and create a security access token. Copy it to a safe place, it will only be shown once. * Add this under your Github repository secrets as `DOCKERHUB_TOKEN`. Also add another secret `DOCKERHUB_USERNAME` to enter your dockerhub username. -* Modify the file `.github/workflows/build_docker_image.yaml` as instructed, adding your username and repository name in the right place. +* Modify the file `.github/workflows/build_docker_image.yaml` as instructed, adding your username and repository name in the right place. You can do this on github with the editor or using vim/nano in the jupyter environment. * add, commit, push. Then create a release on github. See what happens under actions. When the build process is done, you can check your images on dockerhub. -* if you have docker locally installed on your computer you can download and run your image with the following command: `docker run -p 7000:8888 YOURUSERNAME/example_TEM_analysis:latest`. You can then visit `http://localhost:7000` to view the notebook. +* if you have docker locally installed on your computer you can download and run your image with the following command: `docker run -p 7000:8888 YOURUSERNAME/example_TEM_analysis:latest`. You can then visit `http://localhost:7000` to view the notebook. Since this image packages everything together, it should be forever completely reproducible on any platform. + diff --git a/environment.yml b/environment.yml index add0f222f5934583cc26fd3c187279c9a0d36571..d7f553171da94414df4eda147245c7a174a4f05e 100644 --- a/environment.yml +++ b/environment.yml @@ -5,6 +5,8 @@ dependencies: - python - requests - hyperspy=1.6.4 + # see https://github.com/h5py/h5py/issues/1880 + - h5py=3.1.0 - scikit-learn=0.24.2 - atomap=0.3.1 - beautifulsoup4=4.10.0 diff --git a/repository/binder/environment.yml b/repository/binder/environment.yml index 869f1c506c1f7c88cc73b3050c299611f4003cf1..758634a3d7f9fbf71f3809782186f3dbd4190348 100644 --- a/repository/binder/environment.yml +++ b/repository/binder/environment.yml @@ -3,6 +3,7 @@ channels: - conda-forge dependencies: - hyperspy=1.6.4 + - h5py=3.1.0 - scikit-learn=0.24.2 - atomap=0.3.1 - beautifulsoup4=4.10.0