Setup Directions
Below are two steps in preparation to launch JupyterHub that only have to be done once:
STEP 1
1. Create your Virtual Environment.
/apps/anaconda3/conda init bash
/apps/anaconda3/conda create -n my_new_environment_name
/apps/anaconda3/conda activate my_new_environment_name
2. Pick a browser that you normally do not use on your computer.
Some browser names are Chrome, Brave, Firefox, Opera, Edge, etc… This browser is going to be set up differently than your normal browsers. It is going to be set up so that it uses a “proxy”. The proxy is going to be created by ssh.
We chose Firefox as the JupyterHub browser in this example. After starting the browser:
a. Click on the 3 horizontal lines on the upper left of the browser (often call the hamburger)
b. Click on Preferences
c. Click on General on the right
d. Scroll to the bottom of the page
e. Click on Settings under “Network Settings”
f. Select Manual proxy configuration
g. Click on Socks Host box and enter 127.0.0.1
h. In the port box put a random number in that is greater than 1000, like 3456
i. Click OK and save it.
What we just did is force the browser to “surf” through localhost port 3456.
STEP 2
Next are the steps to launch a JupyterHub-notebook job to the cluster and use ssh to connect your local browser to the HPC computer node running JupyterHub.
1. ssh to hpc.haastech.org:
ssh -C -D 3456 [email protected]
2. launch JupyterHub:
/apps/bin/jupturehub
(make note of JobID)
3. run bpeek command after about 10 seconds waiting for JupyterHub to start:
bpeek jobid
Towards the bottom of the output, you will see something like this:
http://haas-hpc10:3456/?token=72abceasyas1230b3b52a2220055eb1662f628e12707c5e3
4. The final step is to enter the above URL into your home browser that is being proxied. But you will run into a problem because your home computer does not know what haas-hpc10 is.
From home you have to put the IP address, so haas-hpc10 becomes 10.10.10.20.
Examples:
haas-hpc01 becomes 10.10.10.11
haas-hpc07 becomes 10.10.10.17
haas-hpc08 becomes 10.10.10.18
haas-hpc09 becomes 10.10.10.19
haas-hpc10 becomes 10.10.10.20
So the output of the third step becomes:
http://10.10.10.20:3456/?token=72abceasyas1230b3b52a2220055eb1662f628e12707c5e3
5. Enjoy your custom JupyterHub!
Since you are in a virtual environment, you may add custom languages and custom libraries. You can close your browser at any time and return days later and pick up where you left off.