z2jh-aws

A take on Zero-to-JupyterHub with Kubernetes using a very specific tech stack, namely:

• Terraform for AWS infrastructure config management
• AWS Elastic Kubernetes Service (EKS) via the

howdio EKS Terraform module

• AWS Elastic Container Registry (ECR) for storing custom JupyterHub

and user images

• Helm and Helmfile for deploying

applications to Kubernetes including external-dns, nginx-ingress, jetstack/cert-manager, and jupyterhub

• Cloudflare for DNS
• Let's Encrypt for TLS certificates
• Homebrew for installing client tools on macOS
• Make for simplifying local commands

Before You Begin

Make sure the tech stack above and all the complexity that comes with it are a good fit for your use case before continuing. Here are some alternatives:

• If you only need a Jupyter Notebook server for yourself, you do not need to set up JupyterHub. You

should instead follow the instructions to install and run a standalone Jupyter Notebook server.

• If you need to run Jupyter Notebook servers for a small group of people and have no interest in

learning about Kubernetes, you will be better off following the guide for The Littlest JupyterHub.

• If you intend to use a Kubernetes provider other than AWS, you should follow the instructions in

the full Zero-to-JupyterHub guide.

• If you are familiar with Kubernetes but have never configured JupyterHub on run on k8s before, you

should follow the instructions in the full Zero-to-JupyterHub guide at least once before using the highly-opinionated shortcuts enabled by this repository.

Prerequisites

• macOS
• Homebrew
• A public domain name

Basic Usage

• Install tools used throughout the rest of the instructions using brew.

make client-prereqs

• Create an

AWS IAM access key and secret for the account that will be used to provision AWS infrastructure.

• Configure a profile (e.g., personal) in ~/.aws/credentials containing the IAM credentials and

desired region for your AWS infrastructure.

[personal]
   awsaccesskeyid=YOURIAMKEYID
   awssecretaccesskey=YOURIAMKEYSECRET
   region=us-east-1

• Configure Elastic Kubernetes Service (EKS) and Elastic Container Registry (ECR) instances on AWS

using Terraform. Fair warning: AWS will start charging you after you run this command.

export AWS_PROFILE=personal
   make k8s-cluster

• Sign up for a (free) CloudFlare account if you don't already have one.
• Add a site to your Cloudflare account matching your domain name (e.g., parente.dev).
• Login to your domain name registrar to configure your domain name to use the CloudFlare

nameservers (currently, alec.ns.cloudflare.com and alice.ns.cloudflare.com).

• Back on the CloudFlare website, click the Crypto icon for your new site and change the SSL

dropdown to Full.

• Also on the CloudFlare site, get your CloudFlare API token from My Profile → API Tokens

→ Global API Key. p.

• Now at the console, apply configurations to Kubernetes for Helm tiller, nginx ingress, Let's

Encrypt certificate issuer, and CloudFlare DNS.

export FQDN=parente.dev
   export CLOUDFLAREAPIKEY=YOURCLOUDFLAREAPI_KEY
   export CLOUDFLARE_EMAIL=parente@gmail.com
   make k8s-services

• Deploy JupyterHub configured to use the jupyter/minimal-notebook Docker image from

jupyter/docker-stacks and FirstUseAuthenticator.

make jupyterhub

• Visit https://YOUR_DOMAIN in your browser.
• Login with username admin and a strong password of your choosing.
• Click File → Hub Control Panel → Admin → Add Users to populate a whitelist of

users allowed to login.

Customization / How do I ...

See what's configured in Kubernetes

make show

Use a different JupyterHub Docker image tag

See https://jupyterhub.github.io/helm-chart/ for available versions and https://github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/master/images/hub for the contents of each image. Substitute the version you want into the command below.

make jupyterhub HUBIMAGETAG=0.8.2

Use a different Docker image as my user environment

Choose any of the images maintained at or derived from https://github.com/jupyter/docker-stacks. Produce one of your own using https://github.com/jupyterhub/repo2docker. Then substitute the image name and tag in the command below.

make jupyterhub USERIMAGE=jupyter/datascience-notebook USERIMAGE_TAG=2ce7c06a61a1

Build private, custom JupyterHub and user Docker images

• Modify the Dockerfile.hub and Dockerfile.user files in this repository so that they define

the Docker images you want to use.

• Run the following commands to build the images and push them to the AWS ECR instances you

provisioned during setup. Specify whatever tag / version number you want to assign to the images (default: latest).

make hub-image HUBIMAGETAG=1
   make user-image USERIMAGETAG=1

• Run the following command to deploy JupyterHub with your custom images:

make jupyterhub-custom HUBIMAGETAG=1 USERIMAGETAG=1

Use a different method of authentication

Modify the auth section in helmfile.yaml based on the guidance in the Authentication section of Zero to JupyterHub.

Use a different DNS, Kubernetes, TLS, etc. provider

Refer to the Zero to JupyterHub with Kubernetes documentation and share your own take on the general instructions in your own GitHub project.

Run these commands on a different operating system

Use the approprate package manager for your platform to install all of the tools instead by make client-prereqs. If make is not any option for you, run the commands specified in the Makefile directly.

Run kubectl commands

cd z2jh-aws
export KUBECONFIG=./output/jupyterhub/kubeconfig-jupyterhub
kubectl get pods

Tear everything down

make destroy