A guide to deploying Jupyter notebooks on Kubernetes

Question

This is the requirement:

1. Mysql, Jupyter Notebook (both should be on Kubernetes cluster)

I need to run a machine learning model using Jupyter Notebook to get data from a Mysql database, and the entire task needs to be scheduled in a Kubernetes cluster environment (just like cron scheduling).

I am new to Kubernetes but understand Docker containerization and have built containerized applications before. I implore you, please help me.

Answer 1

You can schedule resources on Kubernetes using the cornjob function.

Learn more about cronjobs: https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/

If you plan to run a Docker image of a notebook or code, you can simply create a YAML configuration and use a cronjob to run that Docker image as a scheduled task on Kubernetes.

I haven't used Jupyter notebook, so not sure how it works.

To run MySQL on Kubernetes, you can configure the YAML file in the same way and apply the changes in the K8s cluster, and then your container will be deployed to K8s.

Read more here: https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/