Kubernetes

How to Create and Manage Kubernetes CronJobs

Introduction

Kubernetes CronJobs allow you to execute commands at specified times, dates, or intervals. A CronJob creates Jobs on a repeated schedule, and these Jobs are Kubernetes objects that represent one or multiple containerized tasks that run to completion. A typical use case may include backup operations, sending emails, or running cleanup scripts. With CronJobs, you can automate multiple tasks directly within your cluster to ensure consistent configurations and improve scalability.

This article explains how to create and manage Kubernetes CronJobs in a Vultr Kubernetes Engine (VKE) cluster. You are to set up multiple CronJobs and perform different tasks to use the available control policies.

Prerequisites

Before you begin:

  • Deploy a Vultr Kubernetes Engine (VKE) cluster.
  • Deploy a Vultr Ubuntu server to use as the management workstation.
  • Access the server using SSH as a non-root sudo user.
  • Install and Configure Kubectl to access the cluster.

Cron Syntax

Kubernetes CronJobs use schedule expressions based on the standard Cron format that represents time intervals at which a Job is executed. Expressions are divided into five fields that are separated by spaces. Each field represents a different part of the schedule and a valid expression must include the following values:

  1. Minute when the task runs (0 to 59)
  2. Hour (0 to 23)
  3. Day of the month when the task runs (1 to 31)
  4. Month (1 to 12)
  5. Day of the week (0 to 6, or SUN, MON, TUE, and so on)# ┌───────────── minute (0 - 59) # │ ┌───────────── hour (0 - 23) # │ │ ┌───────────── day of the month (1 - 31) # │ │ │ ┌───────────── month (1 - 12) # │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday, OR sun, mon, tue, wed, thu, fri, sat) # │ │ │ │ │ # │ │ │ │ │ # │ │ │ │ │ # * * * * *

Below are examples of valid Cron expressions that run a task depending on the set value:

  • * * * * *: Every minute
  • 30 6 * * *: Every day at 6:30 AM
  • */10 * * * *: Every 10 minutes
  • 45 16 1 * *: Runs the task at 4:45 PM on the first day of every month
  • 0 0 * * 0: Runs the task at midnight every Sunday
  • */5 * * * 1-5: Every 5 minutes on weekdays (Monday to Friday)
  • 0 0 1 1 *: Runs the task at midnight on January 1st

Create a Kubernetes CronJob

To implement Kubernetes CronJobs, create a sample CronJob that runs a container creation task that follows a specific schedule as described in the steps below.

  1. Create a new Cronjob YAML file cron-job.yaml using a text editor such as Nano.CONSOLECopy$ nano cron-job.yaml
  2. Add the following contents to the file.YAMLCopyapiVersion: batch/v1 kind: CronJob metadata: name: example-cronjob spec: schedule: “*/2 * * * *” jobTemplate: spec: template: spec: containers: – name: busybox image: busybox:latest command: – /bin/sh – -c – echo ‘Hello World ‘; date restartPolicy: OnFailure Save and close the file.The above configuration creates a new CronJob that runs a busybox container which prints a Hello World prompt to the console. The task runs every 2 minutes as declared by the */2 * * * * Cron expression:
  3. Apply the CronJob to your cluster.CONSOLECopy$ kubectl apply -f cron-job.yaml

Manage Kubernetes CronJobs

  1. View all cluster CronJobs.CONSOLECopy$ kubectl get cronjobs Output:NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE example-cronjob */2 * * * * False 0 26s 49s
  2. View all jobs created from CronJob tasks.CONSOLECopy$ kubectl get jobs Output:NAME COMPLETIONS DURATION AGE example-cronjob-28398856 1/1 4s 81s
  3. View jobs created by a specific CronJob. For example, example-cronjob.CONSOLECopy$ kubectl get jobs -l cron-job-name=example-cronjob To manually create a new job using your existing CronJob, for example, example-job, run the following command:CONSOLECopy$ kubectl create job –from=cronjob/example-cronjob example-job
  4. View the job container logs to verify the ongoing processes. For example, view the example-job logs.CONSOLECopy$ kubectl logs job/example-job Output:Hello World Sat Dec 30 10:18:50 UTC 2023

Pause a Kubernetes CronJob

To pause the execution process of a Kubernetes CronJob, set the spec.suspend field to true. When paused, future CronJobs won’t execute unless started again. However, existing jobs are not affected by the main Cron execution changes. In this section, pause the example-cronjob resource you created earlier as described in the steps below.

  1. Patch your target CronJob using Kubectl and set the spec.suspend value to true.CONSOLECopy$ kubectl patch cronjob/example-cronjob -p ‘{“spec”: {“suspend”: true}}’ Output:cronjob.batch/example-cronjob patched
  2. View available CronJobs and verify the SUSPEND value attached to your target resource.CONSOLECopy$ kubectl get cronjobs Output:NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE example-cronjob */2 * * * * True 0 92s 7m55sAs displayed in the above output, the example-cronjob execution process is paused. When the SUSPEND value is set to False, the CronJob runs normally.
  3. To start the CronJob again, modify the specification value back to false.CONSOLECopy$ kubectl patch cronjob/example-cronjob -p ‘{“spec”: {“suspend”: false}}’

Set Up CronJob Policies

CronJob policies allow you to modify the behavior of CronJobs within your cluster. This allows you to control multiple jobs, set up execution deadlines, and the restart procedure depending on your set policies which include the following:

  • Concurrency Policy: Manages active Jobs simultaneously attached to the same CronJob. It’s controlled using the .spec.concurrencyPolicy field and accepts the following values:
    • Allow: Enables multiple Jobs to run at the same time
    • Forbid: Disables concurrently running jobs. If a specific job fails to complete, the next Job does not start
    • Replace: Ensures that before starting the next scheduled job, any existing running job stops and replaces it with the new job
  • Starting Deadline Mechanism: Specifies the amount of time in seconds that acts as a deadline to start the CronJob.
  • Job History Limits: Sets the CronJob execution history with the following values:
    • spec.successfulJobsHistoryLimit: Sets the number of completed jobs to keep in memory.
    • spec.failedJobsHistoryLimit: Sets the number of failed Jobs to keep in memory.
  • Backoff Limit: Sets the number of times a failed Job should restart. By default, it’s set to 6. If a Job fails after 6 restarts, it’s marked as failed.
  • Active Deadline Seconds: Sets the maximum duration a Job can run. When the time limit expires, the job terminates. For example, if you set the execution deadline to 3600, the Job can run for up to 1 hour (3600 seconds). After 1 hour, Kubernetes stops the job. This is important when a job is stuck or takes long to complete.

To implement CronJob policies, modify your target rules within the resource definition file. For example, create a new jobpolicy.yaml file.

CONSOLECopy$ nano jobpolicy.yaml

Add the following contents to the file.

YAMLCopyapiVersion: batch/v1 kind: CronJob metadata: name: example-cronjob-2 spec: schedule: “*/5 * * * *” concurrencyPolicy: Forbid startingDeadlineSeconds: 300 jobTemplate: spec: backoffLimit: 3 activeDeadlineSeconds: 1800 template: metadata: labels: app: example-app spec: containers: – name: busybox image: busybox:latest command: – /bin/sh – -c – date restartPolicy: OnFailure successfulJobsHistoryLimit: 5 failedJobsHistoryLimit: 3

Save and close the file.

The code above uses the following policies:

  • concurrencyPolicy: Set to Forbid to avoid concurrent executions
  • startingDeadlineSeconds: Sets 300 seconds as the maximum time allowed for a Job to start
  • backoffLimit: Limits the maximum number of times, 3, to retry failed Jobs
  • activeDeadlineSeconds: Limits the maximum runtime for a single Job to 1800 seconds
  • successfulJobsHistoryLimit: Sets the maximum number of completed job instances to preserve to 5
  • failedJobsHistoryLimit: Sets the number of failed Job instances to preserve to 3

Example: Create a MySQL Database Backup CronJob

  1. Encode a new strong MySQL access password using base64.CONSOLECopy$ echo -n ‘strong-password’ | base64 Copy the generated password string to your clipboard.
  2. Create a new Secret resource file mysql-secret.yaml to store the MySQL root user password.CONSOLECopy$ nano mysql-secret.yaml
  3. Add the following contents to the file. Replace hashed-password with your actual base64 encoded password value.YAMLCopyapiVersion: v1 kind: Secret metadata: name: mysql-db-credentials type: Opaque data: MYSQL_ROOT_PASSWORD: hashed-password Save and close the file.
  4. Apply the Secret to your cluster.CONSOLECopy$ kubectl apply -f mysql-secret.yaml
  5. Create a new PVC resource file pvc.yaml to set Vultr Block Storage as the MySQL data storage method.CONSOLECopy$ nano pvc.yaml
  6. Add the following contents to the file.YAMLCopyapiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-backup-pv-claim spec: accessModes: – ReadWriteOnce resources: requests: storage: 40Gi — apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-pv-claim spec: accessModes: – ReadWriteOnce resources: requests: storage: 40Gi Save and close the file.The above configuration creates two Kubernetes PersistentVolumeClaims (PVCs) mysql-backup-pv-claim and mysql-pv-claim using Vultr Block Storage. Both volumes consist of 40 GB with a ReadWriteOnce access mode that allows the volume to be mounted as read-write by a single node.
  7. Apply the resource to your cluster.CONSOLECopy$ kubectl apply -f pvc.yaml
  8. View the cluster PVCs to verify that the new resources are available.CONSOLECopy$ kubectl get pvc Output:NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE mysql-backup-pv-claim Bound pvc-5095c0a035754f1e 40Gi RWO vultr-block-storage 10s mysql-pv-claim Bound pvc-48284eae3adf445b 40Gi RWO vultr-block-storage 10s
  9. Create a new MySQL resource file mysql.yaml.CONSOLECopy$ nano mysql.yaml
  10. Add the following contents to the file.YAMLCopyapiVersion: apps/v1 kind: Deployment metadata: name: mysql-deployment spec: replicas: 1 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: – name: mysql image: mysql:5.7 env: – name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-db-credentials key: MYSQL_ROOT_PASSWORD – name: MYSQL_DATABASE value: “mydatabase” ports: – containerPort: 3306 volumeMounts: – name: mysql-persistent-storage mountPath: /var/lib/mysql volumes: – name: mysql-persistent-storage persistentVolumeClaim: claimName: mysql-pv-claim — apiVersion: v1 kind: Service metadata: name: mysql-service spec: selector: app: mysql ports: – protocol: TCP port: 3306 targetPort: 3306 Save and close the file.The above configuration consists of a MySQL database Deployment and a Service to run in your cluster on the default port 3306. The MYSQL_ROOT_PASSWORD variable uses the base64 password available in your cluster Secret resource. Together, the configuration creates a MySQL deployment with a persistent storage volume and a service that provides a stable endpoint for accessing the database within the cluster.
  11. Apply the MySQL resources to your cluster.CONSOLECopy$ kubectl apply -f mysql.yaml
  12. View the cluster deployments and verify that your MySQL resource is available.CONSOLECopy$ kubectl get deployments Output:NAME READY UP-TO-DATE AVAILABLE AGE mysql-deployment 1/1 1 1 2m
  13. Create a new CronJob resource file mysql-cronjob.yaml.CONSOLECopy$ nano mysql-cronjob.yaml
  14. Add the following contents to the file.YAMLCopyapiVersion: batch/v1 kind: CronJob metadata: name: db-backup-cronjob spec: schedule: “0 2 * * *” concurrencyPolicy: Replace startingDeadlineSeconds: 100 jobTemplate: spec: template: spec: containers: – name: db-backup image: mysql:5.7 env: – name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-db-credentials key: MYSQL_ROOT_PASSWORD – name: MYSQL_HOST value: “mysql-service” args: – “/bin/bash” – “-c” – “mysqldump -h ${MYSQL_HOST} -u root -p${MYSQL_ROOT_PASSWORD} –all-databases | gzip > /backup/all-databases-$(date +%Y-%m-%d_%H%M%S).sql.gz” volumeMounts: – name: backup-volume mountPath: /backup restartPolicy: OnFailure volumes: – name: backup-volume persistentVolumeClaim: claimName: mysql-backup-pv-claim The above configuration creates a new Kubernetes CronJob db-backup-cronjob that runs every day at 2:00 AM as set with the expression 0 2 * * * and the following policies:
    • concurrencyPolicy: Replace: Specifies that when an existing backup job is running, the new job replaces the active job
    • startingDeadlineSeconds: 100: Terminates the job if the starting time exceeds 100 seconds
    • Within the jobTemplate:
      • The container creates a dump of all databases using mysqldump, compresses the output with gzip, and saves it to the /backup directory with a filename that includes the cluster date and time
      • restartPolicy: OnFailure: Specifies that the Job should restart the container on-failure
      • backup-volume is mounted to the /backup directory within the container which uses mysql-backup-pv-claim for storage to persistently back up files.
  15. Apply the new CronJob to your cluster.CONSOLECopy$ kubectl apply -f mysql-cronjob.yaml
  16. View the cluster Cronjobs and verify that the new resource is availableCONSOLECopy$ kubectl get cronjobs Output:NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE db-backup-cronjob 0 2 * * * False 0 <none> 16s
  17. Create a new MySQL data file example-data.sql.CONSOLECopy$ nano example-data.sql
  18. Add the following contents to the file.SQLCopyUSE mydatabase; CREATE TABLE people ( id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255) NOT NULL, age INT NOT NULL ); INSERT INTO people (name, age) VALUES (‘Alice’, 30); INSERT INTO people (name, age) VALUES (‘Bob’, 25); INSERT INTO people (name, age) VALUES (‘Carol’, 27); Save and close the file.The above script uses the database mydatabase to creates a new MySQL table people. Then, the table is populated with sample data values in 3 columns.
  19. View the cluster Pods and keep note of the MySQL database pod name.CONSOLECopy$ kubectl get pods Your output should look like the one below:NAME READY STATUS RESTARTS AGE mysql-deployment-cc487f97b-4k9n2 1/1 Running 0 26mCopy your MySQL Pod name to your clipboard to use when testing the script. For example, mysql-deployment-cc487f97b-4k9n2.
  20. Copy the MySQL data file to the /tmp/ directory within your MySQL Pod. Replace mysql-deployment-cc487f97b-4k9n2 with your actual Pod name.CONSOLECopy$ kubectl cp example-data.sql mysql-deployment-cc487f97b-4k9n2:/tmp/example-data.sql
  21. When successful, enter the MySQL Pod shell.CONSOLECopy$ kubectl exec -it mysql-deployment-cc487f97b-4k9n2 — /bin/bash
  22. Import the MySQL data file from the /tmp directory to populate your target database mydatabase with the sample data.CONSOLECopy# mysql -u root -p$MYSQL_ROOT_PASSWORD mydatabase < /tmp/example-data.sql
  23. When successful, exit the shell.CONSOLECopy# exit
  24. Manually create a new job using the main db-backup-cronjob.CONSOLECopy$ kubectl create job –from=cronjob/db-backup-cronjob manual-job
  25. Wait at least 1 minute to create the new job, then, view the cluster jobs.CONSOLECopy$ kubectl get jobs/manual-job Output:NAME COMPLETIONS DURATION AGE job 1/1 3s 57sVerify that the COMPLETIONS includes a 1/1 value to confirm that the new job is ready.
  26. Create a new MySQL backup query Pod file backup-check-pod.yaml.CONSOLECopy$ nano backup-check-pod.yaml
  27. Add the following contents to the file.YAMLCopyapiVersion: v1 kind: Pod metadata: name: backup-check-pod spec: volumes: – name: backup-volume persistentVolumeClaim: claimName: mysql-backup-pv-claim containers: – name: busybox image: busybox command: [‘sh’, ‘-c’, ‘echo “Backup Check Pod Running”; sleep 3600’] volumeMounts: – name: backup-volume mountPath: /backup Save and close the file.The above configuration creates a new Pod that runs the busybox container which expires after 3600 seconds.
  28. Apply the Pod to your cluster.CONSOLECopy$ kubectl apply -f backup-check-pod.yaml
  29. Wait at least 1 minute for the pod to finish creating, then, access the Pod shell.CONSOLECopy$ kubectl exec -it backup-check-pod — /bin/sh
  30. Switch to the backup directory.CONSOLECopy# cd /backup
  31. List files in the directory and keep note of your MySQL backup archive name.CONSOLECopy# ls -l Output:total 776 -rw-r--r-- 1 root root 791801 Dec 30 19:55 all-databases-2023-12-30_195511.sql.gzCopy the MySQL backup filename. For example all-databases-2023-12-30_195511.sql.gz to extract contents from the file.
  32. Extract all files from the compressed archive to a new file extracted_backup.sql using the Gzip utility.CONSOLECopy# gzip -d -c database-backup-2023-11-25_200928.sql.gz > extracted_backup.sql
  33. View the contents of the extracted file.CONSOLECopy# cat extracted_backup.sql When your MySQL CronJob is successful, your output should look like the one below:-- -- Current Database: `mydatabase` -- CREATE DATABASE /*!32312 IF NOT EXISTS*/ `mydatabase` /*!40100 DEFAULT CHARACTER SET latin1 */; USE `mydatabase`; -- -- Table structure for table `people` -- -- Dumping data for table `people` -- LOCK TABLES `people` WRITE; /*!40000 ALTER TABLE `people` DISABLE KEYS */; INSERT INTO `people` VALUES (1,'Alice',30),(2,'Bob',25),(3,'Carol',27); /*!40000 ALTER TABLE `people` ENABLE KEYS */; UNLOCK TABLES; /*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */; /*!40101 SET SQL_MODE=@OLD_SQL_MODE */; /*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */; /*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */; /*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */; /*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */; /*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */; /*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */; -- Dump completed on 2023-12-30 19:55:12
  34. Exit the shell.CONSOLECopy# exit

Conclusion

You have created and managed Kubernetes CronJobs to automatically schedule tasks within your cluster. You can set up various CronJob policies to further customize your CronJobs behavior and functionalities.

Introduction Kubernetes CronJobs allow you to execute commands at specified times, dates, or intervals. A CronJob creates Jobs on a repeated schedule, and these Jobs are Kubernetes objects that represent one or multiple containerized tasks that run to completion. A typical use case may include backup operations, sending emails, or…

Introduction Kubernetes CronJobs allow you to execute commands at specified times, dates, or intervals. A CronJob creates Jobs on a repeated schedule, and these Jobs are Kubernetes objects that represent one or multiple containerized tasks that run to completion. A typical use case may include backup operations, sending emails, or…

Leave a Reply

Your email address will not be published. Required fields are marked *