Django + Celery is probably the most popular solution to develop websites that require running tasks in the background. Developing a Django + Celery app locally is complex, as you need to run different services: Django, Celery worker, Celery beat, Redis, databases… docker-compose is a very convenient tool in this case. You can spin up your local environment with docker-compose in just one single command. And thanks to the use of volume mounts, you are able to hot reload your application in seconds.

In this blog post, we want to go a step forward and explain why you should develop your Django + Celery app directly in Kubernetes. The benefits are:

Reduce integration issues by developing in a more production-like environment, consuming Kubernetes manifests, secrets, volumes or config maps from development.

Overcome local development limitations. You will be able to develop against any Kubernetes cluster, local or remote. Microservices make it harder to run development environments locally.

But it is well-known that developing in Kubernetes is tedious. Let’s explore together how to develop in Kubernetes the Cloud Native way 💥💥💥💥.

Deploy the Django + Celery Sample App

Get a local version of the Django + Celery Sample App by executing the following commands in your local terminal:



$ cd math $ git clone https://github.com/okteto/m ath$ cd math

The Django + Celery Sample App is a multi-service application that calculates math operations in the background. It consists of a web view, a worker, a queue, a cache, and a database.

Execute the command below to deploy the application in your Kubernetes cluster:

You can deploy to your own Kubernetes cluster or give Okteto Cloud a try. Okteto Cloud is a development platform for Kubernetes applications. Sign up today to get a free developer account with 4CPUs and 8GB of RAM.

$ kubectl apply -f manifests statefulset.apps "cache" created

service "cache" created

statefulset.apps "db" created

service "db" created

statefulset.apps "queue" created

service "queue" created

deployment.apps "web" created

service "web" created

deployment.apps "worker" created

Wait for a few seconds for the app to be ready. Check that all pods are ready by executing the command below:

$ kubectl get pod NAME READY STATUS RESTARTS AGE

cache-0 1/1 Running 0 2m

db-0 1/1 Running 0 2m

queue-0 1/1 Running 0 2m

web-7bccc4bc99-2nwtc 1/1 Running 0 2m

worker-654d7b8bd5-42rq2 1/1 Running 0 2m

Efficient Kubernetes Development with Okteto

Now that we have the Django + Celery Sample App running in a Kubernetes cluster, we can use it as our development environment. When working in Kubernetes, you would have to rebuild your docker images, push them to a registry and redeploy your application every time you want to test a code change. This cycle is complex and time-consuming. Don’t kill your productivity this way.

If you are familiar with docker-compose, you know how useful is to mount local folders into your containers to avoid the docker build/push/redeploy cycle. Volume mounts are a game-changer for developing Docker applications and they are the missing piece to speed up your Kubernetes development cycle.

We developed Okteto to solve this problem. Okteto is open source, and the code is available in github. Feel free to check it out, contribute, and star it 🤗!

To install the Okteto CLI in your computer, follow the installation instructions and check that it is properly installed by running:

$ okteto version

okteto version 1.8.18

Okteto CLI >= 1.8.18 is required to follow this tutorial.

In order to start developing the Django + Celery Sample App, execute:

$ okteto up

✓ Development environment activated

✓ Files synchronized

Namespace: pchico83

Name: web

Forward: 8080 -> 8080

Database is ready

No changes detected in app ‘myproject’

Created migrations

Operations to perform:

Apply all migrations: auth, contenttypes, myproject, sites

Running migrations:

No migrations to apply.

Migrated DB to latest version

Performing system checks…

System check identified no issues (0 silenced).

June 26, 2019–11:00:02

Django version 1.11.21, using settings ‘myproject.settings’

Starting development server at

Quit the server with CONTROL-C. curl: (52) Empty reply from serverDatabase is readyNo changes detected in app ‘myproject’Created migrationsOperations to perform:Apply all migrations: auth, contenttypes, myproject, sitesRunning migrations:No migrations to apply.Migrated DB to latest versionPerforming system checks…System check identified no issues (0 silenced).June 26, 2019–11:00:02Django version 1.11.21, using settings ‘myproject.settings’Starting development server at http://0.0.0.0:8080/ Quit the server with CONTROL-C.

Let’s have a look at the okteto.yml file to understand what the okteto up command does:

name: web

command: ./run_web.sh

sync:

- .:/app

services:

- name: worker

command: ./run_celery.sh

sync:

- .:/app

forward:

- 8080:8080

okteto up synchronizes your local source code to the containers of the deployment web and worker. Your source code is synchronized to the container path /app . Also, the port 8080 is automatically forwarded between your development environment and your computer. If you want to know more about how Okteto works, follow this link.

Let’s Write some Code and Fix a Bug

Verify that the application is up and running by opening your browser and navigating to http://localhost:8080/jobs/. Go ahead and calculate the fibonacci for the number 5:

Press the POST button to submit the operation. The response payload will include the url of the job. Go to http://localhost:8080/jobs/1/ and you will notice that the result is wrong (hint: the fibonacci number of 5 is not 32). This is because our worker has a bug 🙀!

Typically, fixing this would involve you running the app locally, fixing the bug, building a new container, pushing it and redeploying your app. Instead, we’re going to do it the Cloud Native way.

Open myproject/myproject/models.py in your favorite local IDE. Take a look at the value of the task variable in line 29. It looks like someone hard-coded the name of the operation instead of reading it from the job. Let's fix it by changing it to self.type , as shown below:

task = TASK_MAPPING[‘power’]

to

task = TASK_MAPPING[self.type]

Save your file and go back to http://localhost:8080/jobs/. Submit a new fibonacci calculation for the same values as before. Go to http://localhost:8080/jobs/2/ and verify the result. The result looks correct this time, success!

How did this happen? With Okteto your changes were automatically applied to the remote containers as soon as you saved them. No commit, build, push or redeploy required 💪!