How to Make a Webhook Receiver in Django

Using the Slack Outgoing Webhook as an example

Webhooks (aka HTTP callbacks) are a fantastic way to connect all the various services we as developers depend on now. Unfortunately, I haven’t see a great walk-thru of how to make a HTTP callback receiver/consumer. So while you’re going to see a lot of code below, you’re not going to see a lot of explanation for why I made the choices I did.

Scenario for Consideration

Let’s say we want to capture Slack messages for some reason. Here’s some documentation on what that outbound webhook looks like: https://api.slack.com/outgoing-webhooks

Setup and Install Dependencies

Go create a new django project, and for your database, let’s use Postgres. We’re going to want to store JSON, so let’s install django-hstore. Also, lets suppose we’re going to do something relatively computationally intensive with the data, so we won’t want to slow down our webhook response just to process the content, so let’s install celery too. (All of this stuff will take you a bit. I’ll wait for you to get back.)

Our Webhook App

Now that you’re all done with that, lets start a new app and name it `slack_messages`. (Be sure to add it to your INSTALLED_APPS in your settings file.)

$ python manage.py startapp slack_messages

Here’s a sample model to hold a bunch of generic information related to the webhook event:

# slack_messages/models.py from django.db import models

from django.utils import timezone



from django_hstore import hstore





class WebhookTransaction(models.Model):

UNPROCESSED = 1

PROCESSED = 2

ERROR = 3



STATUSES = (

(UNPROCESSED, 'Unprocessed'),

(PROCESSED, 'Processed'),

(ERROR, 'Error'),

)



date_generated = models.DateTimeField()

date_received = models.DateTimeField(default=timezone.now)

body = hstore.SerializedDictionaryField()

request_meta = hstore.SerializedDictionaryField()

status = models.CharField(max_length=250, choices=STATUSES, default=UNPROCESSED)



objects = hstore.HStoreManager()



def __unicode__(self):

return u'{0}'.format(self.date_event_generated) class Message(models.Model):

date_processed = models.DateTimeField(default=timezone.now)

webhook_transaction = models.OneToOneField(WebhookTransaction)



team_id = models.CharField(max_length=250)

team_domain = models.CharField(max_length=250)

channel_id = models.CharField(max_length=250)

channel_name = models.CharField(max_length=250)

user_id = models.CharField(max_length=250)

user_name = models.CharField(max_length=250)

text = models.TextField()

trigger_word = models.CharField(max_length=250)

def __unicode__(self):

return u'{}'.format(self.user_name)

Here’s a good way to process the incoming webhook and keep a lot of metadata on what happened when for auditing later.

# slack_messages/views.py import copy, json, datetime

from django.utils import timezone

from django.http import HttpResponse

from django.views.decorators.csrf import csrf_exempt

from django.views.decorators.http import require_POST



from .models import WebhookTransaction





@csrf_exempt

@require_POST

def webhook(request):

jsondata = request.body

data = json.loads(jsondata)

meta = copy.copy(request.META)

for k, v in meta.items():

if not isinstance(v, basestring):

del meta[k]



WebhookTransaction.objects.create(

date_event_generated=datetime.datetime.fromtimestamp(

data['timestamp']/1000.0,

tz=timezone.get_current_timezone()

),

body=data,

request_meta=meta

)



return HttpResponse(status=200)

Be sure to connect it up to your project urls. (Yes, I’m skipping putting it in the slack_messages app urls, but that’s just for brevity.)

# project_name/urls.py from django.conf.urls import include, url, patterns

from django.contrib import admin



from slack_webhook import views



urlpatterns = [

url(r'^admin/', include(admin.site.urls)),

url(r'^webhook', views.webhook, name='webhook'),

]

Finally, let’s make a task to process this webhook. You’ll note that I separated out the selection of get_transactions_to_process and process_trans into their own methods because those are the two areas that will vary. The main method run is relatively reusable, so you may want to make a mixin out of it.

# slack_messages/tasks.py from celery.task import PeriodicTask

from celery.schedules import crontab



from .models import Message, WebhookTransaction





class ProcessMessages(PeriodicTask):

run_every = crontab() # this will run once a minute def run(self, **kwargs):

unprocessed_trans = self.get_transactions_to_process()



for trans in unprocessed_trans:

try:

self.process_trans(trans)

trans.status = WebhookTransaction.PROCESSED

trans.save()



except Exception:

trans.status = WebhookTransaction.ERROR

trans.save()



def get_transactions_to_process(self):

return WebhookTransaction.objects.filter(

event_name__in=self.event_names,

status=WebhookTransaction.UNPROCESSED

) def process_trans(self, trans):

return Message.objects.create(

team_id=trans.body['team_id'],

team_domain=trans.body['team_domain'],

channel_id=trans.body['channel_id'],

user_id=trans.body['user_id'],

user_name=trans.body['user_name'],

text=trans.body['text'],

user_id=trans.body['user_id'],

trigger_word=trans.body['trigger_word'],

webhook_transaction=trans

)

Conclusion

So there you have it. Some moderately generalized boilerplate for a webhook receiver, including async processing and audit history. Hope it helps someone!