In this tutorial we’ll write a vibe.d web app that links bugzilla issues to github pull requests.

Let’s start by creating a vibe.d project,

dub init dlang-bot --type = vibe.d git init dlang-bot

which generates a few files and folders.

dub.sdl # the dub package file source/app.d # a hello world vibe.d app public # folder for assets views # folder for frontend templates

Let’s edit app.d and use vibe.d’s URLRouter to add an endpoint for github webhooks, so that the app can receive notifications about new or updated pull requests.

auto router = new URLRouter ; router . get ( "/" , ( req , res ) => res . redirect ( "https://github.com/dlang-bot?tab=activity" )) . post ( "/github_hook" , & githubHook ) ; listenHTTP ( settings , router );

void githubHook ( HTTPServerRequest req , HTTPServerResponse res ) { if ( req . headers [ "X-Github-Event" ] == "ping" ) return res . writeBody ( "pong" ); assert ( req . headers [ "X-GitHub-Event" ] == "pull_request" ); // vibe.d parses application/json post bodies by default auto action = req . json [ "action" ]. get ! string ; logDebug ( "#%s %s" , req . json [ "number" ], action ); switch ( action ) { case "opened" , "closed" , "synchronize" : auto commitsURL = req . json [ "pull_request" ][ "commits_url" ]. get ! string ; auto commentsURL = req . json [ "pull_request" ][ "comments_url" ]. get ! string ; runTask ( toDelegate (& handlePR ), action , commitsURL , commentsURL ); return res . writeBody ( "handled" ); default : return res . writeBody ( "ignored" ); } }

The hook can handle ping and pull_request events and will process interesting pull request actions asynchronously in a separate task.

void handlePR ( string action , string commitsURL , string commentsURL ) { auto comment = getBotComment ( commentsURL ); auto refs = getIssueRefs ( commitsURL ); logDebug ( "%s" , refs ); if ( refs . empty ) { if ( comment . url . length ) // delete any existing comment deleteBotComment ( comment . url ); return ; } auto descs = getDescriptions ( refs ); logDebug ( "%s" , descs ); assert ( refs . map !( r => r . id ). equal ( descs . map !( d => d . id ))); auto msg = formatComment ( refs , descs ); logDebug ( "%s" , msg ); if ( msg != comment . body_ ) updateBotComment ( commentsURL , comment . url , msg ); }

The processing is fairly straightforward. First we get a list of issues references from any of the PR’s commit messages, then ask Bugzilla for a short description of those issues, format the information, and finally create, update, or delete a comment on the PR.

struct IssueRef { int id ; bool fixed ; } // get all issues mentioned in a commit IssueRef [] getIssueRefs ( string commitsURL ) { // see https://github.com/github/github-services/blob/2e886f407696261bd5adfc99b16d36d5e7b50241/lib/services/bugzilla.rb#L155 enum issueRE = ctRegex !( `((close|fix|address)e?(s|d)? )?(ticket|bug|tracker item|issue)s?:? *([\d ,\+&#and]+)` , "i" ); static auto matchToRefs ( M )( M m ) { auto closed = ! m . captures [ 1 ]. empty ; return m . captures [ 5 ]. splitter ( ctRegex ! `[^\d]+` ) . map !( id => IssueRef ( id . to ! int , closed )); } return requestHTTP ( commitsURL , ( scope req ) { req . headers [ "Authorization" ] = githubAuth ; }) . readJson [] . map !( c => c [ "commit" ][ "message" ]. get ! string . matchAll ( issueRE ). map ! matchToRefs . joiner ) . joiner . array . sort !(( a , b ) => a . id < b . id ) . release ; }

This code heavily uses std.algorithm and std.range for pipeline style operations. It’s pretty terse but much more robust (and simple) than writing explicit nested loops. The pipeline fetches all commits, matches all referenced issues in commit messages, converts the matches, joins all references, joins all references of all commits, and sorts them by issue id.

struct Issue { int id ; string desc ; } // get pairs of (issue number, short descriptions) from bugzilla Issue [] getDescriptions ( R )( R issueRefs ) { import std . csv ; return "https://issues.dlang.org/buglist.cgi?bug_id=%(%d,%)&ctype=csv&columnlist=short_desc" . format ( issueRefs . map !( r => r . id )) . requestHTTP . bodyReader . readAllUTF8 . csvReader ! Issue ( null ) . array . sort !(( a , b ) => a . id < b . id ) . release ; }

The code to query Bugzilla is fairly similar but uses csv instead of json.

struct Comment { string url , body_ ; } Comment getBotComment ( string commentsURL ) { auto res = requestHTTP ( commentsURL , ( scope req ) { req . headers [ "Authorization" ] = githubAuth ; }) . readJson [] . find !( c => c [ "user" ][ "login" ] == "dlang-bot" ); if ( res . length ) return deserializeJson ! Comment ( res [ 0 ]); return Comment (); }

I’ll spare the formatting code, you can find it in the full source code.

Getting an existing comment simply searches for the first comment posted by the dedicated dlang-bot user and uses deserializeJson to convert json to a struct.

void sendRequest ( T ...)( HTTPMethod method , string url , T arg ) if ( T . length <= 1 ) { requestHTTP ( url , ( scope req ) { req . headers [ "Authorization" ] = githubAuth ; req . method = method ; static if ( T . length ) req . writeJsonBody ( arg ); }, ( scope res ) { if ( res . statusCode / 100 == 2 ) logInfo ( "%s %s, %s

" , method , url , res . bodyReader . empty ? res . statusPhrase : res . readJson [ "html_url" ]. get ! string ); else logWarn ( "%s %s failed; %s %s.

%s" , method , url , res . statusPhrase , res . statusCode , res . bodyReader . readAllUTF8 ); }); } void deleteBotComment ( string commentURL ) { sendRequest ( HTTPMethod . DELETE , commentURL ); } void updateBotComment ( string commentsURL , string commentURL , string msg ) { if ( commentURL . length ) sendRequest ( HTTPMethod . PATCH , commentURL , [ "body" : msg ]); else sendRequest ( HTTPMethod . POST , commentsURL , [ "body" : msg ]); }

And eventually we need some code to create, update, and delete comments. Notice that requestHTTP is synchronous and uses callbacks only to prevent you from escaping the request/response object.

Now that we’ve implemented our app and tested it locally (using ngrok for example) we can deploy it on Heroku. Though first we have to bind to an external network interface and make the listening port configurable.

auto settings = new HTTPServerSettings ; settings . port = 8080 ; settings . bindAddresses = [ "0.0.0.0" ]; readOption ( "port|p" , & settings . port , "Sets the port used for serving." );

Unfortunately openssl on heroku is configured to use custom config and certificate paths, so we have to explicitly specify Ubuntu’s default CA bundle.

// workaround for openssl.conf on Heroku HTTPClient . setTLSSetupCallback (( ctx ) { ctx . useTrustedCertificateFile ( "/etc/ssl/certs/ca-certificates.crt" ); });

After that we use the Heroku CLI to create a new app, push+deploy our code, set our github token, and start a single dyno to serve the app.

heroku create dlang-bot --buildpack http://github.com/MartinNowak/heroku-buildpack-d.git git push heroku master heroku config:set GH_TOKEN = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx heroku ps:scale web = 1

https://dlang-bot.herokuapp.com/

https://github.com/MartinNowak/dlang-bot