As you might know seriesci is a GitHub application. GitHub uses the standard OAuth 2.0 flow to obtain an authorization code and then exchange it for a token. Our users depend on GitHub to be able to login and also to be able to make requests to their API. This is all great in production but makes development on our own machines harder. Especially problematic is the OAuth callback as GitHub must have access to your server. During development our servers run on localhost and GitHub cannot access them. At the beginning we used ngrok and serveo to open tunnels to our machines. We were assigned random URLs and GitHub could access those. The callback request and also webhooks were then forwarded to our servers running on localhost . However this whole setup is a bit messy, can be expensive and also unreliable. We wanted something easier to we looked at the whole flow very carefully.

At first we will explain how the standard OAuth 2.0 sequence works with GitHub. Afterwards we will describe our own solution to get rid of this dependency and also local tunnels. So let us have a look at the standard implementation.

User visits /login In our case the client renders the login template User clicks on Login with GitHub and makes a GET request to /authorize On our server we do a few things create some random state

store the state in a cookie

generate the auth URL for GitHub

redirect to the generated URL The redirect triggers a GET request to github.com/login/oauth/authorize?... GitHub generates some random code GitHub then redirects and triggers a GET request to /callback Back on our server we do a few more things compare state from GitHub with our own state from cookie

extract code from query parameters

call the exchange method Our server does a POST request to github.com/login/oauth/access_token GitHub validates the code and responds with a real token We put this token into a cookie so only you have access We direct the user to / Our client side web application starts The web app makes a request to our own API route /api/repos On the server we do a few things extract the token from the cookie

create an authenticated client for GitHub Use this client to make requests to the GitHub API, e.g. list all repositories GitHub returns a list of repositories to our server Our server enhances and returns the list to the user Our client web app rerenders and shows you your repositories

Phew! That is a lot to grasp. In pure text it is a bit hard to understand so here is an image.

Source code for sequence diagram User ->> User: GET /login activate User note left of User - render login template end note User ->> -App: GET /authorize activate App note left of App - create new state - store state in cookie - call AuthCodeURL() - redirect to generated URL end note App ->> -GitHub: GET github.com/login/oauth/authorize

?client_id=...

&response_type=code

&scope=...

&state=... activate GitHub note right of GitHub - check client id - generate code end note GitHub ->> -App: GET /callback

?code=...

&state=... activate App note left of App - compare state with cookie - use code to request token - code will expire after 10 mins - call Exchange() end note App ->> -GitHub: POST github.com/login/oauth/access_token activate GitHub note right of GitHub: - generate token GitHub ->> -App: response with:

access_token=...&token_type=bearer activate App note left of App: - store token in cookie App ->> -User: redirect to / activate User note left of User - load index.html - download JavaScript - start web app end note User ->> -App: GET /api/repos activate App note left of App - get token from cookie - create authenticated client - use client for GitHub API end note App ->> -GitHub: client.Repositories.List() activate GitHub note right of GitHub - check token end note GitHub ->> -App: List of Repositories App ->> User: List of Repositories

Our goal is to get rid of the right lane and eliminate the GitHub dependency during development. Generally speaking we have to mock OAuth and also mock the API requests. We are a heavy Go user so we will show you how to do it in Go but it should be the same for other languages. The important packages are github.com/golang/oauth2 and github.com/google/go-github.

oauth2 is responsible for generating a URL to OAuth 2.0 provider's consent page that asks for permissions for the required scopes. It also exchanges the code for a real token. Using this package is pretty straightforward. You just have to create a new instance of oauth2.Config .

import ( "golang.org/x/oauth2" oauth2GitHub "golang.org/x/oauth2/github" ) oauthConfig := & oauth2 . Config { ClientID : "your own client id here" , ClientSecret : "your own client secret here" , Endpoint : oauth2GitHub . Endpoint , RedirectURL : "" , Scopes : [ ] string { "user:email" } , }

Our whole backend application roughly follows Mat Ryer's style How I write Go HTTP services after seven years. We attach the *oauth2.Config instance to our main application struct. That way we can access it within our handlers.

Mock the /login/oauth/authorize request

Here is our source code for the first request to GitHub. As you can see we are doing exactly what is described in the sequence diagram.

Create a new random state Store the state in a cookie Call AuthCodeURL() method to generate the URL Redirect user to URL

func ( app * Application ) Authorize ( w http . ResponseWriter , r * http . Request ) error { state := uuid . NewV4 ( ) . String ( ) session , err := app . SessionStore . Get ( r , sessionName ) if err != nil { return fmt . Errorf ( "session get: %w" , err ) } session . Values [ "state" ] = state if err := session . Save ( r , w ) ; err != nil { return fmt . Errorf ( "session save: %w" , err ) } url := app . OAuthConfig . AuthCodeURL ( state ) http . Redirect ( w , r , url , http . StatusTemporaryRedirect ) return nil }

Since we do not want to change this behavior during development on localhost we have to mock the AuthCodeURL() method. We do not redirect the user to GitHub but back to our own server.

type OAuth2Mock struct { } func ( o * OAuth2Mock ) AuthCodeURL ( state string , opts ... oauth2 . AuthCodeOption ) string { u := url . URL { Scheme : "http" , Host : "localhost" , Path : "login/oauth/authorize" , } v := url . Values { } v . Set ( "state" , state ) u . RawQuery = v . Encode ( ) return u . String ( ) }

Then we use a dummy handler to return some code and redirect to our /callback handler. It acts as a substitution for github.com/login/oauth/authorize . Make sure this handler is only available during development.

func ( app * Application ) DevOAuthAuthorize ( w http . ResponseWriter , r * http . Request ) error { state := r . FormValue ( "state" ) u , err := url . Parse ( "http://localhost/callback" ) if err != nil { return err } v := url . Values { } v . Set ( "code" , "code" ) v . Set ( "state" , state ) u . RawQuery = v . Encode ( ) http . Redirect ( w , r , u . String ( ) , http . StatusTemporaryRedirect ) return nil }

Well done, we have already mocked the first round trip to GitHub.

Mock the /access_token request

Back in our application inside the /callback handler everything still works.

func ( app * Application ) Callback ( w http . ResponseWriter , r * http . Request ) error { session , err := app . SessionStore . Get ( r , sessionName ) if err != nil { return fmt . Errorf ( "session get: %w" , err ) } if r . FormValue ( "state" ) != session . Values [ "state" ] { return fmt . Errorf ( "invalid state: %s" , r . FormValue ( "state" ) ) } token , err := app . OAuthConfig . Exchange ( context . Background ( ) , r . FormValue ( "code" ) ) if err != nil { return fmt . Errorf ( "oauth exchange: %w" , err ) } if ! token . Valid ( ) { return errors . New ( "invalid token" ) } session . Values [ "token" ] = token if err := session . Save ( r , w ) ; err != nil { return fmt . Errorf ( "session save: %w" , err ) } http . Redirect ( w , r , "/" , http . StatusSeeOther ) return nil }

Compare the returned state with the state from the cookie. Then call Exchange() to exchange the code for a real token. The method simply does a POST request to github.com/login/oauth/access_token . github.com/golang/oauth2/blob/.../internal/token.go#L169-L172. Our mock returns a dummy token. We must set AccessToken and Expiry as token.Valid() checks these two fields.

func ( o * OAuth2Mock ) Exchange ( ctx context . Context , code string , opts ... oauth2 . AuthCodeOption ) ( * oauth2 . Token , error ) { return & oauth2 . Token { AccessToken : "AccessToken" , Expiry : time . Now ( ) . Add ( 1 * time . Hour ) , } , nil }

Now that we have a valid token we continue in our sequence diagram. We store the token in a cookie and redirect the user to our index page. We managed to mock the second request to GitHub. The OAuth 2.0 part is done here. All OAuth related requests are handled by us and none of them reaches the GitHub servers. The final step is all about mocking real data request as an authenticated user.

Mock requests to the OAuth 2.0 provider

We have our dummy token and would like to do an authenticated request to GitHub. Although our token is valid, GitHub complains that it is not a real token and we do not have access to their data. So we have to pretend that we are doing an actual request to GitHub and mock this call. The github.NewClient method returns a struct of type *github.Client . We cannot mock this struct so we have to create our own interface that has the same methods. This is common in Go and follows the rule "Accept interfaces, return structs".

#golang top tip: the consumer should define the interface. If you’re defining an interface and an implementation in the same package, you may be doing it wrong. — Dave Cheney (@davecheney) December 18, 2017

Here is a starting point from GitHub issues https://github.com/google/go-github/issues/113. First of all we have to create our GitHub interface.

type RepositoriesService interface { Get ( context . Context , string , string ) ( * github . Repository , * github . Response , error ) } type UsersService interface { Get ( context . Context , string ) ( * github . User , * github . Response , error ) } type GitHubClient struct { Repositories RepositoriesService Users UsersService } type GitHubInterface interface { NewClient ( httpClient * http . Client ) GitHubClient }

Now we need a real type that implements the above mentioned interface GitHubInterface . For the production environment where we do want to communicate with GitHub we pick the interfaces from the client and attach them to our own instance.

type GitHubCreator struct { } func ( g * GitHubCreator ) NewClient ( httpClient * http . Client ) GitHubClient { client := github . NewClient ( httpClient ) return GitHubClient { Repositories : client . Repositories , Users : client . Users , } }

During development we simply return dummy values without making any network requests.

type RepositoriesMock struct { RepositoriesService } func ( r * RepositoriesMock ) Get ( context . Context , string , string ) ( * github . Repository , * github . Response , error ) { return & github . Repository { ID : github . Int64 ( 185409993 ) , Name : github . String ( "wayne" ) , Description : github . String ( "some description" ) , Language : github . String ( "JavaScript" ) , StargazersCount : github . Int ( 3141 ) , HTMLURL : github . String ( "https://www.foo.com" ) , FullName : github . String ( "john/wayne" ) , } , nil , nil } type UsersMock struct { UsersService } func ( u * UsersMock ) Get ( context . Context , string ) ( * github . User , * github . Response , error ) { return & github . User { Login : github . String ( "john" ) , } , nil , nil } type GitHubMock struct { } func ( g * GitHubMock ) NewClient ( httpClient * http . Client ) GitHubClient { return GitHubClient { Repositories : & RepositoriesMock { } , Users : & UsersMock { } , } }

Create an authenticated GitHub client

So where does the *http.Client for NewClient() come from? The Client(ctx context.Context, t *oauth2.Token) *http.Client method from the oauth2 package takes a context and a token and returns an HTTP client. "The go-github library does not directly handle authentication. Instead, when creating a new client, pass an http.Client that can handle authentication for you" https://godoc.org/github.com/google/go-github/github#hdr-Authentication. Use the token from our previous Exchange() method as input.

httpClient := app . OAuthConfig . Client ( context . Background ( ) , token ) client := app . GitHub . NewClient ( httpClient ) repo , res , err := client . Repositories . GetByID ( context . Background ( ) , 1 ) if err != nil { return nil , err }

Voilà! We're done with the second part and are able to make and mock authenticated requests to the GitHub API.

OAuth interface summary

So our final OAuth 2.0 interface looks like this.

type OAuth2ConfigInterface interface { AuthCodeURL ( state string , opts ... oauth2 . AuthCodeOption ) string Exchange ( ctx context . Context , code string , opts ... oauth2 . AuthCodeOption ) ( * oauth2 . Token , error ) Client ( ctx context . Context , t * oauth2 . Token ) * http . Client }

Here is the complete mock again that implements this interface.

type OAuth2Mock struct { } func ( o * OAuth2Mock ) AuthCodeURL ( state string , opts ... oauth2 . AuthCodeOption ) string { u := url . URL { Scheme : "http" , Host : "localhost" , Path : "login/oauth/authorize" , } v := url . Values { } v . Set ( "state" , state ) u . RawQuery = v . Encode ( ) return u . String ( ) } func ( o * OAuth2Mock ) Exchange ( ctx context . Context , code string , opts ... oauth2 . AuthCodeOption ) ( * oauth2 . Token , error ) { return & oauth2 . Token { AccessToken : "AccessToken" , Expiry : time . Now ( ) . Add ( 1 * time . Hour ) , } , nil } func ( o * OAuth2Mock ) Client ( ctx context . Context , t * oauth2 . Token ) * http . Client { return & http . Client { } }

Here is our simplified application code. We define the main Application struct and attach our http handlers to it.

type Application struct { OAuthConfig OAuth2ConfigInterface GitHub GitHubInterface } r . Methods ( "GET" ) . Path ( "/callback" ) . Handler ( app . Callback ) r . Methods ( "GET" ) . Path ( "/authorize" ) . Handler ( app . Login ) if config . Development { r . Methods ( "GET" ) . Path ( "/login/oauth/authorize" ) . Handler ( app . DevOAuthAuthorize ) }

Sequence diagram with mocks in place

Here is the same sequence diagram again. This time we only have two lanes as GitHub is completely mocked away. We achieved our goal, are able to login via OAuth and make API requests without changing the business logic of our code. This provides us with the flexibility we need.

Source code for mocked sequence diagram User ->> User: GET /login activate User note left of User - render login template end note User ->> -App: GET /authorize activate App note left of App - create new state - store state in cookie - call AuthCodeURL() - redirect to generated URL end note # App ->> -GitHub: GET github.com/login/oauth/authorize

?client_id=...

&response_type=code

&scope=...

&state=... # deactivate App App ->> App: GET localhost:8080/.../... # deactivate App note right of App - generate code end note # deactivate App App ->> App: GET /callback

?code=...

&state=... # activate App note left of App - compare state with cookie - use code to request token - code will expire after 10 mins - call Exchange() end note App ->> App: POST localhost:8080/login/oauth/access_token # activate GitHub note right of App: - generate token App ->> App: response with:

access_token=...&token_type=bearer # activate App note left of App: - store token in cookie App ->> -User: redirect to / activate User note left of User - load index.html - download JavaScript - start web app end note User ->> -App: GET /api/repos activate App note left of App - get token from cookie - create authenticated client - use client for GitHub API end note App ->> App: client.Repositories.List() # activate GitHub note right of App - check token end note App ->> App: List of Repositories App ->> User: List of Repositories

Conclusion

Using all those interfaces and mocks might seem a bit complicated. However having the mock in place during development is great. We are able to write code offline on a train or even on a plane. We do not have to set up complicated tools to open a tunnel from our development machine to the outside world. You usually have to do this in order to receive the real OAuth 2.0 callback. The whole architecture also makes testing very easy. Within our tests we can use the existing interfaces and provide dummy implementations. This way we can even test all error paths that are often untested as the OAuth provider usually works. In addition the setup has a superb user experience. Without any network requests response times are super fast. Everything is available as soon as you click on any link.

We hope you enjoyed this deep dive into Go interfaces and dependency injection. We've been using this setup for about half year now and are happy we took the time to add all those abstractions.