Detailed description of the creation of the RESTful JSON API in the Go language

RESTful API is widely used in Web project development. This paper explains how to implement RESTful JSON API step by step in Go language, and also touches on the topic of RESTful design.

Maybe we’ve used all kinds of APIs before, and when we encounter poorly designed APIs, we feel like we’re going to crash. Hope that after this article, we can have a preliminary understanding of the well-designed RESTful API.

What is the JSON API?

Before JSON, many websites exchanged data through XML. If you touch JSON after using XML, you will undoubtedly feel how beautiful the world is. This article does not go into the introduction of JSON API, but you can refer to jsonapi for your interest.

Basic Web Server

Fundamentally, RESTful services are first and foremost Web services. So let’s first look at how the basic Web server in the Go language is implemented. The following example implements a simple Web server that responds to the request’s URL for any request.

package main import ( "fmt" "html" "log" "net/http" ) func main() { http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path)) }) log.Fatal(http.ListenAndServe(":8080", nil)) }

The basic web server above uses two basic functions of the Go standard library, HandleFunc and ListenAndServe.

func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) { DefaultServeMux.HandleFunc(pattern, handler) } func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe() }

Running the basic web services above, you can access http://localhost:8080 directly through the browser.

> go run basic_server.go

Add routing

Although the standard library contains router, I find that many people are confused about how it works. I have used various third-party router libraries in my own projects. Most noteworthy is the MUX router of Gorilla Web ToolKit.

Another popular router is a package called httprouter from Julien Schmidt.

package main import ( "fmt" "html" "log" "net/http" "github.com/gorilla/mux" ) func main() { router := mux.NewRouter().StrictSlash(true) router.HandleFunc("/", Index) log.Fatal(http.ListenAndServe(":8080", router)) } func Index(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path)) }

To run the above code, first use go get to get the source code of MUX router:

> go get github.com/gorilla/mux

The above code creates a basic router that assigns the Index processor to the request “/”, and executes the Index processor when the client requests http://localhost:8080/.

If you are careful enough, you will find that previous basic web services accessing http://localhost:8080/abc can respond normally:’Hello,’/abc’, but after adding routing, you can only access http://localhost:8080. The reason is very simple, because we only added an analysis of “/”, the other routes are invalid routes, so they are 404.

Create some basic routing

Now that we have added routing, we can add more routing.

Suppose we want to create a basic ToDo application, and our code becomes as follows:

package main import ( "fmt" "log" "net/http" "github.com/gorilla/mux" ) func main() { router := mux.NewRouter().StrictSlash(true) router.HandleFunc("/", Index) router.HandleFunc("/todos", TodoIndex) router.HandleFunc("/todos/{todoId}", TodoShow) log.Fatal(http.ListenAndServe(":8080", router)) } func Index(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Welcome!") } func TodoIndex(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Todo Index!") } func TodoShow(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) todoId := vars["todoId"] fmt.Fprintln(w, "Todo Show:", todoId) }

Here we add two other routes: todos and todos/{todoId}.

This is the beginning of RESTful API design.

Note that for the last route, we added a variable called todoId to the end of the route.

This allows us to pass IDs to routing and respond to requests using specific records.

Basic model

Now that the routing is in place, it’s time to create a model that can send and retrieve data. In Go language, model can be implemented by structure, while in other languages, model is usually implemented by class.

package main import ( "time" ) type Todo struct { Name string Completed bool Due time.Time } type Todos []Todo

Above, we define a Todo structure to represent items to be done. In addition, we define a type of Todos, which represents a list to be done, an array, or a fragment.

You will see later that this will become very useful.

Return some JSON

We have a basic model, so we can simulate some real responses. We can simulate some static data lists for TodoIndex.

package main import ( "encoding/json" "fmt" "log" "net/http" "github.com/gorilla/mux" ) // ... func TodoIndex(w http.ResponseWriter, r *http.Request) { todos := Todos{ Todo{Name: "Write presentation"}, Todo{Name: "Host meetup"}, } json.NewEncoder(w).Encode(todos) } // ...

Now we create a static Todos fragment to respond to client requests. Note that if you request http://localhost:8080/todos, you will get the following response:

[ { "Name": "Write presentation", "Completed": false, "Due": "0001-01-01T00:00:00Z" }, { "Name": "Host meetup", "Completed": false, "Due": "0001-01-01T00:00:00Z" } ]

Better Model

For experienced veterans, you may have found a problem. Every key responding to JSON is written in the first letter. Although it seems trivial, capitalizing the first letter of the key responding to JSON is not a customary practice. So here’s how to solve this problem:

type Todo struct { Name string `json:"name"` Completed bool `json:"completed"` Due time.Time `json:"due"` }

In fact, it is very simple to add tag attributes to the structure, which can completely control how the structure is marshalled into JSON.

Split code

So far, all our code is in one file. It looks cluttered. It’s time to split up the code. We can split the code into the following files according to its function.

We are going to create the following file, and then move the corresponding code to the specific code file:

Main.go: Program entry file. Handlers.go: Route-related processors. Routes.go: Routing. Todo.go: Todo-related code.

package main import ( "encoding/json" "fmt" "net/http" "github.com/gorilla/mux" ) func Index(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Welcome!") } func TodoIndex(w http.ResponseWriter, r *http.Request) { todos := Todos{ Todo{Name: "Write presentation"}, Todo{Name: "Host meetup"}, } if err := json.NewEncoder(w).Encode(todos); err != nil { panic(err) } } func TodoShow(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) todoId := vars["todoId"] fmt.Fprintln(w, "Todo show:", todoId) }

package main import ( "net/http" "github.com/gorilla/mux" ) type Route struct { Name string Method string Pattern string HandlerFunc http.HandlerFunc } type Routes []Route func NewRouter() *mux.Router { router := mux.NewRouter().StrictSlash(true) for _, route := range routes { router. Methods(route.Method). Path(route.Pattern). Name(route.Name). Handler(route.HandlerFunc) } return router } var routes = Routes{ Route{ "Index", "GET", "/", Index, }, Route{ "TodoIndex", "GET", "/todos", TodoIndex, }, Route{ "TodoShow", "GET", "/todos/{todoId}", TodoShow, }, }

package main import "time" type Todo struct { Name string `json:"name"` Completed bool `json:"completed"` Due time.Time `json:"due"` } type Todos []Todo

package main import ( "log" "net/http" ) func main() { router := NewRouter() log.Fatal(http.ListenAndServe(":8080", router)) }

Better Routing

During our refactoring, we created a routes file with more functions. This new file takes advantage of a structure that contains multiple routing information. Note that here we can specify the types of requests, such as GET, POST, DELETE, and so on.

Output Web Log

In the split routing file, I also have an ulterior motive. As you will see later, it’s easy to decorate the HTTP processor with other functions after splitting.

First, we need the ability to log web requests, like many popular web servers. In the Go language, there are no web log packages or functions in the standard library, so we need to create them ourselves.

package logger import ( "log" "net/http" "time" ) func Logger(inner http.Handler, name string) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { start := time.Now() inner.ServeHTTP(w, r) log.Printf( "%s\t%s\t%s\t%s", r.Method, r.RequestURI, name, time.Since(start), ) }) }

Above we defined a Logger function that can wrap handler.

This is a very standard idiom in the Go language. In fact, it is also a common way of functional programming. Very effective, we just need to pass the handler into the function, and then it will wrap the handler in to add Web logs and time-consuming statistics.

Applying Logger modifier

To apply the Logger modifier, we can create a router. We just need to simply package all our current routes into it. The NewRouter function is modified as follows:

func NewRouter() *mux.Router { router := mux.NewRouter().StrictSlash(true) for _, route := range routes { var handler http.Handler handler = route.HandlerFunc handler = Logger(handler, route.Name) router. Methods(route.Method). Path(route.Pattern). Name(route.Name). Handler(handler) } return router }

Now run our program again, and we can see that the log is roughly as follows:

2014/11/19 12:41:39 GET /todos TodoIndex 148.324us

This routing file is crazy… Let’s refactor it.



The routing routes file is now slightly larger, so let’s break it down into multiple files:

routes.go router.go

package main import "net/http" type Route struct { Name string Method string Pattern string HandlerFunc http.HandlerFunc } type Routes []Route var routes = Routes{ Route{ "Index", "GET", "/", Index, }, Route{ "TodoIndex", "GET", "/todos", TodoIndex, }, Route{ "TodoShow", "GET", "/todos/{todoId}", TodoShow, }, }

package main import ( "net/http" "github.com/gorilla/mux" ) func NewRouter() *mux.Router { router := mux.NewRouter().StrictSlash(true) for _, route := range routes { var handler http.Handler handler = route.HandlerFunc handler = Logger(handler, route.Name) router. Methods(route.Method). Path(route.Pattern). Name(route.Name). Handler(handler) } return router }

And take on some more responsibilities.

So far, we have some fairly good boilerplate, and it’s time to re-examine our processors. We need a little more responsibility. First, modify the TodoIndex and add the following two lines of code:

func TodoIndex(w http.ResponseWriter, r *http.Request) { todos := Todos{ Todo{Name: "Write presentation"}, Todo{Name: "Host meetup"}, } w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusOK) if err := json.NewEncoder(w).Encode(todos); err != nil { panic(err) } }

Two things have happened here. First, we set the response type and told the client to expect to accept JSON. Second, we explicitly set the response status code.

The net/http server of the Go language will try to guess the type of output content for us (but not always accurate), but since we already know the type of response exactly, we should always set it up by ourselves.

Wait a minute. Where is our database?

Obviously, if we want to create a RESTful API, we need some places to store and retrieve data. However, this is not within the scope of this article, so we will simply create a very simple simulation database (non-thread-safe).

We create a repo.go file, which reads as follows:

package main import "fmt" var currentId int var todos Todos // Give us some seed data func init() { RepoCreateTodo(Todo{Name: "Write presentation"}) RepoCreateTodo(Todo{Name: "Host meetup"}) } func RepoFindTodo(id int) Todo { for _, t := range todos { if t.Id == id { return t } } // return empty Todo if not found return Todo{} } func RepoCreateTodo(t Todo) Todo { currentId += 1 t.Id = currentId todos = append(todos, t) return t } func RepoDestroyTodo(id int) error { for i, t := range todos { if t.Id == id { todos = append(todos[:i], todos[i+1:]...) return nil } } return fmt.Errorf("Could not find Todo with id of %d to delete", id) }

Add ID to Todo

We created a mock database, and we used and gave ids, so we also need to update our Todo structure accordingly.

package main import "time" type Todo struct { Id int `json:"id"` Name string `json:"name"` Completed bool `json:"completed"` Due time.Time `json:"due"` } type Todos []Todo

Update our Todo Index



To use the database, we need to retrieve data in TodoIndex. The modification code is as follows:

func TodoIndex(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusOK) if err := json.NewEncoder(w).Encode(todos); err != nil { panic(err) } }

POST JSON

So far, we’ve only exported JSON, and now it’s time to put some JSON in storage.

Add the following routing to the routes.go file:

Route{ "TodoCreate", "POST", "/todos", TodoCreate, },

Create routing

func TodoCreate(w http.ResponseWriter, r *http.Request) { var todo Todo body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576)) if err != nil { panic(err) } if err := r.Body.Close(); err != nil { panic(err) } if err := json.Unmarshal(body, &todo); err != nil { w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(422) // unprocessable entity if err := json.NewEncoder(w).Encode(err); err != nil { panic(err) } } t := RepoCreateTodo(todo) w.Header().Set("Content-Type", "application/json; charset=UTF-8") w.WriteHeader(http.StatusCreated) if err := json.NewEncoder(w).Encode(t); err != nil { panic(err) } }

First we open the body of the request. Note that we use io. LimitReader. This is a good way to protect servers from malicious attacks. Suppose someone wants to send a 500GB JSON to your server?

After we read the body, we deconstruct the Todo structure. If it fails, we respond correctly, using the appropriate response code 422, but we still use the JSON response back. This allows the client to understand what went wrong and have a way to know what went wrong.

Finally, if everything passes, we respond to the 201 status code to indicate that the entity created by the request has been successfully created. We also respond back to the JSON representing the entity we created, which contains an ID that the client may need to use next.

POST Some JSON

Now that we have pseudo repo and create routing, we need to post some data. We use curl to do this by following commands:

Copy code The code is as follows:

curl -H “Content-Type: application/json” -d ‘{“name”: “New Todo”}’ http://localhost:8080/todos

If you access it again through http://localhost:8080/todos, you will probably get the following response:

[ { "id": 1, "name": "Write presentation", "completed": false, "due": "0001-01-01T00:00:00Z" }, { "id": 2, "name": "Host meetup", "completed": false, "due": "0001-01-01T00:00:00Z" }, { "id": 3, "name": "New Todo", "completed": false, "due": "0001-01-01T00:00:00Z" } ]

What we haven’t done yet

Although we have made a good start, there are still many things to be done:

Version Control: What if we need to modify the API and the results change completely? Maybe we need to add / V1 / prefix at the beginning of our routing? Authorization: Unless these are public/free APIs, we may need authorization. It is recommended to learn about JSON web tokens.

ETag – If you are building something that needs to be extended, you may need to implement eTag.

What else?

For all projects, it was small at first, but soon got out of control. But if we want to take it to another level and get it ready for production, there are some additional things to do:

Refactoring. Create several packages for these files, such as JSON assistants, modifiers, processors, and so on. Testing makes it impossible for you to forget that. We haven’t done any tests here. Testing is necessary for production systems.

Source code: https://github.com/corylanou/tns-restful-json-api

summary

For me, the most important thing to remember is that we need to build a responsible API. Sending appropriate status codes, headers, etc. is the key to the wide adoption of APIs. I hope this article will enable you to start your API as soon as possible.

Reference link

Implementation of RESTful JSON API in Go Language

JSON API

Gorilla Web Toolkit

httprouter

JSON Web Tokens

eTag

The above is the whole content of this article. I hope it will be helpful to everyone’s study, and I hope you will support developpaer more.