dcsimg
September 25, 2016
Hot Topics:

Parsing and Creating JSON with Go

This article was excerpted from Go Web Programming, published by Manning Publications.

JSON (JavaScript Object Notation) is a lightweight, text-based data format based on JavaScript. The main idea behind JSON is that it is both easily read by humans and machines. JSON was originally defined by Douglas Crockford, but is currently described by RFC 7159 as well as ECMA-404. JSON is popularly used in REST-based web services although REST-based web services don't necessarily need to accept or return JSON data.

Go's support for JSON is from the encoding/json library. We'll look into parsing JSON first then we'll see how to create JSON data.

Parsing JSON

The steps to parse JSON data are very similar to parsing XML. We parse the JSON into structs, which we can subsequently extract the data from. This is normally how we parse JSON:

  1. Create structs to contain the JSON data
  2. Use json.Unmarshal to unmarshal the JSON data into the structs

The rules for mapping the structs to the JSON using struct tags are easier than with XML. There is really only one common rule for mapping. If we want to store the JSON value, given the JSON key, we create a field in the struct (with any name) and map it with the struct tag 'json:"<name>"' where <name> is the name of the JSON key. Let's see in action.

This is the JSON file post.json that we will be parsing. The data in this file should be familiar to you—this is same data we used for the XML parsing.

Listing 1: JSON file for parsing

{
   "id" : 1,
   "content" : "Hello World!",
   "author" : {
      "id" : 2,
      "name" : "Sau Sheong"
   },
   "comments" : [
      {
         "id" : 3,
         "content" : "Have a great day!",
         "author" : "Adam"
      },
      {
         "id" : 4,
         "content" : "How are you today?",
         "author" : "Betty"
      }
   ]
}

This is the code that will parse the JSON into the respective structs, in a json.go file. Notice that the structs themselves are not different.

Listing 2: JSON file for parsing

package main

import (
   "encoding/json"
   "fmt"
   "io/ioutil"
   "os"
)

type Post struct {
   Id       int       'json:"id"'
   Content  string    'json:"content"'
   Author   Author    'json:"author"'
   Comments []Comment 'json:"comments"'
}

type Author struct {
   Id   int     'json:"id"'
   Name string  'json:"name"'
}

type Comment struct {
   Id      int    'json:"id"'
   Content string 'json:"content"'
   Author  string 'json:"author"'
}

func main() {
   jsonFile, err := os.Open("post.json")
   if err != nil {
      fmt.Println("Error opening JSON file:", err)
      return
   }
   defer jsonFile.Close()
   jsonData, err := ioutil.ReadAll(jsonFile)
   if err != nil {
      fmt.Println("Error reading JSON data:", err)
      return
   	}

   fmt.Println(string(jsonData))
   var post Post
   json.Unmarshal(jsonData, &post)
   fmt.Println(post)
}

We want to map the value of the key id to the Post struct's Id field, so we append the struct tag 'json:"id"' to after the field.

type Post struct {
   Id       int       'json:"id"'
   Content  string    'json:"content"'
   Author   Author    'json:"author"'
   Comments []Comment 'json:"comments"'
}

This is pretty much what you need to do to map the structs to the JSON data. Notice that we nest the structs (a post has none or more comments) through slices.

Unmarshalling is done with a single line of code, simply a function call.

json.Unmarshal(jsonData, &post)

Let's run our JSON parsing code and see the results. Run this at the console:

go run json.go

You should see the following results.

{1 Hello World!
   {2 Sau Sheong}
   [{3 Have a great day! Adam}
   {4 How are you today? Betty}]}

We looked at unmarshalling using the Unmarshal function. We can also use the Decoder to manually decode XML into the structs, for streaming JSON data. We'll look at that now.

Listing 3: Parsing JSON using Decoder

jsonFile, err := os.Open("post.json")
if err != nil {
   fmt.Println("Error opening JSON file:", err)
   return
}
defer jsonFile.Close()

decoder := json.NewDecoder(jsonFile)
for {
   var post Post
   err := decoder.Decode(&post)
   if err == io.EOF {
       break
   }
   if err != nil {
      fmt.Println("Error decoding JSON:", err)
      return
   }
   fmt.Println(post)
}

We use NewDecoder, passing in an io.Reader containing the JSON data, to create a new decoder. When a reference to the post struct is passed into the Decode method, the struct will be populated with the data and will be ready for use. Once the data runs out, the Decode method returns an EOF, which we can check and exit the loop.

Let's run our JSON decoder and see the results. Run this at the console:

go run json.go

You should see the following results.

{1 Hello World!
   {2 Sau Sheong}
   [{1 Have a great day! Adam}
   {2 How are you today? Betty}]}

So when do we use Decoder vs Unmarshal? That depends on the input. If your data is coming from an io.Reader stream, like the Body of a http.Request, use Decoder. If you have the data in a string or somewhere in memory, use Unmarshal.

Creating JSON

We just went through parsing JSON, which as you can see, is very similar to parsing XML. Creating JSON is also is very similar to creating XML. Here's the code for marshaling the Go structs to JSON.

Listing 4: Marshaling structs to JSON

package main

import (
   "encoding/json"
   "fmt"
   "io/ioutil"
)

type Post struct {
   Id       int       'json:"id"'
   Content  string    'json:"content"'
   Author   Author    'json:"author"'
   Comments []Comment 'json:"comments"'
}

type Author struct {
   Id   int    'json:"id"'
   Name string 'json:"name"'
}

type Comment struct {
   Id      int    'json:"id"'
   Content string 'json:"content"'
   Author  string 'json:"author"'
}

func main() {
   post := Post{
      Id:      1,
      Content: "Hello World!",
      Author: Author{
         Id:   2,
         Name: "Sau Sheong",
      },
      Comments: []Comment{
         Comment{
            Id:      3,
            Content: "Have a great day!",
            Author:  "Adam",
         },
         Comment{
            Id:      4,
            Content: "How are you today?",
            Author:  "Betty",
         },
      },
   }

   output, err := json.MarshalIndent(&post, "", "\t\t")
   if err != nil {
      fmt.Println("Error marshalling to JSON:", err)
      return
   }
   err = ioutil.WriteFile("post.json", output, 0644)
   if err != nil {
      fmt.Println("Error writing JSON to file:", err)
      return
   }
}

Notice that as before, the structs are the same as when we are parsing JSON. First, we create the struct.

post := Post{
   Id:      1,
   Content: "Hello World!",
   Author: Author{
      Id:   2,
      Name: "Sau Sheong",
   },
   Comments: []Comment{
      Comment{
         Id:      3,
         Content: "Have a great day!",
         Author:  "Adam",
      },
      Comment{
         Id:      4,
         Content: "How are you today?",
         Author:  "Betty",
      },
   },
}

Then we call the MarshalIndent function (which works the same way as the one in the xml library) to create the JSON data in a slice of bytes.

output, err := json.MarshalIndent(&post, "", "\t\t")

We can then save the output to file if we want to.

Finally as in creating XML, we can also create JSON manually from the Go structs using an Encoder. The code below, also in json.go will generate JSON from the Go structs.

Listing 5: Creating JSON from structs using Encoder

package main

import (
   "encoding/json"
   "fmt"
   "io"
   "os"
)

type Post struct {
   Id       int       'json:"id"'
   Content  string    'json:"content"'
   Author   Author    'json:"author"'
   Comments []Comment 'json:"comments"'
}

type Author struct {
   Id   int    'json:"id"'
   Name string 'json:"name"'
}

type Comment struct {
   Id      int    'json:"id"'
   Content string 'json:"content"'
   Author  string 'json:"author"'
}

func main() {

   post := Post{
      Id:      1,
      Content: "Hello World!",
      Author: Author{
         Id:   2,
         Name: "Sau Sheong",
      },
      Comments: []Comment{
         Comment{
            Id:      3,
            Content: "Have a great day!",
            Author:  "Adam",
         },

         Comment{
            Id:      4,
            Content: "How are you today?",
            Author:  "Betty",
         },
      },
   }

   jsonFile, err := os.Create("post.json")
   if err != nil {
      fmt.Println("Error creating JSON file:", err)
      return
   }
   jsonWriter := io.Writer(jsonFile)
   encoder := json.NewEncoder(jsonWriter)
   err = encoder.Encode(&post)
   if err != nil {
      fmt.Println("Error encoding JSON to file:", err)
      return
   }
}

We're creating an io.Writer that wraps around a file (since we are writing to file). We use this writer to create an encoder using the NewEncoder function. Then we call the Encode method on the encoder and pass it a reference to the post struct. This will extract the data from the struct and create JSON data, which is then written to the writer we passed in earlier.

JSON

Parsing and Creating JSON with Go

By Sau Sheong Chang

This article was excerpted from Go Web Programming.


Tags: JavaScript, XML, JSON, REST, XML Parser, Go, RFC 7159, ECMA-404, marshal




Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Sitemap | Contact Us

Thanks for your registration, follow us on our social networks to keep up-to-date
Rocket Fuel