JSON #

JSON (JavaScript Object Notation) is the most common data exchange format in modern web APIs. Go provides the encoding/json package in the standard library, which is very capable of encoding (marshaling) and decoding (unmarshaling) JSON. The package works reflectively with Go structs, making the conversion between JSON and Go types very smooth — with a few important nuances to understand.

How JSON Works in Go #

Here’s a visualization of the data flow during serialization (marshaling) and deserialization (unmarshaling) in Go:

flowchart TD
    subgraph serialization["Serialization (Marshal)"]
        A[Go Struct in Memory] -->|"json.Marshal()"| B["[]byte JSON (Text String)"]
    end

    subgraph deserialization["Deserialization (Unmarshal)"]
        C["[]byte JSON (Text String)"] -->|"json.Unmarshal(&dest)"| D[Go Struct in Memory]
    end

Marshal — Go to JSON #

json.Marshal converts a Go value into a JSON representation in []byte:

import "encoding/json"

type Product struct {
    ID       int     `json:"id"`
    Name     string  `json:"name"`
    Price    float64 `json:"price"`
    InStock  bool    `json:"in_stock"`
}

p := Product{ID: 1, Name: "Pro Laptop", Price: 15_000_000, InStock: true}

data, err := json.Marshal(p)
if err != nil {
    log.Fatal(err)
}
fmt.Println(string(data))
// {"id":1,"name":"Pro Laptop","price":1.5e+07,"in_stock":true}

// Pretty print with indentation
pretty, _ := json.MarshalIndent(p, "", "  ")
fmt.Println(string(pretty))
// {
//   "id": 1,
//   "name": "Pro Laptop",
//   "price": 1.5e+07,
//   "in_stock": true
// }

JSON Struct Tags #

Struct tags control how each field is treated during marshal/unmarshal:

type User struct {
    // Basic tag: rename the field in JSON
    ID        int    `json:"id"`
    FirstName string `json:"first_name"`

    // omitempty: this field is ignored if it's the zero value (0, "", false, nil, [], {})
    MiddleName string `json:"middle_name,omitempty"`
    Age        int    `json:"age,omitempty"`

    // "-": this field is always ignored (never appears in JSON at all)
    Password   string `json:"-"`
    // "-,": this field's JSON name is literally "-" (edge case)
    Dash       string `json:"-,"`

    // string: encode numeric/bool values as JSON strings
    Score      float64 `json:"score,string"`

    // Without a tag: the field name is used as-is
    Status string  // → "Status" in JSON

    // Unexported fields are NEVER marshaled (ignored)
    secret string
}

u := User{
    ID: 1, FirstName: "Budi",
    Password: "secret",  // will not appear in JSON
    Score: 9.5,
    Status: "active",
}
data, _ := json.Marshal(u)
// {"id":1,"first_name":"Budi","score":"9.5","Status":"active"}
// MiddleName and Age don't appear (omitempty, zero values)
// Password doesn't appear (the "-" tag)

Unmarshal — JSON to Go #

json.Unmarshal converts a []byte of JSON into a Go struct:

jsonStr := `{
    "id": 42,
    "name": "Sari",
    "email": "[email protected]",
    "age": 28
}`

type Person struct {
    ID    int    `json:"id"`
    Name  string `json:"name"`
    Email string `json:"email"`
    Age   int    `json:"age"`
}

var p Person
if err := json.Unmarshal([]byte(jsonStr), &p); err != nil {
    log.Fatal("unmarshal failed:", err)
}
fmt.Printf("%+v\n", p)
// {ID:42 Name:Sari Email:[email protected] Age:28}

Important Unmarshal Behaviors #

// JSON fields not in the struct → ignored (no error)
// Struct fields not in the JSON → stay as zero values (no error)

// JSON numbers into various Go types
type Example struct {
    IntField   int     `json:"int"`
    FloatField float64 `json:"float"`
    // Careful: large JSON numbers can overflow int
}

// Pointer fields — nil if the JSON is null or the field is missing
type WithPointer struct {
    Name  *string `json:"name"`  // nil if the field is missing
    Score *int    `json:"score"` // nil if "score": null
}

// Nested structs
type Address struct {
    Street string `json:"street"`
    City   string `json:"city"`
}
type UserWithAddr struct {
    Name    string  `json:"name"`
    Address Address `json:"address"`
}

json.Unmarshal([]byte(`{
    "name": "Budi",
    "address": {"street": "Jl. Merdeka", "city": "Jakarta"}
}`), &UserWithAddr{})

In-Memory vs Streaming I/O #

Before using the streaming encoder/decoder, understand the difference in usage scenarios compared to the standard Marshal/Unmarshal:

Characteristic	`json.Marshal` / `Unmarshal`	`json.Encoder` / `Decoder`
Data Source/Destination	Memory variables (`[]byte`, `string`)	I/O streams (`io.Reader`, `io.Writer`)
Memory Usage	Higher (all data loaded into RAM)	Very low (processed incrementally)
Best For	Small data, in-memory processing	HTTP bodies (API requests/responses), large JSON files

flowchart TD
    subgraph in_memory["In-Memory (Marshal/Unmarshal)"]
        A[Data in Memory] -->|"Buffer Operation"| B["json.Marshal() / json.Unmarshal()"]
    end

    subgraph streaming["Streaming I/O (Encoder/Decoder)"]
        C[File / Network Connection / HTTP Request] -->|"Direct Stream (without full buffering)"| D["json.NewEncoder() / json.NewDecoder()"]
    end

Streaming with Encoder and Decoder #

For large JSON or streaming I/O, use json.Encoder and json.Decoder, which work directly with io.Writer and io.Reader without loading all the data into memory:

// Encoder — write JSON to a writer
func writeJSONResponse(w http.ResponseWriter, data interface{}) {
    w.Header().Set("Content-Type", "application/json")
    enc := json.NewEncoder(w)
    enc.SetIndent("", "  ")          // optional: pretty print
    enc.SetEscapeHTML(false)         // disable HTML escaping (<, >, &)
    if err := enc.Encode(data); err != nil {
        log.Println("encode error:", err)
    }
}

// Decoder — read JSON from a reader
func readJSONBody(r *http.Request, dest interface{}) error {
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields()  // error if there are unknown fields

    if err := dec.Decode(dest); err != nil {
        return fmt.Errorf("decode JSON: %w", err)
    }
    return nil
}

// Streaming large arrays — decode one element at a time
func processLargeJSONArray(r io.Reader) error {
    dec := json.NewDecoder(r)

    // Read the opening '[' token
    if _, err := dec.Token(); err != nil {
        return err
    }

    // Read elements one by one
    for dec.More() {
        var item Product
        if err := dec.Decode(&item); err != nil {
            return err
        }
        process(item)  // process without loading everything into memory
    }

    // Read the closing ']' token
    if _, err := dec.Token(); err != nil {
        return err
    }
    return nil
}

`json.RawMessage` — Lazy Parsing #

json.RawMessage is a []byte that isn’t parsed during the parent’s unmarshal, useful for content whose type is only known after reading other fields:

type Event struct {
    Type    string          `json:"type"`
    Payload json.RawMessage `json:"payload"`  // defer parsing
}

jsonData := `{
    "type": "user_created",
    "payload": {"id": 42, "name": "Budi"}
}`

var event Event
json.Unmarshal([]byte(jsonData), &event)

// Now parse the payload based on the type
switch event.Type {
case "user_created":
    var user User
    json.Unmarshal(event.Payload, &user)
    fmt.Println("User created:", user.Name)
case "order_placed":
    var order Order
    json.Unmarshal(event.Payload, &order)
}

Dynamic JSON with `map` and `interface{}` #

For JSON with an unknown structure:

// Parse JSON into a map — flexible but needs type assertions
var result map[string]interface{}
json.Unmarshal([]byte(`{"name":"Budi","age":28,"tags":["go","dev"]}`), &result)

name := result["name"].(string)
age := result["age"].(float64)  // JSON numbers are always float64 in interface{}
ageInt := int(age)
tags := result["tags"].([]interface{})
_ = name; _ = ageInt; _ = tags

// Safer with a type switch
for key, val := range result {
    switch v := val.(type) {
    case string:
        fmt.Printf("%s: string = %s\n", key, v)
    case float64:
        fmt.Printf("%s: number = %v\n", key, v)
    case bool:
        fmt.Printf("%s: bool = %v\n", key, v)
    case []interface{}:
        fmt.Printf("%s: array with %d elements\n", key, len(v))
    case map[string]interface{}:
        fmt.Printf("%s: object\n", key)
    case nil:
        fmt.Printf("%s: null\n", key)
    }
}

// Use json.Number for better numeric precision
dec := json.NewDecoder(strings.NewReader(`{"id": 9999999999999999}`))
dec.UseNumber()  // parse numbers as json.Number, not float64

var data map[string]interface{}
dec.Decode(&data)
id, _ := data["id"].(json.Number).Int64()  // not float64!
fmt.Println(id)  // 9999999999999999 — precision preserved

Custom Marshalers and Unmarshalers #

Implement json.Marshaler or json.Unmarshaler to fully control encoding/decoding:

// A custom date type with a specific format
type Date struct {
    time.Time
}

func (d Date) MarshalJSON() ([]byte, error) {
    return json.Marshal(d.Format("2006-01-02"))
}

func (d *Date) UnmarshalJSON(data []byte) error {
    var s string
    if err := json.Unmarshal(data, &s); err != nil {
        return err
    }
    t, err := time.Parse("2006-01-02", s)
    if err != nil {
        return fmt.Errorf("invalid date format %q: %w", s, err)
    }
    d.Time = t
    return nil
}

type Event struct {
    Name string `json:"name"`
    Date Date   `json:"date"`
}

e := Event{Name: "Independence Day", Date: Date{time.Date(2024, 8, 17, 0, 0, 0, 0, time.UTC)}}
data, _ := json.Marshal(e)
// {"name":"Independence Day","date":"2024-08-17"}

var e2 Event
json.Unmarshal([]byte(`{"name":"Independence Day","date":"2024-08-17"}`), &e2)
fmt.Println(e2.Date.Year())  // 2024

// Enums as strings
type Status int

const (
    StatusActive Status = iota
    StatusInactive
    StatusBanned
)

var statusNames = map[Status]string{
    StatusActive:   "active",
    StatusInactive: "inactive",
    StatusBanned:   "banned",
}

var statusValues = map[string]Status{
    "active":   StatusActive,
    "inactive": StatusInactive,
    "banned":   StatusBanned,
}

func (s Status) MarshalJSON() ([]byte, error) {
    name, ok := statusNames[s]
    if !ok {
        return nil, fmt.Errorf("unknown status: %d", s)
    }
    return json.Marshal(name)
}

func (s *Status) UnmarshalJSON(data []byte) error {
    var str string
    if err := json.Unmarshal(data, &str); err != nil {
        return err
    }
    val, ok := statusValues[str]
    if !ok {
        return fmt.Errorf("invalid status: %q", str)
    }
    *s = val
    return nil
}

API Response Patterns #

A pattern for consistent API responses:

// Standard response wrapper
type Response[T any] struct {
    Success bool   `json:"success"`
    Data    T      `json:"data,omitempty"`
    Error   string `json:"error,omitempty"`
    Meta    *Meta  `json:"meta,omitempty"`
}

type Meta struct {
    Page       int `json:"page"`
    PerPage    int `json:"per_page"`
    Total      int `json:"total"`
    TotalPages int `json:"total_pages"`
}

func WriteSuccess[T any](w http.ResponseWriter, status int, data T) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    json.NewEncoder(w).Encode(Response[T]{Success: true, Data: data})
}

func WriteError(w http.ResponseWriter, status int, msg string) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    json.NewEncoder(w).Encode(Response[any]{Success: false, Error: msg})
}

// Usage
func getProductHandler(w http.ResponseWriter, r *http.Request) {
    id := r.PathValue("id")
    product, err := productRepo.FindByID(id)
    if err != nil {
        WriteError(w, http.StatusNotFound, "product not found")
        return
    }
    WriteSuccess(w, http.StatusOK, product)
}

Complete Example Program #

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "strings"
    "time"
)

// ── Types with custom JSON ───────────────────────────────────

type Money struct {
    Amount   int64  // in cents (Rp 1 = 100 cents)
    Currency string
}

func (m Money) MarshalJSON() ([]byte, error) {
    return json.Marshal(struct {
        Amount   string `json:"amount"`
        Currency string `json:"currency"`
        Display  string `json:"display"`
    }{
        Amount:   fmt.Sprintf("%.2f", float64(m.Amount)/100),
        Currency: m.Currency,
        Display:  fmt.Sprintf("Rp %s", formatIDR(m.Amount)),
    })
}

func (m *Money) UnmarshalJSON(data []byte) error {
    var raw struct {
        Amount   float64 `json:"amount"`
        Currency string  `json:"currency"`
    }
    if err := json.Unmarshal(data, &raw); err != nil {
        return err
    }
    m.Amount = int64(raw.Amount * 100)
    m.Currency = raw.Currency
    return nil
}

func formatIDR(cents int64) string {
    rupiah := cents / 100
    s := fmt.Sprintf("%d", rupiah)
    var result strings.Builder
    n := len(s)
    for i, c := range s {
        if i > 0 && (n-i)%3 == 0 {
            result.WriteByte('.')
        }
        result.WriteRune(c)
    }
    return result.String()
}

type OrderStatus string

const (
    OrderPending    OrderStatus = "pending"
    OrderProcessing OrderStatus = "processing"
    OrderShipped    OrderStatus = "shipped"
    OrderDelivered  OrderStatus = "delivered"
    OrderCancelled  OrderStatus = "cancelled"
)

type Order struct {
    ID        string          `json:"id"`
    CreatedAt time.Time       `json:"created_at"`
    Status    OrderStatus     `json:"status"`
    Total     Money           `json:"total"`
    Items     []OrderItem     `json:"items"`
    Note      string          `json:"note,omitempty"`
    Metadata  json.RawMessage `json:"metadata,omitempty"`
}

type OrderItem struct {
    ProductID   int     `json:"product_id"`
    ProductName string  `json:"product_name"`
    Qty         int     `json:"qty"`
    UnitPrice   Money   `json:"unit_price"`
}

func main() {
    // Marshal: Order to JSON
    order := Order{
        ID:        "ORD-2024-001",
        CreatedAt: time.Date(2024, 7, 28, 10, 30, 0, 0, time.UTC),
        Status:    OrderProcessing,
        Total:     Money{Amount: 3_150_000_00, Currency: "IDR"},
        Items: []OrderItem{
            {
                ProductID:   1,
                ProductName: "Pro Laptop 14",
                Qty:         1,
                UnitPrice:   Money{Amount: 15_000_000_00, Currency: "IDR"},
            },
            {
                ProductID:   2,
                ProductName: "Wireless Mouse",
                Qty:         2,
                UnitPrice:   Money{Amount: 350_000_00, Currency: "IDR"},
            },
        },
        Note:     "Please wrap carefully",
        Metadata: json.RawMessage(`{"source":"web","campaign":"summer_sale"}`),
    }

    data, err := json.MarshalIndent(order, "", "  ")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("=== Marshal Output ===")
    fmt.Println(string(data))

    // Unmarshal: JSON to Order
    jsonInput := `{
        "id": "ORD-2024-002",
        "created_at": "2024-07-28T11:00:00Z",
        "status": "pending",
        "total": {"amount": 500000, "currency": "IDR"},
        "items": [
            {
                "product_id": 3,
                "product_name": "Mechanical Keyboard",
                "qty": 1,
                "unit_price": {"amount": 500000, "currency": "IDR"}
            }
        ]
    }`

    var order2 Order
    if err := json.Unmarshal([]byte(jsonInput), &order2); err != nil {
        log.Fatal("Unmarshal failed:", err)
    }

    fmt.Println("\n=== Unmarshal Result ===")
    fmt.Printf("ID: %s\n", order2.ID)
    fmt.Printf("Status: %s\n", order2.Status)
    fmt.Printf("Total: Rp %s\n", formatIDR(order2.Total.Amount))
    fmt.Printf("Items: %d item(s)\n", len(order2.Items))

    // Streaming: decode several JSON objects from a stream
    fmt.Println("\n=== Streaming Decode ===")
    stream := `{"id":"A","status":"pending"}
{"id":"B","status":"shipped"}
{"id":"C","status":"delivered"}`

    dec := json.NewDecoder(strings.NewReader(stream))
    for dec.More() {
        var o struct {
            ID     string `json:"id"`
            Status string `json:"status"`
        }
        if err := dec.Decode(&o); err != nil {
            log.Fatal(err)
        }
        fmt.Printf("Order %s: %s\n", o.ID, o.Status)
    }

    // json.RawMessage for dynamic payloads
    fmt.Println("\n=== RawMessage / Dynamic JSON ===")
    events := []struct {
        Type    string          `json:"type"`
        Payload json.RawMessage `json:"payload"`
    }{}

    eventJSON := `[
        {"type":"order","payload":{"id":"ORD-001","amount":100000}},
        {"type":"user","payload":{"id":42,"name":"Budi"}},
        {"type":"notification","payload":{"message":"Congratulations!"}}
    ]`

    json.Unmarshal([]byte(eventJSON), &events)
    for _, e := range events {
        fmt.Printf("Type: %-15s | Payload: %s\n", e.Type, string(e.Payload))
    }
}

Summary #

Struct tags json:"name" to rename, omitempty to skip zero values, "-" to always skip, string to encode numbers as strings.
Unexported fields (lowercase) are never marshaled — always use exported fields for structs that need JSON serialization.
json.Encoder / json.Decoder are more efficient for HTTP handlers and large files because they don’t load all the data into memory.
dec.DisallowUnknownFields() for strict validation — errors if the JSON contains fields not in the struct.
json.RawMessage for lazy parsing — parse the content conditionally (type field, version field, etc.).
dec.UseNumber() for large numeric precision — avoids float64 losing precision for large integers.
Custom MarshalJSON/UnmarshalJSON for custom types (dates with a specific format, enums as strings, Money).
map[string]interface{} for dynamic JSON — but remember JSON numbers are always float64, not int.
json.MarshalIndent for human-readable output (debugging, logs, config files).
json.SetEscapeHTML(false) on the encoder to prevent <, >, & from being escaped into \u003c, etc.

← Previous: Mocking Next: YAML →