Bytes #

The bytes package is the mirror of the strings package — almost every function in strings has a counterpart in bytes, but working on []byte instead of string. This matters because in Go, converting between string and []byte always creates a new copy in memory. Code that does a lot of back-and-forth []bytestring conversion will burden the garbage collector. The bytes package lets you work directly with []byte without unnecessary conversions — especially important when processing data from networks, files, or streams that naturally come in byte form. Alongside the manipulation functions, bytes.Buffer is a very frequently used tool for building []byte or string incrementally and efficiently.

An Overview of the bytes Package #

flowchart TD
    B["package bytes"] --> Func["Manipulation Functions\n(like strings)"]
    B --> Buffer["bytes.Buffer\nbuild data incrementally"]
    B --> Reader["bytes.Reader\nio.Reader from []byte"]

    Func --> Search["Search\nContains / Index / Count\nIndexByte / IndexRune"]
    Func --> Transform["Transformation\nToUpper / ToLower / Title\nTrimSpace / Trim / Replace"]
    Func --> Split["Split & Join\nSplit / SplitN / Fields\nJoin / Repeat"]
    Func --> Compare["Comparison\nEqual / Compare\nHasPrefix / HasSuffix"]

    Buffer --> BW["io.Writer\nWrite / WriteByte / WriteString\nWriteRune"]
    Buffer --> BR["io.Reader\nRead / ReadByte / ReadRune\nReadString / ReadLine"]
    Buffer --> BA["Access\nBytes() / String()\nLen() / Cap() / Reset()"]

    Reader --> RR["io.Reader, io.Seeker\nio.ReaderAt\nSeek / ReadAt"]

    style B fill:#4f86c6,color:#fff
    style Func fill:#e8f5e9
    style Buffer fill:#e3f2fd
    style Reader fill:#fff3e0

Basic Functions — Searching and Checking #

package main

import (
    "bytes"
    "fmt"
)

func main() {
    data := []byte("Hello, World! Hello, Go!")

    // Contains — does it contain a sub-slice?
    fmt.Println(bytes.Contains(data, []byte("World")))  // true
    fmt.Println(bytes.Contains(data, []byte("Python"))) // false

    // ContainsAny — contains any byte from the string?
    fmt.Println(bytes.ContainsAny(data, "aeiou")) // true — there are vowels
    fmt.Println(bytes.ContainsAny(data, "xyz"))   // false

    // ContainsRune — contains a specific rune?
    fmt.Println(bytes.ContainsRune(data, '!')) // true

    // Count — count the occurrences
    fmt.Println(bytes.Count(data, []byte("Hello"))) // 2
    fmt.Println(bytes.Count(data, []byte("")))      // 23 (len+1)

    // Index — the position of the first occurrence (-1 if absent)
    fmt.Println(bytes.Index(data, []byte("World"))) // 7
    fmt.Println(bytes.Index(data, []byte("Java")))  // -1

    // LastIndex — the position of the last occurrence
    fmt.Println(bytes.LastIndex(data, []byte("Hello"))) // 14

    // IndexByte — search for one byte (faster than Index)
    fmt.Println(bytes.IndexByte(data, '!')) // 12

    // IndexRune — search for one rune (supports multibyte)
    fmt.Println(bytes.IndexRune(data, 'W')) // 7

    // IndexAny — search for any byte from the set
    fmt.Println(bytes.IndexAny(data, "aeiou")) // 1 — 'e' in "Hello"

    // HasPrefix and HasSuffix
    fmt.Println(bytes.HasPrefix(data, []byte("Hello"))) // true
    fmt.Println(bytes.HasSuffix(data, []byte("Go!")))  // true
}

Transformation — Changing Byte Slice Contents #

data := []byte("  Hello, World!  ")

// Trim — remove characters from both ends
fmt.Println(string(bytes.TrimSpace(data)))              // "Hello, World!"
fmt.Println(string(bytes.Trim(data, " !")))             // "Hello, World"
fmt.Println(string(bytes.TrimLeft(data, " ")))          // "Hello, World!  "
fmt.Println(string(bytes.TrimRight(data, " ")))         // "  Hello, World!"
fmt.Println(string(bytes.TrimPrefix(data, []byte("  ")))) // "Hello, World!  "
fmt.Println(string(bytes.TrimSuffix(data, []byte("  ")))) // "  Hello, World!"

// TrimFunc — remove bytes satisfying a condition
clean := bytes.TrimFunc(data, func(r rune) bool {
    return r == ' ' || r == '!'
})
fmt.Println(string(clean)) // "Hello, World"

// Case conversion
s := []byte("hello world")
fmt.Println(string(bytes.ToUpper(s))) // "HELLO WORLD"
fmt.Println(string(bytes.ToLower([]byte("HELLO WORLD")))) // "hello world"
fmt.Println(string(bytes.ToTitle(s))) // "HELLO WORLD" (title = upper for ASCII)

// Title — capitalize the start of each word (deprecated in strings, but exists in bytes)
fmt.Println(string(bytes.Title(s))) // "Hello World"

// Replace and ReplaceAll
text := []byte("the cat eats fish, the cat is happy")
fmt.Println(string(bytes.Replace(text, []byte("cat"), []byte("dog"), 1)))
// "the dog eats fish, the cat is happy" — replace only the first

fmt.Println(string(bytes.ReplaceAll(text, []byte("cat"), []byte("dog"))))
// "the dog eats fish, the dog is happy" — replace all

// Map — per-rune transformation
result := bytes.Map(func(r rune) rune {
    if r >= 'a' && r <= 'z' {
        return r - 32 // lowercase to uppercase
    }
    return r
}, []byte("hello, world!"))
fmt.Println(string(result)) // "HELLO, WORLD!"

Splitting and Joining #

// Split — split with a separator
data := []byte("apple,mango,orange,banana")
parts := bytes.Split(data, []byte(","))
for _, p := range parts {
    fmt.Println(string(p))
}
// apple
// mango
// orange
// banana

// SplitN — at most N parts
parts2 := bytes.SplitN(data, []byte(","), 2)
fmt.Println(string(parts2[0])) // "apple"
fmt.Println(string(parts2[1])) // "mango,orange,banana"

// SplitAfter — the separator goes with the previous part
parts3 := bytes.SplitAfter(data, []byte(","))
// ["apple," "mango," "orange," "banana"]

// Fields — split by whitespace (like strings.Fields)
sentence := []byte("  hello   world   go  ")
words := bytes.Fields(sentence)
for _, w := range words {
    fmt.Printf("[%s]\n", w)
}
// [hello]
// [world]
// [go]

// FieldsFunc — split by a condition
csv := []byte("one,,two,,,three")
columns := bytes.FieldsFunc(csv, func(r rune) bool {
    return r == ','
})
// ["one" "two" "three"] — empty fields are skipped

// Join — join with a separator
fruits := [][]byte{[]byte("apple"), []byte("mango"), []byte("orange")}
result := bytes.Join(fruits, []byte(", "))
fmt.Println(string(result)) // "apple, mango, orange"

// Repeat — repeat a byte slice
fmt.Println(string(bytes.Repeat([]byte("ab"), 4))) // "abababab"
fmt.Println(string(bytes.Repeat([]byte("-"), 20)))  // "--------------------"

Comparing Byte Slices #

a := []byte("apple")
b := []byte("mango")
c := []byte("apple")

// Equal — are the contents the same?
fmt.Println(bytes.Equal(a, c)) // true
fmt.Println(bytes.Equal(a, b)) // false

// ANTI-PATTERN: compare with string()
// This allocates a string copy!
fmt.Println(string(a) == string(c)) // true but inefficient

// CORRECT: use bytes.Equal
fmt.Println(bytes.Equal(a, c)) // true with no allocation

// Compare — like strcmp: -1, 0, or 1
fmt.Println(bytes.Compare(a, b)) // -1 (apple < mango)
fmt.Println(bytes.Compare(b, a)) // 1  (mango > apple)
fmt.Println(bytes.Compare(a, c)) // 0  (equal)

// EqualFold — case-insensitive comparison
x := []byte("Hello")
y := []byte("hello")
fmt.Println(bytes.EqualFold(x, y)) // true

bytes.Buffer — Building Data Incrementally #

bytes.Buffer is the most frequently used tool from the bytes package. It implements io.Reader and io.Writer, making it very flexible for building byte data incrementally without many allocations.

flowchart LR
    subgraph Write["Writing to the Buffer"]
        W1["Write([]byte)\nwrite a byte slice"]
        W2["WriteByte(byte)\nwrite one byte"]
        W3["WriteString(string)\nwrite a string"]
        W4["WriteRune(rune)\nwrite one rune"]
        W5["fmt.Fprintf(&buf, ...)\nwrite with formatting"]
    end

    subgraph Buffer["bytes.Buffer"]
        B["internal\n[]byte"]
    end

    subgraph Read["Reading from the Buffer"]
        R1["Read([]byte)\nread into a slice"]
        R2["ReadByte()\nread one byte"]
        R3["ReadRune()\nread one rune"]
        R4["ReadString('\n')\nread until a delimiter"]
        R5["ReadLine()\nread one line"]
    end

    subgraph Access["Data Access"]
        A1["Bytes() []byte\nthe buffer contents (no copy)"]
        A2["String() string\nthe contents as a string"]
        A3["Len() int\nthe number of remaining bytes"]
        A4["Reset()\nempty the buffer"]
    end

    Write --> Buffer
    Buffer --> Read
    Buffer --> Access
import (
    "bytes"
    "fmt"
)

// Basic: building a byte slice
var buf bytes.Buffer

buf.WriteString("Hello, ")
buf.WriteString("World")
buf.WriteByte('!')
buf.WriteRune('🌍')

fmt.Println(buf.String()) // "Hello, World!🌍"
fmt.Println(buf.Len())    // 17 (in bytes, not runes)

// Use fmt.Fprintf for formatting
var buf2 bytes.Buffer
for i := 1; i <= 5; i++ {
    fmt.Fprintf(&buf2, "item %d\n", i)
}
fmt.Print(buf2.String())
// item 1
// item 2
// item 3
// item 4
// item 5

// Reset and reuse
buf.Reset()
fmt.Println(buf.Len())    // 0
fmt.Println(buf.Cap())    // the capacity is still there, not reallocated

// Initialize with initial content
buf3 := bytes.NewBuffer([]byte("initial data"))
buf3.WriteString(" appended")
fmt.Println(buf3.String()) // "initial data appended"

// Reading from the buffer
buf4 := bytes.NewBuffer([]byte("first line\nsecond line\nthird line\n"))
line, err := buf4.ReadString('\n')
fmt.Print(line) // "first line\n"
fmt.Println(err) // nil

line, err = buf4.ReadString('\n')
fmt.Print(line) // "second line\n"

Buffer vs strings.Builder — When to Use Which #

flowchart TD
    Q{"What's the output goal?"} --> S["Only need a string\nin the end"]
    Q --> B["Need []byte\nor both"]
    Q --> IO["Need io.Reader\nor io.Writer"]

    S --> SB["strings.Builder\nmore efficient for strings\ncan't Read"]
    B --> BB["bytes.Buffer\nflexible: can read and write\ncan be io.Reader/Writer"]
    IO --> BB2["bytes.Buffer\nimplements both"]

    SB --> SE["String() to get the result"]
    BB --> BE["Bytes() or String()\nto get the result"]

    style SB fill:#e8f5e9
    style BB fill:#e3f2fd
    style BB2 fill:#e3f2fd
// strings.Builder — for building a string (can't be read as a Reader)
var sb strings.Builder
sb.WriteString("Hello, ")
sb.WriteString("World!")
result := sb.String() // get the result as a string

// bytes.Buffer — for building []byte or when you need io.Reader
var buf bytes.Buffer
buf.WriteString("protocol data")
buf.WriteByte(0x00) // can write any byte including null

// Send as an io.Reader to another function
json.NewDecoder(&buf).Decode(&target)
http.Post(url, "application/octet-stream", &buf)

// ANTI-PATTERN: use bytes.Buffer only for a final string
var buf2 bytes.Buffer
for i := 0; i < 100; i++ {
    buf2.WriteString("item") // Buffer works, but Builder is more efficient
}
_ = buf2.String()

// CORRECT: strings.Builder for pure string building
var sb2 strings.Builder
for i := 0; i < 100; i++ {
    sb2.WriteString("item")
}
_ = sb2.String()

bytes.Reader — An io.Reader from []byte #

bytes.Reader turns a []byte into an io.Reader that supports seeking — useful when you have data in memory but the function receiving it expects an io.Reader:

data := []byte(`{"name":"Budi","age":30}`)

// Create a Reader from []byte
reader := bytes.NewReader(data)

// Decode JSON from the Reader (not directly from []byte)
var user struct {
    Name string `json:"name"`
    Age  int    `json:"age"`
}
json.NewDecoder(reader).Decode(&user)
fmt.Println(user.Name, user.Age) // Budi 30

// Seek — go back to a certain position
reader.Seek(0, 0) // back to the beginning
fmt.Println(reader.Len()) // 25 — back to the full length

// ReadAt — read from a certain position without moving the position
buf := make([]byte, 4)
reader.ReadAt(buf, 2)
fmt.Println(string(buf)) // "\"nam"

// Size
fmt.Println(reader.Size()) // 25 — the total size (doesn't change after Seek)

// Use as an io.Reader for HTTP uploads
data2 := []byte("this file's content")
resp, err := http.Post(
    "https://api.example.com/upload",
    "application/octet-stream",
    bytes.NewReader(data2),
)
_ = resp
_ = err

Working with Binary Data #

The bytes package is very useful when processing binary data — network protocols, file formats, or data streams containing a mix of text and binary:

import (
    "bytes"
    "encoding/binary"
    "fmt"
)

// Parsing a simple protocol frame:
// [4 length bytes][1 type byte][N payload bytes]
func parseFrame(data []byte) (kind byte, payload []byte, err error) {
    if len(data) < 5 {
        return 0, nil, fmt.Errorf("frame too short: %d bytes", len(data))
    }

    reader := bytes.NewReader(data)

    // Read the payload length (4 big-endian bytes)
    var length uint32
    if err := binary.Read(reader, binary.BigEndian, &length); err != nil {
        return 0, nil, fmt.Errorf("read length: %w", err)
    }

    // Read the type (1 byte)
    typeByte, err := reader.ReadByte()
    if err != nil {
        return 0, nil, fmt.Errorf("read type: %w", err)
    }

    // Read the payload
    payload = make([]byte, length)
    if _, err := reader.Read(payload); err != nil {
        return 0, nil, fmt.Errorf("read payload: %w", err)
    }

    return typeByte, payload, nil
}

// Building a protocol frame
func makeFrame(kind byte, payload []byte) []byte {
    var buf bytes.Buffer

    // Write the payload length (4 big-endian bytes)
    binary.Write(&buf, binary.BigEndian, uint32(len(payload)))

    // Write the type
    buf.WriteByte(kind)

    // Write the payload
    buf.Write(payload)

    return buf.Bytes()
}

// Usage
frame := makeFrame(0x01, []byte("Hello from Go!"))
kind, payload, err := parseFrame(frame)
if err == nil {
    fmt.Printf("Type: 0x%02X, Payload: %s\n", kind, payload)
}

Processing HTTP Response Bodies #

import (
    "bytes"
    "compress/gzip"
    "io"
    "net/http"
)

func fetchAndProcess(url string) ([]byte, error) {
    resp, err := http.Get(url)
    if err != nil {
        return nil, fmt.Errorf("GET %s: %w", url, err)
    }
    defer resp.Body.Close()

    // Read the whole body into a buffer
    var buf bytes.Buffer
    if _, err := io.Copy(&buf, resp.Body); err != nil {
        return nil, fmt.Errorf("read body: %w", err)
    }

    data := buf.Bytes()

    // Check whether it's gzip-encoded
    if bytes.HasPrefix(data, []byte{0x1f, 0x8b}) {
        // gzip magic bytes
        reader, err := gzip.NewReader(bytes.NewReader(data))
        if err != nil {
            return nil, fmt.Errorf("open gzip: %w", err)
        }
        defer reader.Close()

        var decompressed bytes.Buffer
        if _, err := io.Copy(&decompressed, reader); err != nil {
            return nil, fmt.Errorf("decompress: %w", err)
        }
        return decompressed.Bytes(), nil
    }

    return data, nil
}

Production Usage Patterns #

Template Rendering into a Buffer #

import (
    "bytes"
    "html/template"
)

var tmplEmail = template.Must(template.New("email").Parse(`
To: {{.Name}}

Thank you for registering with our service.
Your verification code: {{.Code}}

This code is valid for {{.DurationMinutes}} minutes.
`))

type EmailData struct {
    Name            string
    Code            string
    DurationMinutes int
}

func renderEmail(data EmailData) ([]byte, error) {
    var buf bytes.Buffer
    if err := tmplEmail.Execute(&buf, data); err != nil {
        return nil, fmt.Errorf("render email: %w", err)
    }
    return buf.Bytes(), nil
}

// Usage
content, err := renderEmail(EmailData{
    Name:            "Budi",
    Code:            "123456",
    DurationMinutes: 10,
})
if err == nil {
    fmt.Println(string(content))
}

Building CSV Manually #

func makeCSV(headers []string, rows [][]string) []byte {
    var buf bytes.Buffer

    // Write the header
    for i, h := range headers {
        if i > 0 {
            buf.WriteByte(',')
        }
        buf.WriteString(escapeCSV(h))
    }
    buf.WriteByte('\n')

    // Write the data rows
    for _, row := range rows {
        for i, cell := range row {
            if i > 0 {
                buf.WriteByte(',')
            }
            buf.WriteString(escapeCSV(cell))
        }
        buf.WriteByte('\n')
    }

    return buf.Bytes()
}

func escapeCSV(s string) string {
    // Quote if it contains a comma, newline, or quote
    if bytes.ContainsAny([]byte(s), ",\"\n\r") {
        return `"` + strings.ReplaceAll(s, `"`, `""`) + `"`
    }
    return s
}

// Usage
csv := makeCSV(
    []string{"ID", "Name", "Email", "City"},
    [][]string{
        {"1", "Budi Santoso", "[email protected]", "Jakarta"},
        {"2", "Ani", "[email protected]", "Bandung, West Java"},
        {"3", "Charlie", `char"[email protected]`, "Surabaya"},
    },
)
os.WriteFile("output.csv", csv, 0644)

A Buffer Pool with sync.Pool #

import "sync"

// A pool of bytes.Buffers to avoid repeated allocations
var bufPool = sync.Pool{
    New: func() any {
        return new(bytes.Buffer)
    },
}

func getBuffer() *bytes.Buffer {
    buf := bufPool.Get().(*bytes.Buffer)
    buf.Reset() // IMPORTANT: reset before use
    return buf
}

func returnBuffer(buf *bytes.Buffer) {
    // Don't return overly large buffers to the pool
    // to avoid holding too much memory
    if buf.Cap() <= 64*1024 { // 64 KB
        bufPool.Put(buf)
    }
}

func processRequest(data []byte) string {
    buf := getBuffer()
    defer returnBuffer(buf)

    // Use buf to build the response
    buf.WriteString(`{"status":"ok","data":`)
    buf.Write(data)
    buf.WriteByte('}')

    return buf.String()
}

Streaming Large Data Without Loading It into Memory #

// Process a large file line by line without loading everything into memory
func processLargeFile(path string) error {
    f, err := os.Open(path)
    if err != nil {
        return fmt.Errorf("open file: %w", err)
    }
    defer f.Close()

    var (
        buf      = make([]byte, 64*1024) // 64 KB buffer
        leftover []byte                  // leftover from the previous read
        lines    int
    )

    for {
        n, err := f.Read(buf)
        if n > 0 {
            // Combine the previous leftover with the new data
            chunk := append(leftover, buf[:n]...)
            leftover = nil

            // Process each complete line
            for {
                idx := bytes.IndexByte(chunk, '\n')
                if idx < 0 {
                    // No newline — save as leftover
                    leftover = chunk
                    break
                }
                lines++
                processLineData(chunk[:idx])
                chunk = chunk[idx+1:]
            }
        }

        if err == io.EOF {
            // Process the final leftover if it doesn't end with a newline
            if len(leftover) > 0 {
                lines++
                processLineData(leftover)
            }
            break
        }
        if err != nil {
            return fmt.Errorf("read file: %w", err)
        }
    }

    fmt.Printf("Processed %d lines\n", lines)
    return nil
}

bytes vs strings — When to Use Which #

flowchart TD
    Q{"What data are you working with?"} --> Str["string\n(immutable, already exists)"]
    Q --> ByteSlice["[]byte\n(mutable, from network/files)"]
    Q --> Both["A mix of both"]

    Str --> UseStr["Use the strings package\n+ strings.Builder\nwithout conversions"]

    ByteSlice --> UseBytes["Use the bytes package\n+ bytes.Buffer\nwithout conversions"]

    Both --> Consider["Decide the primary representation\nminimize conversions\nbytes → string: string(b)\nstring → bytes: []byte(s)"]

    Consider --> Rule["Rule: conversions create copies\nIf a function needs []byte, avoid\nstring → []byte → string"]

    style UseStr fill:#e8f5e9
    style UseBytes fill:#e3f2fd
    style Rule fill:#fff3e0
// ANTI-PATTERN: unnecessary back-and-forth conversions
func processDataBad(data []byte) []byte {
    s := string(data)           // first copy: []byte → string
    s = strings.ToUpper(s)     // process
    s = strings.TrimSpace(s)   // process
    return []byte(s)           // second copy: string → []byte
}

// CORRECT: stay in []byte, use the bytes package
func processDataGood(data []byte) []byte {
    result := bytes.ToUpper(data)
    return bytes.TrimSpace(result)
}

// When conversion IS genuinely needed:
// 1. Map/switch cases with string literals
switch string(data[:4]) {
case "HTTP", "POST", "GET ":
    // process
}

// 2. When an external function only accepts a string
log.Println(string(data)) // log.Println needs a string

// 3. When storing into a string-typed struct field
user.Name = string(nameBytes)

When to Switch to Alternatives #

Keep using bytes if:
  ✓ Manipulating []byte: searching, trimming, splitting, replacing
  ✓ bytes.Buffer for building []byte incrementally
  ✓ bytes.Reader for passing []byte as an io.Reader
  ✓ Binary data processing (protocols, file formats)
  ✓ Working with data from networks or files that naturally comes as []byte

Use strings if:
  ✗ The data is already a string and will stay a string
  ✗ String manipulation: strings.Contains, strings.Split, etc.

Use strings.Builder if:
  ✗ Building a string without needing io.Reader/Writer
  ✗ More efficient than bytes.Buffer for pure string output

Use bufio if:
  ✗ Buffered I/O from files or network connections
  ✗ Reading line by line from large streams
  ✗ Parsing text line by line with Scanner

Use encoding/binary if:
  ✗ Reading/writing integers with specific endianness from []byte
  ✗ Parsing structured binary formats

Summary #

  • bytes is the mirror of strings — almost every strings function has a counterpart in bytes. Choose by data type: stringstrings, []bytebytes.
  • Avoid unnecessary []bytestring conversions — every conversion creates a copy in memory. If the data comes as []byte, process it as []byte until you’re done.
  • bytes.Equal(a, b) is more efficient than string(a) == string(b) — no temporary string allocation, compares byte by byte directly.
  • bytes.Buffer implements both io.Reader and io.Writer — useful when you need a buffer that can be read after writing, or when an external function needs an io.Reader.
  • strings.Builder for pure string building — more efficient than bytes.Buffer when the final output is a string and you don’t need to read it back as a Reader.
  • bytes.Reader to turn a []byte into an io.Reader — supports seeking (Seek), useful for rewinding or re-reading from different positions.
  • Use sync.Pool for bytes.Buffer on HTTP handler hot paths — avoid allocating a new buffer for every request by recycling existing buffers.
  • bytes.Buffer.Reset() empties the buffer without releasing the already-allocated capacity — reusing a buffer with Reset is far more efficient than creating a new one.
  • For large data, process it streaming with a small buffer rather than loading everything into memory — use io.Copy or a manual read loop with a []byte buffer.

← Previous: Sort   Next: Bufio →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact