Os #

Every real-world Go application interacts with the operating system — reading configuration files, writing logs, accessing environment variables, receiving shutdown signals, or checking whether a path exists on the filesystem. The os package is the bridge between Go code and the operating system running it. It provides a uniform interface for all these operations, regardless of whether the program runs on Linux, macOS, or Windows. What makes os important isn’t just its capabilities, but also its consistent error handling approach — every operation that can fail returns an error that can be checked with os.IsNotExist, os.IsPermission, and similar functions. This article covers the entire os package: file operations, directories, environment variables, processes, and signals.

An Overview of the os Package #

The os package organizes its functions by what they operate on — files, directories, the environment, or the process itself.

flowchart TD
    OS["package os"] --> File["File Operations"]
    OS --> Dir["Directory Operations"]
    OS --> Env["Environment & Process"]
    OS --> Signal["OS Signals"]
    OS --> Stdio["Standard I/O"]

    File --> F1["os.Open / os.Create / os.OpenFile"]
    File --> F2["os.ReadFile / os.WriteFile"]
    File --> F3["os.Remove / os.Rename / os.Chmod"]
    File --> F4["os.Stat / os.Lstat"]

    Dir --> D1["os.Mkdir / os.MkdirAll"]
    Dir --> D2["os.ReadDir / os.Getwd"]
    Dir --> D3["os.Remove / os.RemoveAll"]
    Dir --> D4["os.TempDir / os.MkdirTemp"]

    Env --> E1["os.Getenv / os.Setenv / os.Environ"]
    Env --> E2["os.Args — program arguments"]
    Env --> E3["os.Exit / os.Getpid"]
    Env --> E4["os.Hostname / os.Executable"]

    Signal --> S1["os/signal.Notify"]
    Signal --> S2["syscall.SIGINT / SIGTERM"]

    Stdio --> IO1["os.Stdin / os.Stdout / os.Stderr"]

    style OS fill:#4f86c6,color:#fff
    style File fill:#e8f5e9
    style Dir fill:#e3f2fd
    style Env fill:#fff3e0
    style Signal fill:#fce4ec
    style Stdio fill:#f3e5f5

Reading and Writing Files #

File operations are what you’ll do most often with os. Go provides two API levels: high-level functions (ReadFile/WriteFile) for simple cases, and os.File for full control.

ReadFile and WriteFile — The Easiest Way #

For files that fit in memory, os.ReadFile and os.WriteFile are the best choice — one line, no manual open/close needed.

package main

import (
    "fmt"
    "os"
)

func main() {
    // Write a file all at once
    content := []byte("Hello from Go!\nSecond line.\n")
    err := os.WriteFile("output.txt", content, 0644)
    if err != nil {
        fmt.Fprintf(os.Stderr, "failed to write file: %v\n", err)
        os.Exit(1)
    }

    // Read the file all at once
    data, err := os.ReadFile("output.txt")
    if err != nil {
        fmt.Fprintf(os.Stderr, "failed to read file: %v\n", err)
        os.Exit(1)
    }
    fmt.Printf("File content (%d bytes):\n%s", len(data), data)
}

Permission 0644 is the common convention for text files: the owner can read and write, group and others can only read. For executable files use 0755.

os.Open, os.Create, and os.OpenFile #

For more control — reading part of a file, appending, or specific access modes — use these functions:

flowchart LR
    subgraph Shortcut["Shortcut Functions"]
        Open["os.Open(path)\nread-only, O_RDONLY"]
        Create["os.Create(path)\nwrite, O_RDWR|O_CREATE|O_TRUNC"]
    end

    subgraph Full["Full Control"]
        OpenFile["os.OpenFile(path, flag, perm)"]
    end

    subgraph Flags["Common Flags"]
        direction TB
        F1["O_RDONLY — read only"]
        F2["O_WRONLY — write only"]
        F3["O_RDWR — read and write"]
        F4["O_CREATE — create if it doesn't exist"]
        F5["O_TRUNC — truncate when opened"]
        F6["O_APPEND — append at the end"]
        F7["O_EXCL — fail if it already exists"]
    end

    OpenFile --> Flags

    style Shortcut fill:#e8f5e9
    style Full fill:#e3f2fd
    style Flags fill:#fff3e0

// os.Open — read only
f, err := os.Open("config.txt")
if err != nil {
    // check the error type
    if os.IsNotExist(err) {
        fmt.Println("file not found")
    } else {
        fmt.Fprintf(os.Stderr, "failed to open file: %v\n", err)
    }
    return
}
defer f.Close() // always defer Close after a successful Open

// os.Create — create a new file or truncate an existing one
f2, err := os.Create("report.txt")
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to create file: %v\n", err)
    return
}
defer f2.Close()
fmt.Fprintln(f2, "Daily report")

// os.OpenFile — full control with flags
// Append to an existing file, create it if it doesn't exist
f3, err := os.OpenFile("app.log",
    os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to open log: %v\n", err)
    return
}
defer f3.Close()
fmt.Fprintln(f3, "new log entry")

// Create a new file, fail if it already exists (to avoid overwrites)
f4, err := os.OpenFile("data.json",
    os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644)
if os.IsExist(err) {
    fmt.Println("file already exists, skipping")
} else if err != nil {
    fmt.Fprintf(os.Stderr, "error: %v\n", err)
    return
} else {
    defer f4.Close()
    fmt.Fprintln(f4, "{}")
}

Reading Part of a File — The io.Reader Interface #

os.File implements io.Reader, so it can be used with every function that accepts a reader — including bufio.Scanner for reading line by line:

import (
    "bufio"
    "fmt"
    "os"
)

func readLineByLine(path string) error {
    f, err := os.Open(path)
    if err != nil {
        return fmt.Errorf("readLineByLine: %w", err)
    }
    defer f.Close()

    scanner := bufio.NewScanner(f)
    number := 1
    for scanner.Scan() {
        fmt.Printf("%3d: %s\n", number, scanner.Text())
        number++
    }

    // Check the scanner error — not just EOF
    if err := scanner.Err(); err != nil {
        return fmt.Errorf("readLineByLine: error while scanning: %w", err)
    }
    return nil
}

Writing to a File — The io.Writer Interface #

Likewise, os.File implements io.Writer, so fmt.Fprintf and bufio.Writer can be used directly:

import (
    "bufio"
    "fmt"
    "os"
    "time"
)

func writeReport(path string, data []string) error {
    f, err := os.Create(path)
    if err != nil {
        return fmt.Errorf("writeReport: %w", err)
    }
    defer f.Close()

    // Use a bufio.Writer for better performance
    // when writing many small lines
    w := bufio.NewWriter(f)

    fmt.Fprintf(w, "Report — %s\n", time.Now().Format("2006-01-02 15:04:05"))
    fmt.Fprintf(w, "%s\n", "===================")

    for i, item := range data {
        fmt.Fprintf(w, "%3d. %s\n", i+1, item)
    }

    // IMPORTANT: Flush must be called so the data is actually written to the file
    // defer f.Close() does not automatically flush a bufio.Writer
    if err := w.Flush(); err != nil {
        return fmt.Errorf("writeReport: flush failed: %w", err)
    }
    return nil
}

If you use a bufio.Writer, always call w.Flush() before f.Close(). defer f.Close() will not flush the buffer automatically — unflushed data is lost without any error. This is a commonly overlooked source of bugs because the program doesn’t report any error, but the written file is incomplete.

File Information — os.Stat and FileInfo #

os.Stat returns an fs.FileInfo containing file metadata without opening it: size, permissions, modification time, and whether it’s a directory.

info, err := os.Stat("config.yaml")
if err != nil {
    if os.IsNotExist(err) {
        fmt.Println("file doesn't exist")
        return
    }
    fmt.Fprintf(os.Stderr, "stat error: %v\n", err)
    return
}

fmt.Println("Name:", info.Name())          // config.yaml
fmt.Println("Size:", info.Size(), "bytes") // 1024
fmt.Println("Permission:", info.Mode())     // -rw-r--r--
fmt.Println("Directory?", info.IsDir())     // false
fmt.Println("Modified:", info.ModTime().Format("2006-01-02 15:04:05"))

The Difference Between Stat and Lstat #

flowchart TD
    A["Target path"] --> B{"Is it a\nsymlink?"}
    B -- Not a symlink --> C["os.Stat and os.Lstat\nreturn the same\nfile info"]
    B -- Symlink --> D{"Which function\nis used?"}
    D -- "os.Stat()" --> E["Follow the symlink\n→ info of the target file"]
    D -- "os.Lstat()" --> F["Don't follow the symlink\n→ info of the symlink itself"]

    E --> G["info.Mode().IsRegular() → true\nif the target is a regular file"]
    F --> H["info.Mode()&fs.ModeSymlink != 0\n→ true, this is a symlink"]

// Stat — follow the symlink, check the target file
info, _ := os.Stat("link-to-file.txt")

// Lstat — don't follow the symlink, check the symlink itself
infoLink, _ := os.Lstat("link-to-file.txt")
fmt.Println(infoLink.Mode()&os.ModeSymlink != 0) // true if it's a symlink

Checking Whether a File Exists #

// ANTI-PATTERN: check existence then open — there's a race condition
if _, err := os.Stat(path); err == nil {
    f, err := os.Open(path) // the file could be deleted between Stat and Open!
    // ...
}

// CORRECT: open directly, check the error
f, err := os.Open(path)
if err != nil {
    if os.IsNotExist(err) {
        // handle the missing file
        return
    }
    // handle other errors
    return
}
defer f.Close()

Directory Operations #

Creating Directories #

// Mkdir — create one directory, fails if the parent doesn't exist
err := os.Mkdir("output", 0755)
if err != nil && !os.IsExist(err) {
    fmt.Fprintf(os.Stderr, "failed to create dir: %v\n", err)
}

// MkdirAll — create all needed directories at once
// like "mkdir -p" in the shell
err = os.MkdirAll("output/2024/03/reports", 0755)
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to create dir tree: %v\n", err)
}

Reading Directory Contents #

// ReadDir — read all entries in a directory, already sorted by name
entries, err := os.ReadDir(".")
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to read dir: %v\n", err)
    return
}

for _, entry := range entries {
    kind := "file"
    if entry.IsDir() {
        kind = "dir "
    }

    // Info() returns FileInfo — can be nil if the file changed
    info, err := entry.Info()
    if err != nil {
        continue
    }

    fmt.Printf("[%s] %-30s %8d bytes\n",
        kind, entry.Name(), info.Size())
}

Recursive Traversal with os.WalkDir #

os.WalkDir is the most efficient way to traverse an entire directory recursively. It’s more efficient than filepath.Walk because it doesn’t call Lstat for every entry.

import (
    "fmt"
    "io/fs"
    "os"
    "path/filepath"
    "strings"
)

func countFiles(root string) (fileCount, dirCount int, totalSize int64) {
    os.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
        if err != nil {
            // Continue even if one entry has an error
            fmt.Fprintf(os.Stderr, "skip %s: %v\n", path, err)
            return nil
        }

        if d.IsDir() {
            // Skip hidden directories
            if strings.HasPrefix(d.Name(), ".") && path != root {
                return fs.SkipDir
            }
            dirCount++
            return nil
        }

        fileCount++
        info, err := d.Info()
        if err == nil {
            totalSize += info.Size()
        }
        return nil
    })
    return
}

// Collect all .go files in a project
func findGoFiles(root string) ([]string, error) {
    var files []string
    err := os.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
        if err != nil {
            return err
        }
        if !d.IsDir() && filepath.Ext(path) == ".go" {
            files = append(files, path)
        }
        return nil
    })
    return files, err
}

Temporary Files #

// MkdirTemp — create a temporary directory (deleted when done)
tmpDir, err := os.MkdirTemp("", "myapp-*")
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to create tmp dir: %v\n", err)
    return
}
defer os.RemoveAll(tmpDir) // clean up when done

// CreateTemp — create a temporary file
tmpFile, err := os.CreateTemp(tmpDir, "data-*.json")
if err != nil {
    fmt.Fprintf(os.Stderr, "failed to create tmp file: %v\n", err)
    return
}
defer os.Remove(tmpFile.Name()) // remove the file when done
defer tmpFile.Close()

fmt.Fprintf(tmpFile, `{"status": "ok"}`)
fmt.Println("Temporary file:", tmpFile.Name())
// /tmp/myapp-123456789/data-987654321.json

Environment Variables #

Environment variables are the standard way to configure an application without changing code or configuration files. This is a very common practice in container and cloud deployments.

flowchart LR
    subgraph Sources["Config Sources"]
        EnvVar["Environment Variable\nDB_HOST=localhost"]
        DotEnv[".env file\n(with the godotenv library)"]
        Args["os.Args\n--host=localhost"]
    end

    subgraph App["Go Application"]
        GetEnv["os.Getenv('DB_HOST')"]
        LookupEnv["os.LookupEnv('DB_HOST')"]
        Environ["os.Environ()\nall env vars"]
    end

    subgraph Config["Config Struct"]
        C["Config{\n  Host: 'localhost'\n  Port: 5432\n}"]
    end

    Sources --> App --> Config

    style Sources fill:#e3f2fd
    style App fill:#e8f5e9
    style Config fill:#fff3e0

// Getenv — returns an empty string if absent
host := os.Getenv("DB_HOST")
if host == "" {
    host = "localhost" // default value
}

// ANTI-PATTERN: can't distinguish "absent" vs "deliberately empty"
port := os.Getenv("DB_PORT")
if port == "" {
    port = "5432" // could be wrong if DB_PORT="" is intentional
}

// CORRECT: LookupEnv — distinguishes "absent" from "empty value"
portStr, exists := os.LookupEnv("DB_PORT")
if !exists {
    portStr = "5432" // default only when truly absent
}

// Setenv — set an environment variable for this process (and child processes)
os.Setenv("APP_ENV", "production")

// Unsetenv — remove an environment variable
os.Unsetenv("DEBUG_MODE")

// Environ — get all environment variables as a []string of "KEY=VALUE"
for _, env := range os.Environ() {
    parts := strings.SplitN(env, "=", 2)
    if len(parts) == 2 {
        fmt.Printf("%-20s = %s\n", parts[0], parts[1])
    }
}

// Clearenv — remove all environment variables (careful!)
// os.Clearenv() // don't use this casually

Pattern: Config from the Environment #

import (
    "fmt"
    "os"
    "strconv"
    "time"
)

type Config struct {
    DBHost     string
    DBPort     int
    DBName     string
    DBUser     string
    DBPassword string
    MaxConn    int
    Timeout    time.Duration
    Debug      bool
}

func configFromEnv() (*Config, error) {
    cfg := &Config{
        DBHost:  getEnvDefault("DB_HOST", "localhost"),
        DBName:  getEnvDefault("DB_NAME", "myapp"),
        DBUser:  getEnvDefault("DB_USER", "postgres"),
        MaxConn: 10,
        Timeout: 30 * time.Second,
    }

    // Port — needs conversion to int
    portStr := getEnvDefault("DB_PORT", "5432")
    port, err := strconv.Atoi(portStr)
    if err != nil {
        return nil, fmt.Errorf("invalid DB_PORT: %w", err)
    }
    cfg.DBPort = port

    // Password — required
    password, exists := os.LookupEnv("DB_PASSWORD")
    if !exists || password == "" {
        return nil, fmt.Errorf("DB_PASSWORD must be set")
    }
    cfg.DBPassword = password

    // MaxConn — optional with a default
    if maxConnStr, exists := os.LookupEnv("DB_MAX_CONN"); exists {
        maxConn, err := strconv.Atoi(maxConnStr)
        if err != nil {
            return nil, fmt.Errorf("invalid DB_MAX_CONN: %w", err)
        }
        cfg.MaxConn = maxConn
    }

    // Debug flag
    cfg.Debug = os.Getenv("APP_DEBUG") == "true" ||
        os.Getenv("APP_DEBUG") == "1"

    return cfg, nil
}

func getEnvDefault(key, defaultVal string) string {
    if val, exists := os.LookupEnv(key); exists {
        return val
    }
    return defaultVal
}

Program Arguments — os.Args #

os.Args is a slice of strings containing the arguments given when the program runs. os.Args[0] is the program name itself.

// Program: ./myapp --env production --port 8080 --debug

fmt.Println("Program:", os.Args[0])    // ./myapp
fmt.Println("All args:", os.Args[1:]) // [--env production --port 8080 --debug]
fmt.Println("Arg count:", len(os.Args)-1) // 5

// Manual parsing — for simple programs
func parseArgs() map[string]string {
    args := make(map[string]string)
    a := os.Args[1:]

    for i := 0; i < len(a); i++ {
        if strings.HasPrefix(a[i], "--") {
            key := strings.TrimPrefix(a[i], "--")
            if i+1 < len(a) && !strings.HasPrefix(a[i+1], "--") {
                args[key] = a[i+1]
                i++
            } else {
                args[key] = "true" // a flag without a value
            }
        }
    }
    return args
}

For more complex argument parsing, use the flag package from the standard library (covered in its own article) or third-party libraries like cobra (for fuller CLI tools with subcommands). os.Args is usually accessed directly only for very simple programs.

Process Information #

// This process's PID
fmt.Println("PID:", os.Getpid())

// The parent process's PID
fmt.Println("PPID:", os.Getppid())

// Machine hostname
hostname, err := os.Hostname()
if err == nil {
    fmt.Println("Host:", hostname)
}

// Path of this program's executable
execPath, err := os.Executable()
if err == nil {
    fmt.Println("Executable:", execPath)
    fmt.Println("Dir:", filepath.Dir(execPath))
}

// Current working directory
wd, err := os.Getwd()
if err == nil {
    fmt.Println("CWD:", wd)
}

// Change the working directory
err = os.Chdir("/tmp")
if err != nil {
    fmt.Fprintf(os.Stderr, "chdir failed: %v\n", err)
}

Handling OS Signals #

OS signals are the communication mechanism between the operating system and a process. The most important signals to handle in production applications are SIGINT (Ctrl+C) and SIGTERM (shutdown from an orchestrator like Kubernetes). Without proper handling, the application dies immediately upon receiving these signals — database connections aren’t closed, in-flight requests are cut off, and data can be corrupted.

sequenceDiagram
    participant OS as Operating System
    participant Signal as os/signal
    participant App as Go Application
    participant DB as Database/Resources

    OS->>Signal: SIGTERM (from K8s, docker stop, etc.)
    Signal->>App: channel sigChan <- syscall.SIGTERM
    App->>App: receive the signal, start shutdown
    App->>DB: close the database connection
    App->>App: wait for active requests to finish
    App->>App: os.Exit(0)

    Note over App,DB: Graceful shutdown — no data lost

package main

import (
    "context"
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    // Set up a channel to capture signals
    sigChan := make(chan os.Signal, 1)
    signal.Notify(sigChan,
        syscall.SIGINT,  // Ctrl+C
        syscall.SIGTERM, // kill / docker stop / kubernetes
    )

    // Run the application in a separate goroutine
    done := make(chan struct{})
    go func() {
        defer close(done)
        runServer()
    }()

    // Block until a signal is received
    sig := <-sigChan
    fmt.Printf("\nReceived signal: %v\n", sig)
    fmt.Println("Starting graceful shutdown...")

    // Give time for cleanup
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    // Do the cleanup
    if err := cleanupResources(ctx); err != nil {
        fmt.Fprintf(os.Stderr, "cleanup error: %v\n", err)
        os.Exit(1)
    }

    fmt.Println("Shutdown complete")
    os.Exit(0)
}

func cleanupResources(ctx context.Context) error {
    // Simulate cleanup: close DB connections, flush log buffers, etc.
    fmt.Println("Closing database connections...")
    select {
    case <-time.After(2 * time.Second): // simulate the cleanup operation
        fmt.Println("Database connections closed")
        return nil
    case <-ctx.Done():
        return fmt.Errorf("cleanup timeout: %w", ctx.Err())
    }
}

Error Handling in os — Checking Error Types #

The os package provides helper functions for checking the type of errors returned by file and directory operations. This matters because the right action differs depending on the error type.

flowchart TD
    E["error from os.*"] --> Check{"Check the error\ntype"}

    Check --> IsNotExist["os.IsNotExist(err)\nor errors.Is(err, fs.ErrNotExist)"]
    Check --> IsExist["os.IsExist(err)\nor errors.Is(err, fs.ErrExist)"]
    Check --> IsPerm["os.IsPermission(err)\nor errors.Is(err, fs.ErrPermission)"]
    Check --> IsTimeout["os.IsTimeout(err)"]
    Check --> Other["other error\nlog and return"]

    IsNotExist --> A1["Create the file/dir\nor return not found"]
    IsExist --> A2["Skip or\nrename"]
    IsPerm --> A3["Ask for permission\nor run as root"]
    IsTimeout --> A4["Retry with\nbackoff"]

    style E fill:#fce4ec
    style Check fill:#4f86c6,color:#fff
    style IsNotExist fill:#e8f5e9
    style IsExist fill:#e3f2fd
    style IsPerm fill:#fff3e0
    style IsTimeout fill:#f3e5f5

func loadConfig(path string) ([]byte, error) {
    data, err := os.ReadFile(path)
    if err != nil {
        switch {
        case os.IsNotExist(err):
            // File doesn't exist — use the default config
            fmt.Printf("config %s not found, using defaults\n", path)
            return defaultConfig(), nil

        case os.IsPermission(err):
            // No access — this is a serious error
            return nil, fmt.Errorf("no read permission for %s — try running with sudo: %w", path, err)

        default:
            // Other errors — wrap and return
            return nil, fmt.Errorf("failed to read config %s: %w", path, err)
        }
    }
    return data, nil
}

// The modern version uses errors.Is (Go 1.13+)
// errors.Is is more robust because it works with wrapped errors
import "io/fs"

func checkFile(path string) {
    _, err := os.Stat(path)
    if errors.Is(err, fs.ErrNotExist) {
        fmt.Println("doesn't exist")
    } else if errors.Is(err, fs.ErrPermission) {
        fmt.Println("no permission")
    } else if err != nil {
        fmt.Println("other error:", err)
    } else {
        fmt.Println("exists")
    }
}

Other File Operations #

Rename, Copy, and Remove #

// Rename — can also be used to move a file
err := os.Rename("old.txt", "new.txt")
// Or move to another directory
err = os.Rename("tmp/data.json", "output/data.json")

// Remove — delete a file or an empty directory
err = os.Remove("not-needed.txt")
if err != nil && !os.IsNotExist(err) {
    fmt.Fprintf(os.Stderr, "failed to remove: %v\n", err)
}

// RemoveAll — delete a directory and everything in it (like rm -rf)
err = os.RemoveAll("tmp/")
// BE CAREFUL: there's no undo for RemoveAll!

// Chmod — change file permissions
err = os.Chmod("script.sh", 0755) // make it executable

// Truncate — cut a file down to a certain size
err = os.Truncate("file.txt", 0) // empty the file without deleting it

Copying Files #

The os package doesn’t provide a direct copy function — Go deliberately separates read and write operations. The correct pattern uses io.Copy:

import (
    "io"
    "os"
)

func copyFile(src, dst string) error {
    // Open the source file
    source, err := os.Open(src)
    if err != nil {
        return fmt.Errorf("copyFile: open source: %w", err)
    }
    defer source.Close()

    // Create the destination file
    dest, err := os.Create(dst)
    if err != nil {
        return fmt.Errorf("copyFile: create destination: %w", err)
    }
    defer dest.Close()

    // Copy the content
    bytesCopied, err := io.Copy(dest, source)
    if err != nil {
        return fmt.Errorf("copyFile: copy: %w", err)
    }

    // Copy the permissions from the source file
    sourceInfo, err := source.Stat()
    if err == nil {
        os.Chmod(dst, sourceInfo.Mode())
    }

    fmt.Printf("Copied %d bytes from %s to %s\n", bytesCopied, src, dst)
    return nil
}

Production Usage Patterns #

Atomic Writes — Writing Without Corruption Risk #

Writing directly to the target file is risky: if the program crashes mid-write, the file becomes corrupted. The atomic write pattern uses a temporary file as a buffer:

flowchart LR
    A["New data"] --> B["Write to\nfile.tmp"]
    B --> C{"Write\nsuccessful?"}
    C -- Yes --> D["os.Rename\nfile.tmp → file.json"]
    C -- No --> E["os.Remove\nfile.tmp"]
    D --> F["file.json\nalways valid"]
    E --> G["old file.json\nstays intact"]

    style D fill:#e8f5e9
    style E fill:#fce4ec
    style F fill:#e8f5e9
    style G fill:#e3f2fd

func writeAtomic(path string, data []byte) error {
    // Write to a temporary file in the same directory
    // (important: must be on the same filesystem for an atomic rename)
    dir := filepath.Dir(path)
    tmpFile, err := os.CreateTemp(dir, ".tmp-*")
    if err != nil {
        return fmt.Errorf("writeAtomic: create tmp: %w", err)
    }
    tmpPath := tmpFile.Name()

    // Make sure the tmp file is cleaned up on error
    defer func() {
        tmpFile.Close()
        os.Remove(tmpPath) // no-op if the rename succeeded
    }()

    // Write the data to the temporary file
    if _, err := tmpFile.Write(data); err != nil {
        return fmt.Errorf("writeAtomic: write: %w", err)
    }

    // Sync to disk before the rename
    if err := tmpFile.Sync(); err != nil {
        return fmt.Errorf("writeAtomic: sync: %w", err)
    }

    // Rename — an atomic operation on the same filesystem
    // The old file is never in a partial state from a reader's perspective
    if err := os.Rename(tmpPath, path); err != nil {
        return fmt.Errorf("writeAtomic: rename: %w", err)
    }

    return nil
}

Making Sure a Directory Exists Before Writing #

func ensureDirExists(path string) error {
    dir := filepath.Dir(path)
    if err := os.MkdirAll(dir, 0755); err != nil {
        return fmt.Errorf("ensureDirExists %s: %w", dir, err)
    }
    return nil
}

func writeFile(path string, data []byte) error {
    if err := ensureDirExists(path); err != nil {
        return err
    }
    return os.WriteFile(path, data, 0644)
}

Reading Config Files with Fallbacks #

// A common config priority order in production applications
func loadConfig() (*Config, error) {
    locations := []string{
        os.Getenv("CONFIG_PATH"),           // explicitly from the env
        "./config.yaml",                     // the current directory
        filepath.Join(os.Getenv("HOME"),
            ".config/myapp/config.yaml"),    // the user's home directory
        "/etc/myapp/config.yaml",            // system-wide config
    }

    for _, path := range locations {
        if path == "" {
            continue
        }
        data, err := os.ReadFile(path)
        if err != nil {
            if os.IsNotExist(err) {
                continue // try the next location
            }
            return nil, fmt.Errorf("failed to read config %s: %w", path, err)
        }
        fmt.Printf("Using config from: %s\n", path)
        return parseConfig(data)
    }

    fmt.Println("No config found, using default values")
    return configDefault(), nil
}

When to Switch to Alternatives #

Keep using os if:
  ✓ Reading/writing files and directories directly
  ✓ Accessing environment variables and program arguments
  ✓ Handling OS signals for graceful shutdown
  ✓ Process information (PID, hostname, executable path)
  ✓ Filesystem operations: rename, remove, chmod, stat

Consider io/fs and fs.FS if:
  ✗ You want a filesystem abstraction that can be mocked in testing
  ✗ Working with embedded files (go:embed)
  ✗ Creating abstractions usable with virtual filesystems

Consider path/filepath if:
  ✗ Cross-platform path manipulation and joining
  ✗ Finding files with glob patterns
  ✗ Converting relative paths to absolute

Consider bufio if:
  ✗ Reading large files line by line
  ✗ You need buffering to improve I/O performance
  ✗ Parsing complex text formats line by line

Consider external libraries if:
  ✗ Watching file changes in real time → fsnotify
  ✗ Very complex filesystem operations → afero (mock-friendly)

Summary #

os.ReadFile / os.WriteFile are the fastest way to read/write small files all at once — no manual open/close needed, ideal for configs and simple data.
os.OpenFile with flags gives full control: O_APPEND for logs, O_EXCL to avoid overwrites, O_RDWR for reading and writing at once.
Always defer f.Close() right after a successful os.Open or os.Create — don’t delay this declaration.
If you use a bufio.Writer, always Flush() before the file is closed — defer f.Close() doesn’t flush the buffer automatically.
os.LookupEnv is better than os.Getenv when you need to distinguish between a missing variable and a deliberately empty one.
os.MkdirAll is the equivalent of mkdir -p — use it instead of os.Mkdir to avoid errors when the parent directory doesn’t exist yet.
The atomic write pattern (write to tmp then rename) ensures the target file is never in a partial state — important for config files and critical data.
Handle the SIGINT and SIGTERM signals for graceful shutdown — this is a mandatory standard for applications running in containers or the cloud.
os.IsNotExist / errors.Is(err, fs.ErrNotExist) to check filesystem error types — don’t just print the error, check its type and handle it differently.
os.WalkDir is more efficient than filepath.Walk for recursive directory traversal because it avoids extra Lstat calls for every entry.

← Previous: Fmt Next: Time →