EBPF Observability Series

eBPF (Extended Berkeley Packet Filter) started as a network packet filtering tool, but over nearly a decade it has evolved into the most powerful observability framework in the Linux kernel. It allows you to safely inject and execute custom programs without modifying kernel source code or loading kernel modules.

This article kicks off the series, using OOM (Out-of-Memory) monitoring as a concrete entry point to learn the core eBPF concepts and toolchain.

bpftrace is great for quick probing and ad-hoc debugging. For production-grade monitoring tools, you need full eBPF programs. The architecture splits into two layers:

• Kernel side: eBPF program written in C, attached to hook points, collecting event data
• User side: loader written in Go (or Rust / libbpf C), loading the eBPF program and reading events

Architecture

flowchart LR
    classDef kern fill:#E3F2FD,stroke:#1565C0,color:#1565C0
    classDef user fill:#FFF3E0,stroke:#E65100,color:#BF360C
    classDef data fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20

    subgraph kernel["Kernel Space"]
        hook@{ shape: rounded, label: "oom_kill_process (kprobe)" }
        ebpf@{ shape: proc, label: "eBPF Program\nEvent Collection" }
        ring@{ shape: cyl, label: "Ring Buffer" }
    end

    subgraph userspace["User Space (Go)"]
        loader@{ shape: notch-rect, label: "bpf2go Loader" }
        reader@{ shape: proc, label: "RingBuf Reader\nEvent Parsing" }
    end

    hook --> ebpf --> ring
    ring --> reader
    loader -.-> ebpf

    class hook,ebpf,ring kern
    class loader,reader user

eBPF Kernel Program (C)

Name the C file oom_kprobe.bpf.c — the bpf suffix is a cilium/ebpf convention for bpf2go code generation:

The first two articles covered eBPF fundamentals and OOM Killer event tracing. This article goes deeper: container-level OOM pinpointing, real-time memory allocation rate tracking, and implementing the same functionality with the Rust Aya framework.

Container-Level OOM Pinpointing

In Kubernetes, “a Pod OOM’d” is actually a vague statement. A Pod consists of multiple containers, each belonging to different cgroups. eBPF can drill through this layer and precisely identify which container and which process caused the OOM.

The previous articles showed how to use eBPF to observe OOM events. But we could only watch, not intervene. The kernel’s OOM Killer decides who lives and dies based on the oom_badness() algorithm, with no user control.

In 2025, Google engineer Roman Gushchin proposed the BPF OOM kernel patch series, aiming to let eBPF programs fully take over OOM handling policy. This is the biggest change to Linux memory management’s OOM subsystem in nearly two decades.

• Overview:
• How to Analyze a Protocol (MongoDB)
  • Protocol Document Analysis Approach
  • MongoDB Protocol OpCode Reference Table
  • Analyzing the Most Common OpCode OP_MSG
• Extending Protocol Parsing in DeepFlow Agent
  • DeepFlow Agent Development Document Overview
  • Code Guide
    • Define a Protocol with a Constant Identifier
    • Prepare Parsing Logic for the New Protocol
      • Define the Struct
      • Implement L7ProtocolParserInterface
• Extending DeepFlow Protocol Collection Using Wasm Plugins
  • Kafka Protocol Analysis
    • Kafka Header and Data Overview
    • Kafka Fetch API
    • Kafka Produce API
  • Kafka Protocol DeepFlow Agent Native Decoding
  • DeepFlow Agent Wasm Plugin
    • Wasm Go SDK Framework
    • Plugin Code Guide
• Conclusion
  • Native Rust Extension
  • Wasm Plugin Extension
• Appendix

Overview

MongoDB is widely used today, but lacks effective observability capabilities. DeepFlow is an excellent solution for observability, but it lacks support for the MongoDB protocol. This article extends DeepFlow with MongoDB protocol parsing, enhancing observability in the MongoDB ecosystem. It briefly describes the process from protocol document analysis to implementing code parsing within DeepFlow.

Deployment process and instructions reference: pixie install

Pixie Platform Main Components

• Pixie Edge Module (PEM): Pixie’s agent, installed per node. PEMs use eBPF to collect data, which is stored locally on the node.

• Vizier: Pixie’s collector, installed per cluster. Responsible for query execution and managing PEMs.

• Pixie Cloud: Used for user management, authentication, and data proxying. Can be hosted or self-hosted.

• Pixie CLI: Used to deploy Pixie. Can also be used to run queries and manage resources like API keys.