MiBeeSteward v0.3.0 Internals: Docker Networking vs Probe Fidelity, TLS Cert-Chain Collection, and the Three-MIB Topology

The previous post covered what v0.3.0 ships. This one covers why — this release has three seemingly casual decisions with specific engineering trade-offs behind them: how the Docker network mode decides the scanner’s probe fidelity, why TLS cert-chain collection can cover eight protocols with one core, and why L2 topology needs three MIBs to assemble. Let’s take them one at a time.

Docker Network Mode vs Probe Fidelity

This is the hardest engineering finding in this release, and the reason v0.3.0 ships container images at all. Measured on the test LAN (31 devices):

The bridge mode discovers zero device MACs. This isn’t a scanner bug — it’s a network-namespace physical fact.

The scanner’s passive discovery includes a step that reads /proc/net/arp — the kernel’s ARP cache. Inside a default bridge-networked container, the container has its own network namespace; its network stack only sees the docker0 bridge. The ARP table in that namespace contains exactly one entry — the bridge gateway — because the container has never talked directly to any other host on the LAN (traffic is NAT’d). The host’s /proc/net/arp, packed with 30 MAC entries, is invisible to the container.

flowchart TD
    LAN["LAN<br/>30 devices"] --> HOST["Host netns<br/>/proc/net/arp: 30 rows"]
    LAN -->|"NAT'd, invisible"| CTR["Container netns<br/>/proc/net/arp: gateway only"]

    classDef net fill:#E3F2FD,stroke:#1565C0,color:#1565C0
    classDef ok fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20
    classDef bad fill:#FFEBEE,stroke:#F44336,color:#C62828
    class LAN net
    class HOST ok
    class CTR bad

The host network namespace sees the full ARP table; the container namespace sees only the NAT’d gateway — that’s the root cause of 0/26 vs 30/31.

That explains why v0.3.0’s three compose profiles split the way they do:

• bridge (default): the container sits in its own netns; active probes like TCP/SNMP/HTTP go out through the container’s routing table fine, but netns-dependent passive discovery like MAC/ARP is blind. For UI demo and dev — you just want to click around the UI, scanning accuracy isn’t the point.
• host (recommended): --network host shares the host’s netns directly, so /proc/net/arp is the host’s. Probe fidelity ≈ bare metal. This is the only shape production scanning should pick.
• macvlan: gives the container its own LAN IP; the container appears as a device on the LAN, ARPs directly, and naturally collects the full neighbor table. For “the container IS a discovery device on the LAN.”

The unprivileged image ships LLDP/CDP raw-frame send + the eBPF passive observer as no-op stubs for a related but deeper reason: raw frames need CAP_NET_RAW, running eBPF needs CAP_BPF / CAP_NET_ADMIN. Containers don’t have these by default, and calling them would fail. The stub mode ensures the image doesn’t crash under ordinary docker run; anyone who needs these capabilities builds the privileged variant locally with make docker-build-priv. The stub is “degrade on missing capability,” not “pretend to run” — these handlers check capability at startup, skip if missing, and the corresponding fields in the result set are empty.

TLS Cert-Chain Collection Architecture

v0.3.0 adds full cert-chain collection for TLS services; handler count grows 21 → 29 — eight TLS-wrapped handlers (https / ldaps / smtps / imaps / pop3s / ftps / ircs / telnets). It looks like “write one per protocol,” but the reality is one CollectCertChain core shared across all eight handlers.

This works because Go’s crypto/tls fully decouples cert-chain collection from the application protocol. Whether you’re speaking HTTP or SMTP on top, the TLS handshake completes at the tls.Dial step; after a successful handshake, conn.ConnectionState().PeerCertificates returns the full chain ([]*x509.Certificate, index 0 being the leaf). Cert-chain parsing has nothing to do with “what application runs on top,” so eight handlers sharing one core is the natural result — each handler only handles “initiate a handshake on my port and protocol,” and everything after the handshake runs through the same path.

flowchart TD
    DIAL["tls.Dial<br/>handshake done"] --> CS["ConnectionState<br/>.PeerCertificates"]
    CS --> LOOP["Per cert<br/>cert_index 0..N"]
    LOOP --> EXTRACT["Subject/Issuer/SAN<br/>serial / validity<br/>sig algo / key algo<br/>SHA-256 / PEM"]
    EXTRACT --> DB[("host_tls_certs<br/>(ip,port) + not_after indexes")]

    classDef proc fill:#FFF3E0,stroke:#E65100,color:#BF360C
    classDef store fill:#E8F5E9,stroke:#2E7D32,color:#1B5E20
    class DIAL,CS,LOOP,EXTRACT proc
    class DB store

After the handshake, PeerCertificates yields the full chain; per-cert structured fields are extracted (cert_index 0 is the leaf, 1..N are issuers) and batch-inserted into host_tls_certs. Two indexes serve two query paths.

The host_tls_certs table carries two indexes, each serving an independent query path:

• (ip, port) index: serves the main-path query “what certificates live on this device’s ports.” The TLS sub-panel on the device detail page pulls all of a device’s port certs in one call — this index.
• not_after index: serves the sweep “certs expiring within N days.” Certificate expiry inspection is a full-table filter; without the not_after index it’s a table scan; with it, the time-window sweep goes through the index.

The two indexes serve two different query patterns; neither substitutes for the other. Status colors (green = valid / amber = < 15d to expiry / red = expired) are computed on the frontend as not_after - now(), depending on no extra field.

retention.host_tls_certs_days defaults to 30 days. The trade-off is “historical traceability of cert inspection” vs “table growth rate” — certs are typically valid for a year, 30 days is enough to trace back a full inspection cycle, and the table doesn’t grow unbounded. As with v0.2.0’s retention design, days<=0 is a safety guard that won’t trigger a full-table wipe.

Why L2 Topology Needs Three MIBs

v0.2.0 started with Bridge-MIB; going it alone gets you “MAC sits behind which ifIndex,” but that’s only one dimension of a topology. In a real switch environment, Bridge-MIB alone drops three categories of information:

• Vendor neighbor identity: Bridge-MIB gives you MAC↔ifIndex mappings, but ifIndex is a number — what vendor, model, or name the device on the other end has, Bridge-MIB doesn’t say.
• VLAN-aware forwarding: Bridge-MIB walks dot1dTpFdbPort, valid for the VLAN-transparent FDB; but VLAN-aware switches use dot1qTpFdbPort (Q-BRIDGE-MIB), and adjacency for tagged ports lives in a different table.
• Port role: which edge is forwarding, which is STP-blocked, which switch is root — Bridge-MIB doesn’t cover this; you need dot1dStp (STP-MIB).

The three MIBs v0.3.0 adds each cover one slice, with different merge keys:

Different merge keys are the key part: CDP neighbors merge by device id (the device identifier CDP self-reports); Bridge-MIB and Q-BRIDGE neighbors merge by MAC; STP facts key on STP port id. Three data flows each land in device_neighbors, then MAC — the primary key — plus EnrichDeviceByMAC merge facts about the same device from different protocols into one row.

The IF-MIB ifName resolution step is easy to overlook but matters. Ports out of Bridge-MIB / Q-BRIDGE are ifIndex=1011-style numbers, meaningless to humans; IF-MIB’s ifName translates that into GigabitEthernet0/1, so operators can read “port 1 on this device connects to whom.” The port names shown on the topology graph are ifName-resolved.

Closing the neighbor-identity-inference loop: CDP/LLDP neighbors carry a platform string (e.g. Cisco ISR4321 or Hikvision DS-2CD2047G2), which feeds into v0.2.0’s RuleClassifier to infer vendor/model/type. Then EnrichDeviceByMAC lines up “this device in the neighbor table” with “that device in the device table” using MAC — preserving existing non-empty fields on the device row (e.g. a name filled in by hand wins over an inferred one). In v0.2.0, neighbor_device_id was always NULL precisely because this inference chain wasn’t wired up; v0.3.0 resolves it at query time, so the frontend Neighbors panel can show a neighbor table with name/IP/type.

Known Limitations

A few things deliberately not done, to set expectations:

• Single-instance SQLite — the center is single-instance; no multi-center clustering. Not planned until scale demands it. Same as v0.2.0.
• The unprivileged image skips LLDP/CDP raw frames + eBPF — the image you docker pull is the unprivileged variant; these three probes are no-op stubs. If you need them, build the privileged variant locally with make docker-build-priv and grant CAP_NET_RAW / CAP_BPF / CAP_NET_ADMIN at runtime.
• host_tls_certs retains 30 days by default — enough to trace back one cert-inspection cycle; long-term compliance archival is left to external wiring (e.g. periodic dumps to external storage).

Related Links

• MiBeeSteward GitHub
• MiBeeSteward v0.3.0 Release Notes
• Deployment guide (Docker network mode selection)
• v0.3.0 feature overview (promo post)

The engineering trade-offs in v0.3.0 all revolve around one thing each: the scanner is network-namespace-sensitive, TLS cert chains are application-layer-agnostic, and L2 topology can’t be assembled by one MIB alone. The three facts respectively drive the container image’s three profiles, the TLS collection’s one-core-many-handlers design, and the topology’s three-MIB merge. If you’re building a similar discovery system, these root causes are more worth borrowing than the feature list.