Network Development Practice

TCP congestion control is a critical determinant of network transport performance and a cornerstone of internet stability. Whether it’s webpage loading speed in web services, smoothness of live video streaming, inter-container communication in cloud-native applications, or download efficiency in P2P transfers, all rely on TCP congestion control to coordinate bandwidth allocation. A single BitTorrent node, for example, may maintain hundreds of concurrent TCP connections, and choosing the wrong algorithm can severely degrade bandwidth utilization. Since Van Jacobson’s seminal 1988 paper at SIGCOMM, congestion control algorithms have evolved over nearly four decades — from heuristic loss-based methods to precise model-based measurement. This article covers 12 major congestion control algorithms, explaining their core ideas, strengths, weaknesses, and applicable scenarios, from Tahoe and Reno to CUBIC, BBR, and Copa.

BBR (Bottleneck Bandwidth and Round-trip propagation time), developed by Neal Cardwell, Yuchung Cheng, and others at Google, is one of the most advanced model-based congestion control algorithms available today. Unlike traditional loss-based algorithms (Reno, CUBIC), BBR explicitly models the network path by directly measuring bottleneck bandwidth and propagation delay, sending data at the BDP (Bandwidth-Delay Product) rate at the bottleneck point.

BBR’s core insight is that packet loss does not equal congestion. On deep-buffered (Bufferbloat) or wireless links, packet loss can be caused by channel noise or excessive buffer queuing rather than genuine link saturation. BBR actively measures bandwidth and latency to precisely control the sending rate, rather than passively waiting for loss signals.

Peer-to-Peer (P2P) networking is a decentralized architecture where every node acts as both a provider (Server) and consumer (Client). This architecture is widely used in file distribution (BitTorrent), cryptocurrency (Bitcoin), decentralized storage (IPFS), and many other domains.

P2P vs Client-Server Architecture

Before diving into P2P principles, let’s understand the fundamental differences through comparison:

The core advantage of P2P lies in eliminating single points of bottleneck and failure, at the cost of introducing more complex node discovery and data routing mechanisms.

In P2P networks, every node needs to learn the global cluster state—which peers are online, where data is stored, and whether new nodes have joined or old ones left—without relying on any central server. The essence of this problem is: how can information be disseminated efficiently and reliably across an unpredictable, dynamic network?

Gossip protocol (also called Epidemic protocol) offers a decentralized solution: mimic the spread pattern of infectious diseases. Each node randomly selects several neighbors and exchanges the information it knows. The message spreads like a virus, eventually reaching all nodes with high probability. It requires no centralized coordinator and has natural tolerance for network partitions and node failures.

Kademlia is one of the most influential DHT (Distributed Hash Table) protocols, proposed by Petar Maymounkov and David Mazières in 2002. It is widely used in IPFS, BitTorrent, Ethereum, and many other systems. Kademlia’s revolutionary innovation lies in using XOR (exclusive or) as its distance metric, offering elegant mathematical properties and efficient routing algorithms.

XOR Distance Metric

Kademlia maps nodes and resources to the same 160-bit identifier space and defines the XOR distance function:

In the previous article, we explored the core principles of the Gossip protocol in depth—Epidemic propagation models, the distinction between Anti-Entropy and Rumor-Mongering, and the mathematical foundation of the Phi Accrual failure detector. Gossip provides a general mechanism for information dissemination, but to build a complete distributed cluster, information dissemination alone is not enough: every node needs to know who else is in the cluster—who is online, who has left, and who has just joined.

Building on our understanding of P2P core principles and Kademlia DHT, we now dive into two production-proven P2P protocols — the libp2p protocol stack and BitTorrent. They represent two different design philosophies: a general-purpose P2P framework versus a specialized file distribution protocol.

libp2p Modular Architecture

libp2p is the networking layer behind IPFS and Filecoin, providing a modular toolkit for building P2P applications. Its design philosophy is “pluggable network protocol stack for P2P applications” — developers compose transport, security, multiplexing, and application layers like building blocks.

Rust’s ownership model and zero-cost abstractions make it an ideal language for implementing P2P network protocols. The Rust implementation of libp2p (rust-libp2p) provides a complete protocol stack from transport to application layer.

Environment Setup

Configure dependencies in Cargo.toml:

1
2
3
4
5
6
7
8
9

[dependencies]
libp2p = { version = "0.53", features = [
    "tokio", "tcp", "quic", "noise", "yamux", "mplex",
    "ping", "identify", "kad", "gossipsub", "relay", "dcutr", "macros",
] }
tokio = { version = "1.0", features = ["full"] }
tokio-stream = "0.1"
futures = "0.3"
anyhow = "1.0"

Create the project:

Go’s simple concurrency model and fast compilation make it a popular choice for P2P network development. go-libp2p is one of the most complete libp2p implementations and is widely used in large projects like IPFS (Kubo).

Environment Setup

1
2
3

go get github.com/libp2p/go-libp2p
go get github.com/libp2p/go-libp2p-kad-dht
go get github.com/libp2p/go-libp2p-pubsub

go.mod configuration:

Combining theory with practice, we’ll build a real distributed file sharing system. This system will leverage technologies introduced in previous articles — Kademlia DHT for node discovery and metadata distribution, Gossipsub for broadcasting, and a custom file transfer protocol.

System Architecture

flowchart TD
    APP["Application<br/>CLI / REST API"] --> LOGIC["Business Logic<br/>File index / download scheduler / verify"]
    LOGIC --> NET["P2P Network<br/>Kad-DHT / Gossipsub / file transfer"]
    NET -.->|"new peer notification"| LOGIC
    style APP fill:#2196F3,color:#fff
    style LOGIC fill:#FF9800,color:#fff
    style NET fill:#4CAF50,color:#fff

Three strictly separated layers: application faces outward, business logic handles scheduling and verification, and the P2P network layer handles peer discovery, message broadcast, and file transfer.

Moving a P2P system from prototype to production involves engineering challenges across connection management, security, observability, and deployment. This article covers six critical areas with reusable code snippets and actionable guidance.

Connection Management and Resource Limits

P2P nodes must maintain a large number of simultaneous connections. Without resource caps, a node can suffer OOM crashes or file descriptor exhaustion. Production environments require strict control over three dimensions.

Three-Layer Resource Control Model

flowchart TD
    TL["Transport Limits<br/>Inbound 1024 / Outbound 512"] --> Q["Connection Queue"]
    Q --> SL["Stream Limits<br/>Inbound 128 / Outbound 256"]
    SL --> I{"Health Check<br/>Idle 30s/60s"}
    I -->|"Pass"| OK["Update Active Timestamp"]
    I -->|"Fail"| CL["Close Stream/Conn<br/>Release FD & Memory"]

    classDef src fill:#bbdefb,stroke:#2196F3,color:#1B5E20
    classDef proc fill:#fff3e0,stroke:#FF9800,color:#BF360C
    classDef decision fill:#f3e5f5,stroke:#9C27B0,color:#4A148C
    classDef ok fill:#c8e6c9,stroke:#4CAF50,color:#1B5E20
    classDef bad fill:#ffcdd2,stroke:#f44336,color:#B71C1C
    class TL src
    class Q,SL proc
    class I decision
    class OK ok
    class CL bad

Rust Connection Management

Rust’s SwarmBuilder provides a fluent API for connection configuration:

The previous four articles in this series built a complete knowledge foundation—from Epidemic propagation theory in Gossip, to the SWIM membership protocol, to P2P implementations in Rust and Go, and finally to production best practices. Now it is time to apply this knowledge to real distributed systems.

This article examines six representative systems and how they adapt Gossip protocols to different scenarios: from Gossipsub parameter tuning to Raft membership changes, from Redis Cluster PING/PONG to Cassandra’s GossipDigest protocol, and the hidden Gossip routing mechanisms inside message queues.

Building a self-hosted P2P signaling and relay server is the core infrastructure for cross-network connectivity, remote access, and mesh VPN scenarios. This article systematically surveys the complete technology landscape across three dimensions: protocol standards (STUN/TURN/ICE/BEHAVE), mainstream products (Tailscale, Nebula, NetBird, ZeroTier, Headscale, OpenZiti, etc.), and frameworks & algorithms (libp2p, WebRTC, Kademlia DHT).

All technical descriptions are verified against primary sources—RFC originals, academic papers, and official documentation. Key statistics include citations.

How do NVR (Network Video Recorder) systems let mobile apps view surveillance feeds from any network? This is a core requirement for the security industry and smart home ecosystems. This article systematically surveys three dimensions: open-source NVR projects (Frigate, go2rtc, Kerberos.io, Agent DVR, etc.), commercial surveillance vendors (Hikvision, Dahua, EZVIZ, Ubiquiti, Synology, Reolink, etc.), and third-party P2P platforms and security research (TUTK Kalay, iLnkP2P/PPPP, GB/T 28181, key CVEs).

All technical descriptions are verified against primary sources—official documentation, security advisories, and academic papers. Key statistics include citations.

In the MiBee NVR surveillance-dashboard project, should the frontend pull H.264 or H.265 from all 16 cameras? A seemingly simple question that gets deeper the more you dig. H.265 needs only half the bitrate of H.264 at the same quality, saving both bandwidth and storage. But plenty of online sources still claim “H.265 browser support is poor, Firefox doesn’t support it, you need to install extensions.” Which of these claims still hold?

These 14 streaming technologies don’t operate in the same layer. A complete live streaming pipeline typically follows:

Camera/Encoder → Ingest (RTMP / SRT / RIST / WebRTC-WHIP / WebTransport) → Transcode → Delivery (HLS / LL-HLS / DASH / LL-DASH / HESP / HTTP-FLV) → Viewers

Start by identifying which segment of the pipeline you’re working on, not by asking “which is better.”

flowchart TD
    A["Camera/Encoder"] --> B["Ingest Layer<br/>RTMP · SRT · RIST<br/>WebRTC · WebTransport"]
    B --> C["Transcode/Packaging<br/>CMAF · ABR · Multi-bitrate"]
    C --> D["Delivery Layer<br/>HLS · LL-HLS · DASH<br/>HTTP-FLV · WebRTC · HESP"]
    D --> E["Viewer End"]
    style A fill:#2196F3,color:#fff
    style B fill:#FF9800,color:#fff
    style C fill:#9C27B0,color:#fff
    style D fill:#4CAF50,color:#fff
    style E fill:#2196F3,color:#fff

The diagram above shows a five-segment structure for a standard live streaming pipeline. The ingest layer moves signals from cameras to the platform, while the delivery layer distributes processed streams to viewers. RTSP (camera control) and NDI (LAN production) operate outside the main pipeline, each working independently.

Nmap (Network Mapper) is the most widely used open-source network scanning and security auditing tool in the world. Its core identification capabilities rely on seven built-in fingerprint databases that cover operating systems, service versions, protocols, ports, MAC vendors, RPC programs, and NSE script extensions. As of the latest release Nmap 7.99 (March 26, 2026), these databases have evolved into one of the most comprehensive and active fingerprint identification ecosystems in the cybersecurity field.

Nmap has the world’s most comprehensive network fingerprint database — over 6,000 service probe signatures and 5,000+ OS fingerprints. But its fingerprint engine is implemented in C++, tightly coupled to PCRE2 regex and nsock async I/O. Reusing it directly means accepting Nmap’s entire architectural constraints.

Building a custom fingerprint loading tool is valuable when you need to embed the fingerprint database into a standalone binary for offline scanning, integrate fingerprint matching into an automated pipeline, bypass Nmap’s licensing constraints for customized scan strategies, or use it as a foundational component in a security product.

Nmap is the gold standard of network scanning, and its fingerprint databases (nmap-os-db, nmap-service-probes) represent over two decades of accumulated knowledge. However, since Nmap 7.90 (2021), Nmap’s license was changed from GPLv2 to NPSL (Nmap Public Source License), adding many restrictions beyond standard GPL terms.

This means: even open-source projects that read or embed Nmap’s fingerprint data files may constitute derivative works of Nmap and must be released under an NPSL-compatible license. For closed-source commercial products, the compliance risk is even more severe—either open-source the entire project under NPSL, or pay a one-time OEM license fee of $59,980~$119,980.

In network security, fingerprinting is the foundational step for asset discovery and attack surface management. However, a single fingerprint library often has limited coverage—Nmap excels at network-layer service detection but falls short on web technology stacks; Wappalyzer is strong at frontend framework detection but cannot sense underlying protocols; WhatWeb identifies CMS accurately but lacks port scanning capabilities. In practice, a single target may involve network devices, web applications, cloud services, and other asset types simultaneously, so relying on just one fingerprint library inevitably leads to significant omissions.