Hands-On: Building a Distributed File Sharing System

Mon, 18 Mar 2024 00:00:00 +0000

Combining theory with practice, we’ll build a real distributed file sharing system. This system will leverage technologies introduced in previous articles — Kademlia DHT for node discovery and metadata distribution, Gossipsub for broadcasting, and a custom file transfer protocol.

System Architecture

 mermaid
 

 flowchart TD
 subgraph Application
 CLI["CLI Interface"]
 API["REST API"]
 end

 subgraph Business Logic
 Index["File Index Manager"]
 Scheduler["Piece Download Scheduler"]
 Verify["Verify & Reassemble"]
 end

 subgraph P2P Network
 Discovery["Kad-DHT<br/>Peer Discovery + Metadata"]
 Broadcast["GossipSub<br/>Message Broadcast"]
 Transfer["Custom Protocol<br/>File Transfer"]
 end

 CLI --> Index
 API --> Index
 Index --> Scheduler
 Scheduler --> Verify
 Index --> Discovery
 Scheduler --> Transfer
 Broadcast -->|"New Peer Notification"| Index
 

Core design principles:

File Sharing on Mi&Bee Blog

Hands-On: Building a Distributed File Sharing System

System Architecture