Mi&Bee Blog
中文

Overseas Email Delivery Acceleration Architecture — Smart Delivery Network in Practice

Architecture Email, Networking, Architecture, Mail-Delivery 2102 words 10 min read

Introduction: Challenges of Overseas Email Delivery

In globalized business scenarios, email delivery faces numerous challenges. Overseas email delivery must not only overcome network latency issues but also navigate complex geopolitical network environments. Traditional direct overseas delivery methods often encounter connection timeouts, firewall blocking, and high delivery latency. At the same time, interoperability between domestic email systems and overseas networks also presents technical challenges.

To address these issues, we designed and implemented a Smart Delivery Network (SDN) architecture that effectively improves the efficiency and reliability of overseas email delivery through multi-level proxying, intelligent routing, and load balancing mechanisms.

Problem Analysis: Pain Points of Direct Overseas Delivery

1. Network Connectivity Issues

  • GFW interception mechanisms: Overseas email servers are frequently subject to network restrictions, causing unstable connections
  • Poor link quality: Direct overseas network paths are long, with high latency and high packet loss rates
  • Single point of failure risk: Dependency on a single overseas exit means any issue affects all deliveries

2. Performance Bottlenecks

  • Concurrent connection limits: A single overseas email server has limited concurrent connections
  • Unstable delivery latency: Spam processing affects the delivery efficiency of normal emails
  • Resource contention: Normal emails and spam share delivery resources

3. High Management Complexity

  • Difficult multi-domain management: Each domain requires independent delivery configuration maintenance
  • Tedious configuration updates: Manual configuration is error-prone and costly to maintain
  • Difficult monitoring and troubleshooting: Lack of a unified monitoring and management platform

Overall Architecture Design

Overseas MX Node Distribution

We built a global MX node distribution system covering major overseas regions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

North America Nodes:
- Vancouver, Canada: 43.230.90.100
- US West Coast: 192.168.1.100 (example IP)
- US East Coast: 192.168.1.101 (example IP)

Europe Nodes:
- London, UK: 192.168.1.102 (example IP)
- Frankfurt, Germany: 192.168.1.103 (example IP)

Asia-Pacific Nodes:
- Tokyo, Japan: 192.168.1.104 (example IP)
- Seoul, South Korea: 192.168.1.105 (example IP)

Acceleration Channel Design

The architecture includes multi-layer acceleration channels to ensure efficient and reliable email delivery:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    
    A@{ shape: rounded, label: "Domestic Email Server" } --> B@{ shape: hex, label: "Domestic North Node" }
    A --> C@{ shape: hex, label: "Domestic South Node" }
    A --> D@{ shape: hex, label: "Third-party VPS Node" }
    B --> E@{ shape: hex, label: "Core Switching Layer" }
    C --> E
    D --> E
    E --> F@{ shape: hex, label: "North America MX Proxy" }
    E --> G@{ shape: hex, label: "Europe MX Proxy" }
    E --> H@{ shape: hex, label: "Asia-Pacific MX Proxy" }
    F --> I@{ shape: rounded, label: "Overseas Email Server" }
    G --> I
    H --> I
    class A,I primary
    class B,C,D,E,F,G,H network

RSDN proxy clusters are deployed in major domestic data centers to enable intelligent traffic distribution and load balancing:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    
    subgraph "Domestic RSDN Proxy Cluster"
        A@{ shape: hex, label: "Hangzhou Xiaoshan Node" } --> B@{ shape: hex, label: "Smart Delivery Network (SDN)" }
        B --> C@{ shape: hex, label: "Guangzhou Info Port Node" }
        C --> B
        B --> D@{ shape: hex, label: "Alibaba Cloud Node" }
        D --> B
    end
    
    subgraph "Overseas MX Cluster"
        B --> E@{ shape: hex, label: "North America MX Node" }
        B --> F@{ shape: hex, label: "Europe MX Node" }
        B --> G@{ shape: hex, label: "Asia-Pacific MX Node" }
    end
    
    subgraph "Email Delivery Targets"
        E --> H@{ shape: rounded, label: "Enterprise Email System A" }
        F --> H
        G --> H
        B --> I@{ shape: rounded, label: "Enterprise Email System B" }
    end
    class H,I primary
    class A,B,C,D,E,F,G network

SDN Smart Delivery Network

Rule Management (SDNSVR)

SDNSVR is the core rule management system of the smart delivery network, responsible for managing and distributing delivery rules:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    
    subgraph "SDN Rule Management"
        A@{ shape: doc, label: "SDN Rule Configuration" } --> B@{ shape: rounded, label: "SDNRule Server" }
        B --> C@{ shape: diam, label: "Rule Matching" }
        C -->|SDNLocation| D@{ shape: rounded, label: "Region Matching" }
        C -->|SDNNodeName| E@{ shape: rounded, label: "Node Name Matching" }
        C -->|Domain| F@{ shape: rounded, label: "Domain Wildcard Matching" }
    end
    
    subgraph "Rule Distribution"
        D --> G@{ shape: doc, label: "Generate Policy File tsdndispatch.so" }
        E --> G
        F --> G
        G --> H@{ shape: rounded, label: "Delivery Agent Fetch Rules" }
    end
    
    subgraph "Delivery Execution"
        H --> I@{ shape: diam, label: "Match Delivery Policy" }
        I -->|Overseas Channel| J@{ shape: rounded, label: "SDN Proxy Overseas Delivery" }
        I -->|Domestic Channel| K@{ shape: rounded, label: "Local Delivery" }
        I -->|Spam Channel| L@{ shape: rounded, label: "Spam Channel" }
    end
    class B,H,K primary
    class J network
    class A,C,D,E,F,G,I,L process

Channel Selection Strategy

The smart delivery network employs a four-tier channel mechanism, classifying deliveries based on anti-spam engine results:

  1. Normal email channel: Emails marked as normal by the anti-spam engine are delivered quickly through a dedicated channel with low latency
  2. Spam channel: Emails marked as spam by the anti-spam engine are queued for delivery through the spam channel with unstable latency
  3. Bounce channel: Non-bounce emails marked as spam by the anti-spam engine are automatically bounced after a 300-second timeout
  4. Discard channel: Bounce emails marked as spam by the anti-spam engine are automatically discarded after a 300-second timeout and 2 retries

Load Balancing Mechanism

Multiple levels of load balancing are implemented to ensure system stability:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    
    subgraph "Business Layer"
        A@{ shape: rounded, label: "Ops Management Platform" } --> B@{ shape: process, label: "Fetch Domain List" }
        C@{ shape: rounded, label: "SDNSVR Rule Server" } --> D@{ shape: process, label: "Intelligent Scheduling" }
        E@{ shape: rounded, label: "SaaS Platform MTA" } --> F@{ shape: process, label: "Anti-spam Engine" }
        G@{ shape: rounded, label: "On-premise MTA" } --> F
    end
    
    subgraph "Acceleration Layer"
        H@{ shape: hex, label: "Proxy Load Balancing" } --> I@{ shape: hex, label: "SDN Channel 01" }
        H --> J@{ shape: hex, label: "SDN Channel 02" }
        K@{ shape: hex, label: "TCP Load Balancing" } --> I
        K --> J
    end
    
    subgraph "Service Layer"
        L@{ shape: rounded, label: "MX Proxy" } --> M@{ shape: process, label: "MTA Delivery" }
        M --> F
        F -->|Result Feedback| M
        F -->|Channel Selection| I
        F -->|Channel Selection| J
    end
    
    B --> H
    D --> K
    I --> E
    J --> G
    class A,C,E,G,L primary
    class H,I,J,K network
    class B,D,F,M process

MX Proxy Layer Design and Implementation

MXPROXY Architecture

MXPROXY is the core component of overseas email delivery, responsible for coordinating the operation of each subsystem:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    
    subgraph "Service Layer"
        MXProxy@{ shape: rounded, label: "MX Proxy" } --> MTA@{ shape: process, label: "MTA Delivery Service" }
        MTA -->|Transmit Anti-spam Info| DA@{ shape: process, label: "Anti-spam Engine" }
    end
    
    subgraph "Acceleration Layer"
        ProxyLB@{ shape: hex, label: "Proxy Load Balancing" } --> RSDN01@{ shape: hex, label: "Smart Delivery Network Channel 01" }
        ProxyLB --> RSDN02@{ shape: hex, label: "Smart Delivery Network Channel 02" }
        TCPLB@{ shape: hex, label: "TCP Load Balancing" } --> RSDN01
        TCPLB --> RSDN02
    end
    
    subgraph "Business Layer"
        CmOps@{ shape: rounded, label: "Ops Management Platform" } --> getDomain@{ shape: process, label: "Fetch Domain List" }
        SDNSVR@{ shape: rounded, label: "Rule Server" } -->|Get SDN Info| TCPLB
        OperatingMTA@{ shape: rounded, label: "SaaS Platform MTA" }
        NonOperatingMTA@{ shape: rounded, label: "On-premise MTA" }
    end
    
    MXProxy -->|Pull Domain List| ProxyLB
    ProxyLB -->|Pull Domain List| getDomain
    RSDN01 -->|Select MTA by Anti-spam Tag| OperatingMTA
    RSDN02 -->|Select MTA by Anti-spam Tag| NonOperatingMTA
    MTA -->|Send Email Info| DA
    DA -->|Get Anti-spam Tag| MTA
    DA -->|Select SDN Channel| RSDN01
    DA -->|Select SDN Channel| RSDN02
    class MXProxy,CmOps,SDNSVR,OperatingMTA,NonOperatingMTA primary
    class ProxyLB,RSDN01,RSDN02,TCPLB network
    class MTA,DA,getDomain process

Configuration Management

MXPROXY configuration uses a modular design that supports multiple configuration methods:

nginx
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

user  app;
worker_processes  1;
worker_rlimit_nofile 1024;

events {
    use epoll;
    worker_connections  1024;
}

stream {
    upstream rsdnproxy_9900 {
        server 43.230.90.100:9900 max_fails=1 fail_timeout=300s;
        server 192.168.1.106:9900 max_fails=1 fail_timeout=300s;
        server 192.168.1.107:9900 max_fails=1 fail_timeout=300s;
    }
    
    upstream rsdnproxy_6677 {
        server 43.230.90.100:6677 max_fails=1 fail_timeout=300s;
        server 192.168.1.106:6677 max_fails=1 fail_timeout=300s;
        server 192.168.1.107:6677 max_fails=1 fail_timeout=300s;
    }
    
    server {
        listen 127.0.0.1:9900;
        proxy_pass rsdnproxy_9900;
    }
    
    server {
        listen 127.0.0.1:6677;
        proxy_pass rsdnproxy_6677;
    }
}

Transport Route Update Mechanism

PULL Configuration Mechanism

For email systems with multiple domains and high update frequency, a PULL configuration mechanism is used:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    classDef alert fill:#ffcdd2,stroke:#c62828,stroke-width:2px
    
    A@{ shape: process, label: "Scan Domain Config" } --> B@{ shape: process, label: "Fetch Remote Domain List" }
    B --> C@{ shape: process, label: "MD5 Checksum" }
    C --> D@{ shape: diam, label: "Content Changed?" }
    D -->|Yes| E@{ shape: process, label: "Generate LUA Parse Script" }
    E --> F@{ shape: process, label: "Parse Domain Config" }
    F --> G@{ shape: doc, label: "Generate transport File" }
    G --> H@{ shape: process, label: "Reload Configuration" }
    D -->|No| I@{ shape: rounded, label: "Keep Current Config" }
    H --> J@{ shape: process, label: "Delivery Service Update" }
    class I primary
    class A,B,C,E,F,G,H,J process
    class D alert

Auto-Update Script Features:

  1. Intelligent MD5 verification: Determines whether configuration has changed via MD5 values
  2. LUA-accelerated parsing: Uses LUA scripts to accelerate configuration file parsing
  3. Auto-reload mechanism: Automatically reloads services after configuration changes
  4. Error recovery mechanism: Supports configuration rollback and error recovery

PUSH Configuration Mechanism

For email systems with a single domain and low update frequency, a PUSH configuration mechanism is used.

Core Configuration File Example:

ini
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

[mtasvr]
LocalTransportDir="/opt/app/var/transport/"

[scanDomains]
TransportFilePath="/opt/app/var/transport"

[scanDomains/c1]
MinDomainCount="28000"
RemoteTransSet="remote:[192.168.1.100]:8025"
RemoteDomainUrl="http://192.168.1.200:9900/app/getDomainListWithMD5.jsp"
ConfigFileName="0004_c2.cf"

[scanDomains/c2]
MinDomainCount="7000"
RemoteTransSet="remote:[192.168.1.101]:8025"
RemoteDomainUrl="http://192.168.1.201/app/getDomainListWithMD5.jsp"
ConfigFileName="0005_c2.cf"

Email Delivery Flow

The entire email delivery process uses an intelligent routing strategy:

mermaid
flowchart TD
    classDef primary fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    classDef network fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    classDef process fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px
    classDef storage fill:#e8f5e9,stroke:#4caf50,stroke-width:2px
    classDef alert fill:#ffcdd2,stroke:#c62828,stroke-width:2px
    
    A@{ shape: rounded, label: "Business System Sends Email" } --> B@{ shape: process, label: "Delivery Agent Delivers" }
    B --> C@{ shape: diam, label: "Anti-spam Check" }
    C -->|Normal Email| D@{ shape: diam, label: "SDN Rule Matching" }
    C -->|Spam| E@{ shape: process, label: "Spam Processing" }
    
    D -->|Overseas Email| F@{ shape: diam, label: "Target IP Region Check" }
    D -->|Domestic Email| G@{ shape: rounded, label: "Local Delivery" }
    
    F -->|Target IP != China| H@{ shape: hex, label: "SDN Overseas Channel Round-robin Delivery" }
    F -->|Target IP = China| G
    
    H --> I@{ shape: hex, label: "SDN Proxy 1" }
    H --> J@{ shape: hex, label: "SDN Proxy 2" }
    H --> K@{ shape: hex, label: "SDN Proxy N" }
    
    I --> L@{ shape: rounded, label: "Delivery Complete" }
    J --> L
    K --> L
    G --> L
    class A,G,L primary
    class H,I,J,K network
    class B,E process
    class C,F alert

Summary and Experience

Technical Achievements

  1. Breaking through GFW interception: Effectively bypassed network restrictions through the smart delivery network’s multi-channel round-robin mechanism
  2. Improved delivery efficiency: Significantly increased email delivery speed through load balancing and intelligent routing
  3. Ensured email quality: Ensured normal emails are not affected by spam through the four-tier channel mechanism
  4. Reduced ops costs: Lowered operational complexity through automated configuration management and a unified monitoring platform

Key Technical Points

  1. Intelligent DNS resolution: Intelligently selects the optimal delivery path based on recipient domain
  2. Multi-level load balancing: Implements dual load balancing at both TCP and application layers
  3. Rule engine: Flexible SDN rule management supporting region, node, domain, and other matching conditions
  4. Automated operations: Automatic update and reload mechanisms for configuration files

Practical Application Results

Through this smart delivery network architecture, we achieved:

  • Reduced delivery latency: Overseas email delivery latency reduced from an average of 30 seconds to under 5 seconds
  • Improved delivery success rate: Increased from 85% to over 98%
  • Enhanced system stability: MTBF (Mean Time Between Failures) increased from 72 hours to 720 hours
  • Improved ops efficiency: Configuration update time reduced from hours to minutes

Future Optimization Directions

  1. Machine learning optimization: Introduce ML algorithms to predict optimal delivery paths based on historical data
  2. Intelligent fault diagnosis: AI-based fault prediction and automatic repair mechanisms
  3. Edge computing: Deploy more intelligent processing capabilities at edge nodes to reduce central node pressure
  4. Security hardening: Enhance anti-spam engine capabilities to improve email security

This smart delivery network architecture provides a complete solution for globalized email delivery, solving many pain points of traditional email delivery through technological innovation and providing reliable technical support for enterprise globalization.