Hybrid Cloud Cross-Region Monitoring System Governance: Autonomous + Unified Dual-Core Architecture Practice

January 10, 2022 Architecture Hybrid Cloud, Monitoring, Architecture 1259 words 6 min read

In the context of global business expansion and large-scale hybrid cloud deployment, cross-IDC, cross-border, multi-cloud heterogeneous monitoring governance has become a core challenge for stability assurance. Traditional monitoring solutions either rely on expensive dedicated line upgrades that intrude on business architecture, or cannot balance node autonomy with global unification. Meanwhile, as a non-revenue infrastructure, the monitoring system must strictly control resource usage without allowing capability degradation.

This article breaks down a practical cross-region monitoring system governance solution from a real internet company, explaining how to achieve elastic scaling, cross-border coverage, node autonomy, and data unification for the monitoring system without modifying business architecture or incurring business cross-domain costs.

Governance Background and Core Pain Points

As businesses deploy globally across multiple locations, monitoring systems face three critical problems:

Cross-domain management difficulty: No unified monitoring entry for hybrid cloud/transnational nodes, severe multi-cloud fragmentation and data silos
High solution cost: Mainstream industry solutions rely on VPN dedicated line upgrades, high investment, intrusive to business stability architecture
Strong resource constraints: Monitoring systems must strictly control network I/O and computing resources while maintaining monitoring capability
Public network risks: Public network transmission has jitter and security issues, distributed nodes lack unified management

Governance Core Objectives

Possess elastic scaling capability, adaptable to cross-IDC and cross-border deployment
Achieve node autonomy + global unification, single point failure does not affect the entire domain
Zero intrusion into business architecture, no consumption of business cross-domain connectivity costs
Strict resource control, monitoring service without degradation, security compliance

Core Technical Solution Selection

To address these pain points, the solution adopts a public network Mesh + zero-trust network + plug-and-play components three-in-one design, balancing security, performance, and scalability:

Public network Mesh capability: Based on Istio Envoy + Mosn, build a public network service mesh to replace dedicated lines for cross-domain management
Zero-trust network architecture: Consul manages ACL, Token, encryption policies uniformly, ensuring public network transmission security
Plug-and-play scaling: Modular plug-and-play expansion, adaptable to fast integration in heterogeneous environments
Public network performance optimization: TCP BBR algorithm reduces network jitter, Mesh layer implements circuit breaking/degradation
Unified data governance: Thanos cluster enables cross-node data aggregation, storage, and query

Layered Architecture Details (with Mermaid Diagrams)

Public Network Mesh Layer (Cross-Domain Connectivity Core)

All regional nodes communicate securely through port 59080/59443, Envoy handles network proxy, Consul manages policies, Mosn manages transmission rules — no dependency on business VPN dedicated lines.

mermaid
graph LR
    CN1@{ shape: rounded, label: "Domestic Cloud Node 1" } -->|Port 59080| M@{ shape: hex, label: "Envoy+Mosn Mesh Network" }
    CN2@{ shape: rounded, label: "Domestic Cloud Node 2" } -->|Port 59080| M
    CN3@{ shape: rounded, label: "Domestic Cloud Node 3" } -->|Port 59080| M
    FN1@{ shape: rounded, label: "International Cloud Node 1" } -->|Port 59080| M
    FN2@{ shape: rounded, label: "International Cloud Node 2" } -->|Port 59080| M
    M --> F@{ shape: hex, label: "Consul Policy Center" }
    F -->|ACL/Token/Route Sync| M
    %% Bottom optimization
    M --> BBR@{ shape: doc, label: "TCP BBR Algorithm" }
    M --> CB@{ shape: doc, label: "Circuit Breaking/Degradation" }

Each regional node is an independent autonomous unit. Even if disconnected from the primary node, it can still collect, alert, and store normally, preventing domain-wide failures.

mermaid
graph TD
    %% Collection layer
    BE@{ shape: rounded, label: "Business Exporter" } --> P@{ shape: rounded, label: "Prometheus Collection" }
    %% Data storage
    P --> TS@{ shape: cyl, label: "Thanos Sidecar" } --> D@{shape: cyl, label: "S3 Object Storage" }
    P --> TL@{ shape: cyl, label: "TSDB Local Storage" }
    %% Query layer
    P --> TQ@{ shape: rounded, label: "Thanos Query Local Query" }
    %% Alert layer
    P --> AM@{ shape: rounded, label: "Alertmanager Alerts" } --> MN@{ shape: double-circle, label: "Notification Channels" }
    %% Security proxy
    Mosn@{ shape: rounded, label: "Mosn" } --> P & TQ & AM
    Mosn --> PM@{ shape: hex, label: "Public Network Mesh Entry" }

Primary IDC Aggregation Architecture

The primary node has global data aggregation, unified alerting, and global reporting capabilities. Any autonomous node can be quickly promoted to primary node, supporting flexible traffic migration and decommissioning.

mermaid
graph TD
    %% Cross-node data ingestion
    AN1@{ shape: rounded, label: "Autonomous Node 1" } -->|Thanos Receive| MC@{ shape: rounded, label: "Primary IDC Thanos Cluster" }
    AN2@{ shape: rounded, label: "Autonomous Node 2" } -->|Thanos Receive| MC
    ANN@{ shape: rounded, label: "Autonomous Node N" } -->|Thanos Receive| MC
    %% Data processing
    MC --> TS@{ shape: rounded, label: "Thanos Store Query" }
    MC --> TC@{ shape: rounded, label: "Thanos Compact Data Compression" }
    MC --> TR@{ shape: rounded, label: "Thanos Rule Global Alerting" }
    %% Storage
    TS & TC --> S3@{shape: cyl, label: "S3 Object Storage" }
    %% Display and alerts
    TR --> AM@{ shape: rounded, label: "Alertmanager Global Alerts" }
    TS --> GF@{ shape: rounded, label: "Grafana Unified Visualization" }

Overall Cross-Region Monitoring Governance Architecture

mermaid
graph TB
    subgraph Regional Autonomous Nodes
        A1@{ shape: rounded, label: "Prometheus" } --> A2@{ shape: cyl, label: "Thanos Sidecar" }
        A1 --> A3@{ shape: rounded, label: "Alertmanager" }
        A4@{ shape: rounded, label: "Mosn/Envoy" } --> A1
    end
    subgraph Regional Autonomous Nodes
        B1@{ shape: rounded, label: "Prometheus" } --> B2@{ shape: cyl, label: "Thanos Sidecar" }
        B1 --> B3@{ shape: rounded, label: "Alertmanager" }
        B4@{ shape: rounded, label: "Mosn/Envoy" } --> B1
    end
    subgraph Public Network Mesh Control Layer
        C1@{ shape: hex, label: "Consul Policy Center" }
        C2@{ shape: doc, label: "TCP BBR + Circuit Breaking/Degradation" }
    end
    subgraph Primary IDC Aggregation Layer
        D1@{ shape: rounded, label: "Thanos Query/Receive" }
        D2@{ shape: cyl, label: "Thanos Store/Compact" }
        D3@{ shape: rounded, label: "Global Alertmanager" }
        D4@{ shape: rounded, label: "Grafana Unified View" }
    end
    A4 & B4 --> C1
    A2 & B2 --> D1
    D1 --> D2
    D2 --> D3
    D2 --> D4

Key Technical Capability Implementation

Zero-Trust Network Security

Consul manages ACL policies, token authentication, service routing uniformly
Independent certificate encryption per node, full-link encryption for cross-node transmission
Mosn controls port access, strictly limiting data read/write permissions

Public Network Performance Stability

Nodes enable TCP BBR algorithm at the OS level, reducing public network jitter impact
Envoy + Mosn implements circuit breaking, degradation, rate limiting, preventing public network anomalies from crippling monitoring
Data blocks merge and upload periodically (every 2 hours), reducing network I/O usage

Plug-and-Play Elastic Scaling

Monitoring components are modular and plug-and-play for fast onboarding of new regional nodes
Heterogeneous environments (different cloud providers, different architectures) require no modification
Nodes can independently upgrade, decommission, or switch without affecting global monitoring

Autonomous + Unified Dual Mode

Node autonomy: Local collection, local alerting, local storage — still usable when disconnected
Global unification: Primary node aggregates data, unified view, global alerting, centralized reporting

Solution Core Value

Zero business intrusion: No changes to business architecture, no consumption of business dedicated line costs, minimal modification risk
Low-cost implementation: Based on public network Mesh replacing expensive dedicated lines, investment is only 1/3 of traditional solutions
High availability guarantee: Node autonomy eliminates single points of failure, global monitoring stability improved by 90%
Elastic expansion: Plug-and-play components support rapid onboarding of global nodes, adapting to unlimited business expansion
Security compliance: Zero-trust network + full-link encryption, meeting cross-border monitoring security requirements

Summary

This cross-region monitoring governance solution is one of the best practices for monitoring architecture under hybrid cloud, global deployment. It breaks away from the traditional approach of “modifying business and investing in dedicated lines”, centering on public network Mesh + zero-trust network + Thanos data unification, perfectly balancing the four core requirements of scalability, security, cost, and availability. It achieves both global control of cross-region monitoring and independent autonomy of single nodes, providing a replicable implementation template for internet companies’ global monitoring infrastructure.