Multi-instance Deployment on Apache Dubbo

Design Concept for Multi-instance Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Background

Java’s static variable capability allows binding behaviors holding object references to classes, providing significant convenience to developers. Implementation solutions for design patterns like Singleton and Factory rely on static variables. By using static variables, developers can easily access needed object information anytime, anywhere.

public class Test {
 public static Object obj;
}
Test.obj = xxx;

In the long-standing development of the Dubbo framework, static variables have been widely used, such as using a globally shared ConfigManager to store global configuration information, and ServiceRepository to store service information. This design is optimal from the perspective of centralized management of configuration or parameter acquisition. In all versions before Dubbo 2.7, runtime configuration information needed by Dubbo was accessed through global static variables, uniquely identified via the RPC service triplet (interface + version + group).

Definitions of Models and Concepts Related to Multi-Instances

Mon, 01 Jan 0001 00:00:00 +0000

Dubbo Architecture

JVM — Virtual Machine Layer
Purpose: Complete isolation between Dubbo frameworks (ports cannot be reused)

Dubbo Framework — Framework Layer
Purpose: Reuse resources that need global caching (ports, serialization, etc.)

Application — Application Layer
Purpose: Isolate information between applications, including registration center, configuration center, and metadata center

Services — Module Layer
Purpose: Provide hot loading capability, allowing isolation contexts per ClassLoader or Spring Context

Dubbo Concept Alignment

DubboBootstrap
1. Needs to separate export/refer services, ServiceInstance, Metadata/Config, etc. Client
ConfigManager
1. Needs to separate application-level configuration information from module-level configuration information
ApplicationModel
1. Actually stores application layer information, holding a reference to ConfigManager’s application-level configuration information
ConsumerModel
1. Actually stores interface information, held by ModuleModel
ProviderModel
1. Actually stores interface information, held by ModuleModel
ExtensionLoader
1. Needs to load different instance objects based on different levels
Registry
1. Application level sharing, needs to ensure that multi-instance subscriptions work properly (considering unit scenarios)
Router / Filter
1. Module level sharing
Protocol / Remoting
1. Framework level sharing, reusing IO, contributing across multiple applications
Metadata
Application level sharing, considering application-level service discovery
QoS
Framework level sharing, related to IO
Serialization
Framework level sharing, related to IO
ConfigCenter
Application level contribution
ModuleModel (new)
Actually stores module layer information, holding interface-level information
FrameworkModel (new)
Actually stores framework layer information

Configuration Storage Organization

FrameworkModel

Qos, Protocol, Remoting, Serialization, ExtensionLoader

Multiple Instances Make Source Code Development More Complex, Learn How to Correctly Extend SPI Implementations

Mon, 01 Jan 0001 00:00:00 +0000

This article provides a simple summary of the coding-related changes after the multiple-instance transformation of Dubbo 3.

Hierarchical Model

From only ApplicationModel, new ScopeModel/FrameworkModel/ModuleModel are added to express the hierarchical model of multiple instances. Each ScopeModel instance will create and bind its own important members:

ExtensionDirector
BeanFactory
ServiceRepository

ScopeModel, as the most basic model, can hold and pass in SPI/Bean/URL, etc.

SPI Extension

ExtensionScope

The SPI annotation adds the scope attribute to indicate its belonging scope. The correspondence between ExtensionScope and the hierarchical model:

Startup Process and Module Dependency

Mon, 01 Jan 0001 00:00:00 +0000

Application Startup Process
Initialize application configuration, start internal modules, and start other modules.
The application startup methods include: DubboBootstrap.start(), ApplicationModel.getDeployer().start()
Module Startup Process
Starting from ModuleDeployer.start() in the diagram above, automatically initialize application configuration, start internal modules, and then start the current module.
Module startup methods include:

Spring context loads dubbo xml configuration or annotations
Manually start the module: ModuleModel.getDeployer().start()

Service Interface API Startup
ServiceConfig.export() or ReferenceConfig.get() first automatically starts the module, and then executes export/refer service interfaces