Industrial and Commercial Bank of China

Customer Overview

As a leading global commercial bank, its business areas cover deposit and loan, payment and settlement, investment banking, etc., with assets exceeding 44 trillion yuan. It has ranked among the top banks in the world for many years.

The core business of Industrial and Commercial Bank of China needs to expand, but it faces the problems of limited scalability and high costs of traditional centralized databases and mainframes. Therefore, it is urgent to switch to the x86 platform and adopt an open-source database. However, mainstream open-source databases have challenges in terms of high availability and scalability.

Solution Design

Requirements Analysis

Core business expansion requirements: The core business of Industrial and Commercial Bank of China needs to further develop, which requires the IT system to support a larger business volume and more complex data processing.

Cost and scalability considerations: The currently used traditional centralized database and mainframe have limited scalability and high maintenance costs. To reduce costs and improve scalability, Industrial and Commercial Bank of China considers migrating to the x86 platform and introducing an open-source database.

Selection and challenges of open-source databases: Open-source databases have deficiencies in high availability and scalability, which may affect business continuity and future expansion.

Industrial and Commercial Bank of China needs to achieve cost optimization and scalability improvement of the IT system while ensuring business continuity and high availability. This requires full consideration of its performance, reliability, scalability, and cost-effectiveness when selecting and implementing a new database solution, and the problem of insufficient high availability and scalability of the open-source database needs to be solved through self-developed or purchased methods.

Technology Selection

For the problems faced by Industrial and Commercial Bank of China in the expansion of its core business, MySQL is selected as the open-source database.

MySQL serves as the decision basis for the open-source database:

  1. Wide industry application and verification: MySQL is widely used in the Internet industry, and many well-known enterprises such as Google, Taobao, Baidu, Tencent, etc. have adopted the MySQL database. This proves MySQL's ability to handle large-scale data and high-concurrency access.
  2. Open-source characteristics and cost-effectiveness: There are no commercial license fees, and at the same time, the open-source characteristics also mean that it can be customized and optimized according to its own needs.
  3. Community support and rich ecosystem: MySQL has an active open-source community and a rich ecosystem, including various tools, plug-ins, and third-party solutions. When encountering problems, it can quickly get help and use existing solutions to improve database performance and management efficiency.
  4. Version quality and stability: The version quality of MySQL is relatively stable, and its performance and stability have been widely recognized after years of iteration and optimization.

Solutions to the scalability and high availability problems of MySQL:

  1. Distributed architecture:
    • Master-slave replication: Implement read-write separation through the master-slave replication mode, where the master node handles write operations, and the slave nodes handle read operations, thereby sharing the pressure of the master node and improving read performance.
    • Sharding: Horizontally split the database into multiple shardings, and each sharding stores part of the data. This can horizontally expand the processing capacity of the database by increasing the number of shardings.
  2. High availability solution:
    • Third-party high availability solution: Such as the Action Technology CloudTree® database management platform software, this solution can automatically detect and handle master node failures, promote the slave node with the latest data to the master node, and quickly restore services.

To sum up, Industrial and Commercial Bank of China chooses MySQL as the open-source database based on its wide application verification, open-source characteristics, community support, and version quality and stability. For the deficiencies in scalability and high availability of MySQL, it can be solved through distributed architecture and high availability solutions.

Implementation Plan

Business Transformation (Oracle → MySQL)

  1. Vertical splitting of large businesses
  2. Database cluster architecture and performance design
  3. Database migration methods (tools, Full-amount of data, incremental data, job windows)
  4. Data type conversion (field mapping of data types, conversion of SQL functions, etc.)
  5. Data object transformation (tables, users, schemas, indexes, stored procedures, views, sequences, etc.)

Trial Operation of New Business

For the development of new businesses, the MySQL database is adopted in the design stage, and business development is carried out according to the MySQL specifications. Finally, the business goes online, and the MySQL business architecture and development capabilities are reserved.

Implementation and Execution

  1. Unified operation for MySQL operation and maintenance management:
    • Take over some existing database instances.
    • Install database instances according to new standards.
    • Provide database monitoring and alarming.
    • Provide distributed database middleware functions.
    • Provide MySQL instance management functions.
  2. Provide technical consulting for distributed business transformation:
    • Detailed explanation of splitting algorithms.
    • Migration and expansion plans.
    • Business SQL transformation to adapt to distribution.
    • Optimization of complex business SQL to adapt to distributed query plans.
    • High availability schemes for DBLE and MySQL.
  3. Connect with ICBC alarm and other systems.
    • The ICBC alarm system has corresponding alarm specifications, including alarm scenarios, alarm levels, and alarm emergency measures.
  4. Sort out and transform the database operation and maintenance specification baseline within the bank to improve the stability of MySQL.
    • Provide MySQL standard installation specifications.
      • Directory planning.
      • MySQL replication specifications.
      • MySQL configuration specifications.
      • MySQL user specifications.
    • Provide a standard system for MySQL service management.
    • Provide a standard system for MySQL high availability management.
  5. Adapt to ICBC's high availability requirements for five-level disaster recovery:
    • Disaster recovery levels:
      • Five-level disaster recovery: three parks, one master and four slaves, three semi-synchronous replication slaves and one asynchronous replication slave.
      • Four-level disaster recovery: three parks, one master and three slaves, three semi-synchronous replication slaves.
      • Three-level disaster recovery: two parks, one master and two slaves, one semi-synchronous replication slave and one asynchronous replication slave.
    • Disaster recovery scenarios: completely manual, automatic within the same park, manual across parks, automatic across parks.
    • Switching requirements: no data loss, customizable switching priority conditions, automatic switching of database access, customizable SLA agreements.

Achievements and Benefits

Through the Action Technology CloudTree® database management platform software, the unified management of MySQL master-slave clusters, intra-city disaster recovery clusters, and remote disaster recovery clusters has been realized, achieving machine-level, computer room-level, and regional-level disaster recovery protection. The intra-city switching has achieved RPO = 0 and RTO in seconds, meeting the business high availability and disaster recovery requirements, and stably supporting the operation of more than 10 sets of businesses. At the same time, the read-write separation architecture is adopted to make full use of the standby machine resources, improve the resource utilization rate, and significantly improve the system throughput.

  • Significant reduction in host pressure, decreasing year by year
  • Zero growth in the use of commercial databases
  • More than 120 applications, over 20,000 nodes
  • Involving various scenarios such as host's offline from the platform, Oracle migration, and application reconstruction
  • Supporting the core business with an average daily transaction volume of up to 700 million, and peak periods of special holidays such as Double 11 and Spring Festival red envelopes (up to 500,000 TPS)
  • Two places and three centers architecture, intra-city RPO = 0, RTO < 60S