跳到主要內容

Distributed transactions

Table of contents [ hide ] Basic theory  CAP States that any distributed data store can provide only two of the following three guarantees. Consistency Every read receives the most recent write or an error. Availability Every request receives a (non-error) response, without the guarantee that it contains the most recent write. Partition tolerance The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes. Typical architecture of distributed systems When a network partition failure happens, it must be decided  whether to do one of the following: CP: cancel the operation and thus decrease the availability but ensure consistency AP: proceed with the operation and thus provide availability but risk inconsistency. BASE Basically-available, soft-state, eventual consistency. Base theory is the practical application of CAP theory, that is, under the premise of the existence of partitions and copies, through certain syste

Program design template

Designing programs is a common job for every engineer, so templates help simplify the job of designing each program.

Update History

Let everyone know the latest version and update information.

background

Write down why we need this program and what is the background for building this program.

Target

Target functionality

The goal of the program, what function to achieve, the main module and submodule, and these modules' relationship.

Target performance

Specific benchmarks such as QPS or milliseconds to evaluate programs.

Target architecture

Stability.
Readability.
Maintainability.
Extendability.
...

Others Target

Overall design

Design principles and thinking

Explain how and why the program was designed.

Overall architecture

An overall architectural picture.

Dependency

Module dependencies on other modules or programs.

Detail design

Program flow design

Program flow design diagram.

API design

The details of the API, and how to interact with the frontend or backend or modules.

Database design

Concrete SQL syntax.

Implement detail

Explain details not included in the overall design diagram or flowchart.

Compatible design.

Compatible design explanation

Compatible versions between modules, especially whether the client version and server version are compatible.

version dependency

List all version dependencies.

Event Tracking Design

Implementation of program event tracking, how it is collected, and benefits.

Test

The test range

Describe the scope of testing, how many modules, and whether stress testing or stability testing is required.

Test plan

Concrete test cases, including different environments and each test step.

performance test

Test programs under high pressure with large volumes of requests or data.

Online Planning and Risk Assessment

Online Planning

The online version and upgrade steps.

Risk Assessment

If there is a problem with the upgrade, how to deal with it or what are the overall rollback steps.

Unsolved problems

Leaves unsolved or needs discussion questions here.

Schedule and Manpower

attachments

留言

這個網誌中的熱門文章

ShardingSphere

Table of contents [ hide ]  ShardingSphere The distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database. ShardingJDBC ShardingSphere-JDBC is a lightweight Java framework that provides additional services at Java’s JDBC layer. ShardingProxy ShardingSphere-Proxy is a transparent database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Core Concept 1. Virtual Database Provides a virtual database with sharding capabilities, allowing applications to be easily used as a single database 2. Real Database The database that stores real data in the shardingShereDatasource instance for use by ShardingSphere 3. Logic Table Tables used by the application 4. Real Table In the table that stores real data, the data structure is the same as the logical table. The application maintains the mapping between the logical table and the real table. All real tables map to ShardingSpher

Virtual memory

Table of contents [ hide ] Virtual memory Separation of user logical memory from physical memory. To run an extremely large process. Logical address space can be much larger than physical address space. To increase CPU/resource utilization. A higher degree of multiprogramming degree. To simplify programming tasks. A free programmer from memory limitation. To run programs faster. Less I/O would be needed to load or swap. Process & virtual memory Demand paging: only bring in the page containing the first instruction. Copy-on-write: the parent and the child process share the same frames initially, and frame-copy. when a page is written. Allow both the parent and the child process to share the same frames in memory. If either process modifies a frame, only then a frame is copied. COW allows efficient process creation(e.g., fork()). Free frames are allocated from a pool of zeroed-out frames(for security reasons). The content of the frame is erased to 0 Memory-Mapped File: map a fi