Table of contents [ hide ] Basic theory CAP States that any distributed data store can provide only two of the following three guarantees. Consistency Every read receives the most recent write or an error. Availability Every request receives a (non-error) response, without the guarantee that it contains the most recent write. Partition tolerance The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes. Typical architecture of distributed systems When a network partition failure happens, it must be decided whether to do one of the following: CP: cancel the operation and thus decrease the availability but ensure consistency AP: proceed with the operation and thus provide availability but risk inconsistency. BASE Basically-available, soft-state, eventual consistency. Base theory is the practical application of CAP theory, that is, under the premise of the existence of partitions and copies, through certain syste
ShardingSphere
The distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
ShardingJDBC
ShardingSphere-JDBC is a lightweight Java framework that provides additional services at Java’s JDBC layer.
ShardingProxy
ShardingSphere-Proxy is a transparent database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages.
Core Concept
1. Virtual Database
Provides a virtual database with sharding capabilities, allowing applications to be easily used as a single database
2. Real Database
The database that stores real data in the shardingShereDatasource instance for use by ShardingSphere
3. Logic Table
Tables used by the application
4. Real Table
In the table that stores real data, the data structure is the same as the logical table. The application maintains the mapping between the logical table and the real table. All real tables map to ShardingSphere's virtual tables.
5. Distributed primary key algorithm
To create logical table primary keys, ShardingSphere integrates a variety of distributed primary key algorithms and supports the application of custom primary key algorithms.
6. Sharding Strategy
Including the shard key and shard algorithm, the shard key is the vital part of the horizontal shard, without the shard key the ShardingSphere will scan all the tables and lower the performance.
The shard algorithm will find the mapping of the real table by the shard key
Shard Algorithm
1. Inline
2. Standard
3. Complex_inline
4. CLASS_BASED, Custom shard algorithm
5. Hint_inline
The core module of database adaptors
1. SQL Parser
It is divided into the lexical parser and syntactic parser. SQL is first split into indivisible words through a lexical parser.
The syntactic parser is then used to analyze SQL and ultimately extract the parsing context, which can include tables, options, ordering items, grouping items, aggregation functions, pagination information, query conditions, and placeholders that may be modified.
2. SQL Route
The sharding strategy configured by the user is matched according to the parsing context and the routing path is generated. Currently, sharding routers and broadcast routers are supported.
3. SQL Rewriter
Rewrite SQL into statements that can be executed correctly in a real database. SQL rewriting is divided into rewriting for correctness and rewriting for optimization.
4. SQL Executor
It executes asynchronously through a multithreaded executor, focusing more on database connection and memory usage.
5. Result Merger
It merges multiple execution result sets to achieve output through the unified JDBC interface. The resulting merger includes the stream merger, memory merger, and appended merger using decorator mode.
Distributed transactions
ShardingSphere provides a variety of distributed transactions such as XA and SEATA.
留言
張貼留言