跳到主要內容

Distributed transactions

Table of contents [ hide ] Basic theory  CAP States that any distributed data store can provide only two of the following three guarantees. Consistency Every read receives the most recent write or an error. Availability Every request receives a (non-error) response, without the guarantee that it contains the most recent write. Partition tolerance The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes. Typical architecture of distributed systems When a network partition failure happens, it must be decided  whether to do one of the following: CP: cancel the operation and thus decrease the availability but ensure consistency AP: proceed with the operation and thus provide availability but risk inconsistency. BASE Basically-available, soft-state, eventual consistency. Base theory is the practical application of CAP theory, that is, under the premise of the existence of partitions and copies, through certain syste
張貼留言

Redis

 

Redis

To improve response time and avoid the burden on the database, caching is a commonly used tool, and Redis is the most famous and best caching tool.

Basic Operation

SRING

Save string, integer or double.

LIST

Linked list, we can prune nodes like list or queue.

SET

Like HashSet, the values ​​in the set are unique.

HASH

Often objects with multiple properties are saved.

ZSET

Same as linked hashmap, with orderliness.

Advanced Operation

Math operation.
Element operations, such as union and heteroset.
Key operations such as checking existence or getting all keys.
Key expire operation.

Backup

RDB

Snapshot of all data, fast recovery.

AOF

Operation log appending, recovery is slow.

Backup thread operations

save: the operation will stop the client threads.
bgsave: the operation add new a thread to do backup.

After redis4.0, you can use rewrite Aof with Rdb + Aof to reduce recovery time and save volume.

Usage

Single

Testing or basic use of small machines.

Master-slave

Can withstand more operations than smaller machines.

Sentinel

A service that can automatically monitor and switch between master and slave.

Cluster

It has all the functions of master-slave and sentinel, and can easily add or delete nodes.


Distributed lock

In microservices, to avoid inventory problems caused by concurrency, distributed locks are needed to ensure that inventory can be added or subtracted correctly.

Gossip protocol

https://en.wikipedia.org/wiki/Gossip_protocol

留言

張貼留言

這個網誌中的熱門文章

ShardingSphere

Table of contents [ hide ]  ShardingSphere The distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database. ShardingJDBC ShardingSphere-JDBC is a lightweight Java framework that provides additional services at Java’s JDBC layer. ShardingProxy ShardingSphere-Proxy is a transparent database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Core Concept 1. Virtual Database Provides a virtual database with sharding capabilities, allowing applications to be easily used as a single database 2. Real Database The database that stores real data in the shardingShereDatasource instance for use by ShardingSphere 3. Logic Table Tables used by the application 4. Real Table In the table that stores real data, the data structure is the same as the logical table. The application maintains the mapping between the logical table and the real table. All real tables map to ShardingSpher
張貼留言
閱讀完整內容

Program design template

Table of contents [ hide ] Designing programs is a common job for every engineer, so templates help simplify the job of designing each program. Update History Let everyone know the latest version and update information. background Write down why we need this program and what is the background for building this program. Target Target functionality The goal of the program, what function to achieve, the main module and submodule, and these modules' relationship. Target performance Specific benchmarks such as QPS or milliseconds to evaluate programs. Target architecture Stability. Readability. Maintainability. Extendability. ... Others Target Overall design Design principles and thinking Explain how and why the program was designed. Overall architecture An overall architectural picture. Dependency Module dependencies on other modules or programs. Detail design Program flow design Program flow design diagram. API design The details of the API, and how to interact with the frontend
張貼留言
閱讀完整內容

Virtual memory

Table of contents [ hide ] Virtual memory Separation of user logical memory from physical memory. To run an extremely large process. Logical address space can be much larger than physical address space. To increase CPU/resource utilization. A higher degree of multiprogramming degree. To simplify programming tasks. A free programmer from memory limitation. To run programs faster. Less I/O would be needed to load or swap. Process & virtual memory Demand paging: only bring in the page containing the first instruction. Copy-on-write: the parent and the child process share the same frames initially, and frame-copy. when a page is written. Allow both the parent and the child process to share the same frames in memory. If either process modifies a frame, only then a frame is copied. COW allows efficient process creation(e.g., fork()). Free frames are allocated from a pool of zeroed-out frames(for security reasons). The content of the frame is erased to 0 Memory-Mapped File: map a fi
張貼留言
閱讀完整內容