跳到主要內容

Install K8S on VM and spring boot integration.

Installation Environment Vmware Workstation pro  It is recommended to use the  snapshot  to store the state of each installation stage to avoid installation failures and causing the installation to start from scratch. Ubuntu 22.04 windows 11 Hardware settings  create 3 VM: 4 cores and 4G memory and 100G capacity Before installing K8s (All use the root user) set host: 192.168.47.135 master 192.168.47.131 node1 192.168.47.132 node2 set root ssh connection: sudo su - echo "PermitRootLogin yes" >> /etc/ssh/sshd_config systemctl restart sshd sudo passwd ssh-keygen for i in {master,node1,node2}; do  ssh-copy-id root@$i; done set Ipvs and conf  create conf file: for i in {master,node1,node2}; do ssh root@$i 'cat << EOF > /etc/modules-load.d/containerd.conf overlay br_netfilter EOF'; done execute conf: for i in {master,node1,node2}; do ssh root@$i 'modprobe overlay;modprobe br_netfilter;'; done create 99-kubernetes-cri.conf file: for i in {maste...

Mongo DB

Mongo DB

An open-source NoSQL file database. Due to the use of JSON document data storage structure, MongoDB has three advantages: flexibility, easy expansion, and high performance. Compared with traditional relational databases, users do not need to define the data structure in advance when using MongoDB to store data, and can freely add the Key/Value of the data.

Basic concept

SQL vs MongoDB

Database vs Database
Table vs Collection
Row vs Document
Column vs Field
Index vs Index
Primary key vs _id
View vs View
Table Joins vs $lookup

Query Syntax Example

SQL vs MongoDB

a = 1 vs {a: 1}
a<> 1 vs {a: {$ne: 1}}
a > 1 vs {a: {$gt: 1}}
a >= 1 vs {a: {$gte: 1}}
a < 1 vs {a: {$lt: 1}}
a <= 1 vs {a: {$lte: 1}}


Aggregation  Pipeline Stages

Description: Mongo DB vs SQL

Filter criteria: $match vs where
Projection: $project vs as
Left outer join: $lookup vs left outer join
Sort: $sort vs order by
Group: $group vs group by
Pagination: $skip/$limit vs limit 0,10
Expand array $unwind
Graph search: $graphLookup
Paginated search: $facet/$bucket

Index

Single Field Indexes
Compound Index
Multikey Index
Hashed Indexes
Geospatial Index
Text Indexes
Wildcard Indexes

Index attribute

Unique Indexes
Partial Indexes
Sparse Indexes
TTL Indexes
Hidden Indexes

Explain

Same as MySQL, Mongo DB also provides the same function to let us check our query model, to improve our query efficiency.

Cluster Type

Replica Set

MongoDB replication is the process of synchronizing data across multiple servers.

Replication provides redundant backup of data and stores data copies on multiple servers, improving data availability and ensuring data security.

Replication also allows you to recover data from hardware failures and service outages.

MongoDB replication requires at least two nodes. One of them is the master node, responsible for processing client requests, and the rest are slave nodes, responsible for replicating the data on the master node.

Sharding

Sharding is the process of splitting a database and spreading it across different machines. It can store more data and handle larger loads without the need for powerful servers. Cut the collection into smaller pieces out of the total data. , these blocks are dispersed into several slices, each slice only loads a part of the total data and is operated through a routing process of the mongos component that knows the correspondence between the data and the slices.

composition structure

Shard

The part of data storage mostly uses a Replica set rather than a single one.

Config Server

Save all the metadata of the cluster, including shard storage and shard router.

Mongos

The entrance of the cluster routed the request to the right shard.

Strategy

WriteConcern

Write concern describes the level of acknowledgment requested from MongoDB for write operations to a standalone mongod, replica sets, or sharded clusters. In sharded clusters, mongos instances will pass the write concern on to the shards.

ReadPreference

Read preference describes how MongoDB clients route read operations to the members of a replica set.

ReadConcern

The readConcern option allows you to control the consistency and isolation properties of the data read from replica sets and sharded clusters.

Monitor

mongostat

The mongostat  utility provides a quick overview of the status of a currently running mongod or mongos instance. Use mongostat to help identify system bottlenecks

mopngotop

mongotop provides a method to track the amount of time a ongoDB instance mongod spends reading and writing data. 
mongotop provides statistics on a per-collection level. By default, 
mongotop returns values every second.

profiler

The database profiler collects detailed information about Database Commands executed against a running mongod instance. This includes CRUD operations as well as configuration and administration commands.

The profiler writes all the data it collects to a system.profile collection, a capped collection in each profiled database. 

db.currentOp()

Returns a document that contains information on in-progress operations for the database instance.

Raft

https://en.wikipedia.org/wiki/Raft_(algorithm)

留言

這個網誌中的熱門文章

Install K8S on VM and spring boot integration.

Installation Environment Vmware Workstation pro  It is recommended to use the  snapshot  to store the state of each installation stage to avoid installation failures and causing the installation to start from scratch. Ubuntu 22.04 windows 11 Hardware settings  create 3 VM: 4 cores and 4G memory and 100G capacity Before installing K8s (All use the root user) set host: 192.168.47.135 master 192.168.47.131 node1 192.168.47.132 node2 set root ssh connection: sudo su - echo "PermitRootLogin yes" >> /etc/ssh/sshd_config systemctl restart sshd sudo passwd ssh-keygen for i in {master,node1,node2}; do  ssh-copy-id root@$i; done set Ipvs and conf  create conf file: for i in {master,node1,node2}; do ssh root@$i 'cat << EOF > /etc/modules-load.d/containerd.conf overlay br_netfilter EOF'; done execute conf: for i in {master,node1,node2}; do ssh root@$i 'modprobe overlay;modprobe br_netfilter;'; done create 99-kubernetes-cri.conf file: for i in {maste...

ShardingSphere

Table of contents [ hide ]  ShardingSphere The distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database. ShardingJDBC ShardingSphere-JDBC is a lightweight Java framework that provides additional services at Java’s JDBC layer. ShardingProxy ShardingSphere-Proxy is a transparent database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Core Concept 1. Virtual Database Provides a virtual database with sharding capabilities, allowing applications to be easily used as a single database 2. Real Database The database that stores real data in the shardingShereDatasource instance for use by ShardingSphere 3. Logic Table Tables used by the application 4. Real Table In the table that stores real data, the data structure is the same as the logical table. The application maintains the mapping between the logical table and the real table. All real tables map to ShardingS...

Kafka

Table of contents [ hide ] Kafka Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Why use a Message Queue like Kafka 1. Asynchronous The program can continue execution without waiting for I/O to complete, increasing throughput. 2. Decoupling It refers to reducing the dependencies between different parts of the system so that each component of the system can be developed, maintained, and evolved relatively independently. The main goal of decoupling is to reduce tight coupling between components to improve system flexibility, maintainability, and scalability. 3. Peak clipping Peak clipping is essential to delay user requests more, filter user access needs layer by layer, and follow the principle of "the number of requests that ultimately land on the database is as small as possible". Basic Concept 1. Client Includi...