跳到主要內容

Install K8S on VM and spring boot integration.

Installation Environment Vmware Workstation pro  It is recommended to use the  snapshot  to store the state of each installation stage to avoid installation failures and causing the installation to start from scratch. Ubuntu 22.04 windows 11 Hardware settings  create 3 VM: 4 cores and 4G memory and 100G capacity Before installing K8s (All use the root user) set host: 192.168.47.135 master 192.168.47.131 node1 192.168.47.132 node2 set root ssh connection: sudo su - echo "PermitRootLogin yes" >> /etc/ssh/sshd_config systemctl restart sshd sudo passwd ssh-keygen for i in {master,node1,node2}; do  ssh-copy-id root@$i; done set Ipvs and conf  create conf file: for i in {master,node1,node2}; do ssh root@$i 'cat << EOF > /etc/modules-load.d/containerd.conf overlay br_netfilter EOF'; done execute conf: for i in {master,node1,node2}; do ssh root@$i 'modprobe overlay;modprobe br_netfilter;'; done create 99-kubernetes-cri.conf file: for i in {maste...
張貼留言

Mongo DB

Mongo DB

An open-source NoSQL file database. Due to the use of JSON document data storage structure, MongoDB has three advantages: flexibility, easy expansion, and high performance. Compared with traditional relational databases, users do not need to define the data structure in advance when using MongoDB to store data, and can freely add the Key/Value of the data.

Basic concept

SQL vs MongoDB

Database vs Database
Table vs Collection
Row vs Document
Column vs Field
Index vs Index
Primary key vs _id
View vs View
Table Joins vs $lookup

Query Syntax Example

SQL vs MongoDB

a = 1 vs {a: 1}
a<> 1 vs {a: {$ne: 1}}
a > 1 vs {a: {$gt: 1}}
a >= 1 vs {a: {$gte: 1}}
a < 1 vs {a: {$lt: 1}}
a <= 1 vs {a: {$lte: 1}}


Aggregation  Pipeline Stages

Description: Mongo DB vs SQL

Filter criteria: $match vs where
Projection: $project vs as
Left outer join: $lookup vs left outer join
Sort: $sort vs order by
Group: $group vs group by
Pagination: $skip/$limit vs limit 0,10
Expand array $unwind
Graph search: $graphLookup
Paginated search: $facet/$bucket

Index

Single Field Indexes
Compound Index
Multikey Index
Hashed Indexes
Geospatial Index
Text Indexes
Wildcard Indexes

Index attribute

Unique Indexes
Partial Indexes
Sparse Indexes
TTL Indexes
Hidden Indexes

Explain

Same as MySQL, Mongo DB also provides the same function to let us check our query model, to improve our query efficiency.

Cluster Type

Replica Set

MongoDB replication is the process of synchronizing data across multiple servers.

Replication provides redundant backup of data and stores data copies on multiple servers, improving data availability and ensuring data security.

Replication also allows you to recover data from hardware failures and service outages.

MongoDB replication requires at least two nodes. One of them is the master node, responsible for processing client requests, and the rest are slave nodes, responsible for replicating the data on the master node.

Sharding

Sharding is the process of splitting a database and spreading it across different machines. It can store more data and handle larger loads without the need for powerful servers. Cut the collection into smaller pieces out of the total data. , these blocks are dispersed into several slices, each slice only loads a part of the total data and is operated through a routing process of the mongos component that knows the correspondence between the data and the slices.

composition structure

Shard

The part of data storage mostly uses a Replica set rather than a single one.

Config Server

Save all the metadata of the cluster, including shard storage and shard router.

Mongos

The entrance of the cluster routed the request to the right shard.

Strategy

WriteConcern

Write concern describes the level of acknowledgment requested from MongoDB for write operations to a standalone mongod, replica sets, or sharded clusters. In sharded clusters, mongos instances will pass the write concern on to the shards.

ReadPreference

Read preference describes how MongoDB clients route read operations to the members of a replica set.

ReadConcern

The readConcern option allows you to control the consistency and isolation properties of the data read from replica sets and sharded clusters.

Monitor

mongostat

The mongostat  utility provides a quick overview of the status of a currently running mongod or mongos instance. Use mongostat to help identify system bottlenecks

mopngotop

mongotop provides a method to track the amount of time a ongoDB instance mongod spends reading and writing data. 
mongotop provides statistics on a per-collection level. By default, 
mongotop returns values every second.

profiler

The database profiler collects detailed information about Database Commands executed against a running mongod instance. This includes CRUD operations as well as configuration and administration commands.

The profiler writes all the data it collects to a system.profile collection, a capped collection in each profiled database. 

db.currentOp()

Returns a document that contains information on in-progress operations for the database instance.

Raft

https://en.wikipedia.org/wiki/Raft_(algorithm)

留言

張貼留言

這個網誌中的熱門文章

OS basic

Table of contents [ hide ] OS architecture The operating system architecture consists of three parts, user mode, kernel mode, and hardware. User mode is for the application to execute the user's program. Kernel mode is to control all the I/O devices and system stability. Storage device hierarchy System call The system call is a kind of software interrupt, including six categories. Process control. File management. Device management. Information maintenance. Communication. Protection. System calls use three methods to pass parameters. Registers. The table in memory. Push onto the stack. A view of operating system services reference: https://www.amazon.com/-/zh_TW/Operating-System-Concepts-Abraham-Silberschatz/dp/1119800366/ref=sr_1_1?keywords=Operating-System-Concepts&qid=1669538704&s=books&sr=1-1
張貼留言
閱讀完整內容

JVM Tools

Table of contents [ hide ]  Java has many useful tools for debugging at runtime or analyzing an application after it has crashed. Commands jps - JVM process Status Tool Same as the Linux command ps, only to find the process under the JVM. jinfo - Configuration Info for Java Show and adjust the JVM arguments. jmap - Memory Map for Java Getting a memory error dump is the most common technique used at runtime. The dump file can be used to analyze the cause of the failure. jstack - Stack trace for Java Check the thread runtime status. GUI VisualVM VisualVM is a tool that provides a visual interface for viewing detailed information about Java applications while they are running on a Java Virtual Machine (JVM). Memory Analyzer (MAT) The Eclipse Memory Analyzer is a fast and feature-rich Java heap analyzer that helps you find memory leaks and reduce memory consumption. reference: https://www.amazon.com/depth-understanding-Java-Virtual-Machine/dp/7111641248 https://en.wikipedia.org/w...
張貼留言
閱讀完整內容

GC Basic Algorithm

Theory Most GCs follow the theory of generational collection, which is based on the following two: Week generational hypothesis:  Most of the objects are short-living. Strong generational hypothesis:  The more times an object survives GC, the harder it is to die. Mark-Sweep After the marking phase has been completed all space occupied by unvisited objects is considered free and can thus be reused to allocate new objects. There may exist plenty of free regions but if no single region is large enough to accommodate the allocation, the allocation is still going to fail. Mark-Copy To avoid excessive fragmentation, it splits the space into two parts and copies the surviving objects into empty parts. The disadvantage is the need for one more memory region, which should be large enough to accommodate survived objects. Mark-Compact Like mark-copy, it doesn't require another space to copy but a more complex operation to move the object. Reachability analysis First, GC defines some spe...
張貼留言
閱讀完整內容