Backend Structure & Servers Configurations
Deployment Architecture
Meeting on 2024-03-03 11:00:00 Participants:
Masoud, Mohammad, Hosein, Mahdi
Brain Storm:
We must use CDN for DNS level load-balancing
Ingress must have xN replicas, nodePort in multiNode
"Storage" app svc and storage like app svcs must be stateless with using DBs
App stateless services must have multi instances (minimum 2)
Separate client side request from web-socket and using SSE concept for sending server events to client;
Client sends HTTP requests
Server sends web-socket messages(events)
Removing kafka from communication level of services
Using gRPC for communication
We can go on some approaches on managing and maintaining stateful services;
Dedicated cluster
In K8s with CSI
DBaaS
The goal is using Dedicated cluster or DBaaS approach but for now we use In K8s with CSI approach
We must remove using Redis as a DB in our backend services and replace that with a persistent DB
Remove saving states in Redis on Gateway service
Remove saving message-view-count in Redis on message service
Diagram
Current/Old Architecture
New Architecture
Servers Hardware Configurations
According to our monitoring graphs for 1K user we have below table for our needs:
App Server
HW\Users | 1K | 30K |
|---|---|---|
CPU | 8 | 240 |
RAM | 16 | 512 |
Bandwidth | -- | 10G |
Storage | -- | 2000 |
Notes:
DEDICATED SERVER
RAID-1 Hardware level configuration
CPU with high rate single thread performance
Good Iran IP! like 5.160.146.*
DB Server
HW\Users | 1K | 30K | 30K + Elasticsearch |
|---|---|---|---|
CPU | 1 | 32 | 64 |
RAM | 4 | 128 | 256 |
Bandwidth | -- | -- | 10G |
Storage | -- | -- | 2000 |
Notes:
DEDICATED SERVER
RAID-1 Hardware level configuration
CPU with high rate single thread performance
Media Server
Old Architecture (WebRTC)
HW\Users | 1K x1 | 30K x1 | 30K x2 |
|---|---|---|---|
CPU | 8 | 240 | 360 |
RAM | 4 | 120 | 180 |
Bandwidth | 2.5 | 75 | 150 |
New Architecture (WebRTC + RTMP + CDN)
HW\Users | 30K |
|---|---|
CPU | 100 |
RAM | 120 |
Bandwidth | +5G |
Storage | 1000 |
CDN | -- |
Notes
Group calls must have limitation on participants count (e.g. <= 20)
Monitoring
This server includes logging, tracing, metrics, analytics, sentry infra data.
HW\Users | 30K |
|---|---|
CPU | 64 |
RAM | 128 |
Storage | 2000 |
General Registry
Docker (Registry and cache layers), HELM, NPM, Gitlab Runners and etc.
HW\Users | 30K |
|---|---|
CPU | 32 |
RAM | 64 |
Bandwidth | -- |
Storage | 1000 |
Notes:
VM in out-of-country or with active proxy
Test
Test Server
HW\Users | 30K |
|---|---|
CPU | 150 |
RAM | 300 |
Bandwidth | 10G |
Storage | 1000 |
Notes:
DEDICATED SERVER
10G network port
RAID-1 Hardware level configuration
CPU with high rate single thread performance
Test Client
HW\Users | 30K |
|---|---|
CPU | 64 |
RAM | 128 |
Bandwidth | 10G |
Storage | 300 |
Notes:
We need minimum 2 servers for test client
Can be VMs but on different servers and different network (we need 2x 10G ports)