DevOpsDocs Help

Backend Structure & Servers Configurations

Deployment Architecture

Meeting on 2024-03-03 11:00:00 Participants:
Masoud, Mohammad, Hosein, Mahdi

Brain Storm:

  • We must use CDN for DNS level load-balancing

  • Ingress must have xN replicas, nodePort in multiNode

  • "Storage" app svc and storage like app svcs must be stateless with using DBs

  • App stateless services must have multi instances (minimum 2)

  • Separate client side request from web-socket and using SSE concept for sending server events to client;

    • Client sends HTTP requests

    • Server sends web-socket messages(events)

  • Removing kafka from communication level of services

    • Using gRPC for communication

  • We can go on some approaches on managing and maintaining stateful services;

    • Dedicated cluster

    • In K8s with CSI

    • DBaaS

    • The goal is using Dedicated cluster or DBaaS approach but for now we use In K8s with CSI approach

  • We must remove using Redis as a DB in our backend services and replace that with a persistent DB

    • Remove saving states in Redis on Gateway service

    • Remove saving message-view-count in Redis on message service

Diagram

Current/Old Architecture

Communicates with
Communicates with
Rest API
Communicates with
Includes
Includes
Includes
Redis as DB Includes
Kafka
Microservices
Ingress
API + SSE Gateway
StatefulSet Services
Postgres
MongoDB
Elasticsearch
Redis

New Architecture

DNS level load balancing
Server - Sent Events
Communicates with
Communicates with
Communicates with
Communicates with
Includes
Includes
Includes
CDN
Ingress
Web-Socket API Gateway
Service Mesh
Microservices
Rest API Gateway
StatefulSet Services
Postgres
MongoDB
Elasticsearch

Servers Hardware Configurations

According to our monitoring graphs for 1K user we have below table for our needs:

App Server

HW\Users

1K

30K

CPU

8

240

RAM

16

512

Bandwidth

--

10G

Storage

--

2000

Notes:

  • DEDICATED SERVER

  • RAID-1 Hardware level configuration

  • CPU with high rate single thread performance

  • Good Iran IP! like 5.160.146.*

DB Server

HW\Users

1K

30K

30K + Elasticsearch

CPU

1

32

64

RAM

4

128

256

Bandwidth

--

--

10G

Storage

--

--

2000

Notes:

  • DEDICATED SERVER

  • RAID-1 Hardware level configuration

  • CPU with high rate single thread performance

Media Server

Old Architecture (WebRTC)

HW\Users

1K x1

30K x1

30K x2

CPU

8

240

360

RAM

4

120

180

Bandwidth

2.5

75

150

New Architecture (WebRTC + RTMP + CDN)

HW\Users

30K

CPU

100

RAM

120

Bandwidth

+5G

Storage

1000

CDN

--

Notes

  • Group calls must have limitation on participants count (e.g. <= 20)

Monitoring

This server includes logging, tracing, metrics, analytics, sentry infra data.

HW\Users

30K

CPU

64

RAM

128

Storage

2000

General Registry

Docker (Registry and cache layers), HELM, NPM, Gitlab Runners and etc.

HW\Users

30K

CPU

32

RAM

64

Bandwidth

--

Storage

1000

Notes:

  • VM in out-of-country or with active proxy

Test

Test Server

HW\Users

30K

CPU

150

RAM

300

Bandwidth

10G

Storage

1000

Notes:

  • DEDICATED SERVER

  • 10G network port

  • RAID-1 Hardware level configuration

  • CPU with high rate single thread performance

Test Client

HW\Users

30K

CPU

64

RAM

128

Bandwidth

10G

Storage

300

Notes:

  • We need minimum 2 servers for test client

  • Can be VMs but on different servers and different network (we need 2x 10G ports)

Last modified: 07 March 2024