NewStats: 3,265,530 , 8,187,099 topics. Date: Sunday, 15 June 2025 at 03:25 PM 596n6o6382y |
Backend Software Engineers, Let's Talk About System Design (11966 Views)
(1) Go Down)
SilverG33k(m): 9:26pm On Dec 05, 2020 |
Honestly, let's just catch cruise here. Now here's a quick question, you're to build nairaland backend, how would you do that? What technologies would you use, how would you handle scalability issues. Note than, the expected traffic for this system is 1k req/s just for fun tho No yabbing 8 Likes 3 Shares |
talk2hb1(m): 12:50am On Dec 06, 2020 |
Tooh, Waiting for Experience Bosses to Come Tutor us on this. 3 Likes 1 Share |
ULOPVQ: 4:51am On Dec 06, 2020 |
DC49: 6:40am On Dec 06, 2020 |
SilverG33k: In my case I would use a decoupled architecture. That is, separate the front-end from back-end and host them separately. Front end would be an Angular app hosted on firebase, while back-end would be a Django Rest API hosted on Google App Engine with auto-scaling to handle traffic surges. I'd utilize a caching system like Memcached to reduce load on the back-end server, store the generated contents like media files on a separate file storage service like Amazon S3 or Google Cloud Buckets, host the database separately on Google Cloud SQL, and finally use a CDN like Cloudflare. This setup would be expensive though, but for a platform that gets such a huge amount of requests I'm certain the owner would be able to afford it. 88 Likes 13 Shares |
DC49: 6:41am On Dec 06, 2020 |
DC49: |
Karleb(m): 7:20am On Dec 06, 2020 |
Decoupled architecture. Noted! @Mukina2 This deserves front page 1 Like |
SilverG33k(m): 5:16pm On Dec 06, 2020 |
DC49: I love this, interesting, you would definitely be needing devops. Great thinking really 1 Like |
SilverG33k(m): 5:35pm On Dec 06, 2020 |
What if you we try to reduce cost by Using just AWS, or a good old server, right, host the database on a separate server, and the other contents(images, videos, profile pictures), serve the media files with cloudflare or AWS cloudfront, then in cases where the homepage gets accessed by thousands of s, a Memcached would work, I prefer to use Redis however, both does the same thing, so instead of reading the popular topics and posts off the database, it reads from the cache, which reduces load on the database server. For analytics, i always prefer and suggest to use a nosql db like mongo db, cassandra or faunadb, anything flexible actually, obviously our core database is going to be a relational database e.g mysql, oracle so we don't want to put too much load on it, we just need to put in the critical data there, for scalability, so this means that all of those number of views, daily active s, page views, e.t.c would go to a nosql database, s logs as well, requests and responses(if you're keeping track) all of these analytical data should make use of firebase database or any nosql db you want Am i missing anything, yes, anytime i , I'd share, this is how i am going to build mine, its not a general rule that everyone must follow sha, let us understand that 18 Likes 1 Share |
SilverG33k(m): 5:38pm On Dec 06, 2020 |
talk2hb1: lmao look at this boss oh, chai Okay, Sir... How would you handle 10 backend devs working with 4 different languages |
talk2hb1(m): 5:42pm On Dec 06, 2020 |
SilverG33k:This sha want to intimidate me 10backend Dev, 4 different languages. I go dey form big boss ni ooo ![]() 2 Likes |
tensazangetsu20(m): 5:43pm On Dec 06, 2020 |
DC49:I am building something very similar. The frontend is React. The backend is WordPress. |
Fash20: 3:23am On Dec 07, 2020 |
Good... I love to learn from booses... I haven't gotten the chance to work with or learn from one directly but I love what I have read here so far
|
cixak95211: 6:58am On Dec 07, 2020 |
SilverG33k: NL say she has 2.5M , if that is anything to go by then indeed it has to be a microservice. There must be very bad guys amongst these , so an API Gateway is a must for the backend. I'll likely go with Kubernetes so it handles upscaling and downscaling for me without having me loose sleep. NGINX is very much ok as an LB for both frontend and backend, i prefer a round-robin as against weighted since the K8 engine will spin up equally weighted infra. For the DB, i'll likely go with Mongodb since it will allow me run a search engine without the need for Algolia . . oh well Algolia can come later , but at least I can do database sharding right out of the box. You dont expect me to dump 2.5M parent records which will span into hundreds of millions, imagine one has 100 posts, which is way likely to be more each week. Fact is i cannot dump those in one single DB, no maatter hwo powerful it is, It has to be redundant. Redis is needed for caching frontpage topics, so i dont have to serve it all the time, for each request. I could cache with NGinx too but i prefer redis. LRU is powerful. Choice of server infra . . a simple linux image will do, ssd based and most importantly, all servers, FE, BE and DB must be encrypted at rest. I'll use a KMS for that. I'll leave one bastion host available for maintenance work and make sure all servers only communicate via a VPC. 17 Likes 2 Shares |
SilverG33k(m): 8:02am On Dec 07, 2020 |
cixak95211: Database sharding how did I miss that... Excellent point there, very necessary, why would the DB be encrypted tho? Or what part of the DB needs to be encrypted, please share, because I am just considering the speed and complexity 1 Like |
Karleb(m): 8:10am On Dec 07, 2020 |
FE, BE, VPC, KMS What do they mean? |
Amoto05(m): 12:34pm On Dec 07, 2020 |
k
|
Dottore: 12:35pm On Dec 07, 2020 |
Ok
|
Jahzrockballer(m): 12:36pm On Dec 07, 2020 |
Just curious, why is no one talking about ruby on rails ![]() 2 Likes |
Lovelies1: 12:36pm On Dec 07, 2020 |
cinoedhunter: 12:37pm On Dec 07, 2020 |
Please what language are you guys speaking here ![]() ![]() ![]() ![]() ![]() 19 Likes 1 Share |
Olalekank(m): 12:38pm On Dec 07, 2020 |
.
|
JoeEeL(m): 12:40pm On Dec 07, 2020 |
Good. These are the kind of threads that, once in a while, should be popping up on front page and not ridiculously silly-ass posts like Regina daniels prevents Ned's 7th wife, Toke Makinwa stuns with plastic-ass ... Backend, as strictly relates to Web architecture, NodeJS is my go-to. Of course with Express as the de-facto server framework. Request / sec, as far as node is concerned is nearly 2x, response time is as much as 35% faster. Also it abstracts quirks, inconsistencies and is the perfect syntactic sugar. Looking to annex its powers to FE, one has to be thinking in of React -- which leads you to the promised land -- THE MERN stack React helps you define the UI once and for all. Then, when the state of the app changes, the UI is rebuilt to react to the change -- acting on its name. Small manageable components enable to build fairly complex apps but the sweetness of this is, you're still running on vanilla JS anyways. DOM node-hunting is kept at barest minimum; an avoidance of pain you might argue but darn! I was good at nifty DOM selection in my jQuery heydays. Anyways, all you do is maintain the state of your app (with a regular old JavaScript object) and the rest just naturally follows. 18 Likes |
cixak95212: 12:41pm On Dec 07, 2020 |
SilverG33k: 2 Likes |
ipressplay: 12:44pm On Dec 07, 2020 |
DC49: This like you said is very expensive. Why not build with Laravel, its scalable and enjoys massive . Hosting with VPS or dedicated server. Better still use AWS for hosting and different buckets for different files. My thoughts. 7 Likes |
SenorFax(m): 12:46pm On Dec 07, 2020 |
Please guys, start a new one for learners like us. I love the languages you are speaking, but I'm a newbie to software techs
4 Likes |
ElRapido: 12:47pm On Dec 07, 2020 |
I dey laugh here
|
kennyflames(m): 12:49pm On Dec 07, 2020 |
k
|
Opexzy: 12:49pm On Dec 07, 2020 |
I will come back to give my opinion...
|
benedictuyi(m): 12:52pm On Dec 07, 2020 |
Following
|
WiszyFraud: 12:52pm On Dec 07, 2020 |
![]() *Tightens seatbelt* Time to learn 1 Like 1 Share |