How do companies store & parse extremely large data?

Question

Hey community, I was wondering recently about how companies actually store extremely huge data like a giant array, etc. For example, I am curious to know how companies store your likes data for you? How do they store 100000s of posts I liked in the past but without a performance bottleneck when I need to load them?

How do they manage to parse so much data without any performance issue? I know they have huge servers but still how do they manage to handle this amount if data? Especially (if it an array), how do they handle writes of so many posts liked?

Answer 1

Once, same questions arise in my mind, then I asked to my teacher.He told me that, instead of using a single database they distribute the data across multiple machines.
The most important thing is NoSQL, it's a type of database which is mainly designed to handle extremely large amount of unstructured(unarranged) or structured(arranged) or semi-structured data and then told further you will learn in higher classes😅.I told you that I know, may be there some more ways...

Answer 2

Yeah I am also curiously want to know about it but I don't know how they do it🥲

Answer 3

According to my imagination :

Companies store large data as you mentioned (likes on posts) by spreading it across so many servers.

Each server acts like a separate storage space it's like books in blocks at library.

When you like a post, is goes to any of them.

To find your liked posts quickly, the system use programmed algorithm to find it easily.

When lots of people like posts at once, the system handles these updates by sharing the storage loads.

This keeps everything running smoothly without slowing down : )

I guess it work like this and I am curios to know more from other experienced people.

Answer 4

That's a far bigger question than you may realize. You're poking at architecture ;)

If you seriously want to learn more about this topic in depth, read "Designing Data Intensive Applications" and look up the things you don't know as you go.

The answer to your question is probably in the section that mentions how X (referenced by former name Twitter) architects their backend to handle it.

Answer 5

https://www.linkedin.com/pulse/evolution-data-management-luca-de-ioanna