Most databases use B+ Trees to hold the data, but it suffers from a major limitation ⚡
B-trees are designed to keep data sorted and organized, which makes them ideal for storage engines, but they become inefficient when dealing with large datasets; the two main reasons are:
-
they require frequent rebalancing in order to keep data sorted and organized. Maintaining the order is difficult as the datasets become larger.
-
they have relatively higher metadata overhead.
Always remember, there is no one magical solution, ever! The entire domain of computer science is about understanding trade-offs. You get some, you lose some.
⚡ I keep writing and sharing these engineering nuggets, so if you are keen on learning them, follow along.
ps: youtube.com/c/ArpitBhayani