Most databases use B+ Trees to hold the data, but they suffer from a major limitation
B-trees are designed to keep data sorted and organized, which makes them ideal for storage engines that require predictable read latencies and range lookups, but they become inefficient when dealing with large datasets; the two main reasons are:
-
frequent rebalancing to keep data sorted and organized. Maintaining the order is difficult as the datasets become larger.
-
relatively higher metadata overhead.
Always remember, there is no one magical solution, ever! The entire domain of computer science is about understanding trade-offs. You get some, you lose some.