How JunoDB is designed to be a highly secure database

Write an engineering article with a concrete timeline in Markdown format from the following transcript.

so paper recently opens over the key value database named junodb and I spent a few days going through it to understand its features and guarantees in the series of videos I'll be going through the database and talking about the key details and design decisions they took while building this in the process we'll understand how a production grade key value store is built this is the fifth video of the series and in this one I will be talking about how junodb ensures very high security which is a must for payments platform like PayPal without compromising on the performance so let's jump right into it so becoming a better engineer is the need of the art and to help you all reach I have something that you will find amusing eye conduct so nonsense approach the these courses are designed to help you become a better engineer and Ace your career the courses will definitely reorganite your love for engineering and Spark the much needed engineering curiosity some of my most popular courses are of system design and database internals because I operated the no fluff approach my courses are enrolled by folks across all levels from SD ones to tech leads to staff to EMS to VP engineering of some of the most prominent companies out there all the details about the courses like curriculum prerequisites testimonials epic use can be found on the course pages and I have linked them in the description down below so do check them out and I cannot wait to see you all become better engineers and as your engineering career now let's go back to the video let's start with performance junodb delivers a staggering performance at scale it does most of their operations in a single digit millisecond The Benchmark results that they put out on their repository are staggering P99 latency of 4 millisecond with one proxy and three storage servers at 100 000 connections 100 000 persistent connections and 45 000 requests per second insane it's pretty awesome it's pretty awesome and I'm not even there you know if you look in the previous one we saw how Adventure High availability by doing intra cluster application and cross cluster replication across data center application you see your rights and Route should be expensive because it's going on multiple instances and then you are reading from all of them and you are picking the one with the latest value right the rights are waiting for right quorum to establish now that seems quite an expensive thing but because of golang's concurrency go routines cheap go routines they are able to shoot those rights in parallel waiting for them so like basically waiting for the majority of them to complete this is what makes a plus the connections are persistent so this is what is giving them the raw performance of 4 millisecond P99 latency of 4 millisecond at 45 000 requests per second on a three storage servers pretty interesting right this is what like what you it like we see we think how complicated would this be how slow it would make a system to be it would chop down my throughput and all we see it still guarantees to put you just need to tweak your parameters so few things that they do they keep the connections persistent number one and because of which what happens is that you are saving on your three-way handshakes every time right making them making this thing Lightning Fast plus initiating them in parallel using threads right why to do it synchronously remember this when you are building your next system that achieving distributed Quorum is little bit higher on latency side but not humongous because it takes time for us to explain it but for computers to do it it's pretty fast okay that being said let's talk about security very few people like about security but this is very important for a company like PayPal you need to understand why your database needs to provide very high level of security so Juno DB PayPal's key database used by most of their core backend services not in the source of truth but the caching and partial persistence and all it Powers some of them uh it was most of them so it needs to ensure high security for data in transit and data at risk which means whenever is data moving from one component to another component from your client to your to your back end from your backend within your backend it needs to be TLS there needs to be TLS for the transport level which means any payload that you are sending needs to be encrypted so that there is no man in the middle attack no one is able to sniff the data and extract information out of it ensuring very high level of security Security Plus when the data is stored on the disk it needs to be encrypted and stored but encryption is costly it is costly but when it is important you would leave out performance in favor of your security because security takes the utmost precedence especially from a financial platform like PayPal so let's go a little bit deeper into this now payload whatever the payload is sent from the client it is encrypted ideally it is encrypted on the client side if client does not do encryption then the proxy the June of proxy does encryption on the client's behalf but ideally client should do encryption and send it so storage engine when it stores it also encrypts the data and stores it it does not just store Rocky value data on the disk it encrypts and stores so to power this entire suit of encryption across multiple layers unit encryption keys and certificates now how do you get it there has to be a system that is issuing the certificates whenever needed and helps you rotate them because if you're just having a static certificate and if that certificate gets leaked then your entire data is gone right so which is where periodic rotation of keys and certificates is really important so which this all of this is powered by a very common system called key management system let deeper into those details because very important security so the Juno client SDK that your client uses to talk to Juno backend they talk to a key management system we'll talk about key management system in a couple of minutes now whenever your client sends the data it it can be encrypted so using this encryption key issued by key management system the data is encrypted and sent to the Juno DB backend if it is not encrypted due to any reason in a Juno proxy encrypts it for you before it leads it over there but ideally there is a TLS from your client SDK to this no matter what and it needs to be there now we know that one Juno proxy talks to another General proxy over the network this another Journal proxy is across the data center so that is also TLS encrypted that there is also transport level security put over that why because you don't want when your data is moving from one data center to another data center or synchronously you don't want men in the middle attack happening you don't want someone to sniff the data because financial data right so this is where TLS is that now on the storage side when the data is stored this is where your data is stored in an encrypted fashion so your Juno client SDK your zero proxy and your storage server uses a key management system to do this right now few key details about key management system now key management system is takes care of issuing encryption keys to whoever whoever wants it it more importantly keeps track of historical Keys as well for example let's say my storage server has stored has encrypted the data with a particular encryption key and has stored it on the disk so for anyone to read the data even a genuine request they need that key for description for a decryption of the data and then key value is extracted and then sent back to the client now because let's say it was doing it and now the key is rotated which means the key is replaced by another key so now key K2 is used for encryption of data so now when someone is reading the data from the disk if it uses key K2 it would not be able to do decrypt because the data is encrypted by key K1 so how do you handle this this is where your key management system needs to keep track of all the historical keys that exist and you need to know when you are storing the data on the disk which key did you use not actual key but the reference of that key that which key ID if you used to encrypt the data you can use the same thing to decrypt it but what key management system does for you or what you would do is when you know that the key has been rotated whenever you read the data when you found out that it was included with an old key you re-encrypt the data with the new key and write it back all of this needs to be done by your system we take security for granted but it is really critical so that you just don't go and be in like decrypt and encrypt all the data on the data layer you do it lazily whenever a data is requested and you find out that it is encrypted with an old key you decrypt it with the old key and re-encrypt it with the new key ensuring that eventually all the data would be re-encrypted with the new key if the data is not accessed it's not re-encrypted with a new key because you anyway have the old keys in your key management system so that's not a problem right but now you see why this load is CPU intensive because encryption decryption is CPU intensive and now you know why junodb chose to use golang because this is CPU intensive workload because for every keyword needs to do encryption and decryption and whatnot when this happens if you are having a single threaded thing that no matter even if you have 10 or 15 servers or 10 or 15 cores you're still using just one of them so then this would become a bottleneck now you know why the user why they did not go for a single threaded implementation they went for a golang based multi-threaded or rather go routine based implementation over here because their workload is CPU heavy although they are storing key value data but their workload is CPU heavy because of the frequent encryption and decryption of the data that they need to do because they are using it to power a financial platform this the design choice that we talked about at the first video gets answered in the fifth video now you appreciate why picking the right tool the light Technologies the right language the right architecture for your system is so important why thinking of security is so important especially for a financial platform like this we take it for granted we pick any language that we want but understanding what we want the kind of operations that we would do make the best decision depending on that rather than just going for something fancy that hey rust is in fashion let me use rust or Java is sturdy let me use Java think before you make a particular call right and this is all about security and performance I hope you found it this was five part videos it is not sure if I'm going to upload a sixth one or not but up until now this was the five part video series on Juno TV a recently open sourced key value database by PayPal you understand some really nuances of building a distributed data store Key components involved in it why they took a certain decision in a certain way it's really really fascinating out there superb yeah I hope you found it interesting hope you found it amazing that's it for this one I'll see in the next one thanks [Music]

Here's the video ⤵