Now that we understand RESP, and have an idea on what a parser combinator is, we'll need to understand how we can avoid copying in redis oxide. For context, I previously heap allocated almost all bytes in redis-oxide . Without digging too deep into the details, my fundamental types were defined as:

/// These types are used by state and ops to actually perform useful work. pub type Value = Vec < u8 > ; /// Key is the standard type to index our structures pub type Key = Vec < u8 > ;

Which meant my parser needed to output Vec 's, which are heap allocated. Value and Key are used almost everywhere in the application to represent almost all values in a redis command. So we need to change these types to be small, stack allocated items. No matter what direction we take, we need to play nice with tokio 's codec scheme.

Now it should be understood that tokio 's Decoder trait works as follows:

redis-oxide uses the tokio framework which provides services to listen on sockets. One of the APIs provided is the Decoder trait, which you use to carefully read bytes off a socket to produce a type. Tokio maintains a large buffer and will copy bytes received off a socket into this buffer. It will pass this buffer to the parser until the parser signals that a type can be produced. The tokio managed buffer is smart, and allows for several byte slices to be made safely ( Bytes type). The parser will cleave off enough bytes from the buffer when producing a type to allow tokio to safely copy later received bytes. We bypass lifetime issues as the Bytes type maintains reference counts, so we can pass slices up further up the application. Once redis-oxide is done with the produced types, we'll drop the slice references, and memory can be reclaimed.

So our parser will need to dance this careful dance. As described above, we can safely share byte slices of this underlying buffer using the Bytes type. So we'll redefine our fundamental types in terms of Bytes :

/// These types are used by state and ops to actually perform useful work. pub type Value = Bytes ; /// Key is the standard type to index our structures pub type Key = Bytes ;

Aside from a massive related refactoring job, we now need to just write the parser 😛.