My Experience With the Rust Community

Having finished a few chapters of the Rust Book and successfully written a (questionable) linked list, I found myself itching with the desire to begin a Rust project. It was going to be blockchain and use the coolest crates and hell it may even turn into something really big (all the typical over-excited, ambitious, unrealistic pre-side-project thoughts). Before diving into code, I decided to search around Github for some motivation. During this exploration, I stumbled upon the Rust Stellar-SDK project.

I decided to reach out and email the owner of the project, explaining my situation:

Hello, I am currently an intern at Stellar and beginning to learn rust. I was snooping around github and ran into your SDK which looks awesome and also looks like it is under active development. I would love to know more! Ultimately as I said, I am trying to learn rust — so I am looking for projects that would be fun to really dive into with people who would have the patience to work with rust newbies :)

Within just a few hours I received back a reply with:

suggestions for places to start

a full explanation of the project

and a very warm welcome, offering to help me in my rust journey

Fast forward a month…

At this point, I had made 5 contributions, the project was up to 6 contributors, I had begun utilizing this SDK for another side project and I even got to meet one of my fellow contributors in person. Pretty cool eh?

A Story of Community: My Most Recent PR

The story my most recent PR highlights the helpfulness of the rust community.

In the process of using the 0.1.0 release of the SDK’s client library, I realized we were missing a field, specifically the memo field of the transaction resource. So, I opened an issue and took the lead to make the fix.

An Overview:

We are building a Stellar SDK written entirely in Rust. This SDK communicates with the Horizon API server that sits on top of the stellar-core client.

Each endpoint method from the SDK receives data back from the Horizon server. When the SDK receives data, it must deserialize this data into a rust data structure — for this we utilize the wonderful serde library. These data structures are resources in the SDK. Below is the transaction resource. As you can see, this resource has many fields, however it does not have a memo field.

My job, as outlined in the issue linked above, was to add the memo field.

#[derive(Deserialize, Debug, Clone)]

pub struct Transaction {

id: String,

paging_token: String,

hash: String,

ledger: u32,

created_at: DateTime<Utc>,

source_account: String,

#[serde(deserialize_with = "deserialize::from_str")]

source_account_sequence: u64,

fee_paid: i64,

operation_count: u32,

envelope_xdr: String,

result_xdr: String,

result_meta_xdr: String,

fee_meta_xdr: String,

// INSERT MEMO HERE

}

So what is a memo?

As defined in the Stellar Docs:

optional The memo contains optional extra information. It is the responsibility of the client to interpret this value. Memos can be one of the following types:

MEMO_TEXT : A string encoded using either ASCII or UTF-8, up to 28-bytes long.

: A string encoded using either ASCII or UTF-8, up to 28-bytes long. MEMO_ID : A 64 bit unsigned integer.

: A 64 bit unsigned integer. MEMO_HASH : A 32 byte hash.

: A 32 byte hash. MEMO_RETURN : A 32 byte hash intended to be interpreted as the hash of the transaction the sender is refunding.

The Initial Plan:

After discussions with a couple other members of the project, my first thought was to use an intermediate deserialization technique. See here for an example.

So, I created an intermediate transaction, a nearly complete clone of the transaction struct above, but with:

memo_type: String,

memo: Option<String>

This seemed to be a logical solution since all memos have a type, but not every memo has a value. Unfortunately, of the memos that have a value, one type, memo id, is an integer while the rest are strings. Since rust is a strictly typed language, this complicates things.

So, to deal with the possibility of two memo values, an integer or a string, I decided to create yet another intermediate struct, an intermediate memo value. At this point, here is what I had:

Intermediate Transaction Transaction Deserialization Function Intermediate Memo Memo Deserialization Function Memo Struct Transaction Struct

That is a lot of work to just add one field isn’t it? Kind of makes you wonder:

Refactoring Begins:

Once I got a working version (and proved it worked with a series of tests) I submitted a PR explaining this monstrosity and also listing places I especially needed help with:

Through conversations on our gitter and on the PR thread, I began refactoring and immediately cut some of the obvious bloat. However, even after a couple, rounds of refactoring I was still looking at a PR that added 200+ lines of code for the addition of only a single field (not including tests).

The Rust Community Saves the Day:

Even though this code was still ugly, I was proud of the code I had written — remember I am still very new to the rust programming language and I had written all this code myself. So, I decided to post in the r/rust “What’s everyone working on this week thread.” A few days later, the creator of the serde crate, David Tolnay, reviewed my PR.

Here is an approach that avoids duplicating all the fields of Transaction by representing memo as a flattened adjacently tagged enum containing data that is deserialized as an untagged enum.

#[derive(Deserialize, Debug)]

struct Transaction {

id: String,



/* other fields */



#[serde(flatten)]

memo: Memo,

}



#[derive(Deserialize, Debug)]

#[serde(rename_all = "lowercase", tag = "memo_type", content = "memo")]

enum Memo {

Text(String),

Id(i64),

Hash(String),

Return(String),

None,

}

WOW, incredible! With the above revisions, my PR went from:

Intermediate Transaction Transaction Deserialization Function Intermediate Memo Memo Deserialization Function Memo Struct Transaction Struct

To simply:

Memo Struct Transaction Struct

Essentially, using a couple cheeky serde tricks I was able to cut my PR down to less than 50 lines (not including tests).