A few days ago, I had the need to create an async pipeline that consisted of actions (methods) composing over each other. I also wanted the construction of this pipeline to not generate any side effects, including exceptions, object state mutations, IO operations, etc. I had the need for function composition but also wanted Monadic Composition (Kleisli Composition), and did not want to use Task.Run, I do not like doing that in ASP.NET applications.

Having worked in JavaScript with the IO Monad, I decided to do something similar in C#, something that fulfilled my needs. I came up with an initial implementation, and I called it AsyncLazyPipeline. I am not very good at naming things, so feel free to give me your suggestion if you can think of a better name. I did some research to see if there was anything that I could use, and what I found was an implementation that mixed Lazy<T> and Tasks but that on one hand, used Task.Run, and on the other hand, did not fulfill all of my needs expressed in the paragraph above. You can find AsyncLazyPipeline.cs in here https://gist.github.com/ericrey85/da9671a22234ef981e5ee3653face4af#file-asynclazypipeline-cs.

To show how to use this class, I created a POC that consists of a simple domain class (Item), an application Service (ItemService) and a few more complementary classes that we will see in this article. Make sure to have the gist link open in a browser before continuing.

AsyncLazyPipeline member description:

Expression: This property will hold the composition of all the actions (methods) passed into the pipeline. You can think of it this way: given two functions/methods g and f, and an initial value x, the composition in the form f(g(x)) would mean executing g(x), and passing the result out of that into f (executing it too). In JavaScript terms it would also be something like pipe(g, f)(x), or compose(f, g)(x). This Expression property will represent that composition, the pipeline I am talking about. It will be clearer once you see the code examples.

Constructor: You do not really need to call it explicitly, it is being using by CreatePipeLine to create the pipeline.

Flatten: Executes the delegate in the Expression property giving the consumer a Task<T> that represent the async pipeline at this moment.

Select: There are two of them. They allow you to pass delegates that you will pipe at the end of the current pipeline, meaning, delegates that will execute after the ones that have already been queued up.

SelectMany: You will also see two of these methods. One of them, the one with a single parameter, is the equivalent of the function Functional programmers know as bind/chain/flatMap. It allows you to do something similar to one of the Select methods, but the signature of the delegate it receives as a parameter is different, this delegate returns an instance of AsyncLazyPipeline. The last SelectMany, the one that takes two parameters, is the one that allows LINQ Query syntax in C#, which in the functional programming world is known as Monadic/Kleisli Composition. Again, it will become clearer once you see the code.

There are two interfaces that are added just to show you how the creator of the pipeline (ItemService) could interact with other classes. One of them (IOperations) has methods that return Task<T> (in production code these methods would probably belong to some Repository, Façade, IO performing class, etc). The other interface (IFinalNoteAppender) has a method that returns AsyncLazyPipeline<TSource> and it is there to show you how you can do Monadic Composition. Other file you will see on the gist URL is ObjectExtensions, it is just there as a shorter way to convert any object into a Task<T> that wraps the object. This will come in handy when composing non async delegates.

I also wanted to mention that inside AsyncLazyPipeline.cs there is another class, CreatePipeLine. This one is a simple static class that serves as a factory to create AsyncLazyPipeline<TSource> instances without having to deal with the generic parameter TSource.

ItemService:

public class ItemService

{

private IFinalNoteAppender NoteAppender { get; }

private IOperations Operations { get; }



public ItemService(IOperations operations,

IFinalNoteAppender noteAppender)

{

NoteAppender = noteAppender;

Operations = operations;

} public Task<Item> AddNotesToItem()

{

var combineNotes = CreatePipeLine

.With(Operations.GetFirstFileNote)

.Select(Operations.CombineWithSecondFileNote); var appendLastNote =

from notes in combineNotes

from finalNote in NoteAppender.AppendFinalText(notes)

select finalNote; var combinedOperations = appendLastNote

.Select(new Item().AddNotes); return combinedOperations.Flatten();

}

}

Look at the only method this class has, AddNotesToItem. The first step is to get a note from a file, and then combine that step with one that gets another note from a second file and appends those two notes together. The variable combineNotes is of type AsyncLazyPipeline<string>, meaning that if you were to execute the pipeline at this point, you would get a string. Next, you can see the Monadic Composition (LINQ Query syntax) that I have been talking about. It is like you are doing it against IEnumberable<T>, except that you are not, you are doing it against AsyncLazyPipeline<TSource>.

Next, appendLastNote represents the operation of taking the first two notes that have been appended to each other and adding a final note to them. Look at the signature of NoteAppender.AppendFinalText:

AsyncLazyPipeline<string> AppendFinalText(string value)

This means that the pipeline, at some point, will execute this method and will pass into it the value stored in notes (in the LINQ Query). We then use the version of the Select method that does not take an async delegate, but a sync one, to create an Item and add the now processed notes to it. The variable combinedOperations represents everything that needs to happen when the consumer of this pipeline decides to execute it. The last bit of code (the call to Flatten) makes sure that this consumer has access to a Task<Item>, instead of Func<Task<Item>>, although if you like it better (or better suits your scenario), you could return the later instead of the former.

Assuming you have a console app, or an ASP.NET Core controller, or any other caller consuming ItemService, you could have code like this (simplified for demo purposes, and should really use construction injection in Production):

var service = new ItemService(new Operations(),

new FinalNoteAppender()); var pipeline = service.AddNotesToItem();

After doing this, the whole pipeline will have been built, but nothing will have happened yet. No IO to read the notes from the files, no exceptions, not even the Item object will have been created. The whole pipeline was built and there is no change to any system state.

This pipeline’s construction is pure, the AddNotesToItem method is pure. You can call it as many times as you desire and you will not observe side effects or different behavior. This is great news for unit tests, and for everything else that pure functions/methods are good for, which I would need more than an article to talk about, but at the very least I can say they are at the core of Functional Programming and its benefits.

Now, when you are ready to roll, when you are ready to unleash your code and give your system the reputation it deserves in the world, you can simply:

var item = await pipeline;

This line will have the pipeline execute every operation that made its way into it. It will also blow up if any exceptions are thrown in any of those methods that were composed, so make sure you keep that in mind.

Note that when using Select and the first SelectMany method (the one taking one parameter), only one value will be passed into the passed in delegate during the pipeline execution. This is something you may want to consider in your design. Although the SelectMany overload that allows for Monadic Composition allows you to use in LINQ Queries any AsyncLazyPipeline<TSource> returning method, no matter how many parameters it has.

One final word; the approach presented in this article can also provide you with great reusability. AsyncLazyPipeline<TSource> not only allows you to compose methods, it also allows you to compose AsyncLazyPipeline<TSource> instances. This means that you can create sub-pipelines that can be reused and combined at your will to form more complex pipelines.

Happy Coding!