Yesterday while implementing features on the project I'm currently working on, I bumped into the issue - How to upload a file using RxJS? Naturally I wanted to have an upload progress indicator to accompany the file upload process. So I started to look on internet how to implemented it, without reinventing the wheel. I did not find a coherent source of truth that would provide me with idiomatic solution. This article is a compilation of know-how that I accumulated while implementing upload video feature on real world project.

RxJS

Let me give you a quick into into RxJS, what it is and why I use it. RxJS is a library for reactive programming using Observables, to make it easier to compose asynchronous or callback-based code. I use the library because it allows me to you write asynchronous code that looks completely synchronous, allows me to use Functional Programming, uses one interface to deal with various objects in JavaScript (generators, promises, observables, callbacks) and reduces very complex problems into couple of lines of code. When you write code using RxJS you're exercising Reactive Programming. I will not go any deeper here, but for anybody who want to dive into the subject more, I'd recommend reading a book RxJS in Action written by Paul Daniels and Luis Atencio.

Basic upload with RxJS

RxJS has a standard way of making HTTP requests. It wraps XHR object in browser and gives it reactive behavior. The most important thing it does for you - it allows you to cancel any pending HTTP request. That's pretty nifty. Let's dive into the code.

const { ajax } = require('rxjs/ajax'); const crypto = require('crypto'); const data = crypto.randomBytes(1000000); const formData = new FormData(); const blob = new Blob([data], { type: 'video/mp4' }); formData.append('file', blob); const request$ = ajax({ method: 'PUT', url: 'https://httpbin.org/put', body: formData, }); request$.subscribe(console.dir);

This code snippet demonstrate most primitive way how to achieve file upload using RxJS. First we generate some random binary data and assign it to variable data. Then we're using FormData Web API to construct a payload for the file upload. Then we create a request$ observable and subscribe it to the console.dir function. The request$ is lazy observable, which means it's acts as prescription how to do stuff. It's not executed until you subscribe it to the observer, in our case - console.dir. If you execute the code and open your browser console, you'll see the XHR response in there.

Upload progress with RxJS

Now let's add upload progress to our snippet.

const { ajax } = require('rxjs/ajax'); const { Subject } = require('rxjs'); const { merge } = require('rxjs/operators'); const crypto = require('crypto'); const data = crypto.randomBytes(1000000); const progressSubscriber = new Subject(); const formData = new FormData(); const blob = new Blob([data], { type: 'video/mp4' }); formData.append('file', blob); const request$ = ajax({ method: 'PUT', url: 'https://httpbin.org/put', body: formData, progressSubscriber, }); progressSubscriber.pipe(merge(request$)).subscribe(console.dir);

The implementation is simpler than you expected right? We used RxJS Subject and passed it as progressSubscriber property to the ajax function. So what is really happening here ? XHR which is used under the hood of ajax function generates a ProgressEvent while performing file upload. This ProgressEvent is pumped is the progressSubscriber which is instance of Subject, which is technically a multicast observable. Then we merge both the request$ and progressSubscriber into one stream and subscribe to it using our console.dir observer. When you execute the code and open you browser console, you'll see a couple of ProgressEvents and then the last output in the console will be the actual XHR response of the file upload.

Binary data upload with RxJS

The example above uses FormData to upload the file to the server. Sometimes you just really want to push a binary data to some arbitrary URL without using multipart/form-data. This can be achieved in RxJS using following code:

const { ajax } = require('rxjs/ajax'); const { Subject } = require('rxjs'); const { merge } = require('rxjs/operators'); const crypto = require('crypto'); const data = crypto.randomBytes(1000000); const progressSubscriber = new Subject(); const blob = new Blob([data], { type: 'video/mp4' }); const request$ = ajax({ method: 'PUT', url: 'https://httpbin.org/put', headers: { 'Content-Type': 'application/octet-stream', }, body: blob, progressSubscriber, }); progressSubscriber.pipe(merge(request$)).subscribe(console.dir);

We have to explicitly set the Content-Type header to application/octet-stream and set a body either to Blob or File instance. This code will directly pump unmodified binary data to the provided URL.

Real world example

Now that we know the theory behind the file upload using RxJS, I'll show you the code fragment from the real world application I was talking about at the beginning of this article. The application is written using React, Redux and redux-observable. redux-observable is an integration between redux and RxJS and allows you to write declarative Epics that handles your applications side effects.

export const videoUploadEpic = (action$, state$) => { const uploadPipeline$ = videoFile => { const progressSubscriber$ = new Subject().pipe(map(videoUploadProgress)); return merge( progressSubscriber$, of(videoFile).pipe( map(() => { const params = new URLSearchParams(); params.set('original_name', videoFile.name); params.set('size_in_bytes', videoFile.size); params.set('mime_type', videoFile.type); params.set('last_modified', videoFile.lastModifiedDate.toISOString()); return params; }), switchMap(params => requestRestrictedApi$(state$, { url: `/v1/videos/new?${params}`, }).pipe(pluck('response', 'data')) ), switchMap(({ upload_url: uploadUrl, filename }) => ajax({ url: uploadUrl, method: 'PUT', headers: { 'Content-Type': 'application/octet-stream', }, body: videoFile, progressSubscriber: progressSubscriber$, }).pipe( takeUntil(action$.pipe(ofType(cancelVideoUpload))), mapTo(filename) ) ), withLatestFrom(action$.pipe(ofType(videoUploadFormChange))), selectVideoTitle(state$), switchMap(([[filename, values], defaultTitle]) => requestRestrictedApi$(state$, { url: `/v1/videos`, method: 'POST', body: { filename, title: propOr(defaultTitle, 'title', values), description: propOr(defaultTitle, 'description', values), }, }) ), pluck('response'), map(uploadVideoSuccess) ) ); }; return action$.pipe( ofType(uploadVideo), pluck('payload'), switchMap(videoFile => uploadPipeline$(videoFile).pipe( catchError(error => of(uploadVideoFailure(error))) ) ) ); };

I'll explain briefly what is happening here: First we receive a redux action called uploadVideo with the payload of actual video file that the user picked by drag and dropping the video file in application. Then we ask the backend API to give us an URL where to upload a new video file. Subsequently we do the actual video upload. This Epic is generating multiple dispatches of videoUploadProgress redux actions and one dispatch of uploadVideoSuccess redux action. Upload will be cancelled when cancelVideoUpload is dispatched and intercepted by the epic. Imagine doing all this in imperative programming...I personally cannot anymore....

And that's about it folks. Hopefully this article will save you some time when facing the dilemma how to approach file uploads in RxJS. If you have any suggestions to the code I provided (I'm always happy to accept any form of feedback) ping me in the comments and we can make those code fragments better.

Like always, I end my article with the following axiom: Define your code-base as pure functions and lift them only if needed. And compose, compose, compose…