What differentiates nbgallery from other similar products or projects?

We’re certainly not the first to go down this path; however, our internal compute environment presented some unique challenges that required a slightly different approach.

While there are some exciting projects that achieve many of our same overarching goals, the challenge came in attempting to integrate those products into our enterprise data security and compliance frameworks. As you can imagine, our organization deals with a lot of sensitive data and has a strict “need-to-know” framework, meaning that our users should not be allowed to execute notebooks or access data for which they don’t have sufficient clearance or authorization. Additionally we have strict restrictions on co-habitation of analytic input and output data, so nbgallery must prevent the sharing of a notebook’s inputs and outputs. These requirements ruled out any public SAAS platforms and made it difficult to integrate any off-the-shelf product.

The “Run in Jupyter” button connects to a separate Jupyter instance. Our internal version of nbgallery connects to our containerized compute environment, but you can customize this to point to a single notebook server.

nbgallery also allows for a Jupyter execution environment that is independent from the notebook sharing platform. For example, the way we’ve instrumented our user’s Jupyter instances is to use an ephemeral (i.e. short-lived) personalized compute environment. This helps to ensure that while notebooks can be shared widely on the nbgallery server, they are executed in enclaves that maintain data security policy and protections. To prevent a notebook’s output from leaving that enclave, all notebook output is stripped before being saved back to the nbgallery server.

As an open source project from the government, what challenges did your project deal with during development?

We built nbgallery with the intent to release it to the open source community. During its development we made sure it was possible for organizations to adapt it to their own needs, since we ourselves needed customized capabilities to address many of our unique requirements. Our solution was an extension framework for nbgallery that allows for customized plug-ins to support an organization’s unique requirements.

As one simple example, we developed and use an nbgallery extension that requires users to include control markings for each notebook, which can later be used for more fine-grained security access. We also support multiple authentication methods; our internal deployment of nbgallery integrates with our enterprise user authentication service, whereas the open source release available to the public includes support for more standard username/password and OAuth-based authentication.

What features should the larger Jupyter community know about?

Because of the ephemeral execution environment that I previously discussed, the time and speed it takes to install Jupyter was a very important factor for us. To address this, we developed a minimal Jupyter Docker image (<250MBs). The minimal image is based on Alpine Linux and offers a dozen language kernels, most of which are installed dynamically when the user tries to open a new notebook in that language.