To prevent things from getting too confusing — Let us say you have two micro-frontends or two separate apps. They have their own Webpack build, are powered by their own servers, are completely standalone. Let's call them App1 and App2 . I want to use Nav from App1 inside App2 .

You could just take Nav, make it an npm package and install it back into the App1 and App2 , sharing solved. But you still have to re-deploy multiple apps every time you update Nav.

Working out a solution

Taking advantage of built-in code-splitting fundamentals, I wanted to dynamic import() a chunk from another build, as it’s the most logical way and Webpack can already handle importing chunks. However, you can currently only do this from its own build. Webpack doesn’t really know how to handle external imports or a chunk from a foreign bundle. How can App2 import Nav.chunk.js from App1 ?

Challenge 1: Automatic code-splitting

Magic exports

I'll need a way to flag a file as one that I intend to use in another application. Webpack needs to ensure that anything "externalized" is code split regardless of if App1 actually dynamically imports or not — in order to make it efficient to interleave, I need to import as little as possible, not main.bundle.js or vendor.chunk.js . Most things that are not code split end up in a massive chunk, so there needs to be a way to code split a file without having to force that the file to be dynamically imported, possibly changing the developer flow and introducing a very abstract rule. How do we handle this?

Using a magic export, that's how! I’m going to put /*externalize:Nav*/ which I will parse out with Webpack at build time, then handle those files in a different manner.

Magic exports could lead to possible collisions within the same build. I'm considering an alternative which would resemble an interface as you get in other languages like C. The definition would be implemented into package.json and would prevent accidentally calling two chunks the same thing

"interface": {

'./src/components/Menu.js': "Nav"

},

Leveraging splitChunks API

Automate code splitting with cacheGroups . All marked files are automatically code-split into their own cache group, regardless of sync or async imports.

Now, marked files can be recognized and automatically split into their own chunk. App2 would be able to import("someUrl/js/Nav.chunk.js") and it would exist.

Challenge 2: Hashed Module IDs

How Webpack assigns modules by default

By default, Webpack generates a module ID as a number. __webpack_modules__ contains a massive array, each module ID is a position in the array.

This will not work for interleaving, I need to know what the module is.

hashedModuleID optimization option?

Webpack has an internal hashedModuleIDs option, however, this hashing convention does not work. Webpack hashes modules paths interleaving Apps wIth DIfferent FIles STructures Would result in the same modules having different IDs

App2 uses yarn workspaces and is part of a monorepo. App1 is a normal package.

App2 imports React from ../../node_modules and App1 imports react from ../node_modules

Regardless of the versions being the same, when interleaving Nav , Webpack wouldn’t know that it already has React (../node_modules/react) because Nav would ask for a different hashed module due to the hashing depending on the file path. In such a case, App2 would download App1 's chunk which contains React (../../node_modules/react) in order to provide Nav with the module.id it's looking for.

[contenthash] for module ids

Hashing modules based on their contents ensures that I can get matching versions of dependencies across any Webpack build, regardless of its file structure. There is no code duplication because I have a reliable way to check if there is an exact copy of the dependency Nav requires across any other Webpack builds that are already loaded on the page. If it cannot find the dependency it needs, Webpack will fetch the dependency from the origin build that Nav originates from.

While hashing modules based on their contents works for dependencies, the externalized module needs a predictable, human-assigned name so that it can be referenced inside another application when interleaved. Just like magic comments on imports, I’m setting the module name via a magic export comment.

When interleaved, App2 can __webpack_require__("Nav") and it should be able to find it within __webpack_modules__ . It’s important we know the key it's stored as. If we are unable to locate the interleaved export within the Webpack manifest, we cannot call it, even if its already loaded into the manifest.

Getting a foreign chunk into a webpack manifest is only half the battle. You need to be able to call the chunk by a known reference

With these two parts combined:

1) I’m now able to check all apps in a browser and lean on any and all dependencies, regardless of which build they originated from

2) I can find the externalized chunk that was injected in the Webpack manifest and call it by name, much like how dynamic import() works

Tied together, utilities like this could be built.

const ExternalComponent = (props) => {

const {

src, module, export: exportName, cors, ...rest

} = props;

let Component = null;

const [loaded, setLoaded] = useState(false);

const importPromise = useCallback(

() => {

if (!src) return Promise.reject();

if (cors) {

return require("./corsImport").default(src);

}

return new Promise((resolve) => {

resolve(new Function(`return import("${src}")`)());

});

},

[src, cors]

);



useEffect(() => {

require("./polyfill");

if (!src) {

throw new Error(`interleaving error: ${JSON.stringify(props, null, 2)}`);

}



importPromise(src).then(() => {

const requiredComponent = __webpack_require__(module);

Component = requiredComponent.default ? requiredComponent.default : requiredComponent[exportName];

setLoaded(true);

}).catch((e) => {

throw new Error(`dynamic-import: ${e.message}`);

});

}, []);



if (!loaded) return null;

return (

<Component {...rest} />

);

};

Challenge 3: Cache Busting

We need a way to load hashed files that are cache busted. Most builds are cache-busting by hashing the bundle and chunk names, but this makes it hard to load Nav.chunk.js if its name is hashed like Nav.chunk.[contenthash].js

I chose to generate a Javascript file which could be cache busted with a query string (each time). There are no CORS issues with JavaScript files, so its easier to embed manifests across apps. You’ll need to configure CSP if you use it.

Each one has a namespace. Otherwise, the risk of collision between apps is too high.

Challenge 4: Missing Dependency Resolution

What happens if a chunk is interleaved into an app which doesn’t have one of the dependencies the chunk requires. Manifest maps, like above, don't solve dependency tracing. I need to know where the smallest file is at the origin build, containing the missing dependency

Enhancing the manifest map slightly improved reverse dependency lookups and resolution.

Finding what a module depends on within a Webpack plugin.

*There’s probably a better way to do this.

Within the emit hook, I loop over the stats graph, tracking down any externalized files, then looping over all modules in that chunk, and lastly searching for the files dependencies were emitted into.

More complex dependency and nested dependency resolution need to be tested. Cases like externalized components loading other externalized components need to be tested still.

Challenge 5: Tree-shaking and dead code elimination

The Next.js team pointed me to something I had not yet tested, scope-hoisting, tree-shaking, and missing exports.

Unfortunately, Webpack hashes module.ids before tree-shaking, so my [contenthash] modules were not reliable. The hash was based on the contents of what was installed, not what was bundled. I needed a way to manage tree-shaking case by case

Options were limited, turning off tree-shaking would explode the size of a bundle.

Below is an example scenario, App1 tree-shakes an export only used by App2

Interleaving won't be much use if my dependency chains may or may not have their exports, beyond that externalized files may or may not have theirs. Because of when hashing takes place, I can't really determine if a bundle has all the functions and exports required by an interleaved chunk. Versions match, exports might not.

Take control of how Webpack does tree-shaking

The Webpack DLL plugin uses a function that marks a module being used in unknown ways. By applying this function to any externalized files, and perhaps their dependencies, tree shaking will not risk stability.

I also found out how to do this in a more verbose manner. When looping over modules, you can set the module.buildMeta.usedExports to the same as module.providedExports

Imagine being able to download another apps vendors as a fallback to a missing chunk!

Future State

Where is it going?

Self-healing

Implementing a communication bus that allows a querying capability between separate runtimes. If a dependency were to fail to load, or if the dependency chunk is missing from a build for some reason. It should attempt to query any and all other Webpack builds that might be on the page if it can find the dependency in another interleaved build. It could download that chunk from the interleaved downer and use its vendors which were not actually used by the donor. Imagine being able to download other apps vendors as a fallback! Deploying new cache busted files wouldn't have the potential of breaking the download of current users.

SSR middleware

This project works client-side, but server-side is harder. Eval would work, but that sounds incredibly dangerous. I've thought of two solutions that could alleviate this issue. Some middleware could be written to communicate with other MFE’s inside the cluster and relay their independent segments.

Eventual consistency: works for things like a header or footer. Something likely owned by one workstream, but needs to be shared across a fleet of MFE’s. The team could publish their MFE as an npm package. Consumers SSR it, but client-side, browsers will interleave directly from the origin. Which might have deployed more recently than the SSR’d copy which was installed by the consumer.

This isn’t ideal but does work.

Distributed Rendering: The consumer GETs to another MFE via a rendering API that serves JSON containing HTML, CSS, JS, Initial State. With this, Node could pass it in as props to <App> and render the rest. With fragment caching within react-dom, rendering becomes extremely fast.

One could also take advantage of HTTP2 streaming and actually stream SSR from the distributed MFE render cluster. Or could just stream the servers among each other for low latency.

Interleaving can solve many problems, problems that currently require hacky, non-performant, or complicated workarounds.

Suspense implementations! When it’s time

server.get(path, (req, res) =>

serveFragment(

req,

res,

fragmentID => require(`./components/${fragmentID}`).default)

);

A new routing platform

Routing is still one to improve upon. Interleaved router objects would be required the same way a dynamic import is. Adding some utilities and JSX will make it a little easier and abstract away as many complexities.

Mergeable exports

Disabling tree shaking works on a module by module base. But this could lead to bundle sizes getting large, especially if dependency chains were also excluded from tree shaking.

The ability to merge exports would allow tree shaking of node_modules , an external chunk could check if the consumer build has the right exports inside a dependency. If it does not it would merge its own tree shaken version of the same dependency with the exports of the one in the consumer build. Ensuring that minimal code is downloaded but that all functions are available.

Improved utility functions and internalizing manifests