Joe realizes that there are many building blocks to the production-grade Machine learning model.

It was clear for him that building tens of Machine learning applications at times is similar to the game of lego.

Most of the time, you need to reuse the existing blocks and plugin a few new ones.

You need an easy way to compose the system using various building blocks.

Joe is a Machine learning engineer, and he is good at it.

However, he gets tired of having to bug the DevOps engineers.

He needed them to allocate machines, schedule jobs, set up the environments, creating subnets, service discovery, running the necessary services.

Also, to obtain the metrics, setting up storage for the data, get the list of the allocated machines, schedule jobs, set up the environments, creating subnets, service discovery, running the necessary services and obtaining the metrics, setting up storage for the data.

Phew! The list goes on.

Joe wishes that he could have an easy way around solving the DevOps challenges associated with ML/DL.

Joe was primarily using Tensorflow. He realizes that the training of Machine learning algorithms on large amounts of data and serving the trained model as API’s for millions of customers poses a serious scaling issue.

Here are the factors because of which scaling becomes hard,

Storing vast volumes of data.

The computational overhead involved in processing large amounts of data.

Distributing Machine learning training across multiple computational resources.

Cost issues.

Joe found it hard to train on large amounts of data in a reasonable time.

Here were the primary reasons.

Throughput challenge from underlying storage system during training

Procuring of CPU, GPU, and TPU’s and other computational resources to scale up the performance.

Throttling, rate limiting, bandwidth charges and low throughput from public cloud storage systems.

Joe is tired and frustrated with trying to productionize the Machine learning pipeline.

As a last resort, Joe moves to managed services.

It made things more comfortable, and he could get started faster, DevOps was abstracted. It was all great.

Then the bills arrived!

Joe knew that using managed services made things easier.

However, the billing woes added massive pressure on his startup in its goal to be soon profitable.