There are other improvements of Neanderthal's internals that:

make it easier and more consistent;

allow better integration of various matrix types that will come in next versions;

allow easier and more transparent programming of GPUs;

The major new things to expect in versions after this one are (not necessarily in this order):

Various special dense and packed matrices (symmetric, banded, etc.);

Sparse matrices;

CUDA support (although the existing GPU engine already supports Nvidia GPUs!). This is important for the interoperability with Nvidia's libraries for LAPACK and Deep learning.

Tensors are also going to be in Neanderthal, maybe even sooner than you expect ;)

I'll be back once Neanderthal 0.9.0 is ready with more news and details.