The Radeon Open Compute Platform (ROCm) is an open source platform for GPU computing that is language independent and brings modular software development to GPU computing. This provides a real cheaper alternative to Nvidia's CUDA and helps developers in coding compute-oriented software for AMD Radeon GPUs along with converting existing CUDA software to run on GCN hardware.

In the past you almost had to purchase the Intel / Nvidia combo for serious GPU computing (Nvidia doesn't have an x86 license). Now you have AMD Zen as an option on the CPU side and AMD Radeon on the GPU side of things.

For now the compute cards from AMD are the S-series, like the S9300 (2x4096 GCN cores). The fastest APU is stuck at 384 cores. AMD will soon release server based APU's, like Raven Ridge, which will be a much lower cost solution than buying a server CPU and discrete server GPU separately. You should be able to easily port over any previous code that was used for previous hardware.

With Zen AMD could offer a nice HPC ecosystem that has good communication between x86 CPU and GPU, large memory bandwidth and excellent I/O. While Intel may have a faster x86 CPUs and a faster storage medium in Octane - and Nvidia may have faster GPUs along with a more accessible language in CUDA - AMD could have a better integrated HPC system where the sum exceeds the performance of the individual Intel / Nvidia combo parts.

Of course this can be used for deep learning and neural network applications more efficiently at lower cost. Features include:

Multi-GPU coarse-grain shared virtual memory

Process concurrency and preemption

Large memory allocations

HSA signals and atomics

User-mode queues and DMA

Standardized loader and code-object format

Dynamics and offline-compilation support

Peer-to-peer multi-GPU operation with RDMA support

Profiler trace and event-collection API

Systems-management API and tools

LLVM compiler foundation

HCC C++ and HIP for application portability

GCN assembler and disassembler

See: http://developer.amd.com/tools-and-sdks/radeon-open-compute-platform