NNCP: Lossless Data Compression with Neural Networks

The algorithms and results are described in this paper. Warning: the paper only describes the first version of NNCP. Improvements were made in the following versions.

NNCP is based on the LibNC library which allows fast and deterministic evaluation and training of neural networks on x86 CPUs. It is optimized for small batch sizes and low latency. LibNC has no dependency on other libraries and has a C API.

Compression ratio

Program Compr. size

(bytes) Ratio

(bpb) gzip 36 445 248 2.92 xz 24 865 244 1.99 NNCP (2019-11-16) 16 292 774 1.30 CMIX (v18) 14 838 332 1.19

Program Compr. size

(bytes) Ratio

(bpb) Program size(zip, bytes) Total

(bytes) gzip 322 591 995 2.58 38 801 322 630 796 xz 197 331 816 1.58 36 752 197 368 568 NNCP (2019-11-16) 119 167 224 0.95 238 452 119 405 676 CMIX (v18) 115 714 367 0.93 208 961 115 923 328

* The results for the other programs are from the Large Text Compression Benchmark.

Download

Precompiled Windows version: nncp-2019-11-16-win64.zip.

Related Links