Sort Benchmark Home Page

New: We are happy to announce the 2019 winners listed below. The new, 2019 records are listed in green . Congratulations to the winners!

Background

Until 2007, the sort benchmarks were primary defined, sponsored and administered by Jim Gray. Following Jim's disappearance at sea in January 2007, the sort benchmarks have been continued by a committee of past colleagues and sort benchmark winners. The Sort Benchmark committee members include:

Chris Nyberg of Ordinal Technology Corp

Mehul Shah of Amazon Web Services

Naga Govindaraju of Microsoft

Top Results Daytona Indy Gray 2016, 44.8 TB/min Tencent Sort 100 TB in 134 Seconds

512 nodes x (2 OpenPOWER 10-core POWER8 2.926 GHz,

512 GB memory, 4x Huawei ES3600P V3 1.2TB NVMe SSD,

100Gb Mellanox ConnectX4-EN)

Jie Jiang, Lixiong Zheng, Junfeng Pu,

Xiong Cheng, Chongqing Zhao

Tencent Corporation

Mark R. Nutter, Jeremy D. Schaub 2016, 60.7 TB/min Tencent Sort 100 TB in 98.8 Seconds

512 nodes x (2 OpenPOWER 10-core POWER8 2.926 GHz,

512 GB memory, 4x Huawei ES3600P V3 1.2TB NVMe SSD,

100Gb Mellanox ConnectX4-EN)

Jie Jiang, Lixiong Zheng, Junfeng Pu,

Xiong Cheng, Chongqing Zhao

Tencent Corporation

Mark R. Nutter, Jeremy D. Schaub Cloud 2016, $1.44 / TB NADSort 100 TB for $144

394 Alibaba Cloud ECS ecs.n1.large nodes x

(Haswell E5-2680 v3, 8 GB memory,

40GB Ultra Cloud Disk, 4x 135GB SSD Cloud Disk)

Qian Wang, Rong Gu, Yihua Huang

Nanjing University

Reynold Xin

Databricks Inc.

Wei Wu, Jun Song, Junluan Xia

Alibaba Group Inc. 2016, $1.44 / TB NADSort 100 TB for $144

394 Alibaba Cloud ECS ecs.n1.large nodes x

(Haswell E5-2680 v3, 8 GB memory,

40GB Ultra Cloud Disk, 4x 135GB SSD Cloud Disk)

Qian Wang, Rong Gu, Yihua Huang

Nanjing University

Reynold Xin

Databricks Inc.

Wei Wu, Jun Song, Junluan Xia

Alibaba Group Inc. Minute 2016, 37 TB Tencent Sort 512 nodes x (2 OpenPOWER 10-core POWER8 2.926 GHz,

512 GB memory, 4x Huawei ES3600P V3 1.2TB NVMe SSD,

100Gb Mellanox ConnectX4-EN)

Jie Jiang, Lixiong Zheng, Junfeng Pu,

Xiong Cheng, Chongqing Zhao

Tencent Corporation

Mark R. Nutter, Jeremy D. Schaub 2016, 55 TB Tencent Sort 512 nodes x (2 OpenPOWER 10-core POWER8 2.926 GHz,

512 GB memory, 4x Huawei ES3600P V3 1.2TB NVMe SSD,

100Gb Mellanox ConnectX4-EN)

Jie Jiang, Lixiong Zheng, Junfeng Pu,

Xiong Cheng, Chongqing Zhao

Tencent Corporation

Mark R. Nutter, Jeremy D. Schaub Joule

1010 recs 2-way tie: 2019, 163 KJoules TaichiSort 61 K records sorted / joule

Intel i7-9700, 32GB RAM, Nsort, Ubuntu 16.04.3 LTS,

2 Intel DC 3600 series PCIe NVMe SSD (1.2 TB), 1 Intel DC 3600 series PCIe NVMe SSD (2.0 TB)

Ming Liu, Kaiyuan Zhang, Arvind Krishnamurthy

University of Washington

Simon Peter

University of Texas at Austin

2013, 168 KJoules NTOSort 59 K records sorted / joule

Intel i7-3770K, 16GB RAM, Nsort, Windows 8,

16 Samsung 840 Pro 256GB SSDs, 1 Samsung 840 Pro 128GB SSD

Andreas Ebert

Microsoft 2019, 89 KJoules KioxiaSort 112 K records sorted / joule

Intel i9-9900K, 64GB RAM, Ubuntu 19.04 Server,

8 CFD CSSD-M2B1TPG3VNF (1TB), 1 Toshiba XG5-P KXG50PNV2T04 (2TB)

Shintaro Sano, Tomoya Suzuki

Kioxia Corporation

Zaid Mahmoud

Princess Sumaya University for Technology

Common Rules

All the sort benchmarks share the following ground rules:

Must sort to and from operating system files on secondary storage.

No raw disk usage allowed since we are trying to test the IO subsystem.

File or device striping (RAID 0) are allowed (encouraged) to get bandwidth. If file striping is used then the concatenated files must form a sorted file.

The output file must be created as part of the sort.

Time includes the launching of the sort program.

The sort input records must be 100 bytes in length, with the first 10 bytes being a random key.

Use the gensort record generator to create the input records.

The sort output file must be validated for correct key order and checksum.

The hardware used should be commercially available (off-the-shelf), and unmodified (e.g. no processor over or under clocking).

Sort Benchmarks

GraySort Metric: Sort rate (TBs / minute) achieved while sorting a very large amount of data (currently 100 TB minimum). CloudSort Metric: Minimum cost for sorting a very large amount of data on a public cloud. (currently 100 TB).

Complete rules in the CloudSort short paper. MinuteSort Metric: Amount of data that can be sorted in 60.00 seconds or less.

Originally defined in AlphaSort paper. JouleSort Metric: Amount of energy required to sort 108, 109, 1010, or 1012 records (10 GB, 100 GB, 1 TB, or 100TB).

Originally defined in JouleSort paper.

The 108, 109 and 1012 records JouleSort benchmarks are now deprecated. PennySort Metric: Amount of data that can be sorted for a penny's worth of system time.

Originally defined in AlphaSort paper.

PennySort is now deprecated. TeraByte Sort Metric: Elapsed time to sort 1012 bytes of data.

The TeraByte benchmark is now deprecated because it became essentially the same as MinuteSort. Datamation Sort Metric: Amount of time to sort one million records (100 MB).

This is the original sort benchmark, defined in A Measure of Transaction Processing Power With 25 others Datamation, V 31.7, April 1985, pp 112-118. Originally, winners took 1 hour, now 1 second! So the benchmark is deprecated.

Benchmark Categories

For each sort benchmark, there are two categories:

Daytona (stock car) Indy (formula 1) Sort code must be general purpose. Need only sort 100-byte records with 10-byte keys.

Complete Rules

The 2019 rules are currently being adjusted. For the previous version of the detailed rules, see the frequently asked questions (FAQ) .

Process

Entries must include a document describing the algorithm and the hardware in enough detail so that others could reproduce the result. Click on the title of any previous winning sort to view sort description document.

New Entries

The submission deadline for the 2019 contest is 1 September 2019.

This page is maintained by Chris Nyberg (chris dot nyberg at ordinal dot com) and Mehul Shah (mashah at gmail dot com).