HEVC/AV1 Video Codecs Comparison 2019 Fourteen Annual Video-Codecs Comparison by MSU

Video group head: Dr. Dmitriy Vatolin Project head: Dr. Dmitriy Kulikov Measurements, analysis: Dr. Mikhail Erofeev,

Anastasia Antsiferova,

Sergey Zvezdakov,

Denis Kondranin,

Egor Sklyarov,

Stanislav Grokholskiy

100 videos were used for comparison! (report part I, FullHD)

If you want to receive notifications about our reports, please





Navigation

Participated codecs

Codec name Report parts Standard Part I. FullHD Part II. Subjective Part III. 4K Part IV. High Quality 1 aom

AOMedia





AV1 2 arowana xvc

Divideon

Ripping use case



xvc 3 Bytedance V265 Encoder

ByteDance Inc.

Fast, Universal use cases HEVC 4 HW265

Huawei Technologies Co., Ltd. HEVC 5 MainConcept HEVC

MainConcept GmbH HEVC 6 rav1e

rav1e AV1 7 SIF Encoder

SIF Encoder Team

Ripping use case

Universal use case SIF 8 SVT-AV1

Open Visual Cloud

Universal use case AV1 9 SVT-HEVC

Open Visual Cloud HEVC 10 SVT-VP9

Open Visual Cloud VP9 11 sz265

Nanjing Yunyan HEVC 12 Tencent V265 Encoder

Tencent HEVC 13 UC265

Ucodec Inc.

Ripping use case HEVC 14 VP9

The WebM Project (Google)

Ripping use case VP9 15 WZAurora AV1 Encoder

Visionular

Ripping use case

Universal use case AV1 16 x264

x264 Developer Team AVC 17 x265

MulticoreWare, Inc. HEVC 18 xin265

HEVC

Reports

Main report (Objective comparison, FullHD videos)

Three encoding use cases measured on 100 FullHD videos

Released on October, 21 Free version Enterprise version Use cases Universal (partially) Fast, Universal, Ripping Per-sequence-results 2 of 100 sequences (only Universal use case) All 100 sequences for all use cases (in interactive charts) Metric: YUV-SSIM Description of video sequences Codec info (developer, version number, website link) Other objective metrics (in addition to YUV-SSIM)

Y-VMAF(0.6.1), Y-VMAF(0.6.2), Y-VMAF(0.6.3), Y-VMAF(0.6.1, Phone), Y-VMAF(0.6.2, Phone), Y-VMAF(0.6.3, Phone), Y-SSIM, U-SSIM, V-SSIM, YUV-PSNR, Y-PSNR, U-PSNR, V-PSNR Per-frame metrics results (in HTML version of the report)

All metrics for all sequences and use cases (10000+ charts) Relative quality analysis Download links for video sequences Encoders presets description PDF report 58 pages 83 pages HTML report 28 interactive charts 14000+ interactive charts Price Free $950



Descriptions of 100 used videos can be found on this page or in separate PDF (41 MB)





You will receive enterprise versions of all reports (FullHD, Subjective, 4K and High Quality)

Subjective Report

Subjective comparison conducted on Subjectify.us platform

Released on November, 1 11 codecs

Bytedance, sz265, Tencent V265 Encoder, UC265, x265, xin265, arowana xvc, SIF Encoder, VP9, WZAurora AV1 Encoder, x264 Free version











Enterprise version







You will receive enterprise versions of all reports (FullHD, Subjective, 4K and High Quality)

Contact us if you want to buy only Enterprise Subjective Report 732 unique observers

25784 valid answers 5 video sequences

Short fragments from Crowd Run, Kayak Trip, Making Alcohol, Tractor, Wedding Party Special Subjective Encoding Use Case

At least 1 FPS 6 metrics

Subjective score and 5 objective: YUV-SSIM, Y-SSIM, YUV-PSNR, Y-PSNR, Y-VMAF(v.0.6.1) HTML and PDF documents

118 interactive charts and 33 pages

4K Report

Comparison conducted on 4K (UHD) videos

Released on March, 6 12 codecs

Bytedance V265 Encoder, HW265, MainConcept HEVC, SVT-HEVC, sz265, x265, SIF Encoder, SVT-AV1, SVT-VP9, VP9, WZAurora AV1 Encoder, x264 Free version



PDF report





HTML report





Download all in one archive (zip)



Enterprise version







You will receive enterprise versions of all reports (FullHD, Subjective, 4K and High Quality)

Contact us if you want to buy only Enterprise 4K Report 11 4K video sequences

From 24 to 60 FPS Two Encoding Use Cases

4K Universal (1fps) and 4K Fast (20fps) 9 metrics

YUV_SSIM, YUV_PSNR, Y_VMAF (v0.6.2), Y_SSIM, U_SSIM, V_SSIM, Y_PSNR, U_PSNR, V_PSNR HTML and PDF documents

710 interactive charts and 71 pages

Report on high quality encoding

Comparison conducted under slow-speed requireents

Released on March, 30 7 codecs

aom, rav1e, SVT-AV1, SVT-HEVC, SVT-VP9, x264, x265

Enterprise version (for free)



PDF report





HTML report





Download all files in one archive (zip)

PDF, HTML report + download links for videos + HTML report with per-frame metrics results 6 FullHD video sequences

From 24 to 60 FPS Special Encoding Use Case

Formal limitation 0.005 fps (not strict) 9 metrics

YUV_SSIM, YUV_PSNR, Y_VMAF (v0.6.2), Y_SSIM, U_SSIM, V_SSIM, Y_PSNR, U_PSNR, V_PSNR HTML and PDF documents

594 interactive charts and 42 pages

Ovarall Conclusions

Main report (Objective comparison, FullHD videos) summary

According to just quality scores (YUV-SSIM), the best codecs among our competitors evaluated for all three use cases are the following:

First place: HW265

Second place: Tencent V265 Encoder

Third place: sz265

The biggest number of codecs took part in comparison with high-quality encoding presets (Ripping use case). The winners for only high-quality encoding are the following:

First place in high-quality (ripping) use case: HW265

Second place in high-quality (ripping) use case: Tencent V265 Encoder

Third place in high-quality (ripping) use case: VP9

We tested three encoded use cases (see the description in section Test Hardware Characteristics). Here is one of the rate-distortions charts for universal encoding (Cion video sequence):

The universal-encoding use case has two Pareto optimal encoders in terms of mean speed and quality: HW265 and Tencent V265 Encoder. Nevertheless, the differences emerge for particular sequences and use cases.

Free version contains the results for 2 of 100 video sequences, while full results are available in enterprise version.

Subjective report summary

According to subjective quality scores, the best codecs among our competitors evaluated for all three use cases are the following:

First place: WZAurora AV1 Encoder

Second place: Tencent V265 Encoder

Third place: arowana xvc

4K report summary

There is no absolute winner in the comparison, since different encoders take first place at different test video sequences: for example, onthree encoders show Pareto-optimal results:All graphs are available in enterprise version , which is free for all buyers of enterprise main report, and enterprise main report is also free for all buyers of enterprise subjective report.

According to SSIM quality scores, the best codecs among our competitors evaluated for two use cases are the following:

First place: HW265 by Huawei Technologies Co., Ltd.

by Huawei Technologies Co., Ltd. Second place: Bytedance V265 Encoder

Third place: Mainconcept HEVC

The results slightly differ for other objective quality metrics. For example,shows the best result according to VMAF.

Mean overall quality (SSIM):

Overall quality for universal use case (1fps, SSIM):

In this comparison, the results for universal 4K encoding (1fps) differ from overall scores. The following encoders showed the best performance results:

First place: WZAurora AV1 Encoder

Second place: HW265 by Huawei Technologies Co., Ltd.

by Huawei Technologies Co., Ltd. Third place: Mainconcept HEVC

The above plots show only quality gain, hovewer, all encoders slightly differ in encoding speed. The following speed-quality chart shows the picture for Duck Take Off video sequence.

All graphs, including speed-quality trade-off for all videos, are available in enterprise version (which is free for all buyers of enterprise main and subjective reports).

High quality encoding report summary

According to YUV-SSIM, YUV-PSNR and Y-VMAF quality scores, the best codecs among our competitors evaluated for two use cases are the following:

First place: aom

Second place: SVT-AV1

Third place: x265

shows the best scores according to V-SSIM and V-PSNR metrics.

Mean overall quality (SSIM):

As we analysed the case of offline encoding, the limit on encoding speed was not strict in this part of the comparison. The following speed-quality chart shows the landscape of competitors relative speed and quality for all video sequences.

The following plot shows per-frame VMAF scores for axebat video sequence at 2 Mbps. SVT-AV1, SVT-HEVC and SVT-VP9 show quality decrease in a part of the video with high motion and complex scenes (running baseball players, splashing water). Enterprise report includes an HTML report with per-frame metrics results for all videos and bit rates.

Comparison Rules

HEVC codec testing objectives

The main goal of this report is the presentation of a comparative evaluation of the quality of new HEVC codecs and codecs of other standards using objective measures of assessment. The comparison was done using settings provided by the developers of each codec. Nevertheless, we required all presets to satisfy minimum speed requirement on the particular use case. The main task of the comparison is to analyze different encoders for the task of transcoding video – e.g., compressing video for personal use.

Test Hardware Characteristics

CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)

Mainboard: ASRock Z370M Pro4

RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15

OS: Windows 10 x64

For this platform we considered three key use cases with different speed requirements:

Fast/High Density – 1080@60fps

Universal/Broadcast VQ – 1080p@25fps

Ripping/Pristine VQ – 1080p@1fps and SSIM-RD curve better than x264-veryslow

See more on Call-for-codecs 2019 page

Videos

Videos for testing set were chosen from MSU video collection via a voting among comparison participants, organizers and an independend expert. In this year, the collection was enlarged up to 18418 sequences which were chosen from 700000+ user-generated videos posted on Vimeo website. All videos have 4K or FullHD resolution and high bitrate (50 Mbps was selected as a lower bitrate boundary).

Number of videos in MSU video collection

Year # FullHD videos # FullHD samples # 4K videos # 4K samples Total # of videos Total # of samples 2016 3 7 882 2902 885 2909 2017 1996 4638 1544 4561 3540 9299 2018 4342 10330 1946 5503 6288 15833 2019 4945 12402 2091 6016 7036 18418

Bitrate distribution of videos in MSU video collection

Final video set consists of 100 sequences: 8 from the old data set and 92 new ones from Vimeo and media.xiph.org derf's collection. The average bitrate for all sequences in the final set is 218.9 Mbps, median - 143.2 Mbps. City walk (61.5 Mbps), Nancy (67.6 Mbps) and Oman museum (69.7 Mbps) sequences have minimal bitrates.



Descriptions of all test videos are presented on this page and in a separate PDF provided with our reports.

Codec Analysis and Tuning for Codec Developers and Codec Users

Computer Graphics and Multimedia Laboratory of Moscow State University:

15+ years working in the area of video codec analysis and tuning using objective quality metrics and subjective comparisons.

27+ reports of video codec comparisons and analysis (H.265, H.264, AV1, VP9, MPEG-4, MPEG-2, decoders' error recovery).

Methods and algorithms for codec comparison and analysis development, separate codec's features and codec's options analysis.

Strong and Weak Points of Your Codec

Deep encoder parts analysis (ME, RC on GOP, mode decision, etc).

Weak and strong points for your encoder and complete information about encoding quality on different content types.

Encoding Quality improvement by the pre and post filtering (including technologies licensing).

Independent Codec Estimation Comparing to Other Codecs for Different Use-cases

Comparative analysis of your encoder and other encoders.

We have direct contact with many codec developers.

You will know place of your encoder between other newest well-known encoders (compare encoding quality, speed, bitrate handling, etc.).

Encoder Features Implementation Optimality Analysis

Thanks

Special thanks to the following contributors of our previous comparisons

Share with us your opinion about comparison

Contact Information



compression.ru in cooperation

with Lomonosov MSU

Graphics & Media Lab

(Video Group) Dubna State University Institute for Information

Transmission Problems RAS

Subscribe to report updates

>Loading form...

Materials about MSU Codec Comparison

Other Materials

We could perform next tasks for codec developers and codec users.We perform encoder features effectiveness (speed/quality trade-off) analysis that could lead up to 30% increase in the speed/quality characteristics of your codec. We can help you to tune your codec and find best encoding parameters.

Last updated: 02-July-2020