From Is the Order a Rabbit? S2 07 (edited for demonstration purposes)

Looks awful, doesn’t it? But the resolution of the image is 1920x1080, which means that it still counts as Full HD. The reason the image looks terrible is because it was saved as a very low quality JPG, resulting in a very small filesize. Bitrate basically represents the average filesize for the images that make up a video (as video at its core is just a series of images played one after another). The lower the bitrate, the lower the quality is generally going to be. Next, take a look at this image:

This is a lossless PNG image. If you were to take this image and save it as a lossy JPG image, no matter how much filesize you allowed the JPG encoder to use, it would still look terrible. And that’s exactly why source quality is important. The above image shows exaggerated versions of two of the most common artifacts in anime sources: banding and aliasing. While they are basically unfixable here due to the exaggeration, the banding and aliasing generally found in real video sources can actually be dealt with using appropriate video processing techniques, commonly referred to as filtering.

Last but not least, we have encoding settings. The first thing here is the choice of the video format. The format that pretty much everyone uses these days is H.264/AVC. Next up, you need to choose which encoding software to use. This is actually a very important decision, because encoding a video is like building a house. The video format by itself is merely the toolbox you get to build the house with, while the encoding software is a robot that developers must teach to use the tools in the format’s toolbox to build the house with. The better you can teach the encoding software to use the format’s tools, the better the resulting compression quality will be.

That’s why the choice of encoding software matters. Just using the same video format is not enough to guarantee that you will get the same kind of results in terms of compression quality. When it comes to H.264, the encoding software to choose is x264 — the most advanced H.264 encoding software on the planet. This has been proven in many comparisons, like the following for example:

Lower is better. Chart taken from here.

After this, there are still plenty of choices to make, like what profile to use and various instructions you can give to the encoding software on what features to use (video encoding takes a long time, so for example you may want to tell the encoding software to avoid doing something that would add a lot of additional time to the encoding process for very little gain in quality).

So how does Crunchyroll fare in all this?

Not very well. Starting with the source quality, in many cases the sources they get suffer from the banding and aliasing issues mentioned earlier. Crunchyroll has never done anything to try to fix these or any of the other issues the sources might have, even when extremely severe (if you want a great example of this, just look at Saki). There have also been quite a few cases where they get both bad and good sources for a show and consistently pick the bad one to do their encoding from.

For encoding software, they do use x264, which is good, but they also fail to keep their x264 up to date. Their old higher bitrate encodes are done with an x264 version from 2011, while their new low bitrate encodes are done with a version from 2014. Either way, they’re missing out on years of additional improvements in encoding efficiency. You can take a look at the exact encoding settings Crunchyroll uses here. (x264 includes this information in the encoded video files, which is how this information is available.)

What those settings tell to people who understand them is that the new encoding settings are worse in pretty much every way (the only thing you could consider to be better is that they use 5 b-frames instead of just 3). At the end of these settings, we also get to the matter of bitrate: for the old encodes, Crunchyroll uses constant filesize (two-pass) encoding with an average bitrate of ~3000kbps. For the new encodes, Crunchyroll is using constant quality (CRF) encoding with a rate factor of ~22-25. CRF encoding is actually a really good technique, but for 1080p video, truly good quality CRF values will generally be 18 or lower (the lower the better, basically). CRF 22-25 is way above that, which is where the massive bitrate drop comes from in comparison to the old encodes (with the new ones we’re looking at average bitrates of ~1000–2000kbps).

So what can Crunchyroll do about this?

The ways Crunchyroll could (and should!) improve on this are fairly simple: first, they should update their x264 so that it’s not years out of date. Then they should bring the actual x264 settings for the new encodes at least on par with the old encodes, or preferably make them even better. This will increase encoding time on Crunchyroll’s end, but since they are mainly working with video-on-demand, they only need to encode the episodes once (per offered quality level), so this isn’t really that big of a problem. And finally, lower the CRF value used so that the bitrates will at least match up with the old encodes, or preferably go even higher. Remember, Crunchyroll already had the lowest premium video bitrates in the industry, and people have been raising complaints about this for years.

Beyond this, it would also be great if Crunchyroll started filtering the sources they get in order to fix the flaws in them. This would require bigger changes to Crunchyroll’s video pipeline than the simpler encoding fixes, but seeing as Crunchyroll should really have that “world class video team” put together by now that their CEO Kun Gao said they were building two and a half years ago in a Reddit AMA, it really shouldn’t be too much of a problem (or that’s what one could expect with statements like that, anyway). Just having a single filter chain per series (ie. have someone who knows what they’re doing look at the sources for a series at the start of the show and decide on what filtering to apply to them, if any) would go a long way to improve the situation, because right now banding artifacts are more of a rule than the exception with Crunchyroll’s premium video streams.

From Interviews With Monster Girls 09

So what can we do about this?

First of all, spread the word! The more we can make people aware of these issues, the harder time Crunchyroll will have ignoring them. If you’re a paying customer, I highly recommend bringing it up on Crunchyroll’s forums, as well as sending messages about the poor video quality to them using their support system. Even better, vote with your wallet and straight up cancel your premium subscription, citing poor video quality as the reason. As consumers, we deserve better than this.

So what would be better than this?

For this, we can take a look at the show RE:Hamatora, which was licensed by Crunchyroll for English simulcasting. It was also licensed for German simulcasting by a company called Kazé. Here is a comparison where you can see how their respective 1080p streams stack up against each other. The quality difference between the two is quite notable (even when using the old higher bitrate version for Crunchyroll), even though they both had the exact same source files to work with. Why? The explanation is pretty simple: Kazé actually cares about the quality of their video.

2017–03–13 UPDATE: I was linked to another comparison between Crunchyroll’s 1080p video and German simulcaster Anime on Demand’s video which they call “1080p+” (which is an extra high quality option they offer next to their regular 1080p streams). Check out the comparison here. AoD’s video here has probably the best quality I’ve ever seen from a legal digital anime release, only topped by some Italian anime Blu-rays. Notice how they also have actual lyrics and translations for opening & ending songs, something that Crunchyroll basically never bothers to do.