Measuring My Chinese Progress

Last summer I started learning Mandarin Chinese. To start I began taking classes at a Chinese language school in SF. For more practice I started an Instagram @jeffcarp_zh and tried writing a couple blog posts.

Almost a year later, I’m still going to Chinese class on a semi-regular basis (1 hour a week except when I’m taking a break) and keep up a daily spaced-repetition flashcard habit using the Pleco Chinese dictionary app (usually on the train into work).

Since I’m not in college, I don’t have a semester or quarter system to add structure to my learning. So recently I’ve been looking for ways to reliably measure my Chinese learning progress to celebrate wins and keep me motivated to continue studying.

Flashcard Count

To start in my investigation, I downloaded my Pleco backup file containing all my flashcards. Currently I have 1862 flashcards.

The line is going up and to the right, but it doesn't tell me anything about how effective my language skills are.

HSK

The standardized Chinese Proficiency Test is the HSK, or Hànyǔ Shuǐpíng Kǎoshì—汉语水平考试 (over-literal translation: Chinese language water level test 😄). There are 6 HSK levels; level 1 is the easiest. Getting a job in China seems to require around level 4 or 5. The number of vocabulary words per level increases drastically as the levels go up:

Current HSK Level

The HSK is offered twice a year in SF and costs between $20-70, so taking it frequently isn’t a good option for regularly measuring my progress. However, given the HSK vocab lists and my flashcards, I can quickly check how my vocabulary stacks up:

(Note that in the process of making this graph I added a bunch of missing words to my flashcards, which is what caused the big drops recently in HSKs 1, 2, and 3).

And here is the percent of each HSK I’ve completed. Now that I’ve added all the HSK 1-3 vocabulary to my flashcards, I’m about 20% of the way there for HSK 4.

How does this translate to real-world Chinese Progress?

I’m not learning Chinese to pass the HSK. I’m learning Chinese to talk with people who speak Chinese. So the overarching question I want to answer is: given my current vocabulary, how much of day-to-day spoken Chinese can I understand?

I thought a great approximation for this could be Weibo, the Chinese version of Twitter. I downloaded a dataset with 1.1 million “tweets” to see how my vocabulary stacks up. The top 10 most common words in the dataset were those you might expect:

Word Pinyin Percentile Meaning 的 de 100 (possessive particle) 我 wǒ 99.99 I 你 nǐ 99.9979 you 了 le 99.9968 (completed action marker) 是 shì 99.9958 to be 啊 ā 99.9947 interjection (phoenetic) 不 bù 99.9937 no 哈哈 hāhā 99.9926 haha 好 hǎo 99.9916 good 有 yǒu 99.9905 to have

Normalizing all the words in the dataset by their relative frequency, I found that with my current vocabulary, I can read about 63.58% of Weibo.

Using Weibo (and specifically this dataset) isn’t perfect—even if I can read all the words in a sentence, it doesn’t mean I’ll understand its meaning—so I’m making a lot of assumptions.

But I still think there is something here. Standardized language tests came from a time when we didn’t have access to huge datasets of current speech from millions of speakers for large-scale language analysis. The order of the HSK vocabulary words and the order of the most common Chinese words are completely different.

In a future blog post I’d really like to explore in further depth how well the HSK prepares you to read and write the most common day-to-day Chinese words, and ideas for learning based on word frequency.