According to Vintage Technology, both buttons are a way to clear or cancel an entry. The C button will clear all input to the calculator. The CE button clears the most recent entry, so if you make a mistake in a long computation, you don't need to start all over again.

Source

Example

If I now press the CE button, only the 5 is erased. The rest of my computation is still stored.

If I press the C button, my whole computation will be cleared:

History

One might ask why we have these specific keys on our Windows calculator? Why are they not labeled differently?

Luckily, the guys over at Vintage Calculators have an amazing collection of information on the subject.

According to their site, the first electronic calculator was released by Bell Punch Co., Uxbridge, England in 1961. This were the Anita Mk VII and the Anita Mk 8.

Anita Mk VII



Source

Anita Mk 8



Source

For the Mk 8 we get an additional schema:



Source

We can see it has a Clear Register and Clear Keyboard button. Please keep in mind, to my knowledge, this is one of the first electronic calculators that was ever designed.

The terminology was also used in later models, like the Sanyo ICC-0081, which seemed to have a CK (Clear Keyboard) and CA (Clear All) button.



Source

Later models just continue the pattern. For example, the

Canon Pocketronic



Source

We can see a C (Clear) and CI (Cancel Input) button.