It is academic reference season. Every day I receive requests from former and current students, or people whose work I have read or examined, or met in a lift, to write in support of their application for further study or for academic jobs. I'm usually happy to do my bit. But as these references accumulate in files around the world, I do wonder how many of them will ever be read.

Data protection has taken the fun out of reference writing, and hence the fun out of reference reading. Gone are the days when it was possible to write on a plain note card "Grab him if you can," as apparently Gilbert Ryle, professor of philosophy at Oxford, did for one of his students in the 1960s. Or in the strangulated prose of Isaiah Berlin, in his recommendation for the brilliant legal philosopher HLA Hart, "What he is tortured by is the thought that he will never be better than [AC] Ewing and will never hold other views than Ewing. He realises himself that this is not a very exciting state of mind to be in … Nevertheless … he cannot be worse than Ewing, who, after all, is … in his own way, not contemptible."

These days one has to keep in mind that the person you are writing for may eventually see the reference. Accordingly, reference inflation has set in, and everyone is simply wonderful. One reference writer has said of several of his PhD students, "He reminds me of the young Wittgenstein." (That's right! He can never get his shirt to stay tucked in either!) Perhaps the oddest comment I've seen is "Pound for pound she is the best philosopher in the department." What can that mean? She's not very good, but on the other hand she is really small?

The US style of academic reference writing needs a serious look. Typically, job candidates compile a "dossier" that includes perhaps five or six references. And unless a reference is three or four single-spaced pages then the support is regarded as somewhat half-hearted. But usually there is not that much to tell, and so we receive, in effect, six different versions of the candidate's CV in prose form, combined with summaries of his or her PhD thesis, and a few lines of detail designed to convince that the reference writer really knows the candidate well and has been truly, deeply, impressed.

But readers of references are also writers, and so we know the rules. When reading job references our eyes glaze over until we reach the business paragraph: the one where the comparisons are made. The rule is that the more the reference writer sticks out his or her neck, the stronger the recommendation. If you say the candidate is good, that means nothing. If you say "in the top three of the cohort", again, very little. But if you say the candidate is the best this year, or for several years, or for a decade, that means something. One of members of the comparison class might be reading, and be brought up very sharp. The strongest reference of all says the person is better than some starry named figure, albeit done in a rather more positive way than Berlin's comparison between Hart and Ewing.

For all the effort and ink, what, as readers of references, do we hope to find? Only this: is the candidate better or worse than it appears from their CV? Well, no one is going to say: watch out, we should never have admitted this person to our PhD programme and the thesis only passed because the supervisor more or less wrote it. So that's half the reason gone. And we probably won't trust a referee who raves on and on. So the other half bites the dust, too. Hence the value we get from academic references is minuscule compared to the effort involved on all sides.

Perhaps it's time for academia to follow standard practice elsewhere in the world of work, seeking references only at the final stage. A short form, with three questions. Can they be trusted to keep their fingers out of the till? Did they really get so many A stars at GCSE? Are they neat and tidy? The correct answers, of course, are yes, yes, no.

• Jonathan Wolff is professor of philosophy and dean of arts and humanities, University College London