A statistical analysis of the results of 2012 admission tests by scientists at Indian Statistical Institute (ISI) in Kolkata shows that certain Boards, students from high income families, and urban areas bag most of the IITs.

For very long the venerated and coveted IITs and their entrance exams have been considered as the great leveler in India, a country where inequality and diversity are two sides of the coin. Students from villages, from uneducated backgrounds and low economic strata have qualified for these exams. But now this notion seems to have turned on its head.

A statistical analysis of the results of 2012 admission tests by scientists at Indian Statistical Institute (ISI) in Kolkata shows that certain Boards, students from high income families, and urban areas bag most of the IIT seats. In fact, among those who had registered, students from higher income group have a four times higher success rate than those from lower income group.

As for the Boards, IITs picked over 57% of their selected students in 2012 from CBSE, a board that represents 6% of their target group.

Isn't that shrinking the supply base of these institutions to a disastrously narrow band? And if you've followed the rather messy process of "improving" the supply base of IITs - not to get just factory-produced students whose primary (and secondary) aim in life becomes to 'crack' the admission test and then lose steam when the actual learning starts at IITs - then you'd see below how even the new system that was adopted in 2013 beats the entire purpose of admissions revamp downright hollow. Predictably, the preliminary analysis of 2013 JEE-Advanced results shows that CBSE predominance continues.

The table below from ISI shows IITs hardly select an inclusive group. As far as board affiliation is concerned, CBSE and ICSE accounted for less than 7% of all students appearing for Class XII Board examinations in 2012 in India.

The success rates of different sections of candidates (number of qualified candidates divided by the number of registered candidates) tell a similar story:

While no reliable data on the effect of coaching is available, the unusually high success rates of candidates from the higher income group and city-dwellers suggest that access to coaching substantially increases the success rate, says ISI.

The cat and mouse game of 'normalization'

In an exemplary act of transparency (and in the interest of the people), ISI has made public the entire minutes of meetings and deliberations that took place over the last two years leading to the 2013 IIT revamped entrance test. "Why there was a lag between the last meeting and the publication last week", says Debasis Sengupta of ISI. "We just wanted to put forth everything before the public." Beyond that he'd not say anything except lament that all good intention behind the revamp remains just that - an intention. IITs are still not selecting the brightest students, if anything, the reality distortion field for the Boards, other than the CBSE, looks more unfriendly.

Because there are dozens of Class XII Boards, a long-drawn process was adopted to arrive at a new admission process that gave weightage to entrance exam scores and normalized class scores.

"When performances from multiple exams are to be combined, it is usually apples and oranges. So extensive simulations on past data has to be conducted to make sure the outcomes are fair. I very much doubt if any of this was done. More thought and simulation has gone into the Duckworth-Lewis system in cricket," says V Vinay, chairman of LimberLink Technologies and President of the Indian Association for Research in Computing Science.

Debasis Sengupta says a lot of work went into the statistical inputs that ISI gave for the normalisation process. But as the documents show, none of the suggestions was adhered to.

In short, the cut-off scores for IITs are too "weak" and not the best way to select really good students, especially students with analytical and problem solving skills.

However, it's the NIT's where the normalisation formula takes the cake. It's as bizarre as it can get in the world of statistics. You can see the full description in the link above but to summarize it in plain English: One group of experts (from ISI) believes that all Boards, whether central or State, have meritorious students and proposes a normalization formula. Let's call it A. The second group (from the Chair of CBSE) assumes that some Boards (read CBSE) students "are more able" than other boards and proposes another formula. Let's call it B. The two groups don't agree, the latter apparently don't provide scientific answers and the logjam continues. The issue is resolved by taking the mean of both the formulae: Add A and B and divide by 2.

"This is true scientific black comedy", remarks a mathematician from Chennai.

To simplify it further, as one parent explained it to me in utter frustration: "See, this is a case where suppose you have 5 children with different shoe sizes. To find the common size, you find the average but at size 6 ¼ this shoe fits no one."

Hopefully the anomaly is corrected sooner rather than later. As one person close to the developments said, "Things are moving fast and a lot will happen before 2014 exams."

Still, a few facts in this bunch of documents made public need attention: There was a huge effort to make sure the CBSE students did not suffer; there has been a serious conflict of interest with CBSE, which was supposed to be an implementing agency but turned out to be a decision making body, even recruiting people for the decision-making committee; none of the other Boards were represented or even called for consultation; please read points 11 and 12 in this ISI letter to the Government.

A fortnight ago, the NIT admission test row reached the Supreme Court. Whatever the verdict of the apex court, the case is far from settled, in more ways than one.

"As a non-IITian my view has been that the current IIT students barring may be the top 1000 JEE-rankers are simply one-track horses, having spent about four years preparing just for the JEE at the cost of everything else. No amount of tinkering with the selection process is going to fix the core issue that very high career rewards are dependent on one or two numbers (in the current case, JEE advanced and Board normalized score)," says Swami Manohar, managing director of LimberLink Technologies, a start-up that is attempting to re-engineer education in professional engineering colleges.

Unfortunately, after the two-year 'pilgrimage' of IIT admission tests revamp, perhaps everyone is still standing at the purgatory.