Darrin Klimek | Digital Vision | Getty Images

New York is indeed an expensive place, but experts say that alone doesn't explain a recent report that found the city's annual cost per inmate was $167,731 last year — nearly as much as it costs to pay for four years of tuition at an Ivy League university. They say a big part of it is due to New York's most notorious lockup, Rikers Island, and the costs that go along with staffing, maintaining and securing a facility that is literally an island unto itself.

"Other cities don't have Rikers Island," said Martin F. Horn, who in 2009 resigned as the city's correction commissioner, noting that hundreds of millions of dollars are spent a year to run the 400-acre island in the East River next to the runways of LaGuardia Airport that has 10 jail facilities, thousands of staff and its own power plant and bakery. (Read more: Breaking the college tuition monopoly) The city's Independent Budget Office annual figure of $167,731 — which equates to about $460 per day for the 12,287 average daily New York City inmates last year — was based on about $2 billion in total operating expenses for the Department of Correction, which included salaries and benefits for staff, judgments and claims as well as debt service for jail construction and repairs.

But there are particularly expensive costs associated with Rikers. The department says it spends $30.3 million annually alone on transportation costs, running three bus services that usher inmates to and from court throughout the five boroughs, staff from a central parking lot to Rikers jails and visitors to and around the island. There were 261,158 inmates delivered to court last year.

A way to bring down the costs, Horn has long said, would be to replace Rikers Island with more robust jails next door to courthouses. But his attempts to do that failed in part because of political opposition from residential areas near courthouses in Brooklyn, Manhattan and elsewhere.

"My point is: Have you seen a whole lot of outcry on this? Why doesn't anything happen?" Horn said of the $167,731 annual figure. "Because nobody cares."

"That's the reason we have Rikers Island," he said. "We want these guys put away out of public view."

New York's annual costs dwarf the annual per-inmate costs in other big cities. Los Angeles spent $128.94 a day, or $47,063 a year, for 17,400 inmates in fiscal year 2011-12, its sheriff's office said. Chicago spent $145 a day, or $52,925 a year, for 13,200 inmates in 2010, the most recent figures available from that county's sheriff's office. Those costs included debt-service and fringe benefits.

(Read more: ) Experts note that New York's high annual price tag is deceiving because it reflects considerable pensions and salary responsibilities, debt service and the expensive fixed costs. The DOC says 86 percent of its operating costs go for staff wages.