The corner of Christopher Street and Waverly Place, in the heart of Greenwich Village, is one of the most sought after slices of Manhattan. The stoops are tidy, the restaurants are lavish and a town house down the block sold last year for $11.5 million.

Yet at that same intersection, there is a brick building at 165 Waverly Place, pinkish in color and triangular in shape, that has stood empty for more than 20 years. It appears to owe that lonely fate to the quirks of its owners and to an extraordinary set of deed restrictions on its use.

A provision dating to the early 19th century requires that the building, called the Northern Dispensary, be used to serve people who are poor and infirm, according to its owners. This might tangle the plan of any condominium developer. The deed restrictions also rule out “any obscene performances on the premises or any obscene or pornographic purposes.” And they prohibit abortions.

Now, after years of chipping paint and broken windows, there is a bit of movement at this forlorn site. The owners are patching up the roof, cleaning up the inside and trying to figure out where to go from here.