$\begingroup$

This answer evolved over time and got quite long in the process. I've created a cleaned-up, restructured version as an answer to a very similar question on dsp.stackexchange.

Here's my quick&dirty solution. It's a bit similar to @azdahak's answer, but it uses an approximate mapping instead of cylindrical coordinates. On the other hand, there are no manually adjusted control parameters - the mapping coefficients are all determined automatically:

The label is bright in front of a dark background, so I can find it easily using binarization:

src = Import["http://i.stack.imgur.com/rfNu7.png"]; binary = FillingTransform[DeleteBorderComponents[Binarize[src]]]

I simply pick the largest connected component and assume that's the label:

labelMask = Image[SortBy[ComponentMeasurements[binary, {"Area", "Mask"}][[All, 2]], First][[-1, 2]]]

Next step: find the top/bottom/left/right borders using simple derivative convolution masks:

topBorder = DeleteSmallComponents[ImageConvolve[labelMask, {{1}, {-1}}]]; bottomBorder = DeleteSmallComponents[ImageConvolve[labelMask, {{-1}, {1}}]]; leftBorder = DeleteSmallComponents[ImageConvolve[labelMask, {{1, -1}}]]; rightBorder = DeleteSmallComponents[ImageConvolve[labelMask, {{-1, 1}}]];

This is a little helper function that finds all white pixels in one of these four images and converts the indices to coordinates ( Position returns indices, and indices are 1-based {y,x}-tuples, where y=1 is at the top of the image. But all the image processing functions expect coordinates, which are 0-based {x,y}-tuples, where y=0 is the bottom of the image):

{w, h} = ImageDimensions[topBorder]; maskToPoints = Function[mask, {#[[2]]-1, h - #[[1]]+1} & /@ Position[ImageData[mask], 1.]];

Now I have four separate lists of coordinates of the top, bottom, left, right borders of the label. I define a mapping from image coordinates to cylinder coordinates:

Clear[mapping]; mapping[{x_, y_}] := {c1 + c2*x + c3*y + c4*x*y, c5 + c6*y + c7*x + c8*x^2}

This mapping is obviously only a crude approximation to cylinder coordinates. But it's very simple to optimize the coefficients c1..c8:

minimize = Flatten[{ (mapping[#][[1]])^2 & /@ maskToPoints[leftBorder], (mapping[#][[1]] - 1)^2 & /@ maskToPoints[rightBorder], (mapping[#][[2]] - 1)^2 & /@ maskToPoints[topBorder], (mapping[#][[2]])^2 & /@ maskToPoints[bottomBorder] }]; solution = NMinimize[Total[minimize], {c1, c2, c3, c4, c5, c6, c7, c8}][[2]]

This minimizes the mapping coefficients, so the points on the left border are mapped to {0, [anything]}, the points on the top border are mapped to {[anything], 1} and so on.

The actual mapping looks like this:

Show[src, ContourPlot[mapping[{x, y}][[1]] /. solution, {x, 0, w}, {y, 0, h}, ContourShading -> None, ContourStyle -> Red, Contours -> Range[0, 1, 0.1], RegionFunction -> Function[{x, y}, 0 <= (mapping[{x, y}][[2]] /. solution) <= 1]], ContourPlot[mapping[{x, y}][[2]] /. solution, {x, 0, w}, {y, 0, h}, ContourShading -> None, ContourStyle -> Red, Contours -> Range[0, 1, 0.2], RegionFunction -> Function[{x, y}, 0 <= (mapping[{x, y}][[1]] /. solution) <= 1]]]

Now I can pass the mapping directly to ImageForwardTransformation

ImageForwardTransformation[src, mapping[#] /. solution &, {400, 300}, DataRange -> Full, PlotRange -> {{0, 1}, {0, 1}}]

The artifacts in the image are already present in the source image. Do you have a high-res version of this image? The distortion on the left side is due to the incorrect mapping. This could probably be reduced by using an improved mapping function, but I can't think of one that's better and still simple enough for minimization right now.

ADD:

I've tried the same algorithm on the high-res image you linked to in the comment, result looks like this:

I had to make minor changes to the label-detection part ( DeleteBorderComponents first, then FillingTransform ) and I've added extra terms to the mapping formula for the perspective (that wasn't noticeable in the low-res image). At the borders you can see that the 2nd order approximation doesn't fit perfectly, but this might be good enough.

And you might want to invert the mapping function symbolically and use ImageTransformation instead of ImageForwardTransformation , because this is really really slow.

ADD 2:

I think I've found a mapping that eliminates the cylindrical distortion (more or less, at least):

arcSinSeries = Normal[Series[ArcSin[α], {α, 0, 10}]] Clear[mapping]; mapping[{x_, y_}] := { c1 + c2*(arcSinSeries /. α -> (x - cx)/r) + c3*y + c4*x*y, top + y*height + tilt1*Sqrt[Clip[r^2 - (x - cx)^2, {0.01, ∞}]] + tilt2*y*Sqrt[Clip[r^2 - (x - cx)^2, {0.01, ∞}]] }

This is a real cylindrical mapping. I used the Taylor series to approximate the arc sine, because I couldn't get the optimization working with ArcSin directly. The Clip calls are my ad-hoc attempt to prevent complex numbers during the optimization. Also, I couldn't get NMinimize to optimize the coefficients, but FindMinimium will work just fine if I give it good start values. And I can estimate good start values from the image, so it should still work for any image (I hope):

leftMean = Mean[maskToPoints[leftBorder]][[1]]; rightMean = Mean[maskToPoints[rightBorder]][[1]]; topMean = Mean[maskToPoints[topBorder]][[2]]; bottomMean = Mean[maskToPoints[bottomBorder]][[2]]; minimize = Flatten[{ (mapping[#][[1]])^2 & /@ maskToPoints[leftBorder], (mapping[#][[1]] - 1)^2 & /@ maskToPoints[rightBorder], (mapping[#][[2]] - 1)^2 & /@ maskToPoints[topBorder], (mapping[#][[2]])^2 & /@ maskToPoints[bottomBorder] }]; solution = FindMinimum[ Total[minimize], {{c1, 0}, {c2, rightMean - leftMean}, {c3, 0}, {c4, 0}, {cx, (leftMean + rightMean)/2}, {top, topMean}, {r, rightMean - leftMean}, {height, bottomMean - topMean}, {tilt1, 0}, {tilt2, 0}}][[2]]

Resulting mapping:

Unrolled image:

The borders now fit the label outline quite well. The characters all seem to have the same width, so I think there's not much distortion, either. The solution of the optimization can also be checked directly: The optimization tries to estimate the cylinder radius r and the x-coordinate of the cylinder center cx, and the estimated values only a few pixels off the real positions in the image.

ADD 3:

I've tried the algorithm on a few images I've found using google image search. No manual interaction except sometimes cropping. The results look promising:

As expected, the label detection is the least stable step. (Hence the cropping.) If the user marked points inside the label and outside the label, a watershed-based segmentation will probably give better results.

I'm not sure if the mapping optimization is always numerically stable. But it worked for every image I've tried as long as the label detection worked. The original approximate mapping with the 2nd order terms is probably more stable than the improved cylindrical mapping, so it could be used as a "fallback".

For example, in the 4. sample, the radius can not be estimated from the curvature of the top/bottom border (because there is almost no curvature), so the resulting image is distorted. In this case, it might be better to use the "simpler" mapping, or have the user select the left/right borders of the glass (not the label) manually, and set the center/radius explicitly, instead of estimating it by optimizing the mapping coefficients.

ADD 4:

@Szabolcs has written interactive code that can un-distort these images.

My alternative suggestion to improve this interactively would be to let the user select the left&right borders of the image, for example using a locator pane:

LocatorPane[Dynamic[{{xLeft, y1}, {xRight, y2}}], Dynamic[Show[src, Graphics[{Red, Line[{{xLeft, 0}, {xLeft, h}}], Line[{{xRight, 0}, {xRight, h}}]}]]]]

Then I can explicitly calculate r and cx instead of optimizing for them:

manualAdjustments = {cx -> (xLeft + xRight)/2, r -> (xRight - xLeft)/2}; solution = FindMinimum[ Total[minimize /. manualAdjustments], {{c1, 0}, {c2, rightMean - leftMean}, {c3, 0}, {c4, 0}, {top, topMean}, {height, bottomMean - topMean}, {tilt1, 0}, {tilt2, 0}}][[2]] solution = Join[solution, manualAdjustments]

Using this solution, I get almost distortion-free results: