Let r, g, and b be the input values and r', g', b', and a' be the output values, all scaled (for now, as it makes the math prettier) between 1 and 0. Then, by the formula for overlaying colors:

r = a' * r' + 1 - a' g = a' * g' + 1 - a' b = a' * b' + 1 - a'

The 1 - a' terms represent the background contribution, and the other terms represent the foreground. Do some algebra:

r = a' * (r' - 1) + 1 r - 1 = a' * (r' - 1) (r - 1) / (r' - 1) = a' (r' - 1) / (r - 1) = 1 / a' r' - 1 = (r - 1) / a' r' = (r - 1) / a' + 1

Intuitively, it seems that the minimum color value will be the limiting factor in the problem, so bind this to m:

m = min(r, g, b)

Set the corresponding output value, m', to zero, as we want to maximize transparency:

0 = (m - 1) / a' + 1 -1 = (m - 1) / a' -a' = m - 1 a' = 1 - m

So, in javascript (translating from 1 to 255 along the way):

function rgba(r, g, b) { var a = 1 - Math.min(r, Math.min(g, b)) / 255; return [255 + (r - 255) / a, 255 + (g - 255) / a, 255 + (b - 255) / a, a]; }

Note that I'm assuming that a' is opacity here. It is trivial to change it to transparency - just remove the "1 -" from the formula for a'. Anothing thing to note is that this does not seem to produce exact results - it said that the opacity was 0.498 for the example you gave above (128, 128, 255). However, this is extremely close.