For your source image

I would:

create mask of brown stuff Just treshold H,S and ignore V , You already have this. I use integer 255 instead of color (Blue) for latter computation. blur the mask This will remove small clusters of wrongly selected parts. After this you should treshold the mask again because the mask value will be a bit smaller then 255 unless you got fully selected areas. The bigger the area the bigger the value (closer to 255 ). I treshold with >=150 scan the mask by horizontal lines for each line find the center of gravity of all selected pixels After blurring and tresholding again the used mask is in Aqua. So compute the average point x coordinate of all masked pixels in each line. This point is marked with White. regress line through all of the centers of gravity I use mine approximation search for this but you can use any regression you want. The regressed line is marked with Red I used line equation x=x0+y*dx where y=<0,pic1.ys> . And search the solution on intervals: x0=<-pic1.xs,+2*pic1.xs> dx=<-10,+10>

Where pic1.xs,pic1.ys is the image resolution. So as you can see I do not cover full range of angles but I think this would not work on those edge cases anyway (near horizontal directions). For such cases you should do this on the Vertical lines instead and use x=y0+x*dy instead.

Here C++ source I did this with:

picture pic0,pic1; // pic0 - source img // pic1 - output img int x,y,h,s,v,px,pn,*p; color c; // copy source image to output pic1=pic0; pic1.save("cornbot0.png"); // create brown stuff mask for (y=0;y<pic1.ys;y++) // scan all H lines for (x=0;x<pic1.xs;x++) // scan actual H line { c=pic1.p[y][x]; // get pixel color rgb2hsv(c); // in HSV h=WORD(c.db[picture::_h]); s=WORD(c.db[picture::_s]); v=WORD(c.db[picture::_v]); // Treshold brownish stuff if ((abs(h- 20)<10)&&(abs(s-200)<50)) c.dd=255; else c.dd=0; pic1.p[y][x]=c; } pic1.save("cornbot1.png"); pic1.smooth(10); // blur a bit to remove small clusters as marked pic1.save("cornbot2.png"); // compute centers of gravity p=new int[pic1.ys]; // make space for points for (y=0;y<pic1.ys;y++) // scan all H lines { px=0; pn=0; // init center of gravity (avg point) variables for (x=0;x<pic1.xs;x++) // scan actual H line if (pic1.p[y][x].dd>=150) // use marked points only { px+=x; pn++; // add it to avg point pic1.p[y][x].dd=0x00004080; // mark used points (after smooth) with Aqua } if (pn) // finish avg point computation { px/=pn; pic1.p[y][px].dd=0x00FFFFFF;// mark it by White p[y]=px; // store result for line regression } else p[y]=-1; // uncomputed value } // regress line approx x0,dx; double ee; for (x0.init(-pic1.xs,pic1.xs<<1,100,3,&ee); !x0.done; x0.step()) // search x0 for (dx.init(-10.0 ,+10.0 ,1.0,3,&ee); !dx.done; dx.step()) // search dx for (ee=0.0,y=0;y<pic1.ys;y++) // compute actua solution distance to dataset if (p[y]!=-1) // ignore uncomputed values (no brown stuff) ee+=fabs(double(p[y])-x0.a-(double(y)*dx.a)); // render regressed line with Red for (y=0;y<pic1.ys;y++) { x=double(x0.aa+(double(y)*dx.aa)); if ((x>=0)&&(x<pic1.xs)) pic1.p[y][x].dd=0x00FF0000; } pic1.save("cornbot2.png"); delete[] p;

I use my own picture class for images so some members are:

xs,ys size of image in pixels

size of image in pixels p[y][x].dd is pixel at (x,y) position as 32 bit integer type

is pixel at position as 32 bit integer type p[y][x].dw[2] is pixel at (x,y) position as 2x16 bit integer type for 2D fields

is pixel at position as 2x16 bit integer type for fields p[y][x].db[4] is pixel at (x,y) position as 4x8 bit integer type for easy channel access

is pixel at position as 4x8 bit integer type for easy channel access clear(color) - clears entire image

- clears entire image resize(xs,ys) - resizes image to new resolution

- resizes image to new resolution bmp - VCL encapsulated GDI Bitmap with Canvas access

- encapsulated Bitmap with Canvas access smooth(n) - fast blur the image n times

You can further improve this with segmentation (removing small clusters) based on area and position. Also you can ignore too big peaks in avg points between neighbors. Also you can detect sky and ignore whole area where sky is present.

[edit1] smooth

This is how my smooth looks like:

void picture::smooth(int n) { color *q0,*q1; int x,y,i,c0[4],c1[4],c2[4]; bool _signed; if ((xs<2)||(ys<2)) return; for (;n>0;n--) { #define loop_beg for (y=0;y<ys-1;y++){ q0=p[y]; q1=p[y+1]; for (x=0;x<xs-1;x++) { dec_color(c0,q0[x],pf); dec_color(c1,q0[x+1],pf); dec_color(c2,q1[x],pf); #define loop_end enc_color(c0,q0[x ],pf); }} if (pf==_pf_rgba) loop_beg for (i=0;i<4;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])>>2; clamp_u8(c0[i]); } loop_end if (pf==_pf_s ) loop_beg { c0[0]=(c0[0]+c0[0]+c1[0]+c2[0])/ 4; clamp_s32(c0[0]); } loop_end if (pf==_pf_u ) loop_beg { c0[0]=(c0[0]+c0[0]+c1[0]+c2[0])>>2; clamp_u32(c0[0]); } loop_end if (pf==_pf_ss ) loop_beg for (i=0;i<2;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])/ 4; clamp_s16(c0[i]); } loop_end if (pf==_pf_uu ) loop_beg for (i=0;i<2;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])>>2; clamp_u16(c0[i]); } loop_end #undef loop_beg #define loop_beg for (y=ys-1;y>0;y--){ q0=p[y]; q1=p[y-1]; for (x=xs-1;x>0;x--) { dec_color(c0,q0[x],pf); dec_color(c1,q0[x-1],pf); dec_color(c2,q1[x],pf); if (pf==_pf_rgba) loop_beg for (i=0;i<4;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])>>2; clamp_u8(c0[i]); } loop_end if (pf==_pf_s ) loop_beg { c0[0]=(c0[0]+c0[0]+c1[0]+c2[0])/ 4; clamp_s32(c0[0]); } loop_end if (pf==_pf_u ) loop_beg { c0[0]=(c0[0]+c0[0]+c1[0]+c2[0])>>2; clamp_u32(c0[0]); } loop_end if (pf==_pf_ss ) loop_beg for (i=0;i<2;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])/ 4; clamp_s16(c0[i]); } loop_end if (pf==_pf_uu ) loop_beg for (i=0;i<2;i++) { c0[i]=(c0[i]+c0[i]+c1[i]+c2[i])>>2; clamp_u16(c0[i]); } loop_end #undef loop_beg #undef loop_end } }

It simply weight average 3 pixels

(x,y)=(2*(x,y)+(x-1,y)+(x,y-1))/4

and after that do the same with

(x,y)=(2*(x,y)+(x+1,y)+(x,y+1))/4

to avoid shifting of image. Then this whole thing is looped n times and that is all. You can ignore the clamp and pixel-format options in this case it is pf==_pf_rgba but it uses only single channel anyway ... dec_color,enc_color just unpack,pack color channels into/from array of variables to avoid truncating and overflow issues on 8 bit channels and also to format/simplify the code a bit better (for different pixel format support)

btw smooth base is the same as convolution with

0.00 0.25 0.00 0.25 0.50 0.00 0.00 0.00 0.00

and