How to detect an object that blends with the background?


I am a beginner and I am trying to apply an outline to the white remote control on the left that shares the same color with the background.
enter image description here

a = cv2.imread(file_name)
imgGray = cv2.cvtColor(a,cv2.COLOR_BGR2GRAY)

imgGray = cv2.GaussianBlur(imgGray,(11,11),20)

k5 = np.array([[-1,-1,-1],[-1,9,-1],[-1,-1,-1]])
imgGray = cv2.filter2D(imgGray,-1,k5)

while True:
    strel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))
    blocksize = cv2.getTrackbarPos("blocksize","Control")
    c = cv2.getTrackbarPos("c","Control")

    if blocksize%2==0:
        blocksize += 1
    thrash = cv2.adaptiveThreshold(imgGray,255,cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV,blockSize=blocksize,C=c)
    thrash1 = cv2.adaptiveThreshold(imgGray,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,cv2.THRESH_BINARY_INV,blockSize=blocksize,C=c)

    #r,thrash = cv2.threshold(imgGray,150,255,cv2.THRESH_BINARY_INV)
    key = cv2.waitKey(1000)
    if key == 32 or iter == -1:

edges = cv2.Canny(thrash,100,200)
cv2.imshow('grey ',imgGray)
cv2.imshow('thrash ',thrash)
circles = cv2.HoughCircles(imgGray,cv2.HOUGH_GRADIENT,1,60,param1=240,param2=50,minRadius=0,maxRadius=0)
contours,_ = cv2.findContours(thrash,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)

Those are what I have tried, I have also tried morphological operation such as dilation, erosion, opening and closing too but I am still unable to acquire the result.

Below is my best result but the noise is too severe and the remote controller didn’t get fully outlined.
enter image description here


I have thought of a pure image processing approach. But the results are not as accurate as the one depicted by @nathancy


TLDR; I am using Difference of Gaussians (DoG) which is a 2-stage edge detector.

  • Obtain the grayscale image.
  • Perform two different blurring operations on it.
  • Subtract the blurred images

Blurring operation generally acts as a suppressor of high frequencies. By subtracting the result of two different blurring operations we get a band-pass filter. I would like to quote from this blog "Subtracting one blurred image from the other preserves spatial information that lies between the range of frequencies that are preserved in the two blurred images"

I wrote a simple function that returns the difference of two blurred images:

def dog(img, k1, s1, k2, s2):
    b1 = cv2.GaussianBlur(img,(k1, k1), s1)
    b2 = cv2.GaussianBlur(img,(k2, k2), s2)
    return b1 - b2


  • Obtain grayscale image
  • Perform Gaussian blur with different kernel sizes and sigma values
  • Subtract the blurred images
  • Apply Otsu threshold
  • Find contours of sufficiently large area
  • Select contours based on their extent

Note: Extent is a property of a contour which is the ration of the contour area to its corresponding bounding rectangle area. Taken from here

Code & Results

img = cv2.imread('path_to_image', cv2.IMREAD_UNCHANGED)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Function to perform Difference of Gaussians
def difference_of_Gaussians(img, k1, s1, k2, s2):
    b1 = cv2.GaussianBlur(img,(k1, k1), s1)
    b2 = cv2.GaussianBlur(img,(k2, k2), s2)
    return b1 - b2

DoG_img = difference_of_Gaussians(gray, 7, 7, 17, 13)

enter image description here

As you can see, it functions as an edge detector. You can vary the kernel sizes (k1, k2) and sigma values (s1, s2)

# Applying Otsu Threshold and finding contours
th = cv2.threshold(DoG_img ,127,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
contours, hierarchy = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

# Create copy of original image
img1 = img.copy()

# for each contour above certain area and extent, draw minimum bounding box  
for c in contours:
    area = cv2.contourArea(c)
    if area > 1500:
        x,y,w,h = cv2.boundingRect(c)
        extent = int(area)/w*h              
        if extent > 2000:
            rect = cv2.minAreaRect(c)
            box = cv2.boxPoints(rect)
            box = np.int0(box)

enter image description here

As you can see, the result is not perfect. The shadows of the objects are also captured during the edge detection process (Difference of Gaussians). You can try varying the parameters to check if the result gets better.

Answered By – Jeru Luke

Answer Checked By – David Marino (AngularFixing Volunteer)

Leave a Reply

Your email address will not be published.