Python: putting segments on top of the input image

Assume you are working on the image /video segmentation problem. Sooner or later a day is  coming when you need to present your (hopefully good) results to other people on a conference, project meeting, in a paper, whatever. Usually segmentation results are represented by a matrix of integers standing for region labels which are supposed to be unique. A colored representation of segments is derived by assignment of unique colors based on region labels.  In this post I will show you a simple trick how you can impress the audience with your segmentation results without any changes in your segmentation method. Using Alpha Compositing you can combine an input image with colored segments as it’s shown below:

inputsegmentsoutput

For this I will use Python, NumPy, and Python Image Library (PIL). First of all we need a function generating colors for segment labels given by your segmentation technique. It can be done as follows:


#! /usr/bin/env python

import numpy as np
import Image as img

def generate_colors():

  segments_nmb = 3000
  segments_colors = []

  # generate (r,g,b) color values
  for ind in range(segments_nmb):

    r = np.random.randint(255)
    g = np.random.randint(255)
    b = np.random.randint(255)

    segments_colors.append((r,g,b))

  return segments_colors

Here we don’t check the uniqueness of generated colors, but such a check can be done easily. The following function returns colored segments in form of a 2-dimensional numpy array:


def get_colored_segments(segments, colors):

  arr_img = np.zeros([segments.shape[0],segments.shape[1],3], dtype=np.uint8)

  # segment labels which need to be displayed
  labels = set(np.unique(segments)).difference((0,))

  for lb in labels:

    segment_mask = (segments == lb)

    arr_img[:,:,0][ segment_mask.nonzero() ] = colors[int(lb)][0]
    arr_img[:,:,1][ segment_mask.nonzero() ] = colors[int(lb)][1]
    arr_img[:,:,2][ segment_mask.nonzero() ] = colors[int(lb)][2]

  return arr_img

Now we call both functions to get a colored representation of segments and put segments on top of the input image. For this we use paste function from the Python Image Library (PIL) which takes a transparency mask as an optional argument. The transparency value (here 140) is up to you depending on which effect you want to achieve.


im = img.open('input.png')
segments = np.loadtxt('segments.dat',dtype=float)

segment_colors = generate_colors()
colored_segments = get_colored_segments(segments, segment_colors)
im_segments = img.fromarray(np.uint8(colored_segments))

map_mesh = img.new('RGBA', (segments.shape[1], segments.shape[0]), (0, 0, 0, 140))
im.paste(im_segments, (0,0), map_mesh)

im.save('output.png')

Finally I would like to show you a video demonstrating segmentation of a video sequence. Final segments are shown here for each frame in conjunction with the original frame using the alpha compositing. The sequence was segmented by this segmentation technique. Enjoy!

Best wishes,
Alexey

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s