MASKRCNN 自定义数据集

maskrcnn 是图像分割中支持 实例分割的开源框架,使用MATTERPORT MASKRCNN 很容易上手。

图片集包括训练集和验证集。本篇讲述如何使用VIA 标注工具对数据集进行标注。本文章包括一下内容。


  • VIA 介绍
  • 图片标注
  • load_mask分析


VIA介绍

目前使用via版本是 1.6 ,网上在线标注链接 

https://www.robots.ox.ac.uk/~vgg/software/via/via-1.0.6.html 

微信截图_20210727140515.png

标注时需要在下边的 region attibutes 中写上 name 和 类别名称,比如这个井盖 类别是 manhole .

  • 图片标注
标注支持多边形,圆形,正方形,椭圆形等。


需要注意的是 标注后的多边形,圆形椭圆不要超出图像本身的范围。

比如要标注的是圆形,但是当前的图样中只有半圆建议用多边形进行标注。

我使用的数据集是从VGG图像注释器(VIA)创建的。注释文件是.json格式的,其中包含我在图像上绘制的所有多边形的坐标。.json文件如下所示:


{"00b1f292-23dd-44d4-aad3-c1ffb6a6ad5a___RS_LB 4479.JPG21419":{"filename":"00b1f292-23dd-44d4-aad3-c1ffb6a6ad5a___RS_LB 4479.JPG","size":21419,"regions":[{ "shape_attributes":{ "name":"polygon","cx":83,"cy":177,"r":43}, "region_attributes":{ "name":"Horse", "image_quality":{ "good":true,"frontal":true,"good_illumination":true } } }, {"shape_attributes":{ 'all_points_x': [1,2,4,5], 'all_points_y': [0.2,2,5,7],  'name': 'polygon'}, "region_attributes":{ "name":"Man", "image_quality":{ "good":true,"frontal":true,"good_illumination":true}}}, {"shape_attributes":{"name":"ellipse","cx":156,"cy":189,"rx":19.3,"ry":10,"theta":-0.289}, "region_attributes":{"name":"Horse","image_quality":{"good":true,"frontal":true,"good_illumination":true}}}],"file_attributes":{"caption":"","public_domain":"no","image_url":""}},..., ...}  1.6版本  # Load annotations
        # VGG Image Annotator saves each image in the form:
        # { 'filename': '28503151_5b5b7ec140_b.jpg',
        #   'regions': {
        #       '0': {
        #           'region_attributes': {},
        #           'shape_attributes': {
        #               'all_points_x': [...],
        #               'all_points_y': [...],
        #               'name': 'polygon'}},
        #       ... more regions ...
        #   },
        #   'size': 100202
        # }
        #"filename":"image54.jpg",
        #"base64_img_data":"","file_attributes":{},
        #"regions":{
        #"0":{
        #   "shape_attributes":{
        #       "name":"ellipse",
        #       "cx":437,"cy":1007,"rx":278,"ry":166
        #   },
        #   "region_attributes":{}}
        #   }
  


  • load_mask分析
MASKRCNN中自带的ballon.py使用的是多边形 标注。而我们知道 标注的物体多种多样,会有圆形 椭圆等,所以

数据集的mask load 处理需要用户自己进行编写。

 def load_mask(self, image_id):
        """Generate instance masks for an image.
       Returns:
        masks: A bool array of shape [height, width, instance count] with
            one mask per instance.
        class_ids: a 1D array of class IDs of the instance masks.
        """
        # If not a Horse/Man dataset image, delegate to parent class.
        image_info = self.image_info[image_id]
        if image_info["source"] != "object":
            return super(self.__class__, self).load_mask(image_id)

        # Convert polygons to a bitmap mask of shape
        # [height, width, instance_count]
        info = self.image_info[image_id]
        if info["source"] != "object":
            return super(self.__class__, self).load_mask(image_id)
        num_ids = info['num_ids']
        mask = np.zeros([info["height"], info["width"], len(info["polygons"])],
                        dtype=np.uint8)
        for i, p in enumerate(info["polygons"]):
            # Get indexes of pixels inside the polygon and set them to 1
        	rr, cc = skimage.draw.polygon(p['all_points_y'], p['all_points_x'])

        	mask[rr, cc, i] = 1

        # Return mask, and array of class IDs of each instance. Since we have
        # one class ID only, we return an array of 1s
        # Map class names to class IDs.
        num_ids = np.array(num_ids, dtype=np.int32)
        return mask, num_ids #np.ones([mask.shape[-1]], dtype=np.int32)



前面我们讨论过,在标注图片时不应使用多个形状,因为load mask时形状会变得复杂。但是如果存在呢?

https://towardsdatascience.com/mask-rcnn-implementation-on-a-custom-dataset-fd9a878123d4

的网友建议这么写:

def load_mask(self, image_id):
  ...
  ##change the for loop only 
  for i, p in enumerate(info["polygons"]):
    # Get indexes of pixels inside the polygon and set them to 1
      if p['name'] == 'polygon':
        rr, cc = skimage.draw.polygon(p['all_points_y'], p['all_points_x'])            
      elif p['name'] == 'circle':
        rr, cc = skimage.draw.circle(p['cx'], p['cy'], p['r'])
      else: 
        rr, cc = skimage.draw.ellipse(p['cx'], p['cy'], p['rx'], p['ry'], rotation=np.deg2rad(p['theta']))  
      mask[rr, cc, i] = 1
  
  ...
但是 根据 scikit image 提供的接口,https://scikit-image.org/docs/dev/api/skimage.draw.html


>>> from skimage.draw import ellipse
>>> img = np.zeros((10, 12), dtype=np.uint8)
>>> rr, cc = ellipse(5, 6, 3, 5, rotation=np.deg2rad(30))
>>> img[rr, cc] = 1
>>> img
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0],
       [0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0],
       [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0],
       [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0],
       [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)
应该改成

      if p['name'] == 'ellipse':
       
        rr, cc = skimage.draw.ellipse(p['cy'], p['cx'], p['ry'], p['rx'], rotation=np.deg2rad(p['theta']))  
      mask[rr, cc, i] = 1
如果没有角度,
 rr, cc = skimage.draw.ellipse(p['cy'], p['cx'], p['ry'], p['rx'])  
      


sitemap