MASKRCNN 自定义数据集
maskrcnn 是图像分割中支持 实例分割的开源框架，使用MATTERPORT MASKRCNN 很容易上手。
图片集包括训练集和验证集。本篇讲述如何使用VIA 标注工具对数据集进行标注。本文章包括一下内容。
- VIA 介绍
- 图片标注
- load_mask分析
VIA介绍
目前使用via版本是 1.6 ，网上在线标注链接
https://www.robots.ox.ac.uk/~vgg/software/via/via-1.0.6.html
标注时需要在下边的 region attibutes 中写上 name 和 类别名称，比如这个井盖 类别是 manhole .
- 图片标注
标注支持多边形，圆形，正方形，椭圆形等。
需要注意的是 标注后的多边形，圆形椭圆不要超出图像本身的范围。
比如要标注的是圆形，但是当前的图样中只有半圆建议用多边形进行标注。
我使用的数据集是从VGG图像注释器（VIA）创建的。注释文件是.json格式的，其中包含我在图像上绘制的所有多边形的坐标。.json文件如下所示：
{"00b1f292-23dd-44d4-aad3-c1ffb6a6ad5a___RS_LB 4479.JPG21419":{"filename":"00b1f292-23dd-44d4-aad3-c1ffb6a6ad5a___RS_LB 4479.JPG","size":21419,"regions":[{ "shape_attributes":{ "name":"polygon","cx":83,"cy":177,"r":43}, "region_attributes":{ "name":"Horse", "image_quality":{ "good":true,"frontal":true,"good_illumination":true } } }, {"shape_attributes":{ 'all_points_x': [1,2,4,5], 'all_points_y': [0.2,2,5,7], 'name': 'polygon'}, "region_attributes":{ "name":"Man", "image_quality":{ "good":true,"frontal":true,"good_illumination":true}}}, {"shape_attributes":{"name":"ellipse","cx":156,"cy":189,"rx":19.3,"ry":10,"theta":-0.289}, "region_attributes":{"name":"Horse","image_quality":{"good":true,"frontal":true,"good_illumination":true}}}],"file_attributes":{"caption":"","public_domain":"no","image_url":""}},..., ...} 1.6版本 # Load annotations
# VGG Image Annotator saves each image in the form:
# { 'filename': '28503151_5b5b7ec140_b.jpg',
# 'regions': {
# '0': {
# 'region_attributes': {},
# 'shape_attributes': {
# 'all_points_x': [...],
# 'all_points_y': [...],
# 'name': 'polygon'}},
# ... more regions ...
# },
# 'size': 100202
# }
#"filename":"image54.jpg",
#"base64_img_data":"","file_attributes":{},
#"regions":{
#"0":{
# "shape_attributes":{
# "name":"ellipse",
# "cx":437,"cy":1007,"rx":278,"ry":166
# },
# "region_attributes":{}}
# }
MASKRCNN中自带的ballon.py使用的是多边形 标注。而我们知道 标注的物体多种多样，会有圆形 椭圆等，所以
数据集的mask load 处理需要用户自己进行编写。
def load_mask(self, image_id):
"""Generate instance masks for an image.
Returns:
masks: A bool array of shape [height, width, instance count] with
one mask per instance.
class_ids: a 1D array of class IDs of the instance masks.
"""
# If not a Horse/Man dataset image, delegate to parent class.
image_info = self.image_info[image_id]
if image_info["source"] != "object":
return super(self.__class__, self).load_mask(image_id)
# Convert polygons to a bitmap mask of shape
# [height, width, instance_count]
info = self.image_info[image_id]
if info["source"] != "object":
return super(self.__class__, self).load_mask(image_id)
num_ids = info['num_ids']
mask = np.zeros([info["height"], info["width"], len(info["polygons"])],
dtype=np.uint8)
for i, p in enumerate(info["polygons"]):
# Get indexes of pixels inside the polygon and set them to 1
rr, cc = skimage.draw.polygon(p['all_points_y'], p['all_points_x'])
mask[rr, cc, i] = 1
# Return mask, and array of class IDs of each instance. Since we have
# one class ID only, we return an array of 1s
# Map class names to class IDs.
num_ids = np.array(num_ids, dtype=np.int32)
return mask, num_ids #np.ones([mask.shape[-1]], dtype=np.int32)
前面我们讨论过，在标注图片时不应使用多个形状，因为load mask时形状会变得复杂。但是如果存在呢？
https://towardsdatascience.com/mask-rcnn-implementation-on-a-custom-dataset-fd9a878123d4
的网友建议这么写：
def load_mask(self, image_id):
...
##change the for loop only
for i, p in enumerate(info["polygons"]):
# Get indexes of pixels inside the polygon and set them to 1
if p['name'] == 'polygon':
rr, cc = skimage.draw.polygon(p['all_points_y'], p['all_points_x'])
elif p['name'] == 'circle':
rr, cc = skimage.draw.circle(p['cx'], p['cy'], p['r'])
else:
rr, cc = skimage.draw.ellipse(p['cx'], p['cy'], p['rx'], p['ry'], rotation=np.deg2rad(p['theta']))
mask[rr, cc, i] = 1
...
但是 根据 scikit image 提供的接口，https://scikit-image.org/docs/dev/api/skimage.draw.html
>>> from skimage.draw import ellipse
>>> img = np.zeros((10, 12), dtype=np.uint8)
>>> rr, cc = ellipse(5, 6, 3, 5, rotation=np.deg2rad(30))
>>> img[rr, cc] = 1
>>> img
array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0],
[0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0],
[0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0],
[0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0],
[0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)
应该改成
if p['name'] == 'ellipse':
rr, cc = skimage.draw.ellipse(p['cy'], p['cx'], p['ry'], p['rx'], rotation=np.deg2rad(p['theta']))
mask[rr, cc, i] = 1
如果没有角度，
rr, cc = skimage.draw.ellipse(p['cy'], p['cx'], p['ry'], p['rx'])
