TROSD: A New RGB-D Dataset for Transparent and Reflective Object Segmentation in Practice

TCSVT 2023

1Tsinghua University, 2University College London

TROSNet segments transparent and reflective objects from the scene with corresponding RGB and depth input.

Abstract

Transparent and reflective objects are omnipresent in our daily lives, but their unique visual and optical characteristics are notoriously challenging even for state-of-the-art deep networks of semantic segmentation.

To alleviate this challenge, we construct a new large-scale real-world RGB-D dataset called TROSD, which is more comprehensive than existing datasets for transparent and reflective object segmentation. Our TROSD dataset contains 11,060 RGB-D images with three semantic classes in terms of transparent objects, reflective objects, and others, covering a variety of daily scenes.

Together with the dataset, we also introduce a novel network (TROSNet) as a high- standard baseline to assist other researchers to develop and benchmark their algorithms of transparent and reflective object segmentation. Moreover, extensive experiments also clearly show that the proposed TROSD dataset has an excellent capacity to facilitate the development of semantic segmentation algorithms with strong generalizability.

Transparent Subset

We give several examples of samples with transparent objects in the scene.

Reflective Subset

Here are some more examples from our TROSD with reflective objects.

Segmentation Results

Our TROSNet properly segments the target objects from the scene and classifies them according to their type.

BibTeX

@article{sun2023trosd,
  title={Trosd: A new rgb-d dataset for transparent and reflective object segmentation in practice},
  author={Sun, Tianyu and Zhang, Guodong and Yang, Wenming and Xue, Jing-Hao and Wang, Guijin},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  volume={33},
  number={10},
  pages={5721--5733},
  year={2023},
  publisher={IEEE}
}