SEGMENTATION OF COMPLEX STRUCTURES IN POINT CLOUDS USING DEEP NEURAL NETWORKS

Viktor Vdovychenko, Yuriy Furgala

Abstract


Background. Point cloud part segmentation assigns a semantic part label to every point and provides a more detailed description of object geometry than object-level classification. Architecture selection remains difficult because models with similar aggregate benchmark scores may differ substantially in category-level reliability and computational cost.

Materials and Methods. This study compares PointNet++, Dynamic Graph Convolutional Neural Network (DGCNN), and Point Transformer on the ShapeNet Part benchmark, which represents sixteen object categories with fifty part labels. Final evaluation was performed on 2,874 test shapes represented by 2,048 points. Segmentation quality was assessed using point accuracy, class-average accuracy, class mean intersection over union (mIoU), and instance mean intersection over union (mIoU). A separate batch-size-one benchmark measured inference latency, throughput, and graphics memory use.

Results and Discussion. Point Transformer achieved the highest overall quality, with 94.4% point accuracy, 87.7% class-average accuracy, 82.9% class mIoU, and 85.7% instance mIoU. DGCNN reached 85.3% instance mIoU while requiring only 15.87 ms per shape, compared with 53.39 ms for Point Transformer and 344.62 ms for PointNet++. Aggregate differences were small, but category-level results were more pronounced. Point Transformer produced the highest mIoU in eight of sixteen categories, while DGCNN and PointNet++ led in five and three categories, respectively. The largest architecture-dependent differences were observed for Motorbike, Earphone, and Rocket, demonstrating that similar overall scores can conceal distinct failure patterns.

Conclusion. No architecture dominated all practical criteria. Point Transformer provided the strongest segmentation quality, whereas DGCNN offered the most favorable measured speed-quality trade-off. PointNet++ remained competitive in quality but showed substantially higher latency in the evaluated implementation. Model selection should therefore consider category-level behavior, input configuration, and computational constraints together with aggregate mIoU.

Keywords: point cloud segmentation, part segmentation, ShapeNet Part, PointNet++, DGCNN, Point Transformer


Full Text:

PDF

References


[1] Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., & Bennamoun, M. (2021). Deep learning for 3D point clouds: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12), 4338–4364.
https://doi.org/10.1109/TPAMI.2020.3005434

[2] Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). PointNet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652–660. https://doi.org/10.1109/CVPR.2017.16

[3] Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). PointNet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems, 30.
https://doi.org/10.48550/arXiv.1706.02413

[4] Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019). Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 38(5), 1–12.
https://doi.org/10.1145/3326362

[5] Zhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021). Point Transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision, 16259–16268.
https://doi.org/10.1109/ICCV48922.2021.01595

[6] Yi, L., Kim, V. G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., & Guibas, L. (2016). A scalable active framework for region annotation in 3D shape collections. ACM Transactions on Graphics, 35(6), 1–12. https://doi.org/10.1145/2980179.2980238

[7] Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L., & Yu, F. (2015). ShapeNet: An information-rich 3D model repository. arXiv. https://doi.org/10.48550/arXiv.1512.03012

[8] Wu, X., Jiang, L., Wang, P.-S., Liu, Z., Liu, X., Qiao, Y., Ouyang, W., He, T., & Zhao, H. (2024). Point Transformer V3: Simpler, faster, stronger. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4840–4851. https://doi.org/10.1109/CVPR52733.2024.00463

[9] Ando, A., Gidaris, S., Bursuc, A., Puy, G., Boulch, A., & Marlet, R. (2023). RangeViT: Towards vision transformers for 3D semantic segmentation in autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5240–5250.
https://doi.org/10.1109/CVPR52729.2023.00507

[10] Zhang, J., Luo, Y., Zhang, Z., Nie, X., & Li, B. (2025). CamPoint: Boosting point cloud segmentation with virtual camera. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11822–11832. https://doi.org/10.1109/CVPR52734.2025.01104

[11] Yan, X. (2019). Pointnet_Pointnet2_pytorch: PyTorch implementation of PointNet and PointNet++ [Source code]. GitHub. https://github.com/yanx27/Pointnet_Pointnet2_pytorch

[12] Tao, A. (n.d.). dgcnn.pytorch: A PyTorch implementation of Dynamic Graph CNN for Learning on Point Clouds [Source code]. GitHub. https://github.com/antao97/dgcnn.pytorch

[13] Ji, J. (n.d.). Point-Transformer: A PyTorch implementation of Point Transformer for batch-mode point cloud processing [Source code]. GitHub.
https://github.com/ji-code25/Point-Transformer

[14] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., & Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32. https://doi.org/10.48550/arXiv.1912.01703




DOI: http://dx.doi.org/10.30970/eli.35.4

Refbacks

  • There are currently no refbacks.