spaint | A framework for interactive , real-time 3D scene segmentation | Machine Learning library

spaint is a C++ library typically used in Artificial Intelligence, Machine Learning, Deep Learning, Pytorch applications. spaint has no bugs, it has no vulnerabilities and it has low support. However spaint has a Non-SPDX License. You can download it from GitHub.

This is an open-source, real-time implementation of the interactive SemanticPaint system for geometric reconstruction, object-class segmentation and learning of 3D scenes, originally published in [Valentin15]. Using our system, a user can walk into a room wearing a depth camera and a virtual reality headset, and both densely reconstruct the 3D scene [Newcombe11,Niessner13,Kaehler15] and interactively segment the environment into object classes such as 'chair', 'floor' and 'table'. The user interacts physically with the real-world scene, touching objects and using voice commands to assign them appropriate labels. These user-generated labels are leveraged by an online random forest-based machine learning algorithm, which is used to predict labels for previously unseen parts of the scene. The entire pipeline runs in real time, and the user stays 'in the loop' throughout the process, receiving immediate feedback about the progress of the labelling and interacting with the scene as necessary to refine the predicted segmentation. This version of the SemanticPaint concept was implemented by Stuart Golodetz and Michael Sapienza, under the supervision of Professor Philip Torr. More recently, it has been significantly extended by Tommaso Cavallari, Stuart Golodetz and Nick Lord, with the most significant change being the addition of our Grove relocaliser, as described in [Cavallari17]. Our framework is built on top of InfiniTAM v3.5, a highly efficient, open-source 3D reconstruction engine developed by Oxford's Active Vision Group. Anyone interested in InfiniTAM should contact Victor Adrian Prisacariu. The original concept for SemanticPaint was developed by a large number of people both at the University of Oxford and at Microsoft Research: Julien Valentin, Vibhav Vineet, Ming-Ming Cheng, David Kim, Shahram Izadi, Jamie Shotton, Pushmeet Kohli, Matthias Niessner, Antonio Criminisi and Philip H S Torr.