One Billion Points Streamed in AR

ShipReality and Hololight showcase the first holographic visualization of ultra large ship 3D laser scans merged with complex CAD design data – using the most advanced XR streaming technology available.

Ships regularly undergo large-scale retrofits, but shipowners rarely have design data in digital form at hand. To design ship modifications, an engineering accuracy of the as-built ship geometry is required, which means each vessel must be 3D laser-scanned. ShipReality, a company specialized in AR/VR ship design automation and remote ops, synthesizes these large ship laser scans with its CAD software to design directly in 3D – resulting in merged models of CAD in the as-built ship geometry point clouds.

“We want to speed up and optimize retrofit designs for 60,000 ships that require greenhouse gas emissions reduction, energy conversions & ballast water treatment system (BWTS) retrofits in the coming years”, said Georgios Bourtzos, CEO and Co-Founder of ShipReality. “A major challenge we faced designing directly in large point clouds was visualizing entire vessels layered with resulting 3D designs for immersive design reviews on mobile XR devices like Oculus Quest 2 and HoloLens 2.“

Exploring the Use of Point Clouds in AR

Point clouds are precise models of real environments based on 3D laser or photogrammetry scanning. Objects and space are represented in the form of “points”. Millions of such points combined formulate a point cloud scan. The scan is then imported into a 3D modeling platform with the purpose of creating an as-built model. Common CAD software used for ship designs, although incorporating 3D laser scans, still rely on 2D projections and screens to visualize and design in 3D. This often results in incompatibilities with the existing ship geometry, which are only realized during installation, creating substantial delays and high additional costs.

Visualization in 3D is a key issue to address these problems. However, visualizing large point clouds requires substantial CPU and graphics power. The performance requirements are simply too high for rendering it locally on a mobile XR device. It would cause an extremely low frame rate and even software crashes. Dealing with large datasets, ShipReality had to find a solution that could surpass the limited memory, CPU and GPU resources of mobile devices.

XR Streaming Solution that Supports Point Cloud

XR streaming technology outsources the rendering process to a local server or the cloud. But not every solution on the market supports ultra large data or point clouds. 

“We integrated the Hololight Stream into our solution to stream large 3D laser scans merged with CAD retrofit designs to a HoloLens 2.”

Georgios Bourtzos, CEO and Co-Founder of ShipReality

Hololight Stream is a unique remote rendering software component that allows to stream entire augmented and virtual reality applications in real time. “The simplicity and ease of integration of the software development kit worked seamlessly with our large models,” Bourtzos added.

Integrating Hololight Stream into their engine and software, ShipReality was able to visualize a massive model containing more than one billion points. Layered on top was also BWTS CAD design data created by the company’s ShipMR-design software with additional five million polygons. To compare, a mobile XR device could only render about one and half million polygons locally. As the remote rendering server ShipReality used a moderate gaming laptop and the local WiFi, broadcasting on 2.4GHz band. More performance and bandwidth enable even greater visualizations.

Next Level Immersive Experiences

“Hololight Stream has amazing potential for AR/VR visualization of massive digital twins and real-time monitoring of projected complex 3D designs merged with as-built environments in shipping and other industries,” asserted Bourtzos. ShipReality is now able to: 

• visualize ship models that are only suited for high-performance processing
• capture large assets 1:1 and integrate complex 3D CAD designs/data
• visualize detailed models for spatial analytics in augmented reality

“Hololight Stream can save us a lot of time and resources because we can directly use point clouds in mobile AR: some pre-processing steps can be avoided,” Bourtzos further commented. Data availability, accuracy, density, and size of 3D point clouds are also forecast to vastly increase within the next years. “To realize the full potential of immersive point cloud experiences, streaming will play a major role,” concluded Georgios Bourtzos.