Point Cloud Computing Enables Greater Smartness in Precise Reconstruction of Historical Sites

Tsung-Tsong Wu, upon his appointment as Minister of the Ministry of Science and Technology, emphasized the importance of humanity and social sciences for the drive towards an innovative country. Render Farm1 constructed by the National Center for High-Performance Computing (NCHC) under the Ministry of Science and Technology, has assisted in the development of many 3D animation talents by providing the rendering technology required in the cultural and creative industries in Taiwan. At this juncture, the point cloud technology is being applied to the construction of 3D models for a variety of artifacts, historical sites and other buildings. The use of Artificial Intelligence can shorten the timeframe from six months to one month for the correction of abnormal point clouds. The harnessing of technology to reduce the requirement for manual efforts and enhance the capability in historical sites maintenance is the paradigm for the integration of technology and humanity. 

Point clouds are the digital and dimensional records of real-life objects, comprised of millions to billions of points. Each point is the cartesian coordinates of objects obtained at speed, high accuracy and a large quantity, with the scanning and measurement by LiDAR2. These points are aggregated into the 3D model of the objects. This is then combined with colored digital photos to map the correct colors to the right locations for the construction of a digital 3D model in great resemblance with real-life objects. Point clouds can be used in historical sites maintenance, architectural engineering, autonomous driving, and digital cities. 

Introduction of point cloud computing by NCHC Director Chia-chen Kuo

Erroneous color mapping is inevitable based on the existing point cloud technology. For example, swaying tree leaves are often mapped to the color of the skies, and leads to an image that looks like snow falling. The residual of images of passengers is another typical problem. According to the experience of many metrology service providers, there have been efforts to improve the measurement technology and environmental conditions to address these problems. The other methods adopted in the past were to manually recut and recolor the abnormal point clouds in individual regions. Manual coloring was labor-intensive, time-consuming and often not resulting in adequate fidelity.  

To address this issue, the NCHC has developed a method of correcting abnormal point clouds for the point cloud computing services offered by Render Farm, by using Artificial Intelligence to identify abnormal cloud points and automate color corrections. This method fills the void of existing products/technologies. 

The method of correcting abnormal point clouds is to apply Artificial Intelligence for the differentiation of normal colors and abnormal colors for the point clouds. This is followed by the correction based on nearest neighbor regression and laser reflected strengths. Recoloring is administered by overriding the abnormal colors with the right colors. The automation of color corrections can shorten the timeframe of manual processing from six months to one month and achieve greater realism and accuracy for the cloud model. This technology has been used by Japanese Navy’s Sixth Fuel Factory - Hsinchu Branch, and Qi Dong Poetry Salon of Taipei City for the recreation of digital scenes and the assistance in the preservation and utilization of historical sites. 

From the left: Director Kent Chang, Kent Animation; Deputy Director Hsin-Yao Li, Cultural Affairs Bureau, Hsinchu City; Deputy Mayor Hui-Hung Shen, Hsinchu City Government; Director General Shepherd Shi, NCHC; Director Shuo-bin Su, National Museum of Taiwan Literature; Director Chia-chen Kuo, NCHC; General Manager Yin-Jui Huang, LeaderTek

The patent for this technology is pending in Taiwan and the U.S. This technology has been deployed on the NCHC’s Render Farm as a value-added service for point cloud computing to serve the customers’ needs for model corrections by combining the automated visualization powered by high-performance computing on the platform. To assist the across-the-board upgrade of point cloud technologies in Taiwan, The NCHC also hopes to license this technology to cloud software developers, LIDAR manufacturers, metrology service operators, spatial information service providers. 

1.Render Farm: Established by the NCHC in 2011, Render Farm provides high-performance computing services for rendering of animation and special effects. The purpose is to drive the application of technologies to the cultural industries and talent development and to assist in the production of high-quality movie special effects and animation films.   

2.LIDAR: LIDAR stands for light detection and ranging. Working similarly with radar, LIDAR sends off laser beams to objects and picks up reflected light. The cartesian coordinates of an object are determined by calculating the distance between the laser source and the object based on the time difference between sending and receiving.