Infrastructure Robotics: Methodologies, Robotic Systems and Applications(IEEE Press Series on Systems Science and Engineering)

Illuminating resource presenting commonly used robotic methodologies and technologies, with recent developments and clear application examples across different project types Infrastructure Robotics presents state-of-the-art research in infrastructure robotics and key methodologies that enable the development of intelligent robots for operation in civil infrastructure environments, describing sensing, perception, localization, map building, environmental and operation awareness, motion and task planning, design methodologies, robot assistance paradigms, and physical human-robot collaboration. The text also presents many case studies of robotic systems developed for real-world applications in maintaining various civil infrastructures, including steel bridges, tunnels, underground water mains, underwater structures, and sewer pipes. In addition, later chapters discuss lessons learned in deployment of intelligent robots in practical applications overall. Infrastructure Robotics provides a timely and thorough treatment of the subject pertaining to recent developments, such as computer vision and machine learning techniques that have been used in inspection and condition assessment of critical civil infrastructures, including bridges, tunnels, and more. Written by highly qualified contributors with significant experience in both academia and industry, Infrastructure Robotics covers topics such as: Design methods for application of robots in civil infrastructure inspired by biological systems including ants, inchworm, and humans Fundamental aspects of research on intelligent robotic co-workers for human-robot collaborative operations The ROBO-SPECT European project and a robotized alternative to manual tunnel structural inspection and assessment Wider context for the use of additive manufacturing techniques on construction sites Infrastructure Robotics is an essential resource for researchers, engineers, and graduate students in related fields. Professionals in civil engineering, asset management, and project management who wish to be on the cutting edge of the future of their industries will also benefit from the text.

Table of Contents:
Contributors 15 Preface 17 Acronyms 21   I Methodologies 22 1 Infrastructure Robotics: an introduction 23 1.1 Infrastructure Inspection and Maintenance . . . . . . . . . . . . . 24 1.2 Infrastructure Robotics . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.3 Considerations in infrastructure robotics research . . . . . . . . 37 1.4 Opportunities and Challenges . . . . . . . . . . . . . . . . . . . . 40 1.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43   2 Design of Infrastructure Robotic Systems 49 2.1 Special Features of Infrastructure . . . . . . . . . . . . . . . . . . 49 2.2 The Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.3 Types of Robots and their Design and Operation . . . . . . . . . 52 2.4 Software System Design . . . . . . . . . . . . . . . . . . . . . . . . 56 2.5 An example: Development of the CROC Design Concept . . . . 57 2.6 Some Other Examples . . . . . . . . . . . . . . . . . . . . . . . . . 63 2.7 Actuator Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 2.8 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68   3 Perception in complex and unstructured infrastructure environments 71 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.2 Sensor description . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.2.1 2D LiDAR . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.2.2 3D LiDAR . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.2.3 Sonar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.2.4 Monocular camera . . . . . . . . . . . . . . . . . . . . . 76 3.2.5 Stereo camera . . . . . . . . . . . . . . . . . . . . . . . . 77 3.2.6 GRB-D camera . . . . . . . . . . . . . . . . . . . . . . . . 77 3.3 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.4 Theoretical Foundations . . . . . . . . . . . . . . . . . . . . . . . . 80 3.4.1 Extended Kalman filter . . . . . . . . . . . . . . . . . . . 80 3.4.2 Nonlinear least squares . . . . . . . . . . . . . . . . . . 83 3.4.3 Environment representations . . . . . . . . . . . . . . . 87 3.4.4 Mapping techniques . . . . . . . . . . . . . . . . . . . . 89 3.4.5 localization techniques . . . . . . . . . . . . . . . . . . . 94 3.4.6 SLAM techniques . . . . . . . . . . . . . . . . . . . . . . 97 3.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.5.1 localization . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.5.2 SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 3.6 Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.6.1 Mapping in confined space . . . . . . . . . . . . . . . . 107 3.6.2 localization in confined space . . . . . . . . . . . . . . 108 3.6.3 SLAM in underwater bridge environment . . . . . . . . 109 3.7 Conclusion and discussion . . . . . . . . . . . . . . . . . . . . . . 110 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 4 Machine Learning and Computer Vision Applications in Civil Infrastructure Inspection and Monitoring 113 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.2 GNN-based Pipe Failure Prediction . . . . . . . . . . . . . . . . . 115 4.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . 117 4.2.3 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . 118 4.2.4 GNN Learning . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2.5 Failure Pattern Learning . . . . . . . . . . . . . . . . . . 122 4.2.6 Failure Predictor . . . . . . . . . . . . . . . . . . . . . . . 123 4.2.7 Experimental Study . . . . . . . . . . . . . . . . . . . . . 124 4.3 Computer Vision Based Signal Aspect Transition Detection . 126 4.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . 126 4.3.2 Signal Detection Model . . . . . . . . . . . . . . . . . . 127 4.3.3 Track Detection Model . . . . . . . . . . . . . . . . . . . 129 4.3.4 Optimization for Target Locating . . . . . . . . . . . . . 133 4.4 Conclusion and Discussions . . . . . . . . . . . . . . . . . . . . . 138 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140   5 Coverage Planning and Motion Planning of Intelligent Robots for Civil Infrastructure Maintenance 147 5.1 Introduction to Coverage and Motion Planning . . . . . . . . . . 147 5.2 Coverage Planning Algorithms for a Single Robot . . . . . . . . 150 5.2.1 An Off-line Coverage Planning Algorithm . . . . . . . 150 5.2.2 A Real-time Coverage Planning Algorithm . . . . . . . 155 5.3 Coverage Planning Algorithms for Multiple Robots . . . . . . . 161 5.3.1 Base Placement Optimization . . . . . . . . . . . . . . 161 5.3.2 Area Partitioning and Allocation . . . . . . . . . . . . . 166 5.3.3 Adaptive Coverage Path Planning . . . . . . . . . . . . 171 5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178   6 Methodologies in Physical Human-Robot Collaboration for Infrastructure Maintenance 181 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 6.2 Autonomy, tele-operation, and physical human-robot collaboration . . . . . . . . . . . . . . . . . . . . . . 183 6.2.1 Autonomous Robots . . . . . . . . . . . . . . . . . . . . 184 6.2.2 Tele-operated Robots . . . . . . . . . . . . . . . . . . . . 186 6.2.3 Physical Human-Robot Collaboration . . . . . . . . . . 188 6.3 Control Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.3.1 Motion Control . . . . . . . . . . . . . . . . . . . . . . . . 190 6.3.2 Force Control . . . . . . . . . . . . . . . . . . . . . . . . 192 6.4 Adaptive Assistance paradigms . . . . . . . . . . . . . . . . . . . 194 6.4.1 Manually Adapted Assistance . . . . . . . . . . . . . . 196 6.4.2 Assistance-As-Needed paradigms . . . . . . . . . . . . 197 6.4.3 Performance-based assistance . . . . . . . . . . . . . . 198 6.4.4 Physiology-based assistance . . . . . . . . . . . . . . . 199 6.5 Safety framework for physical human-robot collaboration . . . 200 6.6 Performance-based role change . . . . . . . . . . . . . . . . . . . 203 6.7 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 6.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209   II Robotic system design and applications 216   7 Steel Bridge Climbing Robot Design and Development 219 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 7.2 Recent climbing robot platforms developed by the ARA lab . . 225 7.3 Overall Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 7.3.1 Mechanical Design and Analysis . . . . . . . . . . . . . 230 7.4 Overall Control Architecture . . . . . . . . . . . . . . . . . . . . . . 235 7.4.1 Control System Framework . . . . . . . . . . . . . . . . 236 7.5 Experiment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 7.5.1 Switching control . . . . . . . . . . . . . . . . . . . . . . 248 7.5.2 Robot navigation in mobile and Worming transformation  . . . . . . . . . . . 251 7.5.3 Robot Deployment . . . . . . . . . . . . . . . . . . . . . 254 7.6 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . 256 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257   8 Underwater robots for cleaning and inspection of underwater structures 265 8.1 Introduction to maintenance of underwater structures . . . . . 266 8.2 Robot system design . . . . . . . . . . . . . . . . . . . . . . . . . . 268 8.2.1 Hull design and manoeuvring system . . . . . . . . . . 270 8.2.2 Robot arms for docking and water-jet cleaning . . . . 271 8.3 Sensing and perception in underwater environments . . . . . . 274 8.3.1 Underwater Simultaneous Localisation and Mapping (SLAM) around bridge piles . . . . . . . . . . . . . . . . 275 8.3.2 Marine growth identification . . . . . . . . . . . . . . . 277 8.4 Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 280 8.5 Robot navigation, motion planning and system integration . . 282 8.5.1 Localisation and navigation in open water . . . . . . . 282 8.5.2 System integration . . . . . . . . . . . . . . . . . . . . . 284 8.6 Testing in a lab setup and trials in the field . . . . . . . . . . . . . 285 8.6.1 Operation procedure . . . . . . . . . . . . . . . . . . . . 287 8.6.2 Autonomous navigation in narrow environments . . . 289 8.6.3 Vision-based marine growth removing process . . . . 291 8.6.4 Inspection and marine growth identification . . . . . . 294 8.7 Reflection and lessons learned . . . . . . . . . . . . . . . . . . . . 295 8.8 Conclusion and future work . . . . . . . . . . . . . . . . . . . . . . 297 Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298   9 Tunnel structural inspection and assessment using an autonomous robotic system 301 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 9.2 ROBO-SPECT Project . . . . . . . . . . . . . . . . . . . . . . . . . . 304 9.2.1 Robotic System . . . . . . . . . . . . . . . . . . . . . . . 305 9.2.2 Intelligent Global Controller (IGC) . . . . . . . . . . . . 310 9.2.3 Ground Control Station . . . . . . . . . . . . . . . . . . 311 9.2.4 Structural Assessment Tool . . . . . . . . . . . . . . . . 311 9.3 Inspection Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 312 9.4 Extended Kalman Filter (EKF) for Mobile Vehicle Localization . 316 9.5 Mobile Vehicle Navigation . . . . . . . . . . . . . . . . . . . . . . . 319 9.6 Field Experimental Results . . . . . . . . . . . . . . . . . . . . . . 319 9.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325   10 BADGER: Intelligent robotic system for underground construction 329 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 10.2 Boring Systems and Methods . . . . . . . . . . . . . . . . . . . . . 333 10.2.1 Directional Drilling Methods . . . . . . . . . . . . . . . 333 10.2.2 Drilling Robotic Systems . . . . . . . . . . . . . . . . . 334 10.3 Main drawbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 10.4 BADGER System and Components . . . . . . . . . . . . . . . . . 339 10.4.1 Main Systems Description . . . . . . . . . . . . . . . . . 341 10.4.2 BADGER Operation . . . . . . . . . . . . . . . . . . . . . 343 10.5 Future trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348   11 Robots for Underground Pipe Condition Assessment 353 11.1 Introduction to Ferro-Magnetic Pipeline Maintenance . . . . . . 353 11.1.1 NDT Inspection Taxonomy . . . . . . . . . . . . . . . . 355 11.2 Inspection Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 11.2.1 Robot Kinematics and Locomotion . . . . . . . . . . . 358 11.3 PEC Sensing for Ferromagnetic Wall Thickness Mapping . . . . 364 11.3.1 Hardware and Software System Architecture . . . . . 366 11.4 Gaussian Processes for Spatial Regression from Sampled Inspection Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 11.4.1 Gaussian Processes . . . . . . . . . . . . . . . . . . . . 371 11.5 Field Robotic CA Inspection Results . . . . . . . . . . . . . . . . 375 11.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 378 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381   12 Robotics and Sensing for Condition Assessment of Wastewater Pipes          387 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 12.2 Non-destructive Sensing System for Condition Assessment of Sewer Walls . . . . . . . . . . . . . 391 12.3 Robotic Tool for Field Deployment . . . . . . . . . . . . . . . . . . 400 12.4 Laboratory Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 403 12.5 Field Deployment and Evaluation . . . . . . . . . . . . . . . . . . . 406 12.6 Lessons Learned and Future Directions . . . . . . . . . . . . . . 408 12.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412   13 A climbing robot for maintenance operations in confined spaces 417 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 13.2 Robot Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 13.3 Methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 13.3.1 Perception . . . . . . . . . . . . . . . . . . . . . . . . . . 429 13.3.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 13.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 13.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447   14 Multi-UAV systems for inspection of industrial and public infrastructures 449 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450 14.2 Multi-UAV Inspection of Electrical Power Systems . . . . . . . . 454 14.2.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 454 14.2.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 455 14.3 Inspection Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 14.3.1 Vehicle Routing Problem . . . . . . . . . . . . . . . . . . 457 Graph-based representation of the problem . . . . . . . . . . . . . . . 458 MILP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460 14.4 On-board Online Semantic Mapping . . . . . . . . . . . . . . . . . 467 14.4.1 GNSS-endowed Mapping System . . . . . . . . . . . . 468 14.4.2 Reflectivity and Geometry-based Semantic Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 14.4.3 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 14.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476   15 Robotic Platforms for Inspection of Oil Refineries 481 15.1 Refining Oil for Fuels and Petrochemical Basics . . . . . . . . . 482 15.2 The Inspection Process . . . . . . . . . . . . . . . . . . . . . . . . 485 15.3 Inspection and Mechanical Integrity of oil refinery components 490 15.3.1 Liquid Storage Tank Inspection . . . . . . . . . . . . . 491 15.3.2 Pressurized Vessels Inspection . . . . . . . . . . . . . 493 15.3.3 Process Pipping . . . . . . . . . . . . . . . . . . . . . . . 496 15.3.4 Heat Exchanger Bundles . . . . . . . . . . . . . . . . . 498 15.4 Plant Operations, Surveillance, Maintenance Activities, and Others . . . . . . . . . . . . . 499 15.4.1 Surveillance, Operations, and Maintenance of Oil and Gas Refineries . . . . . . . . . . 499 15.4.2 Safety and Security . . . . . . . . . . . . . . . . . . . . . 502 15.4.3 Utilities and Support Activities . . . . . . . . . . . . . . 503 15.5 Robotic Systems for Inspection . . . . . . . . . . . . . . . . . . . 504 15.5.1 Robotics for Storage Tanks . . . . . . . . . . . . . . . . 507 15.5.2 Robotics for Pressure Vessels . . . . . . . . . . . . . . 513 15.5.3 Robotics for Process Piping . . . . . . . . . . . . . . . 521 15.5.4 Robotics Heat Exchanger Bundles . . . . . . . . . . . 525 15.6 Robotics for Plant Operations, Surveillance, Maintenance, and other related activities . . . . . . . . . . . . . . . . . . . . . . . . . 527 15.6.1 Operations, Surveillance, and Maintenance of Oil and Gas Refineries with Robotic systems . . .. . . 527 15.6.2 Safety and Security Robotics . . . . . . . . . . . . . . . 531 15.6.3 Robotics for Utilities and Support Activities . . . . . . 532 15.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533   16 Drone-based Solar Cell Inspection With Autonomous Deep Learning 535 16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 16.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 536 16.1.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . 539 16.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 16.2 Aerial Robot and Detection Framework . . . . . . . . . . . . . . . 542 16.2.1 Simulation Environment . . . . . . . . . . . . . . . . . . 545 16.2.2 Solar Panel Detection . . . . . . . . . . . . . . . . . . . 545 16.2.3 Aerial Robot Trajectory . . . . . . . . . . . . . . . . . . . 548 16.2.4 Sensory Instrumentation for Aerial Robot . . . . . . . 550 16.3 Learning Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 552 16.3.1 Dataset Preparation . . . . . . . . . . . . . . . . . . . . . 553 16.3.2 CNN Architecture . . . . . . . . . . . . . . . . . . . . . . 556 16.3.3 Performance Evaluation Measures . . . . . . . . . . . 557 16.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566   17 Aerial Repair and Aerial Additive Manufacturing 579 17.1 Review of state of the art in additive manufacturing at architectural scales . . . . . . . .. . . . . 580 17.2 Review of demonstrations of aerial manufacturing and repair . 587 17.2.1 Demands and Challenges . . . . . . . . . . . . . . . . . 590 17.2.2 Future Prospects . . . . . . . . . . . . . . . . . . . . . . 594 17.3 Initial Experimental Evaluations . . . . . . . . . . . . . . . . . . . 596 17.4 Conclusions and discussion . . . . . . . . . . . . . . . . . . . . . 598 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599

About the Author :
Dikai Liu, PhD, is a Distinguished Professor at the University of Technology Sydney. Carlos Balaguer, PhD, is a Full Professor at University Carlos III of Madrid (UC3M). Gamini Dissanayake, PhD, is an Emeritus Professor at the University of Technology Sydney. Mirko Kovac, PhD, is Director of the Aerial Robotics Laboratory at Imperial College London and the Head of the Laboratory of Sustainability Robotics at the Swiss Federal Laboratories for Material Science and Technology (Empa).