Illuminating resource presenting commonly used robotic methodologies and technologies, with recent developments and clear application examples across different project types
Infrastructure Robotics presents state-of-the-art research in infrastructure robotics and key methodologies that enable the development of intelligent robots for operation in civil infrastructure environments, describing sensing, perception, localization, map building, environmental and operation awareness, motion and task planning, design methodologies, robot assistance paradigms, and physical human-robot collaboration. The text also presents many case studies of robotic systems developed for real-world applications in maintaining various civil infrastructures, including steel bridges, tunnels, underground water mains, underwater structures, and sewer pipes. In addition, later chapters discuss lessons learned in deployment of intelligent robots in practical applications overall.
Infrastructure Robotics provides a timely and thorough treatment of the subject pertaining to recent developments, such as computer vision and machine learning techniques that have been used in inspection and condition assessment of critical civil infrastructures, including bridges, tunnels, and more.
Written by highly qualified contributors with significant experience in both academia and industry, Infrastructure Robotics covers topics such as:
- Design methods for application of robots in civil infrastructure inspired by biological systems including ants, inchworm, and humans
- Fundamental aspects of research on intelligent robotic co-workers for human-robot collaborative operations
- The ROBO-SPECT European project and a robotized alternative to manual tunnel structural inspection and assessment
- Wider context for the use of additive manufacturing techniques on construction sites
Infrastructure Robotics is an essential resource for researchers, engineers, and graduate students in related fields. Professionals in civil engineering, asset management, and project management who wish to be on the cutting edge of the future of their industries will also benefit from the text.
Table of Contents:
Contributors 15
Preface 17
Acronyms 21
I Methodologies 22
1 Infrastructure Robotics: an introduction 23
1.1 Infrastructure Inspection and Maintenance . . . . . . . . . . . . . 24
1.2 Infrastructure Robotics . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.3 Considerations in infrastructure robotics research . . . . . . . . 37
1.4 Opportunities and Challenges . . . . . . . . . . . . . . . . . . . . 40
1.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2 Design of Infrastructure Robotic Systems 49
2.1 Special Features of Infrastructure . . . . . . . . . . . . . . . . . . 49
2.2 The Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3 Types of Robots and their Design and Operation . . . . . . . . . 52
2.4 Software System Design . . . . . . . . . . . . . . . . . . . . . . . . 56
2.5 An example: Development of the CROC Design Concept . . . . 57
2.6 Some Other Examples . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.7 Actuator Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.8 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3 Perception in complex and unstructured infrastructure environments 71
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2 Sensor description . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.2.1 2D LiDAR . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.2.2 3D LiDAR . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2.3 Sonar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2.4 Monocular camera . . . . . . . . . . . . . . . . . . . . . 76
3.2.5 Stereo camera . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2.6 GRB-D camera . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.4 Theoretical Foundations . . . . . . . . . . . . . . . . . . . . . . . . 80
3.4.1 Extended Kalman filter . . . . . . . . . . . . . . . . . . . 80
3.4.2 Nonlinear least squares . . . . . . . . . . . . . . . . . . 83
3.4.3 Environment representations . . . . . . . . . . . . . . . 87
3.4.4 Mapping techniques . . . . . . . . . . . . . . . . . . . . 89
3.4.5 localization techniques . . . . . . . . . . . . . . . . . . . 94
3.4.6 SLAM techniques . . . . . . . . . . . . . . . . . . . . . . 97
3.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.5.1 localization . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.5.2 SLAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.6 Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.6.1 Mapping in confined space . . . . . . . . . . . . . . . . 107
3.6.2 localization in confined space . . . . . . . . . . . . . . 108
3.6.3 SLAM in underwater bridge environment . . . . . . . . 109
3.7 Conclusion and discussion . . . . . . . . . . . . . . . . . . . . . . 110
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4 Machine Learning and Computer Vision Applications in Civil Infrastructure Inspection and Monitoring 113
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.2 GNN-based Pipe Failure Prediction . . . . . . . . . . . . . . . . . 115
4.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . 117
4.2.3 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . 118
4.2.4 GNN Learning . . . . . . . . . . . . . . . . . . . . . . . . 119
4.2.5 Failure Pattern Learning . . . . . . . . . . . . . . . . . . 122
4.2.6 Failure Predictor . . . . . . . . . . . . . . . . . . . . . . . 123
4.2.7 Experimental Study . . . . . . . . . . . . . . . . . . . . . 124
4.3 Computer Vision Based Signal Aspect Transition Detection . 126
4.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.3.2 Signal Detection Model . . . . . . . . . . . . . . . . . . 127
4.3.3 Track Detection Model . . . . . . . . . . . . . . . . . . . 129
4.3.4 Optimization for Target Locating . . . . . . . . . . . . . 133
4.4 Conclusion and Discussions . . . . . . . . . . . . . . . . . . . . . 138
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5 Coverage Planning and Motion Planning of Intelligent Robots for Civil Infrastructure Maintenance 147
5.1 Introduction to Coverage and Motion Planning . . . . . . . . . . 147
5.2 Coverage Planning Algorithms for a Single Robot . . . . . . . . 150
5.2.1 An Off-line Coverage Planning Algorithm . . . . . . . 150
5.2.2 A Real-time Coverage Planning Algorithm . . . . . . . 155
5.3 Coverage Planning Algorithms for Multiple Robots . . . . . . . 161
5.3.1 Base Placement Optimization . . . . . . . . . . . . . . 161
5.3.2 Area Partitioning and Allocation . . . . . . . . . . . . . 166
5.3.3 Adaptive Coverage Path Planning . . . . . . . . . . . . 171
5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
6 Methodologies in Physical Human-Robot Collaboration for Infrastructure Maintenance 181
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.2 Autonomy, tele-operation, and physical human-robot collaboration . . . . . . . . . . . . . . . . . . . . . . 183
6.2.1 Autonomous Robots . . . . . . . . . . . . . . . . . . . . 184
6.2.2 Tele-operated Robots . . . . . . . . . . . . . . . . . . . . 186
6.2.3 Physical Human-Robot Collaboration . . . . . . . . . . 188
6.3 Control Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.3.1 Motion Control . . . . . . . . . . . . . . . . . . . . . . . . 190
6.3.2 Force Control . . . . . . . . . . . . . . . . . . . . . . . . 192
6.4 Adaptive Assistance paradigms . . . . . . . . . . . . . . . . . . . 194
6.4.1 Manually Adapted Assistance . . . . . . . . . . . . . . 196
6.4.2 Assistance-As-Needed paradigms . . . . . . . . . . . . 197
6.4.3 Performance-based assistance . . . . . . . . . . . . . . 198
6.4.4 Physiology-based assistance . . . . . . . . . . . . . . . 199
6.5 Safety framework for physical human-robot collaboration . . . 200
6.6 Performance-based role change . . . . . . . . . . . . . . . . . . . 203
6.7 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
II Robotic system design and applications 216
7 Steel Bridge Climbing Robot Design and Development 219
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.2 Recent climbing robot platforms developed by the ARA lab . . 225
7.3 Overall Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
7.3.1 Mechanical Design and Analysis . . . . . . . . . . . . . 230
7.4 Overall Control Architecture . . . . . . . . . . . . . . . . . . . . . . 235
7.4.1 Control System Framework . . . . . . . . . . . . . . . . 236
7.5 Experiment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
7.5.1 Switching control . . . . . . . . . . . . . . . . . . . . . . 248
7.5.2 Robot navigation in mobile and Worming transformation . . . . . . . . . . . 251
7.5.3 Robot Deployment . . . . . . . . . . . . . . . . . . . . . 254
7.6 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . 256
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
8 Underwater robots for cleaning and inspection of underwater structures 265
8.1 Introduction to maintenance of underwater structures . . . . . 266
8.2 Robot system design . . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.2.1 Hull design and manoeuvring system . . . . . . . . . . 270
8.2.2 Robot arms for docking and water-jet cleaning . . . . 271
8.3 Sensing and perception in underwater environments . . . . . . 274
8.3.1 Underwater Simultaneous Localisation and Mapping (SLAM) around bridge piles . . . . . . . . . . . . . . . . 275
8.3.2 Marine growth identification . . . . . . . . . . . . . . . 277
8.4 Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 280
8.5 Robot navigation, motion planning and system integration . . 282
8.5.1 Localisation and navigation in open water . . . . . . . 282
8.5.2 System integration . . . . . . . . . . . . . . . . . . . . . 284
8.6 Testing in a lab setup and trials in the field . . . . . . . . . . . . . 285
8.6.1 Operation procedure . . . . . . . . . . . . . . . . . . . . 287
8.6.2 Autonomous navigation in narrow environments . . . 289
8.6.3 Vision-based marine growth removing process . . . . 291
8.6.4 Inspection and marine growth identification . . . . . . 294
8.7 Reflection and lessons learned . . . . . . . . . . . . . . . . . . . . 295
8.8 Conclusion and future work . . . . . . . . . . . . . . . . . . . . . . 297
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
9 Tunnel structural inspection and assessment using an autonomous robotic system 301
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
9.2 ROBO-SPECT Project . . . . . . . . . . . . . . . . . . . . . . . . . . 304
9.2.1 Robotic System . . . . . . . . . . . . . . . . . . . . . . . 305
9.2.2 Intelligent Global Controller (IGC) . . . . . . . . . . . . 310
9.2.3 Ground Control Station . . . . . . . . . . . . . . . . . . 311
9.2.4 Structural Assessment Tool . . . . . . . . . . . . . . . . 311
9.3 Inspection Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 312
9.4 Extended Kalman Filter (EKF) for Mobile Vehicle Localization . 316
9.5 Mobile Vehicle Navigation . . . . . . . . . . . . . . . . . . . . . . . 319
9.6 Field Experimental Results . . . . . . . . . . . . . . . . . . . . . . 319
9.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
10 BADGER: Intelligent robotic system for underground construction 329
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
10.2 Boring Systems and Methods . . . . . . . . . . . . . . . . . . . . . 333
10.2.1 Directional Drilling Methods . . . . . . . . . . . . . . . 333
10.2.2 Drilling Robotic Systems . . . . . . . . . . . . . . . . . 334
10.3 Main drawbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
10.4 BADGER System and Components . . . . . . . . . . . . . . . . . 339
10.4.1 Main Systems Description . . . . . . . . . . . . . . . . . 341
10.4.2 BADGER Operation . . . . . . . . . . . . . . . . . . . . . 343
10.5 Future trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
11 Robots for Underground Pipe Condition Assessment 353
11.1 Introduction to Ferro-Magnetic Pipeline Maintenance . . . . . . 353
11.1.1 NDT Inspection Taxonomy . . . . . . . . . . . . . . . . 355
11.2 Inspection Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
11.2.1 Robot Kinematics and Locomotion . . . . . . . . . . . 358
11.3 PEC Sensing for Ferromagnetic Wall Thickness Mapping . . . . 364
11.3.1 Hardware and Software System Architecture . . . . . 366
11.4 Gaussian Processes for Spatial Regression from Sampled Inspection Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
11.4.1 Gaussian Processes . . . . . . . . . . . . . . . . . . . . 371
11.5 Field Robotic CA Inspection Results . . . . . . . . . . . . . . . . 375
11.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 378
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
12 Robotics and Sensing for Condition Assessment of Wastewater Pipes 387
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
12.2 Non-destructive Sensing System for Condition Assessment of Sewer Walls . . . . . . . . . . . . . 391
12.3 Robotic Tool for Field Deployment . . . . . . . . . . . . . . . . . . 400
12.4 Laboratory Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 403
12.5 Field Deployment and Evaluation . . . . . . . . . . . . . . . . . . . 406
12.6 Lessons Learned and Future Directions . . . . . . . . . . . . . . 408
12.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
13 A climbing robot for maintenance operations in confined spaces 417
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
13.2 Robot Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
13.3 Methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
13.3.1 Perception . . . . . . . . . . . . . . . . . . . . . . . . . . 429
13.3.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
13.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
13.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
14 Multi-UAV systems for inspection of industrial and public infrastructures 449
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
14.2 Multi-UAV Inspection of Electrical Power Systems . . . . . . . . 454
14.2.1 Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 454
14.2.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 455
14.3 Inspection Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
14.3.1 Vehicle Routing Problem . . . . . . . . . . . . . . . . . . 457
Graph-based representation of the problem . . . . . . . . . . . . . . . 458
MILP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
14.4 On-board Online Semantic Mapping . . . . . . . . . . . . . . . . . 467
14.4.1 GNSS-endowed Mapping System . . . . . . . . . . . . 468
14.4.2 Reflectivity and Geometry-based Semantic Classification
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
14.4.3 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
14.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
15 Robotic Platforms for Inspection of Oil Refineries 481
15.1 Refining Oil for Fuels and Petrochemical Basics . . . . . . . . . 482
15.2 The Inspection Process . . . . . . . . . . . . . . . . . . . . . . . . 485
15.3 Inspection and Mechanical Integrity of oil refinery components 490
15.3.1 Liquid Storage Tank Inspection . . . . . . . . . . . . . 491
15.3.2 Pressurized Vessels Inspection . . . . . . . . . . . . . 493
15.3.3 Process Pipping . . . . . . . . . . . . . . . . . . . . . . . 496
15.3.4 Heat Exchanger Bundles . . . . . . . . . . . . . . . . . 498
15.4 Plant Operations, Surveillance, Maintenance Activities, and Others . . . . . . . . . . . . . 499
15.4.1 Surveillance, Operations, and Maintenance of Oil and Gas Refineries . . . . . . . . . . 499
15.4.2 Safety and Security . . . . . . . . . . . . . . . . . . . . . 502
15.4.3 Utilities and Support Activities . . . . . . . . . . . . . . 503
15.5 Robotic Systems for Inspection . . . . . . . . . . . . . . . . . . . 504
15.5.1 Robotics for Storage Tanks . . . . . . . . . . . . . . . . 507
15.5.2 Robotics for Pressure Vessels . . . . . . . . . . . . . . 513
15.5.3 Robotics for Process Piping . . . . . . . . . . . . . . . 521
15.5.4 Robotics Heat Exchanger Bundles . . . . . . . . . . . 525
15.6 Robotics for Plant Operations, Surveillance, Maintenance, and other related activities . . . . . . . . . . . . . . . . . . . . . . . . . 527
15.6.1 Operations, Surveillance, and Maintenance of Oil and Gas Refineries with Robotic systems . . .. . . 527
15.6.2 Safety and Security Robotics . . . . . . . . . . . . . . . 531
15.6.3 Robotics for Utilities and Support Activities . . . . . . 532
15.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533
16 Drone-based Solar Cell Inspection With Autonomous Deep Learning 535
16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
16.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
16.1.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . 539
16.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542
16.2 Aerial Robot and Detection Framework . . . . . . . . . . . . . . . 542
16.2.1 Simulation Environment . . . . . . . . . . . . . . . . . . 545
16.2.2 Solar Panel Detection . . . . . . . . . . . . . . . . . . . 545
16.2.3 Aerial Robot Trajectory . . . . . . . . . . . . . . . . . . . 548
16.2.4 Sensory Instrumentation for Aerial Robot . . . . . . . 550
16.3 Learning Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 552
16.3.1 Dataset Preparation . . . . . . . . . . . . . . . . . . . . . 553
16.3.2 CNN Architecture . . . . . . . . . . . . . . . . . . . . . . 556
16.3.3 Performance Evaluation Measures . . . . . . . . . . . 557
16.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
17 Aerial Repair and Aerial Additive Manufacturing 579
17.1 Review of state of the art in additive manufacturing at architectural scales . . . . . . . .. . . . . 580
17.2 Review of demonstrations of aerial manufacturing and repair . 587
17.2.1 Demands and Challenges . . . . . . . . . . . . . . . . . 590
17.2.2 Future Prospects . . . . . . . . . . . . . . . . . . . . . . 594
17.3 Initial Experimental Evaluations . . . . . . . . . . . . . . . . . . . 596
17.4 Conclusions and discussion . . . . . . . . . . . . . . . . . . . . . 598
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599
About the Author :
Dikai Liu, PhD, is a Distinguished Professor at the University of Technology Sydney.
Carlos Balaguer, PhD, is a Full Professor at University Carlos III of Madrid (UC3M).
Gamini Dissanayake, PhD, is an Emeritus Professor at the University of Technology Sydney.
Mirko Kovac, PhD, is Director of the Aerial Robotics Laboratory at Imperial College London and the Head of the Laboratory of Sustainability Robotics at the Swiss Federal Laboratories for Material Science and Technology (Empa).
