Engineering UAS Applications: Sensor Fusion, Machine Vision and Mission Management

Unmanned aerial systems (UAS) have evolved rapidly in recent years thanks to advances in microelectromechanical components, navigation, perception, and artificial intelligence, allowing for a fast development of autonomy. This book presents general approaches to develop, test, and evaluate critical functions such as navigation, obstacle avoidance and perception, and the capacity to improve performance in real and simulated scenarios. It provides the practical knowledge to install, analyze and evaluate UAS solutions working in real systems; illustrates how to use and configure complete platforms and software tools; and reviews the main enabling technologies applied to develop UAS, possibilities and evaluation methodology. You will get the tools you need to evaluate navigation and obstacle avoidance functions, object detection, and planning and landing alternatives in simulated conditions. The book also provides helpful guidance on the integration of additional sensors (video, weather, meteorological) and communication networks to build IoT solutions. This is an important book for practitioners and researchers interested in integrating advanced techniques in the fields of AI, sensor fusion and mission management, and anyone interest in applying and testing advanced algorithms in UAS platforms.

Table of Contents:
Chapter 1. Introduction and state of the art 1.1 Introduction and types of UAVs 1.2 Main technologies used in UAS 1.2.1 Navigation 1.2.2 Communications 1.2.3 Machine vision 1.2.4 Coordination. Swarms of UAVs and applications 1.2.5 Simulation 1.3 Summary and structure of this book References   Chapter 2: Components of UAS 2.1 Introduction 2.2 Flight controller 2.2.1 Logic components 2.2.1.1 Control 2.2.1.2 Navigation 2.2.1.3 Guidance 2.2.2 Physical components 2.3 Communications 2.3.1 Radio communication technologies 2.3.2 Communication protocols 2.3.3 UAV messaging protocols 2.4 Payload 2.4.1 Payload types 2.4.1.1 Image sensors 2.4.1.2 Wave sensors 2.4.1.3 Internet of Things (IoT) 2.4.1.4 Actuators 2.4.2 Payload positioning 2.4.2.1 Front sensor-FPV 2.4.2.2 Bottom sensor 2.4.2.3 Mounted fixed sensor 2.4.2.4 360º sensor 2.5 Mission management units 2.5.1 Ground station 2.5.2 Companion computer 2.5.3 Control APIs 2.6 Obstacle avoidance use case 2.6.1 Phase 1. Assembly of the physical components 2.6.2 Phase 2. Setting up the software 2.6.3 Phase 3. Mission design from the ground station 2.6.4 Phase 4. Mission start 2.6.5 Phase 5. Obstacle detection and avoidance References   Chapter 3: Simulation of UAS 3.1 Introduction 3.2 From development to reality 3.2.1 Simulation software 3.2.2 Software-in-the-loop 3.2.3 Hardware-in-the-loop 3.2.4 External Hardware-in-the-loop 3.2.5 Simulation in Hardware 3.2.6 Vehicle in the loop 3.3 UAS simulators 3.3.1 Current simulators 3.3.2 jMAVsim 3.3.3 JBSim 3.3.4 FlightGoggles 3.3.5 Gazebo 3.3.5.1 Gazebo Ignition 3.3.6 FlightGear 3.3.7 MATLAB UAV toolbox 3.3.7.1 RflySim 3.3.8 AirSim 3.3.8.1 Aerial Autonomy: Project AirSim 3.3.8.2 Unreal Engine addons 3.3.9 Simulators comparison 3.4 AirSim simulation examples 3.4.1 Framework required programs 3.4.2 Simulation environment 3.4.3 AirSim settings 3.4.4 SimpleFlight simulation 3.4.5 Mission 1. Using SimpleFlight SITL 3.4.5.1 Python mission 3.4.5.2 Data gathering 3.4.5.3 Multiple drones 3.4.5.4 Manual control using joystick or USB controller 3.4.6 PX4 simulation 3.4.7 Mission 2. Using PX4 SITL 3.4.7.1 QGroundControl mission 3.4.7.2 MavSDK mission 3.4.7.3 Multiple PX4 SITL 3.4.8 Mission 3. Using PX4 HITL 3.4.8.1 Manual control using receiver and transmitter 3.4.9 Flight analysis References   Chapter 4: Navigation systems on UAS 4.1 Introduction 4.2 Reference Frame Systems 4.2.1 Global frames (WGS84 and ECEF) and local frame at tangent point ENU and NED 4.2.2 Geodetic to ECEF transformation 4.2.3 ECEF to geodetic transformation 4.2.4 ECEF to local Cartesian (ENU and NED) transformation 4.2.5 Local Cartesian (ENU or NED) to ECEF transformation 4.3 Attitude mathematical concepts 4.3.1 Attitude representation 4.3.1.1 Direction cosine matrix (DCM) 4.3.1.2 Euler angles 4.3.1.3 Quaternions 4.3.2 Attitude Kinematics 4.3.2.1 DCM Kinematics 4.3.2.2 Euler Angles 4.3.2.3 Quaternions 4.4 Fusion of the INS and GNSS 4.4.1 State estimation 4.4.2 INS State vector 4.4.3 GNSS State vector 4.4.4 Fusion of the INS and GNSS 4.5 Application: Search for the best navigation parameters 4.5.1 Fusion quality metrics 4.5.1.1 Based on ground truth 4.5.1.2 Without ground truth 4.5.2 PX4 navigation system 4.5.3 Search best EKF parameters 4.5.3.1 Design test mission and flight plan 4.5.3.2 Analysis of results References   Chapter 5: Machine vision systems of UAS 5.1 Introduction 5.2 Computer vision system 5.2.1 Pinhole camera 5.2.2 Camera calibration 5.2.3 AirSim camera calibration 5.3 Image stabilization 5.3.1 Mechanical stabilization 5.3.1.1 Example of mechanical stabilization in AirSim 5.3.2 Computational stabilization 5.3.2.1 Digital image motion 5.3.2.2 Example of computational correction 5.4 Object detection 5.4.1 Problems of object detection 5.4.2 How to evaluate object detection? 5.4.3 Object detection example 5.5 Visual object tracking 5.5.1 Visual object tracking algorithms 5.5.2 Drone VOT system in AirSim simulator 5.5.2.1 Evaluation References   Chapter 6: Illustrative examples of UAS 6.1 Introduction 6.2 Sense and avoid illustrative example 6.2.1 Simulation architecture 6.2.2 Software modules 6.2.3 Detection System 6.2.4 Occupancy map 6.2.5 Avoidance strategy 6.2.6 Evaluation 6.2.7 Conclusions 6.3 Decentralized drone swarm illustrative example 6.3.1 Simulation architecture 6.3.2 Software modules 6.3.3 Swarm generator 6.3.4 Communication loop 6.3.5 Mission vote system 6.3.6 Formation generation avoiding collisions 6.3.7 Swarm formation to target 6.3.8 Mission movement 6.3.9 Swarm obstacle avoidance 6.3.10 Mission recalculation 6.3.11 Evaluation 6.3.12 Conclusions 6.4 Terrain reconstruction illustrative example 6.4.1 Simulation architecture 6.4.2 Software modules 6.4.3 Multi-UAV path planning 6.4.4 UAV Mission 6.4.5 Real-time orthoimagery generation 6.4.6 Evaluation 6.4.7 Conclusions References   Chapter VII: Conclusions and future works