103,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Serie: Wiley - IEEE
- Sprache: Englisch
Offers a step-by-step guide to building autonomous vehicles and robots, with source code and accompanying videos The first book of its kind on the detailed steps for creating an autonomous vehicle or robot, this book provides an overview of the technology and introduction of the key elements involved in developing autonomous vehicles, and offers an excellent introduction to the basics for someone new to the topic of autonomous vehicles and the innovative, modular-based engineering approach called DragonFly. Engineering Autonomous Vehicles and Robots: The DragonFly Modular-based Approach covers everything that technical professionals need to know about: CAN bus, chassis, sonars, radars, GNSS, computer vision, localization, perception, motion planning, and more. Particularly, it covers Computer Vision for active perception and localization, as well as mapping and motion planning. The book offers several case studies on the building of an autonomous passenger pod, bus, and vending robot. It features a large amount of supplementary material, including the standard protocol and sample codes for chassis, sonar, and radar. GPSD protocol/NMEA protocol and GPS deployment methods are also provided. Most importantly, readers will learn the philosophy behind the DragonFly modular-based design approach, which empowers readers to design and build their own autonomous vehicles and robots with flexibility and affordability. * Offers progressive guidance on building autonomous vehicles and robots * Provides detailed steps and codes to create an autonomous machine, at affordable cost, and with a modular approach * Written by one of the pioneers in the field building autonomous vehicles * Includes case studies, source code, and state-of-the art research results * Accompanied by a website with supplementary material, including sample code for chassis/sonar/radar; GPS deployment methods; Vision Calibration methods Engineering Autonomous Vehicles and Robots is an excellent book for students, researchers, and practitioners in the field of autonomous vehicles and robots.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 316
Veröffentlichungsjahr: 2020
Ähnliche
Table of Contents
Cover
1 Affordable and Reliable Autonomous Driving Through Modular Design
1.1 Introduction
1.2 High Cost of Autonomous Driving Technologies
1.3 Achieving Affordability and Reliability
1.4 Modular Design
1.5 The Rest of the Book
1.6 Open Source Projects Used in this Book
References
2 In-Vehicle Communication Systems
2.1 Introduction
2.2 CAN
2.3 FlexRay
2.4 CANopen
References
3 Chassis Technologies for Autonomous Robots and Vehicles
3.1 Introduction
3.2 Throttle-by-Wire
3.3 Brake-by-Wire
3.4 Steer-by-Wire
3.5 Open Source Car Control
3.6 OpenCaret
3.7 PerceptIn Chassis Software Adaptation Layer
References
4 Passive Perception with Sonar and Millimeter Wave Radar
4.1 Introduction
4.2 The Fundamentals of mmWave Radar
4.3 mmWave Radar Deployment
4.4 Sonar Deployment
References
5 Localization with Real-Time Kinematic Global Navigation Satellite System
5.1 Introduction
5.2 GNSS Technology Overview
5.3 RTK GNSS
5.4 RTK-GNSS NtripCaster Setup Steps
5.5 Setting Up NtripServer and NtripClient on Raspberry Pi
5.6 Setting Up a Base Station and a GNSS Rover
5.7 FreeWave Radio Basic Configuration
References
6 Computer Vision for Perception and Localization
6.1 Introduction
6.2 Building Computer Vision Hardware
6.3 Calibration
6.4 Localization with Computer Vision
6.5 Perception with Computer Vision
6.6 The DragonFly Computer Vision Module
References
7 Planning and Control
7.1 Introduction
7.2 Route Planning
7.3 Behavioral Planning
7.4 Motion Planning
7.5 Feedback Control
7.6 Iterative EM Plannning System in Apollo
7.7 PerceptIn's Planning and Control Framework
References
8 Mapping
8.1 Introduction
8.2 Digital Maps
8.3 High-Definition Maps
8.4 PerceptIn's π-Map
References
9 Building the DragonFly Pod and Bus
9.1 Introduction
9.2 Chassis Hardware Specifications
9.3 Sensor Configurations
9.4 Software Architecture
9.5 Mechanism
9.6 Data Structures
9.7 User Interface
References
10 Enabling Commercial Autonomous Space Robotic Explorers
10.1 Introduction
10.2 Destination Mars
10.3 Mars Explorer Autonomy
10.4 Challenge: Onboard Computing Capability
10.5 Conclusion
References
11 Edge Computing for Autonomous Vehicles
11.1 Introduction
11.2 Benchmarks
11.3 Computing System Architectures
11.4 Runtime
11.5 Middleware
11.6 Case Studies
References
12 Innovations on the Vehicle-to-Everything Infrastructure
12.1 Introduction
12.2 Evolution of V2X Technology
12.3 Cooperative Autonomous Driving
12.4 Challenges
References
13 Vehicular Edge Security
13.1 Introduction
13.2 Sensor Security
13.3 Operating System Security
13.4 Control System Security
13.5 V2X Security
13.6 Security for Edge Computing
References
Index
End User License Agreement
List of Tables
Chapter 2
Table 2.1 Comparisons between CAN and FlexRay.
Chapter 9
Table 9.1 DragonFly Pod chassis specification.
Table 9.2 DragonFly bus chassis specification.
Chapter 12
Table 12.1 Summary of V2X solutions for autonomous driving.
Chapter 13
Table 13.1 Summary of security threats.
List of Illustrations
Chapter 1
Figure 1.1 Cost breakdown of existing autonomous driving solutions.
Figure 1.2 Modular design of a DragonFly Pod.
Figure 1.3 Modular design architecture.
Chapter 2
Figure 2.1 Modular design architecture.
Figure 2.2 CAN protocol layers.
Figure 2.3 CAN message format.
Figure 2.4 Flowchart of a typical CANopenNode implementation.
Chapter 3
Figure 3.1 Modular design architecture.
Figure 3.2 Electronic throttle control.
Figure 3.3 Kia Soul drive-by-wire steering.
Figure 3.4 PerceptIn chassis interface.
Figure 3.5 CAN bus connection setup.
Figure 3.6 DragonFly Pod software interface.
Chapter 4
Figure 4.1 Modular design architecture.
Figure 4.2 FMCW radar block diagram: 1, synthesizer; 2, TX antenna; 3, RX an...
Figure 4.3 Sample configuration of mmWave radar.
Figure 4.4 Detection range of mmWave radar.
Figure 4.5 Hardware set.
Figure 4.6 Radar detection UI.
Figure 4.7 Radar hardware specifications.
Figure 4.8 Sample configuration of sonar.
Figure 4.9 Detection range of sonar.
Figure 4.10 Hardware set.
Figure 4.11 Sonar detection UI.
Figure 4.12 Sonar sensor specifications.
Chapter 5
Figure 5.1 Modular design architecture.
Figure 5.2 RTK GNSS.
Figure 5.3 RTK GNSS architecture.
Figure 5.4 GPS antenna installation requirements.
Figure 5.5 Raspberry Pi setup.
Figure 5.6 SD card setup.
Figure 5.7 Connecting Raspberry Pi to the GNSS station.
Figure 5.8 Setting up the base station.
Figure 5.9 Components: 1, antenna; 2, GNSS receiver module; 3, power supply ...
Figure 5.10 Setup steps.
Figure 5.11 Connection to PC.
Figure 5.12 Placement of a GNSS base station antenna.
Figure 5.13 Device setup interface.
Figure 5.14 Input of the freset command.
Figure 5.15 Freset command successful.
Figure 5.16 “gpgga com1 1” command successful.
Figure 5.17 “mode base time 60 1.5 2.5” command successful.
Figure 5.18 GPGGA fix status.
Figure 5.19 Rover hardware setup.
Figure 5.20 Rover hardware components: 1, GNSS antenna; 2, GNSS receiver mod...
Figure 5.21 Rover station antenna installation.
Figure 5.22 Cable connection.
Figure 5.23 Connecting the GNSS module.
Figure 5.24 Rover station output on the screen.
Figure 5.25 RTK-GNSS data transmission scheme by FreeWave radio.
Figure 5.26 900 MHz or 2.4 GHz radio programming screen on CoolTerm.
Figure 5.27 900 MHz or 2.4 GHz radio programming screen on GtkTerm.
Figure 5.28 LED configuration.
Chapter 6
Figure 6.1 Modular design architecture.
Figure 6.2 Seven layers of technologies to build computer vision hardware.
Figure 6.3 Synchronization.
Figure 6.4 Aprilgrid.
Figure 6.5 DragonFly sensor module.
Figure 6.6 DragonFly+ hardware architecture.
Chapter 7
Figure 7.1 Modular design architecture.
Figure 7.2 Planning and control module architecture.
Figure 7.3 Weighted graph data structure.
Figure 7.4 Dijkstra's algorithm pseudocode.
Figure 7.5 A* algorithm pseudocode.
Figure 7.6 Value iteration algorithm pseudocode.
Figure 7.7 Example of a POMDP.
Figure 7.8 RRT algorithm illustration.
Figure 7.9 RRT algorithm pseudocode.
Figure 7.10 RRT* algorithm pseudocode.
Figure 7.11 Feedback control.
Figure 7.12 A PID controller.
Figure 7.13 PID pseudocode.
Figure 7.14 Model predictive control.
Figure 7.15 Path and trajectory definition.
Figure 7.16 ST-graph example.
Figure 7.17 Modules in the Iterative EM planning.
Figure 7.18 DP-based path planning in the Iterative EM planning algorithm.
Figure 7.19 DP path and speed computation results are interpreted as decisio...
Figure 7.20 PerceptIn's planning and control framework.
Chapter 8
Figure 8.1 Modular design architecture.
Figure 8.2 OSM architecture.
Figure 8.3 JOSM user interface.
Figure 8.4 Adding a tag in JOSM.
Figure 8.5 HD map creation.
Figure 8.6 Example of our map's topology.
Figure 8.7 The Anteater Recreation Center. The solid line is the lane map th...
Figure 8.8 The resulting map in the JOSM.
Chapter 9
Figure 9.1 DragonFly two-seater pod.
Figure 9.2 DragonFly eight-seater bus.
Figure 9.3 DragonFly two-seater pod sensor configuration.
Figure 9.4 DragonFly eight-seater bus sensor configuration.
Figure 9.5 DragonFly system software architecture.
Figure 9.6 DragonFly system mechanism.
Figure 9.7 DragonFly UI when the vehicle is static.
Figure 9.8 DragonFly UI when the vehicle is moving.
Chapter 10
Figure 10.1 The future of commercial space exploration.
Figure 10.2 Grid-based traversability map.
Figure 10.3 Mars 2020 explorer.
Figure 10.4 Mars navigation map generation.
Chapter 12
Figure 12.1 V2X communications in crossroads.
Guide
Cover
Table of Contents
Begin Reading
Pages
iii
iv
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
158
156
157
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
199
200
201
202
203
204
Engineering Autonomous Vehicles and Robots
The DragonFly Modular-based Approach
Shaoshan Liu
PerceptInFremont, CAUSA
This edition first published 2020
© 2020 John Wiley & Sons Ltd
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.
The right of Shaoshan Liu to be identified as the author of this work has been asserted in accordance with law.
Registered Offices
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK
Editorial Office
The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK
For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other formats.
Limit of Liability/Disclaimer of Warranty
While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Library of Congress Cataloging-in-Publication Data
Names: Liu, Shaoshan, author.
Title: Engineering autonomous vehicles and robots : the DragonFly modular-based approach / Shaoshan Liu.
Description: First edition. | Hoboken : Wiley-IEEE Press, 2020. | Includes bibliographical references and index.
Identifiers: LCCN 2019058288 (print) | LCCN 2019058289 (ebook) | ISBN 9781119570561 (hardback) | ISBN 9781119570554 (adobe pdf) | ISBN 9781119570547 (epub)
Subjects: LCSH: Automated vehicles. | Mobile robots.
Classification: LCC TL152.8 .L585 2020 (print) | LCC TL152.8 (ebook) | DDC 629.04/6–dc23
LC record available at https://lccn.loc.gov/2019058288
LC ebook record available at https://lccn.loc.gov/2019058289
Cover Design: Wiley
Cover Images: Courtesy of Shaoshan Liu; Background © Chainarong Prasertthai/Getty Images
1Affordable and Reliable Autonomous Driving Through Modular Design
1.1 Introduction
In recent years, autonomous driving has become quite a popular topic in the research community as well as in industry, and even in the press, but besides the fact that it is exciting and revolutionary, why should we deploy autonomous vehicles? One reason is that ridesharing using clean-energy autonomous vehicles will completely revolutionize the transportation industry by reducing pollution and traffic problems, by improving safety, and by making our economy more efficient.
More specifically and starting with pollution reduction: there are about 260 million cars in the US today. If we were to convert all cars to clean-energy cars, we would reduce annual carbon emissions by 800 million tons, which would account for 13.3% of the US commitment to the Paris Agreement [1]. Also, with near-perfect scheduling, if ridesharing autonomous vehicles could be deployed, the number of cars could be reduced by 75% [2]. Consequently, these two changes combined have the potential to yield an annual reduction of 1 billion tons in carbon emission, an amount roughly equivalent to 20% of the US Commitment to the Paris Agreement.
As for safety improvement, human drivers have a crash rate of 4.2 accidents per million miles (PMM), while the current autonomous vehicle crash rate is 3.2 crashes PMM [3]. Yet, as the safety of autonomous vehicles continues to improve, if the autonomous vehicle crash rate PMM can be made to drop below 1, a whopping 30 000 lives could be saved annually in the US alone [4].
Lastly, consider the impact on the economy. Each ton of carbon emission has around a $220 impact on the US GDP. This means that $220 B could be saved annually by converting all vehicles to ride-sharing clean-energy autonomous vehicles [5]. Also, since the average cost per crash is about $30 000 in the US, by dropping the autonomous vehicle crash rate PMM to below 1, we could achieve another annual cost reduction of $300 B [6]. Therefore, in the US alone, the universal adoption of ride-sharing clean-energy autonomous vehicles could save as much as $520 B annually, which almost ties with the GDP of Sweden, one of the world's largest economies.
Nonetheless, the large-scale adoption of autonomous driving vehicles is now meeting with several barriers, including reliability, ethical and legal considerations, and, not least of which, affordability. What are the problems behind the building and deploying of autonomous vehicles and how can we solve them? Answering these questions demands that we first look at the underlying design.
1.2 High Cost of Autonomous Driving Technologies
In this section we break down the costs of existing autonomous driving systems, and demonstrate that the high costs of sensors, computing systems, and High-Definition (HD) maps are the major barriers of autonomous driving deployment [7] (Figure 1.1).
1.2.1 Sensing
The typical sensors used in autonomous driving include Global Navigation Satellite System (GNSS), Light Detection and Ranging (LiDAR), cameras, radar and sonar: GNSS receivers, especially those with real-time kinematic (RTK) capabilities, help autonomous vehicles localize themselves by updating global positions with at least meter-level accuracy. A high-end GNSS receiver for autonomous driving could cost well over $10 000.
LiDAR is normally used for the creation of HD maps, real-time localization, as well as obstacle avoidance. LiDAR works by bouncing a laser beam off of surfaces and measuring the reflection time to determine distance. LiDAR units suffer from two problems: first, they are extremely expensive (an autonomous driving grade LiDAR could cost over $80 000); secondly, they may not provide accurate measurements under bad weather conditions, such as heavy rain or fog.
Cameras are mostly used for object recognition and tracking tasks, such as lane detection, traffic light detection, and pedestrian detection. Existing implementations usually mount multiple cameras around the vehicle to detect, recognize, and track objects. However, an important drawback of camera sensors is that the data they provide may not be reliable under bad weather conditions and that their sheer amount creates high computational demands. Note that these cameras usually run at 60 Hz, and, when combined, can generate over 1 GB of raw data per second.
Figure 1.1 Cost breakdown of existing autonomous driving solutions.
Radar and sonar: The radar and sonar subsystems are used as the last line of defense in obstacle avoidance. The data generated by radar and sonar show the distance from the nearest object in front of the vehicle's path. Note that a major advantage of radar is that it works under all weather conditions. Sonar usually covers a range of 0–10 m whereas radar covers a range of 3–150 m. Combined, these sensors cost less than $5000.
1.2.2 HD Map Creation and Maintenance
Traditional digital maps are usually generated from satellite imagery and have meter-level accuracy. Although this accuracy is sufficient for human drivers, autonomous vehicles demand maps with higher accuracy for lane-level information. Therefore, HD maps are needed for autonomous driving.
Just as with traditional digital maps, HD maps have many layers of information. At the bottom layer, instead of using satellite imagery, a grid map is generated by raw LiDAR data, with a grid granularity of about 5 cm by 5 cm. This grid basically records elevation and reflection information of the environment in each cell. As the autonomous vehicles are moving and collecting new LiDAR scans, they perform self-localization by performing a real time comparison of the new LiDAR scans against the grid map with initial position estimates provided by GNSS [8].
On top of the grid layer, there are several layers of semantic information. For instance, lane information is added to the grid map to allow autonomous vehicles to determine whether they are on the correct lane when moving. On top of the lane information, traffic sign labels are added to notify the autonomous vehicles of the local speed limit, whether traffic lights are nearby, etc. This gives an additional layer of protection in case the sensors on the autonomous vehicles fail to catch the signs.
Traditional digital maps have a refresh cycle of 6–12 months. However, to make sure the HD maps contain the most up-to-date information, the refresh cycle for HD maps should be shortened to no more than one week. As a result, operating, generating, and maintaining HD maps can cost upwards of millions of dollars per year for a mid-size city.
1.2.3 Computing Systems
The planning and control algorithms and the object recognition and tracking algorithms have very different behavioral characteristics which call for different kinds of processors. HD maps, on the other hand, stress the memory [9]. Therefore, it is imperative to design a computing hardware system which addresses these demands, all within limited computing resources and power budget. For instance, as indicated in [9], an early design of an autonomous driving computing system was equipped with an Intel® Xeon E5 processor and four to eight Nvidia® K80 graphics processing unit (GPU) accelerators, connected with a Peripheral Component Interconnect-E (PCI-E) bus. At its peak, the whole system, while capable of delivering 64.5 Tera Operations Per Second (TOPS), consumed about 3000 W, consequently generating an enormous amount of heat. Also, at a cost of $30 000, the whole solution would be unaffordable (and unacceptable) to the average consumer.
1.3 Achieving Affordability and Reliability
Many major autonomous driving companies, such as Waymo, Baidu, and Uber, and several others are engaged in a competition to design and deploy the ultimate ubiquitous autonomous vehicle which can operate reliably and affordably, even in the most extreme environments. Yet, we have just seen that the cost for all sensors could be over $100 000, with the cost for the computing system another $30 000, resulting in an extremely high cost for each vehicle: a demo autonomous vehicle can easily cost over $800 000 [10]. Further, beyond the unit cost, it is still unclear how the operational costs for HD map creation and maintenance will be covered.
In addition, even with the most advanced sensors, having autonomous vehicles coexist with human-driven vehicles in complex traffic conditions remains a dicey proposition. As a result, unless we can significantly drop the costs of sensors, computing systems, and HD maps, as well as dramatically improve localization, perception, and decision-making algorithms in the next few years, autonomous driving will not be universally adopted.
Addressing these problems, a reliable autonomous vehicle has been developed by us and for low-speed scenarios, such as university campuses, industrial parks, and areas with limited traffic [11,12]. This approach starts with low speed to ensure safety, thus allowing immediate deployment. Then, with technology improvements and with the benefit of accumulated experience, high-speed scenarios will be envisioned, ultimately having the vehicle's performance equal that of a human driver in any driving scenario. The keys to enable affordability and reliability include using sensor fusion, modular design, and high-precision visual maps (HPVMs).
1.3.1 Sensor Fusion
Using LiDAR for localization or perception is extremely expensive and may not be reliable. To achieve affordability and reliability, multiple affordable sensors (cameras, GNSS receivers, wheel encoders, radars, and sonars) can be used to synergistically fuse their data. Not only do these sensors each have their own characteristics, drawbacks, and advantages but they complement each other such that when one fails or otherwise malfunctions, others can immediately take over to ensure system reliability. With this sensor fusion approach, sensor costs are limited to under $2000.
The localization subsystem relies on GNSS receivers to provide an initial localization with sub-meter-level accuracy. Visual odometry can further improve the localization accuracy down to the decimeter level. In addition, wheel encoders can be used to track the vehicles' movements in case of GNSS receiver and camera failures. Note that visual odometry deduces position changes by examining the overlaps between two frames. However, when a sudden motion is applied to the vehicle, such as a sharp turn, it is possible that visual odometry will fail to maintain localization due to the lack of overlapping regions between two consecutive frames.
The active perception subsystem seeks to assist the vehicle in understanding its environment. Based on this understanding and a combination of computer vision and of millimeter wave (mmWave) radars to detect and track static or moving objects within a 50 m range, the vehicle can make action decisions to ensure a smooth and safe trip. With stereo vision, not only can objects including pedestrians and moving vehicles be easily recognized but the distance to these detected objects can be accurately pinpointed as well. In addition, mmWave radars can also detect and track fast-moving objects and their distances under all weather conditions.
The passive perception subsystem aims to detect any immediate danger and acts as the last line of defense of the vehicle. It covers the near field, i.e. a range of 0–5 m around the vehicle. This is achieved by a combination of mmWave radars and sonars. Radars are very good moving object detectors and sonars are very good static object detectors. Depending on the current vehicle speed, when something is detected within the near field, different policies are put into place to ensure the safety of the vehicle.
1.3.2 Modular Design
In the recent past, designs of autonomous driving computing systems have tended to be costly but affordable computing solutions are possible [9]. This has been made possible by the application of modular design principles which push computing to the sensor end so as to reduce the computing demands on the main computing units. Indeed, a quad-camera module such as the DragonFly sensor module [11] alone can generate image data at a rate of 400 Mbps. If all the sensor data were transferred to the main computing unit, it would require this computing unit to be extremely complex, with many consequences in terms of reliability, power, cost, etc.
Our approach is more practical: it entails breaking the functional units into modules and having each module perform as much computing as possible. This makes for a reduction in the burden on the main computing system and a simplification in its design, with consequently higher reliability. More specifically, a GPU SoM (System on Module) is embedded into the DragonFly module to extract features from the raw images. Then, only the extracted features are sent to the main computing unit, reducing the data transfer rate a 1000-fold. Applying the same design principles to the GNSS receiver subsystem and the radar subsystem reduces the cost of the whole computing system to less than $2000.
1.3.3 Extending Existing Digital Maps
Creating and maintaining HD maps is another important component of deployment costs. Crowd-sourcing the data for creating HD maps has been proposed. However, this would require vehicles with LiDAR units, and we have already seen that LiDARs are extremely expensive and thus not ready for large-scale deployment. On the other hand, crowd-sourcing visual data is a very practical solution as many cars today are already equipped with cameras.
Hence, instead of building HD maps from scratch, our philosophy is to enhance existing digital maps with visual information to achieve decimeter-level accuracy. These are called HPVMs. To effectively help with vehicle localization, HPVMs consists of multiple layers:
The bottom layer can be any of the existing digital maps, such as Open Street Map; this bottom layer has a resolution of about 1 m.
The second layer is the ground feature layer. It records the visual features from the road surfaces to improve mapping resolution to the decimeter level. The ground feature layer is particularly useful when in crowded city environments where the surroundings are filled with other vehicles and pedestrians.
The third layer is the spatial feature layer, which records the visual features from the environments; this provides more visual features compared with the ground feature layer. It also has a mapping resolution at the decimeter level. The spatial feature layer is particularly useful in less-crowded open environments such as the countryside.
The fourth layer is the semantic layer, which contains lane labels, traffic light and traffic sign labels, etc. The semantic layer aids vehicles in making planning decisions such as routing.
1.4 Modular Design
Before we go into the details of the rest of this book, let us briefly go over the modular design methodology and introduce each module. Hopefully with this introduction, readers will be able to easily follow the contents of this book.
Figure 1.2 shows a DragonFly Pod [13], a low-speed autonomous passenger pod built utilizing the modular design methodology described in this book. This vehicle consists of multiple components, a RTK GNSS module for localization, a DragonFly computer vision module for localization (using visual inertial odometry technology) and active perception, a mmWave radar and a sonar for passive perception, a planning and control module for real-time planning, and a chassis module. Figure 1.3 shows the architecture diagram of this design and shows how the modules interact with each other.
Figure 1.2 Modular design of a DragonFly Pod.
Figure 1.3 Modular design architecture.
1.4.1 Communication System
First, to connect different modules to form a working system, a reliable communication system is needed. The Controller Area Network (CAN) bus is the most widely used in-vehicle communication network today due to its simplicity, and it can be used to connect Electronic Control Units (ECUs), sensors, and other components to enable communication with each other. Before going into the details of other components, readers should first understand how the CAN bus works.
1.4.2 Chassis
The traditional vehicle chassis utilizes mechanical control, such as mechanical cables, hydraulic pressure, and other ways of providing a driver with direct, physical control over the speed or direction of a vehicle.
However, for autonomous driving to work, we need a drive-by-wire-ready chassis such that the chassis can apply electronic controls to activate the brakes, control the steering, and operate other mechanical systems. Specifically, the chassis module provides the basic application program interfaces for the planning and control module, such that the planning and control module can perform steer, throttle, and brake actions to make sure that the vehicle travels on the planned trajectory.
1.4.3 mmWave Radar and Sonar for Passive Perception
For mid-range obstacle detection, we can apply 77 GHz mmWave radar such that the planning and control module can make decisions when obstacles are detected. Similarly, sonars cover near-range obstacles and act as the very last line of defense; once sonars detect an obstacle, they directly signal the chassis to stop to minimize risks of an accident.
mmWave radar and sonar sensors can be combined and used for passive perception. By passive perception, we mean that when obstacles are detected, the raw data are not fed to the planning and control module for decision making. Instead, the raw data are directly sent to the chassis through the CAN bus for quick decision making. In this case, a simple decision module is implemented in the chassis to stop the vehicle when an obstacle is detected within a short range.
The main reason for this design is that when obstacles are detected in close range, we want to stop the vehicle as soon as possible instead of going through the complete decision pipeline. This is the best way to guarantee the safety of passengers as well as pedestrians.
1.4.4 GNSS for Localization
The GNSS system is a natural choice for vehicle localization, especially with RTK capability, GNSS systems can achieve very high localization accuracy. GNSS provides detailed localization information such as latitude, longitude, altitude, as well as vehicle heading. Nonetheless, GNSS accuracy suffers when there are buildings and trees blocking an open sky, leading to multipath problems. Hence, we cannot solely rely on GNSS for localization.
1.4.5 Computer Vision for Active Perception and Localization
Computer vision can be utilized for both localization and active perception. For localization, we can rely on visual simultaneous localization and mapping (VSLAM) technologies to achieve accurate real-time vehicle locations. However, VSLAM usually suffers from cumulative errors such that the longer the distance the vehicle travels, the higher the localization error. Fortunately, by fusing VSLAM and GNSS localizations, we can achieve high accuracy under different conditions, because GNSS can be used as the group-truth data when it is not blocked, and VSLAM can provide high accuracy when GNSS is blocked.
In addition, computer vision can be used for active perception as well. Using stereo vision, we can extract spatial or depth information of different objects; using deep learning techniques, we can extract semantic information of different objects. By fusing spatial and semantic information, we can detect objects of interest, such as pedestrians and cars, as well as getting their distance to the current vehicle.
1.4.6 Planning and Control
The planning and control module receives inputs from perception and localization modules, and generates decisions in real time. Usually, different behaviors are defined for a planning and control module and under different conditions, one behavior is chosen.
A typical planning and control system has the following architecture: first, as the user enters the destination, the routing module checks the map for road network information and generates a route. Then the route is fed to the behavioral planning module, which checks the traffic rules to generate motion specifications. Next, the generated route along with motion specifications are passed down to the motion planner, which combines real-time perception and localization information to generate trajectories. Finally, the generated trajectories are passed down to the control system, which reactively corrects errors in the execution of the planned motions.
1.4.7 Mapping
A mapping module provides essential geographical information, such as lane configurations and static obstacle information, to the planning and control module. In order to generate real-time motion plans, the planning and control module can combine perception inputs, which detect dynamic obstacles in real time, localization inputs, which generate real-time vehicle poses, and mapping inputs, which capture road geometry and static obstacles.
Currently, fully autonomous vehicles use high definition 3D maps. Such high precision maps are extremely complex and contain a trillion bytes of data to represent not only lanes and roads but also semantic and locations of 3D landmarks in the real world. With HD maps, autonomous vehicles are able to localize themselves and navigate in the mapped area.
1.5 The Rest of the Book
In the previous sections we have introduced the proposed modular design approach for building autonomous vehicles and robots. In the rest of the book, we will delve into these topics, and present the details of each module as well as how to integrate these modules to enable a fully functioning autonomous vehicle or robot.
The first part of the book consists of Chapters 2–8, in which we introduce each module, including communication systems, chassis technologies, passive perception technologies, localization with RTK GNSS, computer vision for perception and localization, planning and control, as well as mapping technologies.
Chapter 2: In-Vehicle Communication Systems
Chapter 3: Chassis Technologies for Autonomous Robots and Vehicles
Chapter 4: Passive Perception with Sonar and mmWave Radar
Chapter 5: Localization with RTK GNSS
Chapter 6: Computer Vision for Perception and Localization
Chapter 7: Planning and Control
Chapter 8: Mapping
The second part of the book consists of Chapters 9 and 10, in which we present two interesting case studies: the first one is about applying the modular design to build low-speed autonomous vehicles; and the second one is about how NASA builds its space robotic explorer using a modular design approach.
Chapter 9: Building the DragonFly Pod and Bus
Chapter 10: Enabling Commercial Autonomous Space Robotic Explorers
From our practical experiences, the capabilities of autonomous vehicles and robots are often constrained by limited onboard computing power. Therefore, in the final part of the book, we delve into state-of-the-art approaches in building edge computing systems for autonomous vehicles and robots. We will cover onboard edge computing design, vehicle-to-everything infrastructure, as well as autonomous vehicle security.
Chapter 11: Edge Computing for Autonomous Vehicles
Chapter 12: Innovations on the Vehicle-to-Everything Infrastructure
Chapter 13: Vehicular Edge Security
1.6 Open Source Projects Used in this Book
As you can see, an autonomous driving system is a highly complex system that integrates many technology pieces and modules. Hence, it is infeasible and inefficient to build everything from scratch. Hence, we have referred to many open source projects throughout the book to help readers to build their own autonomous driving systems. Also, throughout the book we have used PerceptIn's autonomous driving software stack to demonstrate the idea of modular design. The open source projects used in this book are listed below:
CANopenNode
[
14
]: This is free and open source CANopen Stack is for CAN bus communication.
Open Source Car Control
[
15
]: This is an assemblage of software and hardware designs that enable computer control of modern cars in order to facilitate the development of autonomous vehicle technology. It is a modular and stable way of using software to interface with a vehicle's communications network and control systems.
OpenCaret
[
16
]: This is an open source Level-3 Highway autopilot system for Kia Soul EV.
NtripCaster
[
17
]: A GNSS
NTRIP
(
Networked Transport of RTCM via Internet Protocol
) Caster takes GNSS data from one or more data stream sources (Base Stations referred to as NTRIP Servers) and provides these data to one or more end users (often called rovers), the NTRIP Clients. If you need to send data to more than one client at a time, or have more than one data stream, you will need a Caster.
GPSD
(
GPS Daemon
)
[
18
]: This is a service daemon that monitors one or more GNSS receivers attached to a host computer through serial or USB ports, making all data on the location/course/velocity of the sensors available to be queried on Transmission Control Protocol port 2947 of the host computer. With GPSD, multiple location-aware client applications can share access to supported sensors without contention or loss of data. Also, GPSD responds to queries with a format that is substantially easier to parse than the NMEA 0183 emitted by most GNSS receivers.
Kalibr
[
19
]: This is a toolbox that solves the following calibration problems:
– Multiple camera calibration: intrinsic and extrinsic calibration of a camera system with non-globally shared overlapping fields of view.
– Visual-inertial calibration (camera-IMU): spatial and temporal calibration of an IMU with respect to a camera system.
– Rolling shutter camera calibration: full intrinsic calibration (projection, distortion, and shutter parameters) of rolling shutter cameras.
OpenCV
[
20
]: OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
ORB-SLAM2
[
21
]: This is a real-time SLAM library for Monocular, Stereo and RGB-D cameras that computes the camera trajectory and a sparse 3D reconstruction. It is able to detect loops and relocalize the camera in real time.
libELAS
[
22
]: This is a cross-platform C++ library with MATLAB wrappers for computing disparity maps of large images. Input is a rectified grayscale stereo image pair of the same size. Output is the corresponding disparity maps.
Mask R-CNN
[
23
]: This is a deep learning model for object detection and instance segmentation on Keras and TensorFlow.
Baidu Apollo
[
24
