The DARPA Grand Challenge:
The US government, seeking unmanned ground vehicles for battlefield safety, invested in research and development, but the systems were limited and brittle. DARPA created the Grand Challenge: a race across the desert for a robot to drive itself without human intervention.

The First Grand Challenge (2004):
106 teams competed, including top universities, companies, and individuals. The course was 142 miles, with GPS breadcrumbs as guidance but inaccurate enough to require steering and velocity decisions. The race finished at mile 7.3, with Carnegie Mellon’s vehicle reaching a ridge and getting stuck.

Stanford’s Entry (2005):
Stanford assembled a team and started with no vehicle, no money, and no resources. They created a course, “DARPA Grand Challenge,” where students would build a self-driving car in one year for course credit. Volkswagen provided Stanford with a Touareg vehicle.

Equipping the Car:
The car was equipped with inertial guidance systems, combining GPS receivers and inertial measurement units for navigation. A simple control mechanism was devised to keep the vehicle on the GPS-defined route, adjusting the front tires’ alignment.

Cross Track Error:
The vehicle can stay on course by measuring and adjusting the cross track error. PID control with differential and integral terms is used for fine-tuning.

Laser Obstacle Detection:
Rotating mirrors direct laser beams to a planar area, where they reflect off objects. The time of flight is measured to determine distance.

Motion Planning:
Simple motion planning involves following a road while avoiding obstacles detected by lasers. Lateral offset to the reference trajectory can be adjusted to avoid obstacles. Different speeds and swerves can be simulated to find the best path.

Initial Tests:
Early tests in Arizona showed promising results, reaching the level of 2004 competitors in just five weeks. Laser-based obstacle avoidance was implemented for improved safety.

Challenges and Failures:
In a test at a parking garage, Stanley successfully navigated around obstacles. In the desert, Stanley encountered numerous obstacles and struggled to maintain its course, ultimately failing after 8.5 miles. The failure occurred near the same spot where Carnegie Mellon had previously failed.

Stanford’s Team and Architecture:
Sebastian Thrun assembled a team of 60 people, including software engineers, computer vision experts, planning and optimization specialists, and alumni. Android provided sponsorship and various resources like software, vehicles, cameras, and caffeine. Stanley’s architecture featured a traditional cognitive architecture, with sensors, perceptual models, state estimation, planning and control modules, and an interface.

Site Visit and Selection Process:
DARPA received 195 submissions and aimed to select 40 promising teams for further support in the race. Stanford submitted a video showcasing their system, including a demonstration of navigating around an obstacle (trash bin) without a human driver. Carnegie Mellon submitted two vehicles, increasing their chances of selection, and passed the site visit with flying colors. Other participants included car enthusiasts focused on speed and track performance, as well as a team called Team Axiom, known for their humorous approach and inflatable palm tree.

Stanford’s Progress and Challenges:
Stanford achieved their goal and was selected as one of the 43 semifinalists, along with Carnegie Mellon’s Red Team. The team focused on eliminating bugs and improving the system’s reliability to achieve hundreds of miles of autonomous driving without intervention. They started with a limited range of about a mile before needing intervention, aiming to increase it significantly.

Laser Interpretation:
Stanley used a scanning laser to build a 3D model of its environment, inferring obstacles from z-elevation differences. Vehicle pitch caused scan lines to reverse order, leading to erroneous obstacle detection due to pitch angle error.

Probabilistic Obstacle Detection:
Classical data processing considered sensor readings as rock solids, leading to false obstacle detection. To address this, probabilities were introduced to determine the likelihood of two points being in vertical alignment, reducing false positives. The error model was learned using discriminative machine learning, minimizing false positives and maximizing correct positive rates.

Camera-Based Road Detection:
To drive faster, Stanley needed to see further than the lasers allowed. A camera was used to find roads, but existing AI methods were insufficient for the task. A novel approach was developed to detect roads by identifying regions of the color spectrum and smoothness. Henrik Dahlkamp’s idea of “detecting some of rote finding” led to the use of the laser system to identify roads closer to the car, which were then used to inform the camera’s road detection further out.

Laser System for Obstacle Detection:
A laser system can provide samples of the world’s current appearance. By extracting a drivable region from the laser data, a mixture of Gaussian model can be trained and applied to the entire image to estimate drivable areas up to the horizon. This approach extends the perceptual range of the self-driving car beyond the limited range of the laser system.

Benefits of Adaptive Vision:
Increased driving speed due to the extended perceptual range. Earlier detection of obstacles, leading to safer driving. Improved performance in various conditions, such as changing road colors and lighting conditions.

Adaptation to Different Surfaces:
The adaptive vision system can recognize and adapt to different surfaces, such as grass and hay bales, allowing the car to drive reliably on various terrains.

Challenges in Computer Vision:
Computer vision is a complex field with many challenges, making it difficult to develop reliable vision systems. Adaptive vision systems, however, can overcome these challenges by continuously adapting to changing conditions, such as lighting and road colors.

Real-World Results:
The adaptive vision system demonstrated impressive results in real-world testing, including qualification events where the car had to transition between different surfaces. The system successfully adapted to various conditions, enabling reliable driving in challenging environments.

Vehicle Speed Control:
The autonomous vehicle faced challenges with excessive speed in treacherous terrains, posing safety concerns. Adaptive mechanisms for speed control were developed to address this issue. Competitors employed a team of editors and management personnel to manually label terrain data and specify appropriate speeds.

Machine Learning for Speed Control:
The team utilized machine learning to train the vehicle’s speed control system. Data was collected by having a human driver navigate through challenging terrains, demonstrating the desired driving behavior. A controller was designed to monitor shock, terrain steepness, and narrowness, adjusting speed accordingly. The system gradually increased speed after encountering a shock until another shock was detected.

Testing and Performance:
Extensive testing was conducted in Arizona, balancing development efforts with personal lives. The vehicle demonstrated competence in long-distance driving, completing a 200-mile run in one day. Average speeds of around 22 miles per hour were achieved, which was significant in the desert environment. The system’s steering was slightly delayed compared to human drivers but functioned effectively overall.

Drunk Programming and Legal Implications:
Mike, a team member, noted the irony that it is illegal to drive drunk but not to program a car while intoxicated. This highlights the evolving legal and ethical considerations surrounding autonomous vehicles.

The Competitors:
Carnegie Mellon: Practiced in Pittsburgh’s industrial wasteland. Won the challenge. Berkeley: Anthony Lewandowski was a notable competitor. Faced challenges in controlling the motorcycle. Developed a unique steering technique for the motorcycle. Stanford: Faced difficulties due to a major rainfall in the Mojave Desert. Visibility was limited due to water splashing up. The lasers on the roof were the only visible elements.

The Race:
Final race took place in Fontana, Speedway, and Prim, Nevada. Fontana was a qualifying event to select the final 20 teams. Stanford chose blue shirts because Carnegie Mellon had already chosen red.

Challenges on the Course:
The DARPA Grand Challenge course was designed to resemble the challenges of desert racing. The course included a narrow gate, an abandoned car, a high-speed section, a tunnel, a power pole, and hay bales.

The Tunnel:
The tunnel was the most difficult obstacle on the course. Cars lost GPS coverage inside the tunnel, which made it difficult to navigate. Cars had to carefully integrate their data to avoid errors in their maps after exiting the tunnel.

Solutions to the GPS Problem:
Some teams used a full-throttle approach, accelerating to 60 miles per hour in the tunnel. Carnegie Mellon University miscalibrated their sensors, causing them to go over hay bales.

Teams’ Struggles:
Many teams failed to make it through the first gate or the second run into a car. Some teams had difficulty organizing their software, leading to problems with driving.

Unmanned Vehicles on the Course:
At one point, there were two unmanned vehicles stuck in the underpass. A university team used a unique approach, with each student controlling one laser scanner and one laptop in the trunk, and taking a majority vote to drive.

The Race Day:
On October 8, the DARPA Grand Challenge race took place, marking a significant day for the Stanford team. The team had limited time to process the race course data obtained on a CD at 4 AM, with Carnegie Mellon focusing on hand labeling and waypoint editing while Stanford smoothed the course using a least square smoother. The race began at 6:30 AM, with Stanford’s Stanley starting second after Carnegie Mellon, which had two cars in the race.

The Emotional Experience:
Sebastian Thrun describes the emotional impact of witnessing Stanley drive autonomously for the first time after a year of development and training. He compares the feeling to raising a college kid and trusting them to make the right decisions on their own. Despite making a few mistakes, Stanley survived the race, and the team was fortunate to have helicopter footage from NOVA documenting the event.

Teams and Their Challenges:
Carnegie Mellon and Stanford University teams dominated the race initially, but Carnegie Mellon faced engine issues, causing them to fall behind. Caltech experienced a GPS glitch, leading to their failure. Team Dead got stuck behind a rock, while Team Enscore lost perception due to a loose laptop. Cunningham-Allen got stuck, allowing Stanley to take the lead.

Stanley’s Victory:
Stanley’s win was largely due to Carnegie Mellon’s engine failure, highlighting the randomness of the outcome. A crucial moment came at mile 103 when Carnegie Mellon was paused, allowing Stanley to pass. Stanley successfully navigated the treacherous mountain pass, making life-or-death decisions to avoid danger.

The Finish Line and Celebration:
Teams gathered at the finish line, eagerly awaiting the arrival of the vehicles. A helicopter and dust cloud signaled Stanley’s approach, creating a thrilling moment for the team. Stanley crossed the finish line first, achieving a historic victory for the robotics field.

Adaptive Vision and Obstacles:
Stanley’s adaptive vision system was showcased during the race. The only man-made obstacle encountered was Carnegie Mellon’s paused car, which Stanley successfully classified as non-drivable.

Equal Winners and the Importance of Luck:
Five teams finished the race, four within 30 minutes of each other, highlighting the equality of their achievements. The differences in finishing times were insignificant and largely influenced by luck and random factors. This outcome emphasized the remarkable accomplishment of multiple teams successfully completing the challenging course.

DARPA’s Urban Challenge:
DARPA created the Urban Challenge to promote the development of self-driving cars in a city environment with traffic. The competition will take place on November 3, 2007, and Google is participating with its self-driving car.

Societal Benefits of Self-Driving Cars:
Reducing traffic accidents: Self-driving cars could significantly reduce traffic accidents caused by human error, potentially saving lives. Improving commuting: Self-driving cars could free up commuting time, allowing people to work, relax, or engage in other productive activities during their commutes. Addressing aging population challenges: Self-driving cars could provide mobility options for people who cannot drive due to age, disability, or other factors. Increasing highway throughput: Self-driving cars could improve highway capacity by driving more efficiently and reducing traffic congestion.

Challenges of the Urban Challenge:
Driving in traffic: The Urban Challenge is more difficult than the previous Grand Challenge because it requires self-driving cars to navigate in traffic with other vehicles.

Stanford’s Involvement:
Stanford University is actively involved in the development of self-driving cars and is participating in the Urban Challenge.