Overview:
Sebastian Thrun, a renowned expert in AI, machine learning, and robotics, presents his work on autonomous cars, emphasizing their significance and potential impact.

Thrun’s Research and Contributions:
Sebastian Thrun’s research spans various fields, including AI, machine learning, and robotics, earning him recognition as a highly influential figure in these areas. He pioneered the field of probabilistic robotics, focusing on robots’ ability to handle uncertainty and noise in their sensors by representing variables randomly and reasoning through distribution estimations. Thrun’s successful robot deployments in museums, nursing homes, coal mines, and autonomous driving vehicles showcased the effectiveness of his approach. He achieved notable achievements in the DARPA Grand Challenge, winning the first successful completion and securing second place in the DARPA Urban Challenge.

Thrun’s Role in Street View:
Thrun’s contributions to Google’s Street View project remained confidential due to his employment agreement. Despite the secrecy surrounding his involvement, it is widely believed that Thrun played a crucial role in the development and success of Street View.

Motivation for Autonomous Car Research:
Thrun emphasizes the transformative impact of the internet in the past 15 years, revolutionizing data storage and transport, affecting various aspects of human communication. He anticipates an even faster pace of technological change in the future.

Transportation Needs a Revolution:
The transportation sector has not experienced significant transformation in the last 60 years, leading to the decline of the American automotive industry due to lack of innovation.

Transportation as a Major Expense:
Transportation is the second biggest expense for most people, surpassing food and other life pursuits.

The Current Transportation System is Uneconomical and Dangerous:
42,000 people die annually in traffic accidents, equivalent to a monthly September 11 tragedy. 30% of a vehicle’s weight is dedicated to safety equipment, which could be reduced if cars were inherently safer. Highways are vastly underutilized, with only 8% of space occupied by cars at peak capacity. Our bad driving habits, such as texting and emailing, contribute to the need for extra space between cars, reducing highway efficiency.

Simple Innovations Can Solve Highway Capacity Issues:
Technological solutions that improve lane-keeping and reduce following distances could double the capacity of the US highway system. Moving vertically is uneconomical compared to simple innovations that can improve highway utilization.

Addressing the Number of Cars:
The next topic will address the issue of the excessive number of cars on the road.

Sharing Cars:
Sebastian Thrun emphasizes the underutilized resource of personal cars, with most cars remaining idle for 97% of the time.

Zipcar:
Zipcar is mentioned as an existing car-sharing concept, but its success is limited to densely populated urban areas.

Self-Driving Cars:
Thrun envisions a future where self-driving cars can be summoned on demand, increasing car utilization and eliminating the need for individual ownership.

Benefits:
Sharing cars could free up parking space, reduce traffic congestion, and save time spent commuting.

Economic Impact:
Self-driving cars could improve productivity by allowing people to work or engage in other activities during their commute.

Collaboration:
Thrun emphasizes the need for collaboration among engineers, policymakers, businesspeople, and lawyers to address the complex challenges involved in implementing self-driving cars.

The Grand Challenge:
DARPA, the entity behind the internet research, initiated the Grand Challenge, a bold car race to promote autonomous driving technology.

DARPA’s Motivation:
DARPA aimed to bypass traditional funding instruments and incentivize rapid progress in autonomous driving by setting challenging performance metrics.

Funding and Performance Goals:
Instead of providing substantial funding upfront, DARPA offered a $1 million prize for the first team to complete a challenging autonomous driving course.

Course Design:
The original route from Los Angeles to Vegas was changed to Barstow to Prim due to population density concerns. The course consisted of 2,700 GPS waypoints defining a complex desert trail with speed limits and corridor width specifications.

Team Participation:
106 teams worldwide registered, showing a correlation between states with registered teams and those that voted for Obama in the recent election.

First Grand Challenge Results:
15 teams were selected for the final competition, including universities, car enthusiasts, and individuals modifying existing vehicles. The heaviest vehicle, Terramax, had backup issues, while the smallest, a motorcycle from Cal, completed seven miles before crashing. All vehicles struggled to stay on the road, highlighting the challenges of road detection and navigation.

Second Grand Challenge:
DARPA increased the prize money for the second challenge, hoping to attract more teams and drive innovation.

Sebastian Thrun’s Involvement:
Sebastian Thrun, a researcher at Stanford, decided to build a team to compete in the Grand Challenge despite lacking funding.

Starting a Self-Driving Car Project with Limited Funding:
Sebastian Thrun, facing a lack of funding for his autonomous car project, utilized a unique business model within a university setting. He created CS 294, “Projects in Artificial Intelligence,” where students paid tuition to participate in the project, essentially funding the initiative.

The Selection Process for Dedicated Students:
Approximately 40 students initially enrolled in the course, with around 20 remaining after the first class. Thrun believes that the students who remained were the more dedicated and talented individuals, who were willing to commit to the challenging project.

Ambitious Goal and Collaboration with Volkswagen:
Thrun set an ambitious goal of building a self-driving car capable of navigating one desert mile autonomously within a two-month timeframe. Volkswagen provided Touareg vehicles for the project, which were equipped with computers and various sensors.

Lifesaving Button and Vehicle Equipment:
A crucial safety feature was a button that allowed Thrun to switch from robotic control to human control in case of software malfunctions. The vehicle was outfitted with sensors to monitor the vehicle’s position and motion, as well as sensors for environmental perception.

Importance of GPS and Inertial Measurement Units:
GPS and inertial measurement units were essential for determining the vehicle’s position and motion accurately. These technologies had been extensively developed in the field of avionics over the preceding 40 years.

Simple Yet Effective Steering Controller:
A basic P controller was used to guide the vehicle along the desired path. This controller adjusted the steering based on the cross-track error, helping to keep the vehicle on course.

Laser-Based Environment Perception:
The team utilized a laser-based sensor to perceive the environment. The laser emits invisible light beams that reflect off objects, providing range measurements with centimeter accuracy. The laser’s rotating mirror allows for 3D mapping of the surroundings. Vertical obstacles are detected by comparing and storing past laser points.

Obstacle Avoidance:
A simple algorithm was employed to detect and avoid obstacles. The vehicle can draw internal paths and select a path that avoids collision. This approach was tested in a parking garage, with limited success.

Challenges and Failures:
The vehicle’s autonomous driving was initially unsuccessful due to various factors, including a drunk squirrel causing oversteering and understeering. The complexity of laser data and the accumulation of software bugs and conceptual errors posed significant challenges. The team realized they had neglected the principles of probabilistic robotics, which are essential for dealing with uncertainty in sensor data. Errors in pitch and roll estimates led to misinterpretations of the terrain, affecting the vehicle’s perception and decision-making.

Challenges and Solutions in Autonomous Driving:
Error Accumulation in Laser Measurements: The team encountered issues with error accumulation in laser measurements taken at different times. To address this, they devised a rule based on a mark of random fields that analyzed the accumulated error and rejected laser measurements with high probability of error.

Addressing Range Limitations:
Limited Range of Laser Sensors: The team recognized the limited range of laser sensors, which constrained the robot’s ability to detect obstacles at longer distances. To overcome this, they explored the use of computer vision for obstacle detection.

Challenges in Computer Vision for Road Detection:
Unsuccessful Color-Based Approach: The team initially attempted to use color-based computer vision to identify the road, but this proved unreliable due to varying lighting conditions, vegetation, and pavement colors. Smoothness-Based Approach Fails: They also tried a smoothness-based approach, which failed because the sky was often the smoothest part of the image, leading to the development of a sky driving system instead of a road driving system.

Solution: Adaptive Computer Vision:
Adapting to the Environment: Eventually, the team found a solution inspired by Dean Pomerleau’s work in the early 1990s. They formulated computer vision as an adaptive problem, continuously adjusting to the environment just like humans adapt their visual ability to different lighting conditions. Unknown Training Data: The challenge of adapting to the world’s color variations led to the question of where to obtain training data, as the environment’s color is unknown.

Adaptive Vision:
Sebastian Thrun and his team developed an adaptive vision system that mapped laser readings into the visual image to extract the drivable region. A classifier was trained on the area just before the robot, where it was too late to react, and then applied further in the image to find more drivable areas. The system was not perfect, but it was reliable enough to be used to gate speed.

Speed Control:
The team wanted to find a balance between speed and safety in the desert environment. They copied a human driver’s speed by setting up a controller that gradually increased speed until it hit a bump, then slowed down to a safe z-acceleration threshold. Parameters of the controller were learned from a human driver, starting with a careful American driver and eventually reaching a reckless German profile.

Competition:
Carnegie Mellon took a different approach, manually labeling the terrain and using aerial imagery and 3D LiDAR scans to get perfect speed settings for different terrain areas. Thrun’s team had a more environmentally friendly vehicle that could drive around obstacles, save every bush, and swerve around large insects or birds.

Team Diversity:
Teams varied widely in their backgrounds and approaches, from universities to companies and car enthusiasts. Some teams focused solely on hardware, while others prioritized software and artificial intelligence.

Carnegie Mellon University (CMU) Team:
CMU deployed two Hummers, exploiting a loophole in the rules to increase their chances of winning. They achieved a significant milestone by completing a 200-mile run in challenging terrain before anyone else. However, their emphasis on speed and hardware limitations hindered their overall performance.

University of California, Berkeley (UC Berkeley) Team:
UC Berkeley’s team attempted to navigate a motorcycle autonomously, facing unique challenges due to the motorcycle’s instability. They employed reinforcement learning techniques to control the motorcycle’s balance and movement. Despite impressive results, the team experienced numerous crashes, highlighting the difficulty of autonomous motorcycle operation.

Challenges and Risks:
Operating autonomous vehicles in harsh desert environments presented significant risks, with teams relying on their software to ensure safety. Heavy rainfall during the competition resulted in limited visibility, further testing the capabilities of the autonomous systems.

First Competition:
The initial DARPA Grand Challenge took place in a controlled desert environment in Fontana, California. Teams showcased their autonomous vehicles navigating a 2.2-mile course with various obstacles, including passing other vehicles and avoiding abandoned robots. The competition highlighted the importance of computer vision, obstacle detection, and high-speed maneuverability in autonomous driving.

Tunnel Obstacles:
The course included three tunnels that caused GPS reception loss, making orientation difficult. Anthony Lewandowski’s motorcycle had a software bug that caused it to accelerate in tunnels to regain GPS.

Carnegie Mellon’s Mishaps:
Carnegie Mellon had a calibration error and a Hummer that got penalized for not handling hay bales. Some of their vehicles struggled with obstacles and got stuck.

Stanley’s Journey:
Despite a GPS glitch at the start, Stanley moved out and eventually took the lead. Carnegie Mellon’s leading vehicle faced an engine problem, giving Stanley a chance to gain an advantage. Stanley circumnavigated an obstacle placed by Carnegie Mellon, formally taking the lead.

Beer Bottle Pass:
The final obstacle was a treacherous mountain pass with a cliff on one side and a mountain on the other. Stanley successfully navigated the pass despite limited visibility due to dust.

Victory and Aftermath:
Stanley became the first vehicle to finish the 131-mile race, followed by Carnegie Mellon’s two vehicles. The victory brought recognition and funding to the Stanford team and led to the DARPA Urban Challenge. The Urban Challenge aimed to make autonomous driving more realistic by introducing urban scenarios and other traffic.

Introduction:
DARPA’s Urban Challenge was a race where autonomous vehicles competed in tasks such as delivering packages in urban environments.

Technology in the Urban Challenge:
The Urban Challenge required precise perception of the environment, including lane markers and other vehicles. The vehicles used a rotating laser sensor to scan the environment and build detailed maps. Machine learning was used to detect obstacles and classify underpasses. Tracking of other vehicles was done using particle filters, a probabilistic technique.

Merging Scene in the Urban Challenge:
The presentation showed a scene where a robot had to make a left turn onto a narrow lane through traffic. The robot had to wait for gaps in traffic, obey traffic laws, and drive precisely along the lane. The precision required for this task was beyond what GPS could provide.

Looping the Course:
The vehicles had to complete the same loop multiple times, including a left turn across traffic at the other end.

New Sensor Technology:
The vehicles were equipped with a rotating laser sensor that scanned the environment 10 times per second with 64 scan lines. This sensor allowed for the creation of detailed environment maps and the detection of obstacles and underpasses.

Tracking Other Vehicles:
Particle filters were used to track other vehicles on the road. This technique allowed the vehicles to estimate the location and trajectory of other vehicles.

Lane Passing Decisions:
Stanley’s decision-making and planning capabilities were showcased in various situations, including lane passing. The system considered factors such as abandoned cars on the road and adjusted its behavior accordingly.

Collision Avoidance:
Stanley was equipped with a hierarchy of behaviors to handle rare cases like collisions in intersections. If the road was blocked, it would invoke a general-purpose planner to find a path, regardless of other cars or rules.

Continuous Path Planning:
Stanley utilized A-star based generalizations of path planning in continuous spaces. This allowed it to dynamically replan and navigate through impasses.

Urban Challenge Overview:
The Urban Challenge involved 90 competitors, with 40 admitted to the semifinals and 13 making it to the race. The race took place in a carefully manicured network of streets inhabited by robots and stunt-driven cars.

Parking Maneuvers:
Stanley demonstrated its parking capabilities in designated spots, maneuvering in and out of parking spaces.

Robotic Traffic Jam:
The Urban Challenge witnessed the first robotic traffic jam, captured from a helicopter view. This event highlighted the safety of the robot race compared to human driving.

Laser Calibration Issues:
While using multiple lasers at different angles is ideal for road detection, calibration challenges were encountered. Mounting multiple lasers on top of each other required precise alignment to avoid systematic errors.

Communication with Other Vehicles:
The speaker inquired about the possibility of vehicles communicating with each other for cooperative driving. Sebastian Thrun acknowledged the potential benefits but emphasized the current focus on autonomous navigation.

Challenges in Implementing Car-to-Car Communication:
Practical challenges are holding back the implementation of car-to-car communication, including ongoing discussions about standards and involvement of various agencies. Commercial interests of different car companies hinder the establishment of a universal standard.

Benefits of Car-to-Car Communication:
Improved road safety by enabling vehicles to warn each other about hazards, such as fog, and prevent collisions. Enhanced traffic management through communication with traffic lights and intersections to optimize traffic flow and reduce accidents. Increased convenience for drivers, allowing them to relax and engage in other activities while the vehicle safely navigates the road.

Next Steps for Autonomous Vehicle Development:
Focus on improving the reliability and performance of autonomous vehicles to match or exceed human drivers in terms of safety and accident prevention. Collaborate with policymakers to address regulatory and legal challenges to facilitate the adoption of autonomous vehicle technology. Foster the development of new markets and applications for autonomous vehicles, showcasing their benefits and encouraging public acceptance.

Benefits of Autonomous Vehicles:
Enhanced safety by eliminating human error as a major cause of accidents. Reduced traffic congestion and improved traffic flow through optimized coordination between vehicles and infrastructure. Increased productivity and convenience for individuals, allowing them to engage in other activities during their commute or travel.

Conclusion:
The development of autonomous vehicles and car-to-car communication holds immense potential for improving road safety, traffic management, and overall transportation efficiency. Collaborative efforts among researchers, engineers, policymakers, and industry stakeholders are crucial to overcome current challenges and bring this technology to fruition.

DARPA Challenge as a Seed for Innovation:
The DARPA Urban Challenge is no longer active, and the $2 million prize is no longer available. DARPA’s role in the challenge was to serve as a seed starter for the field of autonomous vehicles. The challenge successfully attracted and engaged a community of experts who understand the vision and potential of autonomous vehicles.

The Need for More People in the Field:
Sebastian Thrun emphasizes the importance of having more people involved in the development of autonomous vehicles. He views it as a great challenge and a fantastic robotic endeavor. Thrun believes that this is the only project that can genuinely and credibly claim to have a transformative impact on society.