Welcoming Address by Johannes Frohlich:
Johannes Frohlich, Vice Rector of Research, welcomed attendees to the third Godel Lecture at the Vienna University of Technology. He highlighted the significance of the lecture series, which invites renowned computer scientists to share their insights. Frohlich emphasized the strength of the Faculty of Informatics at the university, ranking among the top 14 in Europe. He noted the importance of ICT and its contribution to Vienna’s economy, surpassing tourism in terms of value-added.

Address by Faculty Dean, Professor Steinert:
Professor Steinert welcomed attendees to the prestigious Godel Lecture Series, aiming to showcase computer science’s role in developing information and knowledge societies. He introduced Kurt Godel, a famous Viennese logician who inspired the lecture series. Steinert warmly welcomed Peter Norvig, the guest speaker, known for his contributions to artificial intelligence, machine learning, and information retrieval. He highlighted Norvig’s widely used textbook on artificial intelligence and his interests in new approaches to learning and teaching, such as MOOCs.

Introduction of Peter Norvig by Stefan Seider:
Seider briefly introduced Peter Norvig, emphasizing the impact of his book on artificial intelligence as a standard textbook shaping generations of researchers. He mentioned the upcoming fourth edition of the book and its recognition as the fourth textbook of the 21st century. Seider concluded by highlighting Norvig’s enthusiasm for solving tricky problems and how it inspires other researchers.

Godel’s Connection Between Mathematics and the Real World:
Godel established a link between the real world and the world of mathematics, allowing for the translation of observations into mathematical expressions. This enabled the logical inference of new facts by combining multiple observations, even without direct real-world observation.

Formalizing Proofs and Computer Programs:
Godel developed a formal method for specifying mathematical proofs, ensuring their validity and precision. This laid the foundation for computer programs, which are also highly structured and precise.

Limitations of Computer Programs:
While computer programs can solve various problems, some challenges may leave programmers uncertain about how to proceed.

Example of Anna’s Interaction with Her Phone:
Anna speaks into her phone, and the phone recognizes her speech. The phone takes action based on Anna’s request, such as playing music.

Introduction of Machine Learning:
Machine learning is a subfield of artificial intelligence where computers learn to solve problems without explicitly being programmed to do so. It involves developing algorithms that can learn from data and improve their performance over time.

Example-Based Learning:
In machine learning, we provide a computer with examples and let it learn the underlying patterns. The output of machine learning algorithms is a new program that can perform a specific task.

Understanding Word Senses:
Word senses refer to different meanings of a word in different contexts. Machine learning can be used to distinguish between different word senses. By feeding a machine learning algorithm with a list of sentences containing the word and its definitions, it can learn to identify the correct word sense in new sentences.

Benefits of Machine Learning:
Machine learning enables computers to solve problems that are difficult or impossible for humans to solve using traditional programming methods. It allows computers to learn from data and improve their performance over time. Machine learning has wide applications in various domains, including speech recognition, image recognition, natural language processing, and more.

Introduction of the Bag of Words Model:
Peter Norvig presents a simple yet effective approach to word sense disambiguation called the bag of words model. He starts with the definitions of different senses of a word, cuts them into individual words, and puts each word into a bag. This creates multiple bags, each representing a specific sense of the word.

Developing the Model:
To determine the sense of a new sentence, Norvig compares the words in the sentence to the words in each bag. The bag with the most matching words is considered the most likely source of the sentence. As more sentences are processed, the bags are updated to include new words and strengthen the association between words and their senses.

Simplicity and Usefulness of the Model:
Norvig acknowledges that the bag of words model is a simplistic representation of language, but emphasizes its usefulness in word sense disambiguation. He cites the famous quote from statistician George Box: “All models are wrong, but some are useful.”

Updating the Model:
As new sentences are processed, words from those sentences are added to the appropriate bags. This continuous update improves the model’s accuracy over time and allows it to handle new sentences effectively.

Conclusion:
Norvig concludes by highlighting the simplicity and effectiveness of the bag of words model for word sense disambiguation. He credits Sesame Street’s character The Count for inspiring his approach.

Performance of Machine Learning Techniques:
Machine learning algorithms perform better with more training data. Four different machine learning techniques were compared using one million words of training data, with the best performing at 85%. Performance improved for all techniques when trained with 10 million words of data.

Increased Data Leads to Improved Performance:
With 100 million words of training data, three out of four techniques performed better than the best with one million words. At one billion words of training data, performance asymptoted, with the best technique achieving 97% accuracy.

Simple Algorithms with Sufficient Data:
Even with a simple algorithm and minimal initial knowledge, feeding it billions of examples can lead to high performance. This pattern is observed in many problems, where a simple algorithm with enough data outperforms more complex approaches with less data.

Translation between Languages:
Translation involves finding correspondences between phrases in different languages. This can be done by analyzing millions of examples of translated documents. A model is created that captures the frequency of phrase correspondences, language probabilities, and word movement patterns.

Components of the Translation Model:
Translation Model: Tracks the frequency of phrase correspondences between languages. Target Language Model: Estimates the probability of phrases in the target language. Movement Model: Accounts for word order differences between languages.

Applying the Model to New Sentences:
Break the sentence into phrases. Choose translations for each phrase independently. Choose whether to move words or phrases based on the movement model. Consider multiple choices and select the best translation based on probabilities.

Overview:
This chapter delves into the captivating world of machine learning, showcasing how it empowers computers to perform tasks typically requiring human intelligence. The chapter explores examples like language translation and image recognition, demonstrating how machines can learn from vast amounts of data to make informed decisions and solve complex problems.

Machine Translation:
Machine translation, a core application of machine learning, allows computers to translate text from one language to another. The system estimates the probability of each sentence and selects the most probable translation. Additional techniques enhance the translation process, but the fundamental principle remains the same: feeding the machine with examples to learn and generate translations.

Image Recognition:
Image recognition is a challenging task for computers, requiring them to identify and categorize objects within images. The ImageNet database, containing 20,000 categories and 14 million images, serves as a valuable resource for training machine learning algorithms. Machine learning systems excel in recognizing objects with consistent appearances, such as Blenheim Spaniels, but struggle with objects exhibiting significant variations, like water bottles.

Parable of Tile Mosaics:
The chapter presents a parable to illustrate how machine learning algorithms select the most effective pieces for creating tile mosaics. The goal is to choose a limited set of tiles that can be combined to approximate various pictures uploaded by customers. The algorithm analyzes past customer orders to identify frequently requested elements and selects tiles that can be combined to recreate these elements.

Optimization Objective:
The task for the computer is to select a set of pieces of a specific size from a larger pool, ensuring that this set can be used to create near-replicas of every picture customers provide. The objective is to optimize the selection of these pieces to maximize the coverage of customer requests.

The Puzzle Analogy:
Peter Norvig uses the analogy of creating a puzzle to explain image processing. Instead of choosing individual tiles, he allows each piece to be a combination of multiple pieces, similar to stained glass. The goal is to minimize the difference between the copy and the original image.

Discovering the Building Blocks of Images:
An algorithm was developed to determine the key pieces needed to represent the entire world in images. The result revealed that pictures are primarily composed of lines, emphasizing the significance of edge detection in image processing.

The Multi-Layer Approach:
The next step was to create multiple layers of inventory, starting with smaller bits and combining them into larger ones. This approach led to more interesting results, with the algorithm identifying meaningful objects like eyes, noses, faces, doors, wheels, and cars.

Machine-Learned Image Recognition:
In 2006, Hinton and Ng’s work demonstrated the first image recognition algorithm that produced meaningful results. The algorithm could identify faces, cars, and other objects in a mixture of images.

Practical Results and YouTube Experiment:
By 2010, practical results started to emerge, and by 2012, machine-learned image recognition took over the field. Google conducted an experiment using 10 million YouTube video frames, revealing that “cats” were the most prominent category.

Automating Image Categorization:
The system learned to identify important objects in images without being explicitly labeled. It could distinguish cats, people, and other objects based solely on their physical appearance.

The ImageNet Competition:
In 2012, Hinton and his team achieved a 16% error rate in the ImageNet competition, where the goal was to correctly categorize images into 20,000 categories. By 2014, Google’s team improved the error rate to 6.6%.

Combining Language and Image Understanding:
The final example demonstrated the integration of language and image understanding. The system could identify objects in images and generate natural language descriptions of them.

Unsupervised Learning:
Unsupervised learning is a type of machine learning where the computer is given a bunch of input data and attempts to find patterns and categories within the data without being explicitly told what to look for.

Supervised Learning:
Supervised learning is a type of machine learning where the computer is given a bunch of input data along with labels indicating the correct answer. The computer learns to map the input data to the correct answer.

Reinforcement Learning:
Reinforcement learning is a type of machine learning where the computer learns by interacting with its environment. The computer is given rewards for good actions and punishments for bad actions, and it learns to adjust its behavior accordingly.

Examples of Machine Learning:
Image captioning: A machine learning algorithm can be trained to generate captions for images by being given a large number of images with corresponding captions. Object detection: A machine learning algorithm can be trained to detect and classify objects in images by being given a large number of images with labeled objects. Playing Atari games: A machine learning algorithm can be trained to play Atari video games by being given the game’s pixels and the score as input. The algorithm learns to map the pixels to the appropriate actions to take in the game.

Challenges of Machine Learning:
Outtakes: Machine learning algorithms can sometimes make mistakes, leading to humorous or nonsensical results. Limited understanding: Machine learning algorithms are often able to perform tasks very well, but they may not have a deep understanding of the concepts they are working with. Bias: Machine learning algorithms can be biased if they are trained on data that is not representative of the real world.

Important Pairs of Languages for Machine Learning Models:
Machine learning models can translate between 75 languages, creating over 5,000 language pairs. Some language pairs work better than others due to the availability of training examples. Insufficient training examples lead to multiple-step translation, which can introduce errors. Languages with different word orders and those that rely heavily on individual words pose challenges for translation models. Translating from a language with less information to one with more information (e.g., adding gender to pronouns) can be difficult.

Factors Influencing Machine Learning Training Time and Capacity:
Training time varies widely depending on the complexity of the task. Some tasks may require only hundreds or thousands of training examples. Access to large computational resources like hundreds or thousands of computers can significantly speed up training. Home computers can handle some machine learning tasks, but data centers allow for faster iteration and experimentation.

Challenges and Future Directions in Machine Learning:
Machine learning has made significant progress, especially in perceptual tasks like image and speech recognition. The challenge lies in applying these models to more complex tasks like reasoning and planning. Researchers are exploring ways to combine symbolic and statistical approaches to address these challenges. The goal is to develop models that can learn from small datasets, adapt to new situations quickly, and explain their reasoning.

Neural Networks:
Neural networks have shown great success in tasks like playing video games and recognizing images, but they struggle with long-term planning. Combining probabilistic, fuzzy, perceptual work with logical planning is a promising approach to overcome this limitation.

Brain vs. Machine:
The logo of the Google Brain team features a brain, leading to confusion between biological brains and neural networks. Neural networks solve mathematical equations and find optimal results, while biological brains may use different methods. Mimicking the brain’s physiology is important for tasks like color balancing, but engineering solutions should be evaluated independently.

Symbolism:
In the paper bag example, words are represented individually with a count of their occurrences. Other approaches use word embeddings to represent words in a multi-dimensional space.

Logic-Based Approaches:
Logic-based approaches are limited by Kurt Godel’s incompleteness theorems. Symbolic approaches are useful for tasks requiring logical reasoning and knowledge representation.

Logic’s Limitations in Real-World Scenarios:
Logic struggles to provide definitive answers when dealing with real-world scenarios due to their inherent complexity and numerous variables. For example, while we know that most birds can fly most of the time, we can’t rely on logic to guarantee that a specific bird, like Tweety, can fly. Thus, probability becomes more valuable in these uncertain situations, as it allows us to express the likelihood of an event occurring rather than claiming absolute certainty.

Sturgeon’s Law and Internet Data Quality:
Sturgeon’s Law posits that 90% of everything is “crap,” and this extends to the vast amounts of data available on the Internet. Google and other search engines face the challenge of guiding users through this overwhelming and often low-quality data, without the responsibility of eliminating it entirely. Adversaries and those with vested interests may attempt to promote their content, even if it’s considered “trash,” leading to ongoing battles between search engines and those seeking visibility.

Risks Associated with Algorithmic Inaccuracies:
The inherent inaccuracies of AI algorithms pose risks to individuals, especially when agencies use these algorithms to make decisions that impact lives. Agencies may possess extensive data and employ algorithms to identify potential threats, but these algorithms are prone to errors and biases. Inaccurate algorithms can result in false positives, where innocent individuals are wrongly targeted or subjected to harmful actions, potentially leading to deadly consequences.

Inaccuracy:
Lack of reliability in AI systems due to unreliable components. Software industry needs to prioritize reliability over speed of development. Adhering to engineering processes can lead to better results, as seen in civil engineering. Customers’ preference for quick updates over reliability poses a challenge.

Privacy:
Privacy concerns arise when errors involve inaccurate perceptions about individuals. Societal and legal regulations are needed to address privacy issues, rather than solely relying on technological solutions.

Medical and Biological Research:
Big data approaches in medicine and biology lead to a higher likelihood of false positives. The field needs to learn to manage false positives and conduct independent verification for accuracy.

Google’s Use of AI Techniques Internally:
Question from the audience: Does Google use AI techniques internally for financial planning or recruiting?

Data-Driven Analysis of Hiring and Performance:
Google uses data analysis techniques to examine the relationship between resume features and interview performance, as well as subsequent job performance. This analysis helps identify factors that contribute to successful job performance and improve the hiring process.

Surprising Findings about Programming Contest Winners:
Surprisingly, winning programming contests was found to be a negative factor in predicting job performance. This suggests that while contest winners are highly skilled, they may lack the reflective and meticulous approach needed for effective job performance.

Data-Driven Insights for Improved Hiring and Development:
The analysis of recruitment and employee performance data provides valuable insights for improving hiring practices and employee development. By identifying characteristics and behaviors that correlate with success, Google can make more informed hiring decisions and tailor development programs to address specific needs.

Conclusion:
Google’s data-driven approach to recruitment and employee performance analysis enables the company to make evidence-based decisions, improve hiring outcomes, and foster a high-performing workforce.