Speaker Introduction:
Jen Scanlon, Interim Dean for Academic Affairs and Professor of Gender, Sexuality, and Women’s Studies, welcomes everyone to Bowdoin College’s Common Hour. Today’s Common Hour serves as the keynote for the President’s Summer Research Symposium, featuring research presentations by 150 students. Dr. Peter Norvig, Director of Research at Google, is introduced as the keynote speaker.

Dr. Norvig’s Background and Achievements:
Dr. Norvig has held various leadership positions in the field of computer science, including at Google and NASA Ames. He has received numerous awards, including the NASA Exceptional Achievement Award and the Distinguished Alumni Award from UC Berkeley. Dr. Norvig is a fellow of multiple prestigious organizations, including the Association for the Advancement of Artificial Intelligence and the American Academy of Arts and Sciences. He has contributed significantly to the field of artificial intelligence, including co-teaching a MOOC with 160,000 students and authoring the leading textbook, “Artificial Intelligence, A Modern Approach.”

Dr. Norvig’s Palindromic Sentence:
Dr. Norvig is also known for creating the world’s longest palindromic sentence.

Keynote Title:

Dr. Norvig’s talk is titled “The Challenges of Creating Software with Machine Learning.”

Traditional vs. Machine Learning Approach in Software Development:
Traditional approach: Programmers define rules and debug them. Machine learning approach: Computers create models based on data.

Machine Learning as the Ultimate Agile Software Development:
Fast and flexible development, minimizing bureaucracy. Need for caution and precautions due to rapid development.

Classic Machine Learning: Word Sense Disambiguation as an Example:
Word sense disambiguation: Determining the meaning of a word in a sentence. Gathering data from the web to train the machine learning model.

Bag of Words Model:
A simple model of language, considering only word occurrences. Words are cut from sentences and collected in bags. Words associated with each meaning of a word are counted.

Model Improvement:
Updating the model with new data to improve accuracy. Collecting more data leads to more comprehensive bags of words.

Counting as the Core of Machine Learning:
Machine learning tasks involve counting and aggregating data.

Accuracy of Word Sense Disambiguation:
High accuracy (80-95%) achieved with sufficient training data.

Data-Driven Success:
Many problems show improved performance with more data. Google’s success in finding problems that benefit from more data.

Data Collection:
Bilingual texts (sentence-to-sentence translations) are gathered from sources like street signs or the internet. These text pairs provide a basis for building translation models.

Breaking Up Sentences into Phrases:
To overcome the challenge of translating entire sentences directly, sentences are broken down into phrases or chunks. This allows for more flexible and adaptable translations.

Creating a Model:
A translation model is constructed using the collected bilingual data. This model consists of three components: Translation Model: Captures the frequency of phrase translations between the two languages. Target Language Model: Assesses the commonality of phrases in the target language. Movement Model: Tracks word movement (shifts to the right or left) during translation.

Translating New Sentences:
When translating a new sentence, it is broken down into phrases using known common phrases. For each phrase, a translation is chosen based on previous observations. Movement of phrases (right, left, or no movement) is determined based on observed patterns.

Considering Multiple Possibilities:
During translation, various choices are made, including phrase segmentation, translation selection, and phrase movement. The model evaluates multiple possibilities and chooses the most common or probable option.

Conclusion:
The presented approach to machine translation involves breaking down sentences into phrases, building a statistical model, and selecting the most likely translation and movement options for each phrase. This method allows for accurate and flexible translation between languages.

Deep Learning for Language and Vision Tasks:
A deep learning model for translation works as a refrigerator magnet model, shuffling small pieces of language around. Hierarchical learning approaches, like deep learning, enable computers to recognize objects by finding pieces at multiple levels and assembling them into more complex representations. Deep learning systems can identify regularities in data and find useful representations without being explicitly told what to look for. Google Photos uses deep learning to automatically categorize photos, saving users time and effort in manual organization. Deep learning systems can also generate captions for images by understanding the content and generating grammatically correct sentences. Deep learning models may make mistakes, such as misidentifying objects or generating inaccurate captions, but they continue to improve with more data and training.

Understandability of Deep Learning Models:
The inner workings of deep learning models are often opaque, making it difficult to understand their predictions and decisions. Lack of understanding hinders trust in the models and limits their application in critical domains. Developing better tools for analyzing and visualizing deep learning models is crucial to improve understandability.

Privacy, Security, and Fairness Concerns:
Deep learning models are often trained on data collected from individuals, raising questions about data ownership and privacy. Sharing data for model training poses challenges in maintaining privacy and preventing misuse. Ensuring fairness in deep learning models is important to avoid bias and discrimination. Aggregation and summarization techniques can help protect privacy while enabling data sharing.

Data Management and Software Development Challenges:
Deep learning models rely heavily on data, requiring new approaches to data management and software development. Existing software tools and debugging techniques may not be suitable for deep learning systems. Developing new tools and methodologies specifically designed for data-centric software development is necessary.

Managing Change and Adaptability:
Deep learning models trained on outdated data may become ineffective over time as the world changes. Continual learning and retraining strategies are needed to keep models up-to-date and adaptable to changing environments. Transfer learning techniques can facilitate adaptation to new tasks without requiring extensive data collection and training.

Exploration vs. Exploitation Tradeoff:
Deep learning models often face a tradeoff between exploration (trying new strategies) and exploitation (sticking to what works). Finding the right balance is crucial for achieving optimal performance and avoiding stagnation. Exploration is more feasible in non-real-world domains, while real-world applications demand careful exploration to ensure safety and reliability.

Human-Machine Partnerships:
Human involvement in deep learning systems can take various forms, from designing and training models to providing assistance during operation. Efficient allocation of human resources is important, especially when human assistance is limited. Mechanical Turk and other crowdsourcing platforms can be used to obtain human input for training and evaluation.

Introduction of New Approaches in Machine Learning:
Machine learning involves a paradigm shift from instructing computers “how to do things” to specifying “what to do.” This approach allows machines to independently determine the best course of action to achieve a desired outcome.

Challenges in Expressing Human Desires to Machines:
Difficulty in articulating human desires in a manner that machines can comprehend. The need for a suitable language to bridge the gap between human intentions and machine understanding.

The Role of Machine Learning in Addressing Global Issues:
Machine learning has the potential to contribute to resolving global challenges such as world peace, hunger, equality, and diversity. However, the current ecosystem prioritizes user preferences and desires, leading to a disconnect between what we want and what we need.

Building an Ecosystem for Aligned Desires and Outcomes:
The importance of creating an ecosystem that facilitates the expression of human desires and aligns them with broader societal goals. Encouraging the development of technologies that enable more effective communication between humans and machines.

Conclusion:
The need for continued research and innovation in machine learning to address the challenges of expressing human desires and achieving desired outcomes. The potential of machine learning to contribute to positive global change depends on the alignment of human desires with societal needs.

Differences Between the Brain and Deep Learning:
The term “deep learning” is derived from the metaphor of the brain, with neurons and signals traveling between them. There is a lack of precision in this metaphor, leading to a separation of the two fields.

Focus on Results Rather Than Replication:
The primary objective in machine learning is to achieve optimal results, rather than precisely replicating the brain’s processes. Input from neuroscientists is welcomed, but the focus remains on practical outcomes.

Collaboration with Neuroscientists:
Neuroscientists provide valuable insights, particularly in areas like vision, speech, and hearing recognition. Human perception is used as a benchmark for machine learning systems, aiming to match or exceed human capabilities.

Hardware Innovations for Machine Learning:
Traditional general-purpose computers are not optimized for machine learning tasks. Specialized hardware, such as graphics processing units (GPUs), have been repurposed for machine learning due to their efficiency in handling large volumes of data in parallel. Collaboration with hardware manufacturers like Intel and NVIDIA is ongoing to develop specialized hardware for machine learning.

Challenges of Multiple Players in Machine Learning:
The involvement of numerous players, including Amazon, Microsoft, Google, and Apple, poses a challenge in maintaining consistency and interoperability. The diversity of players can also lead to a lack of standardization, resulting in a potential “Tower of Babel” scenario.

App-Based User Interfaces:
Modern interactions are primarily through app interfaces, where users switch between different apps for various tasks. This app-centric approach simplifies user interactions but presents challenges for seamless integration and data sharing across different apps and services.

Terminology for Communication Between Apps:
Third-party providers have developed a wide range of apps, but the industry is shifting towards assistant-driven approaches. Challenges lie in how to communicate with these assistants to complete tasks effectively. The lack of a standardized language hinders communication between apps and assistants, making it difficult for users to access the desired capabilities.

Bias in Data Collection:
The increasing reliance on data collection and analysis raises concerns about bias. Bias can arise due to the exclusion of non-digital entities, leading to their invisibility. Majority bias occurs when improvements are focused on the dominant group, neglecting minority groups. The pursuit of overall performance metrics can result in neglecting the needs of specific groups. Striking a balance between addressing bias and defining relevant groups remains a challenge.

Machine Learning and Software:
Machine learning is becoming more prevalent in software, particularly in areas like data analysis. As assistance-based ecosystems become more common, software developers will need to adapt and interface with machine learning, even if it’s not their core focus.

Feedback and Reinforcement Learning:
Machine learning models can learn through supervised learning (with labeled data), unsupervised learning (finding patterns in unlabeled data), reinforcement learning (receiving rewards or punishments for actions), and engineer intervention (trial and error). The choice of learning approach depends on the domain, available data, and expertise.

Google Search Results:
Google search results include web pages, specialized results (maps), and knowledge graph objects (structured information about entities). The knowledge graph is used to provide more comprehensive and relevant search results, but Google is cautious about displaying inferences that might be incorrect.

The Singularity:
The speaker does not believe in the idea of the singularity, where AI surpasses human intelligence and consciousness. The speaker argues that exponential growth trends in technology do not necessarily lead to unbounded progress and that there are limitations and challenges that AI will face.

Limits of Intelligence and Exponential Growth:
Speaker 2 highlights that exponential growth cannot continue indefinitely in the real world. There are theoretical limits to what can be solved, even with exponential capabilities. Some problems are inherently hard and may not be solvable with sufficient computation.

Intelligence as a Limited Attribute:
Speaker 2 cautions against overemphasizing intelligence as the sole attribute for solving world problems. Intelligence is just one of many attributes that matter. The tendency to focus on intelligence may stem from the fact that those predicting the singularity are often highly intelligent.

Challenges of Superintelligence:
Speaker 2 expresses skepticism about the ability of superintelligent computers to solve all world problems. Trading on Wall Street might be feasible, but complex issues like peace negotiations may not be resolvable solely through intelligence.

The Role of Regulation in Tech and AI:
The tech community often views regulation as slow and unable to keep up with rapid technological advancements. Self-regulation is seen as a potential solution, but it may not be sufficient. Existing regulations can be applied to tech settings, such as anti-discrimination laws and privacy regulations. Striking a balance between protecting users and stifling innovation is a challenge.

Evidence of Meaning Detached from Language in Machine Translation:
Machine translation models demonstrate the ability to process and reformulate input language into different output languages, suggesting the existence of an intermediate form that retains meaning. This intermediate form remains relatively consistent when translating from the input language to multiple output languages, despite the significant differences in the output languages.

Interpreting the Intermediate Form:
The intermediate form may represent meaning independent of language, or it could be considered a reformulation of the input language that has been transformed in a specific way. The intermediate form’s ability to generate accurate translations in multiple output languages supports the idea that it captures some form of meaning or semantic content.

Conclusion:
Machine translation models provide evidence that suggests the possibility of meaning existing independently of language, challenging the traditional notion that meaning is solely tied to language descriptions.