Main Points:
Dr. Peter Norvig, Director of Research at Google, gave a lecture on creating software with machine learning at Stevenson Institute of Technology. Traditional software development involves a programmer carefully writing down all the steps of how to do something, while machine learning involves a computer writing the program based on examples and data. Machine learning models are like black boxes that can be used to generalize from examples and find answers to new cases that have never been seen before. Machine learning allows for faster development and the ability to approach problems in new ways, enabling computers to do things that humans may not be able to program explicitly. Word-sense disambiguation is an example of a task where machine learning can be used to determine the correct meaning of a word in a given context.

Bag of Words Model for Word Sense Disambiguation:
The bag of words model is a simple approach to word sense disambiguation, where the meaning of a word is determined by the words that co-occur with it. The model assumes that sentences are constructed by randomly selecting words from a bag containing all the words that can be used in that context. To build the model, dictionary definitions of different senses of a word are cut into individual words and put into separate bags. Sentences containing the ambiguous word are then used to improve the model by adding words from those sentences to the appropriate bag. The model can then be used to disambiguate the sense of a word in a new sentence by comparing the probability of the sentence being generated from each bag.

Bag of Words Model Applications in Computer Vision:
The bag of words model has also been successfully applied to computer vision tasks such as object recognition and image classification. In these applications, the model is used to represent images as a collection of visual words, which are extracted from the image using local features such as edges, corners, or textures. The visual words are then used to build a histogram, which represents the frequency of each visual word in the image. The histogram is then used to compare images and classify them into different categories.

Data Threshold and the Importance of Large Datasets:
The bag of words model demonstrates the phenomenon of the data threshold, where performance significantly improves as the amount of training data increases. This highlights the importance of large datasets for machine learning tasks, as they allow models to learn more effectively and achieve higher accuracy.

Applications of Computer Vision in Self-Driving Cars and Image Translation:
Computer vision is used in self-driving cars to detect pedestrians, other vehicles, and obstacles on the road, enabling safe navigation. Computer vision is also used in applications like Google Translate, where it allows users to point their phones at signs or text in a foreign language and have it translated and displayed on their phones.

Google Photos and Automatic Image Organization:
Google Photos uses computer vision to automatically organize photos into categories and allows users to search for specific objects or scenes within their photo collection. The model is able to accurately categorize and tag photos, making it easier for users to find and manage their images.

Inventory Optimization for Custom Tile Displays:
Custom tile displays are popular, but manufacturing them one at a time is expensive. Maintaining an inventory of tiles to assemble custom orders reduces costs. Determining the optimal inventory of tiles involves identifying the most commonly used components.

The Primitive Elements of Pictures:
An initial attempt to identify the basic components of pictures resulted in lines, which was not a useful discovery.

Hierarchical Decomposition:
The breakthrough came with the idea of hierarchical decomposition, involving multiple levels of components. This approach led to the identification of higher-level components, such as eyes, noses, and faces.

Expanding the Hierarchy:
Current systems employ hierarchical decomposition with more than 20 levels, enabling the creation of complex components.

Application to Language and Computer Vision:
The same principles of decomposition and composition can be applied to tasks involving language and computer vision.

Model Limitations:
AI models for caption generation are trained on large datasets of images and corresponding captions. While these models can generate impressive results, they are not perfect and can make mistakes. Common errors include misidentifying objects, misinterpreting context, and creating captions that are inaccurate or nonsensical.

Challenges in Understanding Errors:
It can be difficult to understand the reasons behind the errors made by AI caption generation models. This is because these models are complex and make decisions based on statistical patterns in the data they have been trained on. As a result, it can be challenging to identify the specific factors that lead to errors and to develop strategies for correcting them.

Addressing the Challenges:
Researchers are working on developing new techniques to improve the accuracy and reliability of AI caption generation models. One approach is to use larger and more diverse datasets for training, which can help the models to learn a more comprehensive understanding of the relationship between images and captions. Another approach is to develop new algorithms that are better able to interpret the context of an image and to generate captions that are both accurate and meaningful.

Challenges of Rapid Machine Learning Development:
As machine learning models advance, so do the challenges associated with their deployment. One emerging concern is the presence of adversaries who actively seek to defeat these models rather than passively observing them.

The Notion of Adversaries:
Adversaries can exploit vulnerabilities in machine learning models by providing examples specifically designed to confuse or mislead the model. These attacks can lead to incorrect predictions and potentially have severe consequences in applications such as autonomous vehicles or medical diagnosis.

Decision Boundary and Misclassification:
A machine learning model classifies data points based on their location relative to a decision boundary. Adversaries can identify data points near the decision boundary and make small changes to shift them across the boundary, resulting in misclassification.

Minimal Perturbations for Maximum Impact:
Adversaries can use mathematical techniques to determine the minimum number of pixels or features that need to be adjusted to cause misclassification. These perturbations are often imperceptible to humans but can significantly affect the model’s output.

Visual Imperceptibility of Adversarial Changes:
Despite fooling the machine learning model, adversarial changes may not be noticeable to the human eye. This characteristic poses a significant challenge in detecting and mitigating adversarial attacks.

The Changing Nature of Machine Learning:
Machine learning programs operate differently than humans, making bizarre mistakes due to high-frequency changes that the visual system cannot detect. Rather than clear boundaries between image types, there are narrow corridors where machine learning excels and vast areas where it struggles. Most possible images are unknown to machine learning systems, and statistical accuracy stems from similar pictures taken by many people.

Challenges in Debugging, Explainability, and Privacy:
Machine learning systems lack clear bug locations, making debugging and modification difficult. Data security and privacy concerns arise due to the need for data collection. Research continues to address these issues.

Fairness and Ethical Considerations:
Optimizing for overall accuracy can unintentionally favor majority groups, disadvantaging minority groups. Defining fairness and determining deserving groups for fair treatment pose significant challenges.

Non-Stationarity and Evolving Data:
Machine learning systems trained on historical data can degrade over time as the world changes. Continuous system updates may result in the loss of old knowledge and introduce security and reliability challenges.

Visualization and Understanding:
Machine learning excels at decision-making but struggles with visualization and explaining why certain decisions are made. Tools for summarizing large amounts of data can aid in understanding machine learning outcomes. Predictive coding and compression are alternative metaphors for machine learning.

Clustering in Machine Learning:
Clustering is a type of unsupervised learning, where data is grouped into similar categories without labeled data. The Google Photos application uses clustering to group similar pictures together. Clustering can be hierarchical, where similar clusters are grouped into larger clusters.

Supervised Learning:
Supervised learning involves providing the machine learning system with both data and the correct answers. The Google Photos application uses supervised learning to label pictures with the correct labels. Supervised learning is often used in tasks such as image classification and language translation.

Reinforcement Learning:
Reinforcement learning involves providing the machine learning system with feedback on its performance rather than the correct answers. Game playing is an example of reinforcement learning, where the system learns to play the game by receiving rewards for winning and penalties for losing. Reinforcement learning is also used in robotics and other applications where the system needs to learn through trial and error.

The Role of Humans in Machine Learning:
Humans are still essential in the development and deployment of machine learning systems. Humans define the problem, provide data, and design the learning algorithms. Machine learning systems are most effective when they are used in combination with human expertise.

Legal and Ethical Challenges of Machine Learning:
Machine learning systems can be difficult to explain and understand, making it challenging to assess their reliability and safety. This raises legal and ethical questions about liability in cases where machine learning systems make mistakes. Developers need to ensure that machine learning systems are reliable, explainable, and understandable.

Generalized Human Intelligence and Superhuman Performance:
Creating generalized human intelligence with emotions and all human abilities is still a mystery. Instead of creating human-like AI, the focus is on developing superhuman performance in specific tasks. This approach aims to achieve superhuman accuracy, speed, and capabilities beyond human limitations.

Challenges of Interacting AI Systems with Different Value Systems:
As AI systems become more prevalent, they may encounter situations where they have conflicting value systems. This can lead to conflicts and challenges in decision-making. Resolving these conflicts requires careful consideration of the ethical and societal implications of AI systems.

The Subjective Nature of Human Values:
Speaker 03 emphasizes that individual human values vary greatly, leading to disagreements even on fundamental societal issues. This diversity of values exists among humans and will likely extend to artificial intelligence systems as well. The speaker questions whether the involvement of machines will significantly alter the dynamics of value-based conflicts.

Machine Learning in Job Hiring:
Speaker 04 inquires about the challenges in using machine learning in the job hiring process. Speaker 03 recognizes the potential of machine learning as a tool to assist in job selection decisions. The collaboration between machine learning, expert admissions personnel, and technical translators is suggested to optimize the process.

Overcoming Implicit Biases:
Speaker 03 highlights the prevalence of implicit biases in human decision-making, affecting everything from societal issues to individual preferences. These biases can lead to inaccurate judgments and unfair treatment. Machine learning can potentially help identify and quantify implicit biases, aiding in the creation of fairer decision-making processes.

Technology’s Impact on Human Biases:
Speaker 13 expresses concerns about machine learning systems amplifying existing human biases. The speaker suggests that automated decisions based on biased data may outpace our ability to correct them, leading to reinforced societal prejudices.

Navigating Ethical Challenges:
Speaker 13 inquires about potential frameworks for determining when human intervention is necessary in automated decision-making processes. Speaker 03 acknowledges the complexity of this issue, comparing it to the metaphor of race cars, where speed can lead to crashes if not handled responsibly. The speaker emphasizes the importance of addressing the “garbage in, garbage out” problem, ensuring that machine learning systems are trained on data that reflects desired values and outcomes.

Ethical Considerations in Machine Learning:
It is challenging to eliminate all biases in AI systems. Curation of data to remove bias is never perfect, and post-processing is necessary to ensure fair and unbiased decisions. Adjudication of decisions made by different programs is necessary to catch biases and ensure accuracy.

Prioritizing Sustainability and Long-Term Effects:
Machine learning can be used to address long-term effects, such as pollution and urban planning, to avoid future problems. Balancing speed and sustainability is crucial to ensure long-term viability.

Machine Learning for Societal Problems:
Machine learning may not be the solution for all societal problems, as some issues are rooted in societal factors. Information technology can play a role in addressing societal problems by simplifying processes and making them more accessible. Non-profit organizations use technology to help people with criminal records get back on track, highlighting the positive impact of technology on societal issues.

Machine Learning and Stream Analytics:
Machine learning is an integral part of stream analytics, but there are complications due to the dynamic nature of data in stream analytics. The integration of machine learning and stream analytics requires careful consideration and expertise in both areas.

Static vs. Temporal AI:
AI has progressed from static image analysis to systems that can learn and progress over time, keeping track of recent and long-term information. However, there are limitations in representing concepts like “every other Tuesday,” as AI still struggles with understanding time conceptually.

Buffering as a Solution:
AI systems still rely on buffering to understand and run through models, indicating that further development is needed in this area.

Control and Apathetic Goals:
The concern that AI systems might prioritize self-preservation and resist being turned off is not yet relevant since current systems do not have such goals. AI programs can be turned off and on, duplicated, and modified without the same existential implications as living beings.

Communication with AI Systems:
Developing better languages for communicating with AI is crucial to clearly convey goals and policies. AI systems should be able to identify contradictions and inconsistencies in instructions and break rules when necessary.

The Challenge of Changing Rules:
AI systems need to understand what can and cannot be changed in policies and rules. Balancing priorities and describing goals in a way that AI can comprehend is an ongoing challenge.

Understanding Human Desires:
Humans often contradict their stated preferences and desires, making it difficult to accurately communicate them to AI systems. Computers observing human behavior may not accurately infer true desires, as they might mistake mistakes for goals.

Distinguishing Good and Bad Intentions:
AI systems need to learn to distinguish between positive and negative human behaviors and intentions. Research on this topic can help AI systems align with human values and goals.

Model Complexity and Verification:
The increasing complexity of AI models requires higher-dimensional spaces. Lack of a foolproof method for verifying model accuracy. Experimental analysis and replication studies gain confidence in model performance. Combining experimentation and theory strengthens confidence in model correctness.

Video Processing for AI:
Video captures more information than text or photos. Videos provide a more comprehensive view of the world. AI techniques for video processing face computational constraints. Faster computing power will enable processing of large video datasets.

Experimental Approach in Computer Science:
Shift from mathematical to experimental science in computer science. Mathematical understanding remains important. Emphasis on experimentation and data analysis. Balancing empirical validation and theoretical explanations.

Data and Machine Learning:
The power of machine learning algorithms lies in the availability of large amounts of data. More data leads to increased effectiveness and accuracy in learning.

Concern about Data Access Imbalance:
Some organizations, like Google, Amazon, Facebook, and the National Security Agency, have vast amounts of data. This raises the question of whether those with data access will have an unfair advantage over those without.

Increased Accessibility of Computing Resources:
Computer access and power are becoming more accessible to individuals. Many machine learning tasks can be performed on personal laptops. Even with a modest budget, individuals can acquire GPUs for more advanced tasks.

Potential for Data Sharing:
Big companies may share or rent out their data centers and computing power to customers. This can reduce the need for individuals and organizations to make large capital investments in computing infrastructure.

APIs and Data Sharing Services:
Companies like Google, Microsoft, and IBM offer APIs that allow users to access their machine learning models. Users can train their own models on top of these pre-existing models.

Data Sharing Challenges:
Sharing data raises security and privacy concerns. Determining how to share data, manage access, and handle donated data is an ongoing challenge.

Predicted Trend: Data Sharing Rather than Hoarding:
Commercial providers are more likely to share data rather than hoard it. National security agencies may be an exception to this trend.

Award Presentation:
Dr. Peter Norvig received the President’s Medal from Stevens Institute of Technology for his significant impact on society, contributions to science and technology research, and widespread influence on business and education.

Recognition of Dr. Norvig’s Work:
Dr. Norvig’s ability to explain complex concepts in simple terms using imagery and graphics was lauded for making machine learning accessible to both laypeople and experts.

Establishment of the Dr. Norvig Scholarship:
In honor of Dr. Norvig’s profound influence, Stevens Institute of Technology will award a scholarship in his name to a deserving and talented student. Dr. Norvig will be involved in selecting the scholarship recipient and may visit the student during his frequent trips to New York.

Symbolic Gesture:
Instead of presenting Dr. Norvig with a traditional gift like a tie or a mug, the scholarship was chosen as a more meaningful way to recognize his contributions and create a lasting impact.

Media Attention:
Photographers requested that the award presentation be moved closer to their position to capture better photographs of the event.