Building AI to See What Humans See:
Computer vision aims to build AI systems that can see and understand the world like humans. Cognitive and neuroscience studies have provided valuable insights into the human visual system, guiding the development of computer vision algorithms. Experiments have shown the remarkable speed and robustness of human object detection and categorization. Research has identified neural correlates for objects in the human brain, supporting the development of object recognition algorithms.

From Simple Organisms to the Cambrian Explosion:
540 million years ago, simple organisms lived in a primordial soup, lacking sensory capabilities and complex behaviors. During the Cambrian Explosion, a sudden evolution of vision triggered an evolutionary arms race, leading to a diverse array of animal species. Vision became a primary sensory system for animals, enabling them to actively interact with their environment.

The History of Computer Vision:
Computer vision is a relatively young field, with its origins in the 1960s Summer Vision Project at MIT. Despite significant progress, fully solving computer vision remains a challenge, but it has led to advancements in self-driving cars, image understanding, and AI-generated vision.

The Future of AI and Computer Vision:
The future of AI and computer vision lies in expanding our understanding and capabilities beyond human vision. AI systems should be able to see what humans don’t see, such as microscopic or infrared images, and generate visual content that humans desire. As we move towards a future empowered by AI, it is essential to consider the values and ethical considerations that should guide the development and use of these technologies.

The First Era of Object Recognition: Hand-Designed Features and Models:
Computer scientists hypothesized that objects could be represented by geometric shapes configured in specific ways. Models such as geode theories, generalized cylinders, and parts and springs were proposed. Researchers manually designed and parametrized these models and attempted to use them for object recognition. This approach faced challenges due to its reliance on manual feature engineering and limited success in real-world scenarios.

The Second Era of Object Recognition: Machine Learning:
The introduction of machine learning brought significant advancements to object recognition. Hand-designed features, such as SIFT and HOG, were used in conjunction with statistical machine learning models. Models like base net support vector machines, AdaBoost, and conditional random fields gained popularity. This era marked the beginning of the major flourishing of machine learning for computer vision. The availability of data, particularly with the rise of the internet, played a crucial role in enabling machine learning approaches.

The Importance of Data in Object Recognition:
Prior to the 21st century, obtaining data for object recognition was challenging. The lack of data limited the ability of machine learning algorithms to learn and generalize effectively. The availability of large and diverse datasets, facilitated by the internet, became a key factor in the success of machine learning for object recognition.

Data and the Birth of ImageNet:
The internet revolutionized access to data, leading to the creation of datasets for training and benchmarking in computer vision. Early datasets were small and focused on a limited number of object classes. The recognition of the need for more diverse and comprehensive data led to the creation of the ImageNet project.

The Role of Data in Machine Learning:
Data is considered a first-class citizen in machine learning, alongside algorithms. The availability of large and diverse datasets has been a key factor in the success of deep learning algorithms.

The Third Phase of Computer Vision and Object Recognition:
The advent of end-to-end models has eliminated the need for hand-designing features and learning models. These models can learn both the feature representation of the visual world and the end task, such as object recognition. This has led to significant advancements in deep learning, particularly in the field of computer vision.

Toronto’s Contribution to Deep Learning:
Toronto has played a significant role in the development of deep learning. The city is home to several leading researchers and institutions in the field. This has contributed to the growth and success of deep learning as a whole.

Data, Compute, and Neural Networks:
Data, compute, and neural network algorithms ushered in the deep learning revolution in AI. ImageNet challenge in 2012 marked the beginning of the deep learning revolution. ImageNet led to significant progress in computer vision, including object recognition, scene graphs, and image captioning. Deep learning revolutionized computer vision, leading to advances in 3D vision, dense pose estimation, semantic segmentation, and generative AI.

Beyond ImageNet:
Computer vision research extended beyond ImageNet to address more challenging tasks. Fine-grained recognition aims to recognize objects at a more detailed level, such as bird species or car models. Fine-grained recognition algorithms can be used to analyze societal trends by correlating car recognition results with census data.

Human Visual Recognition:
Human visual recognition goes beyond object recognition to include relationships, attributes, and dynamic interactions. Scene graphs represent visual scenes as a collection of objects, attributes, and relationships. Visual relationships are essential for human visual memory, understanding, and downstream tasks. Visual relationships are challenging to learn due to their infinite variety. Composable learning enables zero-shot or few-shot learning of new relationships by composing typical relationships.

Pushing AI Beyond Human Abilities:
AI research aims to push AI capabilities beyond human abilities. Fine-grained recognition is an example of a task that is challenging for humans and requires AI systems with specialized knowledge. AI systems can be used to analyze societal trends and provide insights into various aspects of human society.

AI Augmenting Human Capabilities:
AI’s potential to amplify human capabilities rather than replace them. Labor shortage in healthcare, particularly nurses, presents an opportunity for AI to augment human care. Dark spaces in healthcare can be addressed with ambient intelligence, using smart sensors and machine learning to provide health-critical insights.

Visual Illusions and Human Perception:
Human vision has limitations, such as missing fine-grained objects, overlooking items in plain sight, and struggling with visual attention. Visual illusions demonstrate the fallibility of human perception, influenced by context and bias.

Visual Bias in AI:
AI inherits visual biases from human data and historical context. Countering visual bias in AI requires addressing both technical and social aspects.

Privacy in Computer Vision:
Privacy is a critical consideration in various applications, including healthcare and surveillance. Privacy-preserving computer vision techniques are being developed, including face blurring, dimensionality reduction, body masking, federated learning, homomorphic encryption, and virtual privacy algorithms. Hardware-software approaches, such as lenses that filter images while preserving activity recognition, offer potential solutions for privacy-protected computer vision.

Forecasting and Guiding AI’s Impact on Society:
The need for a multidisciplinary approach to study, forecast, and guide AI’s impact on people and society. AI scientists must commit to considering the profound issues that AI may exacerbate, such as bias and socioeconomic changes, and work towards mitigating these effects.

Conclusion:
Shifting the narrative of AI to a more positive and constructive one, focusing on AI’s potential to enhance human capabilities and address societal challenges.

Hand Hygiene Monitoring:
Hand hygiene is crucial for reducing hospital-acquired infections. Lack of proper hand hygiene by clinicians is a major problem. Traditional monitoring methods, such as human monitors with clipboards, are expensive, biased, and unsustainable. RFID technology has been tried but is not specific enough. Vision-based monitoring can understand human posture and activity. A pilot study at Stanford Children’s Hospital using sensors and deep learning algorithms showed that automatic monitoring outperforms human monitors in accuracy.

Patient Vital Signs Monitoring in the ICU:
The ICU is where patients fight for life and death.

ICU Mobility Monitoring:
ICU patients often need mobility assistance, but monitoring their movement can be challenging. A collaboration between Stanford and Utah hospitals aimed to address this issue by using connect sensors to track patient activity. The system could detect four basic activities: getting out of bed, getting in bed, getting out of a chair, and getting in a chair. This technology can help ensure that patients receive the necessary mobility interventions to aid their recovery.

Home Care Applications:
Ambient intelligence technology can extend beyond hospitals and into homes, particularly for seniors or chronically ill individuals. Computer vision can be used for early infection detection, mobility pattern understanding, sleep pattern understanding, and diet monitoring. These technologies can provide critical support for home care and improve the quality of life for individuals with healthcare needs.

Personal Robotic Assistants:
Fei-Fei Li envisions personal robotic assistants that augment the capabilities of doctors, clinicians, and family members. However, current robotic research faces challenges in dealing with the ecological complexity, dynamism, and social interactivity of the real world.

Behavior Benchmark for Everyday Household Activities:
To address these challenges, Li and her team at Stanford developed a new North Star for robotic learning: an ecological robotic learning environment called Behavior. Behavior consists of a large, diverse set of activities that robots may need to perform in household settings. The benchmark includes 1,000 tasks ranked highest by human users in terms of their desire for robotic assistance. A simulation environment called Omni-Gibson was created to provide a realistic and interactive platform for robotic learning within Behavior.

Research Using Behavior:
The Behavior benchmark has facilitated various research projects, including benchmarking state-of-the-art algorithms, exploring reinforcement learning and language-based approaches, and investigating multisensory behavior. The environment has also been used for cognitive science studies and economic analyses related to household robots.

Sim2Real Transfer:
Li and her team are also working on transferring knowledge from simulation to real-world settings. They have a mobile manipulation robot at Stanford that can operate in both simulated and real environments.

Conclusion:
Fei-Fei Li’s research focuses on building AI systems that can see and do what humans want them to do. Her work spans various domains, including healthcare, aging population, and robotics, with the ultimate goal of creating personal robotic assistants that can augment human capabilities and improve the quality of life.

Introduction to Human-Centered AI:
Fei-Fei Li emphasizes the importance of combining scientific goals with the aim of benefiting society when conducting AI research. Human-centered AI focuses on understanding and forecasting the societal impact of AI, augmenting human capabilities, and developing technology inspired by human intelligence.

Stanford’s Human-Centered AI Institute (HAI):
Stanford launched the HAI four years ago to bring together faculty from various disciplines to lead research and education in human-centered AI. HAI conducts research in areas such as digital economy, robotics, foundation models, and ethics in AI. The institute has a review process for all research projects to ensure their positive impact on society.

Education and Outreach:
HAI embeds ethics into its undergraduate and graduate courses in computer science and AI. It expands education beyond Stanford, reaching out to lawyers, business leaders, journalists, and policymakers.

Policy Advocacy:
HAI engages with policymakers in Washington, D.C., to inform and influence AI-related policies. Stanford led the successful lobbying effort to establish the National AI Research Resource, a cloud task force, under President Biden’s administration.

Key Challenges in Applying AI Algorithms in Healthcare and Privacy:
Fei-Fei Li acknowledges the question about challenges in applying AI algorithms in healthcare and privacy. The specific challenges and their formulation as machine learning problems are not discussed in detail in the provided transcript.

Building Trust:
Establishing trust between technologists and healthcare professionals is a critical challenge. Lack of trust and communication can hinder collaboration and adoption of AI technologies. Attitudes of superiority or condescension from technologists can further impede trust-building.

Technical Challenges:
Long Tail and Out of Sample Problem: Medical errors are rare and diverse, making it difficult to train AI models with sufficient data. Privacy: Differential privacy, a method for protecting patient privacy, can be computationally slow. Vision-Based Computing: Fine-grained activity recognition using vision-based AI is still facing technical challenges.

Balancing Augmentation and Challenge:
While AI can augment human skills and improve society, it is important to consider how AI can also challenge humans to improve their own capabilities. Striking a balance between augmentation and challenge is crucial for fostering human growth and development.

Human Augmentation and the Loss of Essential Skills:
We rely on technology to augment our capabilities, but this can lead to the loss of essential skills. For example, GPS has made it easier to navigate, but if it fails, we may struggle to find our way without it. Similarly, AI may lead to the loss of skills such as writing essays, as students can use AI tools like ChatGPT to complete their assignments.

The Impact of AI on Human Agency:
As we rely more on AI, it raises profound questions about human agency and the organization of our society. For example, our political structure may be impacted when humans and machines have a different relationship. It is essential to consider the broader implications of AI on human society and to foster multidisciplinary research to address these challenges.

The Need for Nuanced Public Discourse on AI:
The public discourse on AI is often polarized, with extreme views dominating the conversation. There is a need for a more nuanced and balanced discussion that acknowledges both the potential benefits and risks of AI. Instead of calling for a pause on AI research, we should focus on developing thoughtful regulatory frameworks that address the specific areas where AI impacts human users directly.

The Role of Research Community in Shaping the Future of AI:
The research community has a responsibility to engage in public discourse and provide evidence-based insights on AI. Researchers can contribute to the development of regulatory frameworks and policies that guide the responsible development and use of AI. By actively participating in the public conversation, researchers can help shape the future of AI in a way that benefits society.

Public Perception of AI:
Fei-Fei Li highlights two common themes in public discourse about AI: concerns over AI’s impact on humanity and AI regulations. She argues that instead of extreme discussions, there needs to be a focus on regulating specific AI applications, such as those related to drugs and food.

AI and Humanity:
Li emphasizes that humanity has a history of self-destructive tendencies, including wars, social inequality, and wealth disparity. She suggests that instead of blaming AI for potential negative outcomes, it is more productive to focus on making positive changes within human endeavors.

Importance of Engagement:
Li encourages researchers to engage in thoughtful discourse with policymakers and business leaders to provide informed perspectives on AI. She emphasizes the need for policymakers to hear from experts in the field to make informed decisions rather than relying solely on extreme voices.

Rationalizing Augmented Collective Intelligence:
Li highlights the importance of developing guardrails and standards to ensure the responsible use of AI in various applications. She suggests creating a set of best practices and guidelines for the integration of AI into different sectors, such as healthcare, education, and finance.

Future Steps:
Li emphasizes the need for ongoing research and development to advance AI technologies while addressing potential risks and concerns. She stresses the importance of collaboration between researchers, policymakers, and industry leaders to shape the future of AI in a responsible and beneficial manner.

Challenges in Ethical AI Development:
Lack of understanding among technologists about the broader implications of their work. Specialized education: gap between technical fields and humanities. Ethical considerations often neglected in the early stages of AI research. Balancing innovation and responsible development.

Proposed Solutions:
Multidisciplinary collaboration: involving law scholars, bioethicists, and ethicists. Self-governance and adherence to national regulatory frameworks. Embedding ethics into technical courses and educating humanities and social scientists. Creating “bilingual” students with both technical and ethical understanding. Educating journalists, CEOs, CTOs, congressional staffers, and policymakers.

Global Task Force on Regulating AI:
The need for a global task force is being discussed at national and international levels. Potential challenges in coordinating and implementing regulations across different countries.

Role of Researchers in the Discourse on AI:
Researchers are often a silent majority in the discourse on AI. The need for researchers to actively engage in discussions and policymaking.

Global AI Governance:
Fei-Fei Li emphasizes the need for global AI governance, similar to how nuclear governance is handled, to address the horizontal nature and international implications of AI. She points out the efforts of organizations like OECD in this regard but stresses the need for more collaboration and dialogue between countries, especially those with close relations like Canada and the U.S.

Lifting the Voices of the Silent Majority:
Li highlights the importance of amplifying the voices of underrepresented groups in AI, including women and researchers working on non-glamorous but impactful areas like healthcare and climate. She mentions Stanford’s efforts to host journalist days and engage in thoughtful policy discussions to promote diverse perspectives and research.

Resource Concerns in AI Research:
Li acknowledges the growing concern that the development of AI has become a resource game, with large models requiring immense compute power and financial resources. She emphasizes the need for government resourcing in the public sector to ensure a healthy ecosystem where students, entrepreneurs, and researchers have access to adequate compute resources. She advocates for a balance between large-scale models and smaller, more focused research to address the concentration of resources in a few entities.

Call for Collaboration and Resource Allocation:
Li concludes by reiterating the need for collaboration and resource allocation to address the challenges and opportunities of AI, ensuring that the benefits of AI are widely shared and that the risks are effectively managed.

AI Research Beyond Big Tech:
Fei-Fei Li emphasizes the importance of creative research in AI that goes beyond the field of big tech companies. She highlights her team’s healthcare and robotics research as examples of innovative work that might not be feasible in a corporate setting. Li stresses the need for a multi-faceted approach to AI research, involving both individual innovation and government support.

Income Inequality and AI:
Li expresses concern about the potential for AI to exacerbate income inequality. She points to the digital divide and limited access to AI resources as contributing factors to this issue. The automation brought about by AI is also seen as a disruptor of the current economic landscape, particularly affecting knowledge workers. Li emphasizes the need for further research and analysis to understand the impact of AI on various socioeconomic groups, including low-income populations.

HAI’s Economic Lab:
To address these concerns, the Human-Centered AI Institute (HAI) has established a digital economy lab. This lab focuses on studying the economic implications of AI, including income inequality and the impact on different socioeconomic groups. The goal is to gain a deeper understanding of these issues and develop strategies to mitigate negative consequences.