Introduction:
Satya Nadella highlights the rapid pace of AI innovation, particularly generative AI, which is now moving beyond theoretical discussions to real-world product development and deployment. The focus is on co-pilots, with examples of companies like Airbnb, Duolingo, BT, Bear, Dentsu, Goodyear, Lumen, Microsoft, LCEG, Epic, Rockwell Automation, and Siemens using Microsoft CoPilot.

Productivity Gains:
Microsoft shares data on the productivity gains driven by CoPilot, resulting in faster task completion, reduced information search time, more effective meetings, and enhanced collaboration.

CoPilot as the New UI:
CoPilot is envisioned as the new user interface, enabling access to global and organizational knowledge while acting as an agent for taking action based on that knowledge.

100 Updates Across the Stack:
Microsoft introduces 100 new updates across the entire technology stack to realize the vision of CoPilot as a comprehensive co-pilot stack spanning infrastructure, foundation models, data, tool chains, and the co-pilot itself.

AI Infrastructure Layer:
Azure as the world’s computer, with a comprehensive global infrastructure of 60+ data center regions. Collaboration with industry partners to incorporate the best innovations in power, data center, racks, networks, and AI accelerators.

Renewable Energy:
Microsoft’s commitment to carbon neutrality and erasing historical carbon emissions. On track to generate 100% of data center energy from zero-carbon sources by 2025.

Network Connectivity:
Advanced and extensive network connecting data centers. Breakthrough hollow core fiber technology improves speed by 47%.

Data Center Innovation:
Manufacturing hollow core fiber in the world’s only dedicated factory. First live deployment connecting data centers in the United Kingdom.

Azure Boost:
Azure Boost, a new system, enables improved networking, remote storage, and local storage throughput. Azure becomes the best cloud for high-performance workloads, while also strengthening security.

Partnerships with AMD and Intel:
Azure leverages AMD and Intel’s innovations, such as AMD’s use for high-end compute and Intel’s MVV3 virtual machines for SAP deployments. Microsoft learns from customer workloads to optimize the entire stack for improved performance and efficiency.

Azure Cobalt:
Azure Cobalt, Microsoft’s first custom in-house CPU series, is designed specifically for the Microsoft Cloud. The 64-bit 128-core ARM-based Cobalt 100 is the fastest CPU among cloud providers. Cobalt 100 powers Microsoft Teams, Azure Communication Services, and Azure SQL.

AI Accelerators:
Azure partners with industry leaders, including NVIDIA, to make Azure the best cloud for both training and inference. Azure has the most powerful AI supercomputing infrastructure using NVIDIA GPUs. Azure was the largest submission to the MLPerf Benchmarking Consortium, delivering better performance than any other cloud. Azure adds NVIDIA’s latest GPU AI accelerator, H200, for larger model inferencing with low latency.

Azure Confidential GPU VMs:
Azure introduces the first preview of Azure confidential GPU VMs, co-designed with NVIDIA. Confidential GPU VMs enable AI models to run on sensitive data sets on Azure, with end-to-end protection.

AMD MI300X AI Accelerator:
AMD’s flagship MI300X AI accelerator comes to Azure, offering more choice for AI-optimized VMs. MI300X offers industry-leading memory speed and capacity, enabling faster serving of large models with fewer GPUs.

Azure Maya:
Azure Maya, Microsoft’s first fully custom in-house AI accelerator, is announced. Maya 100, designed for Cloud AI workloads, is manufactured on a 5-nanometer process and has 105 billion transistors. Maya 100 includes an end-to-end rack for AI, featuring a tailored cooling unit and rack-level closed-loop liquid cooling.

Introduction to Maya: Satya Nadella introduces Maya, a cutting-edge technology designed to enhance data center capabilities. Maya integrates advanced silicon packaging, high-bandwidth networking, modern cooling, power management, and sophisticated algorithmic design in both hardware and software. This innovation aligns with Microsoft’s commitment to zero waste and is a step forward in their data center evolution.

Maya’s Implementation and Impact: Nadella highlights the implementation of Maya accelerators across Microsoft’s infrastructure, initially supporting internal workloads before expanding to third-party applications. This move underscores Microsoft’s commitment to silicon diversity, powering powerful foundation models and diverse AI workloads, from GitHub Copilot to custom AI applications. Nadella emphasizes the holistic nature of these innovations, encompassing everything from hardware to software.

Comprehensive System Approach: Nadella elaborates on the comprehensive approach Microsoft is taking. This includes developing next-generation fiber optic cables, sourcing renewable energy, and creating new thermal distribution methods. The innovations extend to silicon technology, with the overarching goal of maximizing efficiency, performance, and scalability, delivered through Microsoft and its partners.

Foundation Models and AI Partnerships: The advancements in systems and technologies pave the way for foundation models in AI. Nadella points out that these models range from trillions of parameter large language models (LLMs) to smaller, task-specific models, all supported by powerful GPUs in Azure. He emphasizes the versatility of these models in meeting various cost, latency, and performance requirements. The partnership with OpenAI is highlighted as a key element in advancing AI models, demonstrating Microsoft’s commitment to leading in AI innovation and collaboration.

Azure’s Commitment to OpenAI Innovations: Satya Nadella promises to integrate all OpenAI innovations into Azure AI, highlighting the introduction of GPT-4 Turbo and GPT-4 Turbo with Vision to Azure OpenAI service. He emphasizes GPT-4 Turbo’s features like lower pricing, structured JSON formatting, and extended prompt length capable of handling up to 300 pages of text.

Enhancements in Azure OpenAI Service: Nadella announces the preview of GPT-4 Turbo on Azure OpenAI Service and its token pricing parity with OpenAI. He also discusses the upcoming feature that allows GPT-4 Turbo to connect with Azure AI Vision, enabling the use of video images and text as prompts.

Customization and Open Source Integration: Nadella introduces the capability to fine-tune GPT-4 on Azure OpenAI Service using custom data. He emphasizes Microsoft’s commitment to open source, highlighting the broad selection of models in Azure’s catalog including stable diffusion for images, Code Llama for code generation, Mistral 7B for text translation and summarization, and NVIDIA’s Nemetron 3 models for general-purpose AI apps.

New Models as a Service Offering: Azure is launching a new service that provides hosted APIs for large open-source models, easing the burden on developers by eliminating the need to provision GPUs. This service begins with Lama 2 and will include other models like Mistral’s premium models and JS, a high-quality Arabic language model.

Advancements in SLMs and PHY 2 Announcement: Nadella highlights the importance of Small Language Models (SLMs), mentioning PHY 1.5, a Microsoft Research model that performs well despite its smaller size. He announces PHY 2, an improved version with better performance and mathematical reasoning capabilities, which will also be open source.

Tooling and Safety in Azure AI Studio: Azure AI Studio is presented as a comprehensive toolchain for building, customizing, training, evaluating, and deploying AI models, with a strong focus on safety. It extends its capabilities to endpoints, starting with Windows, allowing customization of SLMs for various applications.

NVIDIA Partnership and Generative AI Foundry Service: Nadella speaks about Microsoft’s deepening partnership with NVIDIA, focusing on making Azure the best cloud for AI training and inference. This collaboration is expanding to include NVIDIA’s Generative AI Foundry Service on Azure, combining NVIDIA’s foundation models, tools, and AI supercomputing services to create generative AI models.

NVIDIA and Microsoft Partnership:
NVIDIA and Microsoft’s partnership involves multiple facets, including supplier-customer relationships, co-development of fundamental computer science, and hosting of NVIDIA’s native stack on Azure. This partnership enables NVIDIA’s ecosystem of developers to seamlessly run their applications on Azure, amplifying NVIDIA’s ecosystem.

Three Waves of Generative AI:
The first wave is led by startups and cloud internet services, focusing on generative AI applications. The second wave is driven by co-pilots, such as those in Office or Windows 365, and targets enterprise applications. The third wave, which is expected to be the largest, will focus on heavy industries and the integration of NVIDIA’s Omniverse and generative AI to enable digitalization.

Generative AI as a Platform Shift:
Generative AI is seen as the most significant platform transition in computing history, surpassing the impact of PCs, mobile devices, and the internet. It creates a new type of datacenter dedicated to running AI models and generating intelligence, known as AI factories. The TAM expansion in AI factories and co-pilots represents a significant growth opportunity for the industry.

AI Foundry:
NVIDIA aims to become a foundry for AI models, similar to TSMC’s role in chip manufacturing. This service will enable customers to build their own custom large language models, leveraging NVIDIA’s AI expertise, end-to-end workflow, and AI factories.

Confidential Computing and Azure Integration:
NVIDIA and Microsoft collaborate on advancing fundamental computer science, including confidential computing and generative AI. Hosting NVIDIA’s native stack on Azure allows NVIDIA’s developers to seamlessly integrate with Azure APIs and services, such as secure storage and confidential computing.

Microsoft Fabric:
Microsoft Fabric unifies data and analytic workloads into a single experience, allowing teams to connect and work from the same copy of data across engines. It provides a cost-effective and efficient integration experience for data professionals. Fabric integrates with Microsoft Purview to govern and protect data, ensuring security and compliance. AI-powered features, such as Copilot, enhance productivity by automating tasks like data flow creation, SQL statement writing, and report building.

Mirroring in Fabric:
Fabric introduces a new mirroring capability that enables frictionless integration of existing cloud data warehouses and databases from various sources, including Cosmos DB, Azure SQL DB, Mongo, and Snowflake. Data is synchronized automatically between the source and Fabric in the open-source Apache Parquet and Delta Lake formats.

Example: Electric Car Charging Company:
A real-world example demonstrates how an electric car charging company uses Fabric to proactively maintain charging stations. Real-time IoT data from charging stations is streamed into Cosmos DB and mirrored to Fabric, ensuring data consistency. Fabric combines data from various sources, including maintenance schedules and weather data, into a single lake house. A preventive maintenance model is built using Azure AI Studio to predict when stations require servicing, based on data analysis and modeling. Power Apps are used to deliver alerts to maintenance crews and gather additional context through a chat function.

AI Integration in the Data Stack:
Microsoft integrates the power of AI across the entire data stack, enabling users to leverage AI capabilities within their data applications. Vector indices are added to Cosmos DB and PostgreSQL, allowing for efficient vector search and retrieval augmented generation (RAG). Azure AI extensions for PostgreSQL simplify the management of AI-powered indices within the database, unlocking the full potential of relational data. Azure AI Search provides a comprehensive vector search solution with state-of-the-art re-ranking technology, delivering high-quality search results beyond traditional vector search methods.

Reimagining Teams in the AI Era:
Microsoft’s vision for Teams is to create a unified platform for collaboration, chat, meetings, and calling, enhancing productivity and connectivity. New Teams, reimagined for the AI era, offers improved performance, reduced resource usage, and a streamlined user experience. It serves as a foundation for next-generation AI-powered experiences, transforming the way people work. Teams is now available across multiple platforms, including Windows, Mac, and mobile devices. Beyond communication and collaboration, Teams serves as a multiplayer canvas, integrating business processes into the workflow. The Teams Store hosts over 2,000 apps from leading providers, with more than one million monthly active users. Organizations have developed over 145,000 custom line-of-business applications within Teams. Presence is a key aspect of Teams, motivating the integration of Mesh technology to enhance employee connectivity and collaboration across various devices.

Mesh:
Mesh will be generally available in January. Allows users to express themselves with avatars in 2D Teams meetings and immersive 3D spaces. Immersive spaces enable new ways to connect, with spatial audio for directionality and proximity. Custom spaces can be created for specific needs like employee events, training, and product showcases. Mesh Toolkit and no-code editor promote employee connection.

Microsoft Copilot:
Microsoft’s vision is a co-pilot for everyone and everything. Microsoft Copilot provides a unified experience across surfaces, understanding context on the web, devices, and at work. Copilot offers various functions like shopping, coding, analyzing, learning, and creating. Search is integrated into Copilot, bringing the context of the web. Bing Chat is now Copilot, a standalone destination that works on various platforms. Enterprise version with commercial data protection is available at no additional cost to eligible Enter ID users. Copilot for Microsoft 365 reasons across the Microsoft Graph, answering tasks, and integrating with applications. Plugins and GPTs can be used to tailor Copilot experiences.

Copilot Studio:
Copilot Studio allows users to build custom GPTs, create plugins, orchestrate workflows, and manage customizations. It comes with pre-built plugins for SAP Workday, ServiceNow, and other applications. Users can connect to databases, custom backends, and on-premise systems. Copilot Studio enables organizations to extend Copilot with capabilities unique to their needs.

Copilot for Different Roles and Functions:
GitHub Copilot helps developers code 55 times faster by turning natural language into programming language. Copilot assists SecOps teams in responding to threats at machine speed. Copilot helps sellers close more deals by enriching customer interactions with CRM data. Copilot for Service helps customer service agents resolve cases faster by providing access to relevant knowledge.

Integration with Existing Tools:
Copilot integrates with Teams, Outlook, Salesforce, ServiceNow, Zendesk, and Dynamics 365. Copilot plugins are available for Confluence, Jira, Mural, RAM, and Trello. Bayer researchers can use Copilot to ask questions about crop science models within Teams.

AI and Mixed Reality:
Combining AI and mixed reality creates a more immersive and intuitive user experience. Frontline workers can use voice, gestures, and eye movements to interact with Copilot and complete tasks. Dynamics 365 users can leverage mixed reality and AI to enhance their productivity and efficiency.

Quantum Computing and AI:
Azure Quantum Elements uses Graphformers to generate entirely new chemical compounds. It can compress 250 years of progress in chemistry and material science into the next 25 years. Scientists can use Copilot to reason over quantum results, narrow down candidates, and design novel molecules. Microsoft’s collaboration with Photonic expands the quantum networking and computing capabilities within Azure.

AI for Empowering Individuals and Organizations:
Microsoft’s mission is to empower every person and every organization with a Copilot. Copilot can provide personalized guidance, medical advice, and mentorship to individuals. Anton, a freelance developer from Ukraine, shares his story of how Copilot has empowered him despite his disability.