MIT’s Transformation:

Jaron Lanier reflects on how MIT has changed over the years. He reminisces about the “funky” atmosphere the campus once had, a stark contrast to its current “highfalutin” ambiance. The dialogue also suggests that this transformation is not just architectural but cultural, affecting how space is used and what is considered important.

Rising Living Costs:

Lanier notes that the “world’s rent” has gone up, alluding to the increasing cost of living and perhaps the rising barriers to innovation and creative freedom. This rise in cost affects not just institutions like MIT but potentially the world at large.

Importance of Musical Instruments:

The conversation touches on the lack of musical instruments in the modern MIT environment, with Lanier emphasizing that musical instruments should be ubiquitous. His perspective is that music can be an essential component for fostering creativity and should “overwhelm every aspect of living entirely.”

Virtual Reality and Music:

The talk refers to a packed room where Jaron Lanier had previously discussed his vision for virtual reality. Interestingly, the motivation for Lanier’s venture into this field was partly to create musical instruments that had never been invented. This anecdote reveals Lanier’s multi-disciplinary approach, where technology serves artistic ends.

Overall, the conversation offers insights into how places of innovation like MIT have evolved, the impact of rising costs on creative spaces, and the intricate relationship between technology and art in Jaron Lanier’s career.

Introduction:

Jaron Lanier, a prominent figure in the tech world, discusses his long-standing interest in virtual reality (VR) and the metaverse. He delves into the complexities, challenges, and potential of these technologies, from the creation of immersive experiences to the inherent problems that come with the development and sustenance of virtual worlds.

Post-Symbolic Communication:

Lanier talks about his early vision for VR as a platform for “post-symbolic communication,” a kind of interaction that goes beyond traditional symbolic references. In this form of communication, people could create a shared experience by “playing reality into existence” through instruments in virtual reality. The objective is to engage in a more direct, shared experience rather than relying solely on symbols and text.

Challenges with VR Persistence:

One of the main issues facing VR development is the lack of persistence and adaptability in virtual worlds. Due to ever-changing technologies, like new headsets and graphics cards, it’s difficult for a virtual experience to sustain itself over time. This problem has been a “hidden drag” on VR, making developers hesitant to invest time and resources into the medium.

Prompt-Based Virtual Worlds:

Lanier suggests that one solution could be “prompt-based virtual worlds,” where users could request modifications to the virtual environment to suit their current situation. This would help the VR world adapt and persist, making it more feasible for long-term development and user engagement.

VR for New Types of Communication:

Lanier theorizes that merging prompt-based worlds with VR could lead to a new form of communication that is both “unbounded” and intentional, like a “shared waking state lucid dream.” This would bring a new layer of connection between people, bridging the gap between our isolated experiences.

Technological Dangers:

Lastly, Lanier raises ethical concerns about the future of technology. As tech becomes cheaper and more powerful, the potential for misuse grows, creating a dilemma for society. Lanier questions how we could prevent destructive behavior without imposing extreme restrictions on individual freedoms.

In summary, Jaron Lanier offers a deep and nuanced look at the challenges and potentials of virtual reality and the metaverse, including their roles in ushering in new forms of communication and societal challenges that accompany advancing technology.

Human Experience and Technology:

Jaron Lanier opens by discussing the human inclination towards progress and technological advancement. He posits that technology becomes significant only when people experience it. He theorizes about a machine that could create experiences so intense they surpass the allure of power or violence, emphasizing the role of art in human life.

Prompt-Based Virtual Reality and Interactivity:

Lanier explores the evolving nature of prompt-based technologies, stating that they move from simple image creation to complex 3D objects and scenes. He argues that the future lies in making these virtual worlds highly interactive, going beyond just static scenes. He highlights the difference between gaming and fully-realized virtual worlds, stating the latter need to be deeply interactive, dynamic, and haptic.

Test Case: Virtual Tennis:

Lanier describes a test case for virtual reality: a virtual tennis game where players can change the ball and their equipment in real-time. This serves as a proof-of-concept, showing that prompt-based technology can be interactive and dynamic, challenging the players to adapt continuously.

Importance of Inclusivity:

One of Lanier’s significant concerns is the risk of excluding individuals who may have specific cognitive or physical needs. He argues that prompt-based approaches can adapt experiences for these individuals in real-time. Lanier criticizes the current methods, which involve committees trying to account for various disabilities inefficiently.

Individuality and Adaptability:

He suggests that the future should involve using prompt-based technology to adapt to each individual’s unique needs. This stands in contrast to the one-size-fits-all solutions often deployed today, which are inefficient and fail to cater to individual differences.

Overall, Jaron Lanier is concerned with how technology can enhance human experience, emphasizing the need for prompt-based, interactive, and inclusive design.

Flexible Interfaces:

Jaron Lanier discusses the need for more flexible types of interfaces in computing, suggesting that we’ve reached a point where this can be achieved. He mentions his focus on virtual reality (VR) as one potential area for such advancements.

Critical Perspective:

Lanier also shares his journey from a technology enthusiast to a critic. He explains that his criticism mainly stems from the harmful social effects of algorithms, particularly recommendation algorithms that can be manipulated.

The Power of Prompts:

Lanier highlights that prompt-based techniques, like the ones used in conversational models like GPT-4, could revolutionize the way we interact with the digital world. He believes this could empower users rather than diminishing their agency, which is a common concern with AI systems.

VR and Human Experience:

Lanier talks extensively about VR, including its cyclic history of hype and disappointment. Despite the cycles, he believes in VR’s intrinsic value, especially the unique experience it offers when transitioning back to physical reality. He thinks that this transition enhances our sensory experience and contradicts the notion that VR will ever be indistinguishable from reality.

Commercial and Social Challenges in VR:

Lanier observes similar patterns in what works and what doesn’t in VR stores like Meta’s, based on his experience. He notes a peculiar finding that avatar-based social experiences in VR only seem to attract a minority of users, despite seeming naturally appealing.

Transformative Possibilities:

In summary, Lanier argues for a shift in focus from the content generated by AI to the user interface itself. He believes that prompt-based systems can transform our interaction with technology, both economically and politically, if developed correctly.

Through the talk, Jaron Lanier offers a multidimensional view on the future of computing interfaces, particularly focusing on the transformative potential of virtual reality and prompt-based interactions.

VR’s Emotional Challenge:

Jaron Lanier emphasizes that the power dynamic in virtual reality (VR) can make people uncomfortable. Unlike cinema where one is at a remove, VR engulfs the user, making experiences like horror or action too intense. This impacts the emotional calculus and makes VR less universally appealing as a gaming medium.

Creative vs. Consumptive VR:

Lanier observes that VR games gaining popularity often focus on creativity, such as Tilt Brush where users can create art. He admits that his perspective might be biased, as he favors VR as a creative rather than a consumptive medium.

Industry Focus on Visuals:

Lanier points out that the VR industry is primarily focused on enhancing graphics—more pixels, faster rendering, etc. He argues that haptics and interactivity are being overlooked, which are crucial for a more immersive experience in VR.

Adaptability in a Changing Field:

Lanier advises aspiring VR professionals to be adaptable and find the core reason for their interest in VR. He suggests that during market downturns (referred to as ‘VR winters’), one should be able to pivot their skills to other areas, such as AI.

Post-symbolic Communication vs. Language Models:

In response to a question about whether large language models move toward post-symbolic communication, Lanier clarifies that language models are inherently symbolic. However, he leaves room for the possibility that their use could lead to post-symbolic outcomes.

Hype Cycles and Historical Perspective:

Lanier acknowledges that technology often goes through hype cycles. He cites a historical example of stereographic photography from the 19th century but emphasizes that the cycles have become more pronounced in recent times.

Questions of Technological Interaction:

Finally, Lanier discusses the philosophical questions around our interaction with technology. He mentions Sherry Turkle’s perspective that the focus should shift from what computers can do for us to what we can do with computers.

This segment reflects Lanier’s nuanced views on the challenges and opportunities within the realm of virtual reality and broader technological trends.

Open Source and Its Limitations:

Jaron Lanier critiques the open-source model, particularly in the realm of AI. He argues that while the ethos of sharing in open source is admirable, it suffers from a “context loss” and a “provenance-free approach.” In this model, code can be shared without adequately tracing its origins or understanding how it will be used, which Lanier sees as a potential risk, especially when combined with AI technologies.

Code vs. Data:

Lanier emphasizes that in contemporary AI models, the source code is not as important as the data that feeds it. He argues that the actual programming involved in big language models is relatively straightforward and uninteresting compared to the data they utilize. He points out that open-source culture has been historically more focused on code rather than data, which could be a limitation as data becomes increasingly central in AI development.

Scale and Centralization:

One of the defining features of powerful AI models like GPT-4, according to Lanier, is their massive scale. Achieving this scale requires significant financial and logistical resources, which are typically only available to large, centralized organizations. This concentration of power raises questions about governance and control over these technologies. Lanier argues that while smaller models may offer some benefits in specific applications, their larger counterparts are generally more functional, especially in specialized domains.

Role of Tech Giants:

Lanier observes that the current reality leans toward a handful of giant tech companies controlling the future of large-scale AI models. He hints that this concentration of power is problematic and that relying on these companies to be “good citizens” is not a long-term solution. Lanier suggests that something will have to give in terms of governance or decentralization, but doesn’t specify what that might be.

The Duality of Function and Size:

Lastly, Lanier states that for the time being, the functionality of an AI model is largely determined by its size. While there is hope in the AI community that future insights may allow for smaller, equally functional models, the current ideology emphasizes the importance of scale. This, Lanier suggests, reinforces the need for centralized organizations to be at the forefront of AI development, at least for the moment.

Machine Learning and Its Impact:

Jaron Lanier emphasizes the transformative potential of machine learning, particularly in enhancing our understanding of the world. Machine learning algorithms have significantly altered data analysis, object recognition, and predictive models, among other applications.

Challenges in Virtual Reality (VR):

Lanier mentions that while VR has significant potential, it presents the challenge of needing to “fill up” the 3D spaces created. VR in its initial state is empty, requiring content to make it engaging or useful. This can be especially challenging given the complexity of 3D graphics and user interactions.

The Role of AI and Marketplaces:

The second speaker refers to a fine-grained marketplace like the one in Second Life to address the issue of empty VR spaces. The market enabled creators to specialize in smaller aspects, like “hands” or “wallpaper,” allowing for quicker, more varied development. The speaker also suggests that AI could play a role in automatically populating these spaces, though no platform has yet scaled this effectively.

Mixed Reality and Pandemic’s Impact:

Mixed reality, a blend of real and virtual worlds, is highlighted as an even more intriguing area for storytelling. However, efforts in this area have been hampered by the pandemic, as shared physical settings are less feasible when people are staying home.

Importance of Identity in Web3:

In the context of Web3, a decentralized internet, the focus shifts to the concept of “identity” and “provenance.” The speaker argues that tracking the origin (“provenance”) of digital artifacts will benefit people more than an obsessive focus on privacy. He suggests that identity could be thought of as one’s timeline or history, rather than just physical or genetic traits.

Asymptotic Approach to Unity:

When asked about humans evolving to achieve a unity of experience (“qualia”), the speaker suggests that while post-symbolic communication could move towards direct sharing of experiences, this would be an asymptotic approach, never fully reaching complete unity.

The conversation is a comprehensive exploration of the challenges and opportunities presented by emerging technologies, particularly in the realms of VR, machine learning, and digital identity.

Mystical Dualism:

Jaron Lanier describes himself as a “mystical dualist,” asserting that aspects of human experience like qualia are beyond technical understanding. He believes this perspective is both pragmatic and consistent with observations, claiming it simplifies thinking about complex issues. He argues that this view considers people and qualities as “supernatural,” operating outside conventional rational frameworks.

Terminology and Language:

Lanier reflects on his past contributions to terminology, particularly in coining the terms for “reality” and “mixed reality.” While he acknowledges the importance of terminology, he indicates a reluctance to create more terms, suggesting that others should contribute by adopting terms from diverse cultural backgrounds.

Early Virtual Reality (VR) Experiments:

In the early ’80s, Lanier founded the first VR startup. At that time, he and his co-founders fantasized about using VR to raise children who would grow up with a four-dimensional intuition, aiming to make them exceptional mathematicians. Though he didn’t follow through with this idea, it exemplifies his adventurous approach to technology and its potential impact on human cognition.

Relationship with Marvin Minsky:

Lanier also touches on his interactions with Marvin Minsky, a notable figure in artificial intelligence. He notes that their philosophical approaches to AI were almost entirely opposite, yet they enjoyed arguing and debating these ideas over many years.

Personal Anecdotes and Future Conversations:

Lanier shares a personal story about his daughter’s reaction to not being raised in a VR environment. He also exchanges pleasantries about future meetings to discuss musical instruments, indicating a diverse range of interests and conversations that extend beyond technology.

The segments collectively offer a window into Lanier’s unique viewpoints, his early role in VR, and his engagement with other leading thinkers.