Alan Kay (VPRI Co-founder) – Is it really “Complex”? Or did we just make it “Complicated”? (Jul 2014)


Chapters

00:00:06 Conceptualizing Complexity and Complication in Software Systems
00:12:13 Science, Systems, and the Limits of Understanding
00:18:27 Connecting Biology, Math, and Engineering for Innovation
00:25:37 Inventive Effort for Higher-Level Languages and Architectures
00:32:45 Reevaluating Programming Crew Size in the Era of Advanced Technology
00:38:22 The Evolution of Interactive Computer Graphics and Personal Computing
00:47:50 Developing New Languages for Personal Computing: Creating Efficient Graphics Systems
01:03:21 New Approaches to Programming: Making Code More Efficient and Creative
01:09:25 Next Generation Internet Computing
01:17:14 Key Advances in Computing
01:20:46 Innovations in Programming and Computing: From Dynabook to AI
01:24:52 Apple's Stifling of Creativity and Innovation
01:28:35 Abstraction and Optimization in Computing Systems
01:32:35 Revolutionizing Computing with Field Programmable Gate Arrays
01:40:21 Understanding Hardware Challenges for Nile's Execution

Abstract

Bridging Complexity: The Evolution and Impact of Alan Kay’s Perspectives on Computing and Science, with Supplemental Insights

In a series of thought-provoking talks, Alan Kay, a renowned figure in computing and science, delved deep into the intricacies of code complexity, the evolution of programming languages, and the convergence of science and engineering. By drawing analogies from personal computing, biological systems, and historical innovations, Kay emphasized the crucial need for strategic thinking in the field of software engineering and the creation of more efficient, user-centric computing systems. He critiqued the current landscape, including large codebases and their maintenance, while advocating for a paradigm shift towards domain-specific languages (DSLs) and customized hardware solutions. This article synthesizes these insights, highlighting Kay’s profound impact on how we perceive and approach complex systems in the digital age.

Organizing Main Ideas

1. The Complexity and Complications in Code: Kay highlighted the gap between system complexity and added complications in software, citing code size and the challenges in managing large codebases as significant issues. In his work at Xerox PARC, he demonstrated how software can be developed with significantly fewer lines of code, thereby reducing complexity and enhancing understanding.

2. Science and Systems Understanding: Kay’s unique perspective on science as a relationship between phenomena and explanatory stories underlines the importance of a holistic approach to understanding complex systems, particularly in biology. By drawing parallels between the study of biology and the development of computing systems, Kay emphasizes the importance of understanding the system’s architecture and dynamics, rather than focusing solely on individual components.

3. Convergence of Science and Engineering: Emphasizing the artistic nature of this convergence, Kay draws parallels between biological evolution and engineering design, urging for a molecular biology-like approach in scaling systems. He envisions a future where science, engineering, and art converge to create elegant and efficient computing systems.

4. Shift from Tactical to Strategic Thinking: Kay critiqued the prevalent tactical approach in computing, advocating for strategic thinking as exemplified by the invention of the arch and the need for high-level programming languages. He highlights the arch as an example of strategic thinking in architecture, allowing for more intricate and durable structures compared to simple stacking of bricks.

5. The Role of Crew Size in Software Development: Questioning the necessity of large programming teams, Kay stresses the importance of clear, debuggable requirements in software development. He argues that the large programming crews common in software development today are a relic of the past, when code was written in machine code and there was much more code to be written. Today, programming should be done in terms of specifications or requirements, which are often not runnable or debuggable.

6. Innovative Experiments in Personal Computing: Through his experiments, Kay demonstrated efficient and novel approaches to personal computing, emphasizing the potential of DSLs and meta-compilers in simplifying complex code. His work on a highly efficient personal computing system, requiring minimal lines of code, demonstrates the potential for significant advancements in software design.

7. Reimagining Computing Systems: Kay criticizes the stagnation in computing systems development and urges for the creation of better languages and environments, leveraging his insights from the Sketchpad and Frank System. He highlights the limitations of current systems like PowerPoint and advocates for leveraging computers’ full capabilities as simulators.

8. The Future of Computing: Envisioning a future with loosely coupled systems, semantic compilers, and n-dimensional CAD-like development, Kay outlines the path for more dynamic and efficient computing. He envisions a future where computing involves more dynamic relationships and efficient design processes, made possible by technologies like semantic compilers and n-dimensional printing.

9. Alan Kay’s Insights on Programming Languages: Kay addresses the challenges and potential in developing usable programming languages, criticizing the persistence of outdated paradigms and advocating for innovative approaches. He discusses the spectrum of programming languages from machine code to higher-level languages, emphasizing that as we move towards higher-level languages, the programming effort should decrease. However, inventing these languages requires significant upfront investment.

10. Apple’s Role in Personal Computing: Kay’s critique of Apple’s restrictive policies highlights the need for more open, creative computing environments for learning and innovation. He criticizes Apple’s policies for stifling creativity and learning in computing, reflecting on the ethical implications of their approach.

11. High-Level Abstractions and System Performance: Discussing theories on system development, Kay emphasizes the significance of high-level language efficiency and the role of custom hardware in optimizing performance. He illustrates the need for innovation in both software and hardware design, discussing the efficiency of high-level languages and the potential of custom hardware.

12. Exploring the Design Principles and Implementation of a Personal Computing System with Alan Kay: Kay discusses the conceptual framework, design principles, and implementation details of a personal computing system that aims to simplify and enhance the process of understanding complex systems. The system leverages DSLs (Domain-Specific Languages) to express various aspects of computing, enabling modularity and code reduction. The graphics system is implemented with 435 lines of code, significantly smaller than traditional approaches, demonstrating the power of DSLs.

13. Language Creation and Optimization: Kay discusses the concept of using a metalanguage to create languages, demonstrating the creation of a language with only 76 lines of code. He also presents an example of a TCPIP implementation in only 160 lines of code, achieved by viewing TCPIP as a non-deterministic parser. Additionally, Kay describes an experiment where simple rules applied to individual text characters caused them to organize themselves into a text justifier and editor, demonstrating how simple rules can lead to complex behavior in a distributed system.

14. Insights and Key Points from Alan Kay’s Discussion on Computing Systems and Design: Kay conceptualizes computing systems as intercommunicating processes, emphasizing adaptability and dynamic bindings. He proposes a recursive internet model with next-generation publish and subscribe mechanisms and negotiation processes to handle differences between objects. Kay advocates for a shift from solvers to constraints and meanings, assisted by automatic libraries and expert systems. He draws analogies to 3D printing and n-dimensional printing, envisioning future development systems resembling CAD systems for comprehensive design and testing.

15. Summary of Alan Kay’s Presentation on Computing and Programming Languages: Kay’s presentation highlights the potential of future computing devices with significantly increased power, enabling the testing of prototypes and the consideration of designs for shipping. He discusses the availability of his programming language, Ometa, for experimentation, including a user-friendly version that can be downloaded and run in a browser. Kay also discusses past attempts at creating specification languages that can be automatically compiled into high-level languages, emphasizing the sophistication of Sketchpad and the challenges of developing programming languages that minimize unnecessary code.

16. Alan Kay on Programming Language Evolution: Kay acknowledges the complexity of developing programming languages that minimize unnecessary code and allow for efficient real-time usage. He highlights the significance of NILE’s early development in enabling real-time use and the challenges of transitioning from a demo to a usable tool. Kay also discusses the evolution of compiler optimization techniques, emphasizing the potential of modern optimizers in achieving performance comparable to statically typed languages. However, he criticizes the lack of significant progress in computing, despite the identification of problems with programming languages decades ago, resulting in immense codebases and high maintenance costs.

17. Apple’s Restrictive Policies and the iPad’s User Interface: Kay critiques Apple’s restrictive policies, which prevent children from sharing their creations on the internet and downloading creations from other children, hindering personal computing and creativity. He also criticizes the iPad’s user interface, which caters to extreme ends of the user spectrum and neglects the needs of individuals who want to learn and use the device for more than just consuming media. Kay highlights the lack of stylus support, which makes it inconvenient for creative purposes, and the impact on education, as the iPad fails to offer the educational capabilities of computers. He draws parallels between Apple’s approach and Esau’s decision to trade his birthright for a bowl of soup, emphasizing the importance of long-term benefits over immediate convenience. Kay also criticizes Apple for intentionally creating a device that caters to convenience rather than educational needs and blames schools for failing to educate themselves about the iPad’s limitations as a learning tool. He mentions his efforts to make systems like Squeak and Scratch compatible with the iPad but faced obstacles due to Apple’s restrictive policies. Despite Steve Jobs’s partial concession to allow interpreters on the iPad, Kay criticizes Apple’s consumer-centric approach, which views consumers as sources of revenue rather than individuals capable of learning and creating.

Background and Additional Information

In concluding, this article reflects on the broader implications of Kay’s insights for the future of computing. His emphasis on strategic thinking and his calls for the development of better languages and environments have been instrumental in shaping the current landscape of computing. Kay’s vision for the future of computing, with its focus on dynamic relationships and efficient design processes, promises to usher in a new era of innovation and creativity in the field.

First-Order vs. Second-Order Theories:

Two schools of thought exist regarding the development of operating systems and programming languages. The first-order theory suggests avoiding creating your own operating system or programming language due to the risk of getting lost in the details and forgetting the original purpose. On the other hand, the second-order theory argues that skilled individuals should create their own operating systems and programming languages to avoid dealing with vendors and external ideas.

The Importance of Implementation and Debugging:

In computing, it is difficult to prove things mathematically. The most interesting aspects of computing often cannot be proven. Therefore, the best way to assess the quality of an idea is to implement it, debug it, and evaluate its performance.

Five Smart People Can Prove Suppositions Wrong:

To address the concern that high-level abstraction may result in suboptimal code for embedded systems, Alan Kay suggests assigning five smart people to work on the problem. He believes that there is a 90% chance that they would disprove the supposition and create something that rivals hand-crafted code in terms of speed and debuggability.

Benchmarking Real Programs:

Integer benchmarks are not always meaningful for evaluating a system’s performance. A more effective approach is to run real programs on the system and measure the actual ups and downs of the hardware. This provides a more accurate assessment of the system’s performance under realistic conditions.

Hardware and Software Progression:

Commercial hardware companies like Intel lack the incentive to innovate and improve their products, leading to stagnation in hardware development.

FPGA Technology:

Field Programmable Gate Arrays (FPGAs) have undergone significant advancements in recent years, providing a new approach to hardware design. FPGAs offer a large number of gates, allowing for the implementation of complex functions. Optimization techniques have improved the efficiency of FPGA programming, enabling better utilization of chip resources.

BEE3 and B-Cube:

Chuck Thacker, a former Xerox PARC researcher, developed the BEE3, a modern equivalent of the Xerox PARC machine using FPGA technology. B-Cube, the company behind the BEE3, offers a powerful system with four Xilinx FPGA chips, 170 gigabytes of RAM, and a compact form factor.

Hardware as Crystallized Software:

Alan Kay views hardware as software that is “crystallized early” due to its early implementation in the process. He emphasizes the importance of considering hardware design as an integral part of the overall system design process.

Hardware and Software Convergence:

In the 1960s, computer science required proficiency in both hardware and software, recognizing the interconnectedness of these disciplines. The distinction between hardware and software has led to separate cultures and approaches to system design.

Nile System and High-Level Language Efficiency:

The Nile system automatically adapts to the number of cores and hardware threads available, optimizing performance without manual intervention. Automatic SIMD generation and parallelization techniques enhance the efficiency of high-level programs. High-level languages can be used effectively for efficient system development, challenging the notion that low-level languages are necessary for performance.

Intel’s Multi-Core Approach:

Intel’s multi-core strategy was a reactive measure in response to a chip meltdown incident. Cache line management in the Intel architecture can be challenging due to its complex design.

Difficulties of Feeding Multiple Cores:

Intel’s memory architecture makes it challenging to efficiently feed even one core, let alone multiple cores with double threading.

Challenges Faced by the Nile Development Team:

The team faced significant difficulties in optimizing Nile’s performance on Intel hardware due to the lack of control over the hardware. The team spent a considerable amount of time and effort in efficiently distributing computations across Intel processors.

Initial Plan for a Proof of Concept:

The team initially intended to use FPGA (Field-Programmable Gate Array) hardware for a proof of concept.

Practical Considerations and the Importance of Custom Hardware:

For practical applications, it’s beneficial to have the option of designing and rolling out custom hardware. Developing simple ASICs (Application-Specific Integrated Circuits) can significantly improve performance, especially for tasks with high overhead in virtual machine execution.

Advantages of Byte Codes and Custom Hardware:

Byte codes are more compact than regular instructions, and executing them as fast as regular instructions can provide significant performance gains. Custom hardware can accelerate the execution of byte codes, leading to improved overall performance.



The development of Nile on Intel hardware highlighted the importance of having control over hardware and the potential benefits of custom hardware for improving performance.


Notes by: Rogue_Atom