Alan Kay (VPRI Co-founder) – Doing with Images Makes Symbols (Apr 2015)
Chapters
00:01:54 Early Innovations in User Interfaces: From Sketchpad to Engelbart's System
Dawn of User Interface: In the early 1960s, user interface emerged as a separate field, aiming to create understandable and efficient machines for users.
Link: The First Personal Computer: The Link, designed by Wes Clark in 1962, is considered the first personal computer. It featured a display, keyboard, and deck tapes, prioritizing a compact design that doesn’t intimidate users.
Computer Graphics Revolution: In 1962, Ivan Sutherland’s Sketchpad introduced groundbreaking concepts in computer graphics. Sketchpad enabled real-time manipulation of objects, non-procedural programming, and the first window-based system.
Sketchpad’s Features: Rubber bands for flexible manipulation of objects. Non-procedural programming through sketching and applying constraints. Master drawings and instances for creating and manipulating multiple copies of objects. Object-oriented programming principles.
Challenges and Uniqueness of Sketchpad: Sketchpad’s advanced concepts were ahead of its time and remain unmatched even today. It ran on a massive machine, highlighting the computational limitations of the era.
Douglas Engelbart’s Contributions: Engelbart introduced the mouse, black-on-white display, and hypertext. He emphasized studying user behavior and designing systems that adapt to natural working styles.
Engelbart’s Demonstrations: Engelbart’s system allowed remote collaboration and navigation using a lap board integrated into a chair. He showcased the potential of instantly responsive computer systems for intellectual work.
00:11:53 Origins and Development of Personal Computers
Engelbart’s Vision of Collaborative Computing: Engelbart envisioned a computing environment that facilitated collaboration, sharing, and direct communication among users. He believed in designing user interfaces that recognized people’s natural desire for communication and collaboration.
Multimodal Interaction: Engelbart’s system featured a combination of input devices, including a mouse, regular keyboard, and a five-finger keyboard, each serving different purposes.
Modeless Interaction: Kay highlighted the limitations of the mouse accuracy in Engelbart’s system, leading to a less efficient command process requiring multiple steps.
The Flex Machine: Kay’s Flex Machine was a significant early personal computer, notable for its powerful interactive object-oriented language and high-quality graphics.
Challenges with the Flex Machine: Despite its advanced features, the Flex Machine faced user resistance due to its complex user interface, which Kay attributed to a lack of visual appeal and intuitiveness.
Inspiration from Seymour Papert’s Logo Work: Kay gained insights into improving user interfaces through early exposure to Seymour Papert’s Logo work, which emphasized visual programming and engaging educational experiences.
00:22:49 Computer Innovation Developments in the 1960s
Engelbart’s Influence: Alan Kay’s initial perception of computers shifted from viewing them as complex machines to perceiving them as media accessible to children, similar to paper and pencil. This shift was inspired by Seymour Papert’s work on children’s programming.
RAND Corporation’s Iconic Programming: The first iconic programming system, GRAIL, was developed at RAND Corporation in response to economists’ complaints about the lack of user-friendly interfaces. GRAIL introduced the first data tablet, called the RAM tablet, and utilized iconic and analogic commands.
Graphical Input Language (GRAIL): GRAIL allowed users to interact with graphical elements directly without using a keyboard. It featured a modalist system that enabled users to modify commands without issuing additional ones. Kay emphasizes the system’s remarkable intimacy, allowing users to feel like they were directly manipulating information structures.
Flat Panel Displays and the DynaBook: In 1968, the University of Illinois developed the first tiny one-inch square flat panel display. Kay recognized the potential of combining this technology with the electronics of the flux machine to create a portable device, which he later termed the DynaBook.
00:27:09 Early Ideas and Experiments in Personal Computing
Metaphor of the Pencil: Kay’s goal was to create a machine that would be as ubiquitous as the pencil. He realized that adults were too entrenched in existing paradigms to adapt to this new technology, so he shifted his focus to children.
Kitty Comp: Kay’s early concept for the Dynabook was called Kitty Comp. It was designed to be built around a Sony Trinitron display.
Xerox PARC and Beanbags: Xerox PARC provided funding and a supportive culture for Kay’s research. Beanbag chairs were used to promote relaxation and creativity among researchers.
High-Resolution Display and Font Editor: Kay’s team developed a high-resolution display that allowed for crisp images and text. They also created the first font editor, enabling users to customize their own handwriting.
Sketching and Drawing: The bitmap display facilitated sketching and drawing, enabling real-time design on the computer. This eliminated the need for designing on paper and then transferring the design to the computer.
Minicom: In 1971, Kay’s team developed a prototype called Minicom. This system represented a significant step forward in the development of the Dynabook concept.
00:30:57 The Journey of Dynabook: A Revolutionary Computer
Sketchpad and Simula: Sketchpad, a groundbreaking interactive drawing program, inspired many concepts in the design of personal computers. Simula, an object-oriented programming language, served as a foundation for Smalltalk, a language developed at Xerox PARC.
The Alto Computer: The Alto computer, built by Chuck Thacker at Xerox PARC, was the first workstation. It incorporated an improved version of Engelbart’s mouse and laid the groundwork for subsequent workstations, including the Risa and the Macintosh.
Children’s Involvement: PARC invited children to work on the Alto machines, providing valuable insights and feedback. The machines were also taken to local schools, where hundreds of children had the opportunity to interact with them.
Animated Demonstrations: The presentation includes animated versions of programs and activities that ran on the Alto computer. These demonstrations showcase the machine’s capabilities, including a font editor, a painting program, and hand character recognition.
Dynamic Animation: The Alto computer was capable of dynamic animation, which allowed users to create interactive and engaging programs. This feature set it apart from earlier computers and paved the way for modern video games.
Inspiration for the Dynabook: The Dynabook model served as the inspiration for the Alto computer and subsequent personal computers. It envisioned a device that combined the portability of a notebook with the power of a computer.
Hand Character Recognition: The Alto computer featured hand character recognition, allowing users to draw letters and symbols to input commands and draw shapes. This intuitive input method made the computer more accessible to users of all ages and skill levels.
Customization and Creativity: The Alto computer’s software allowed users to customize their brushes and tools within the painting program. This flexibility encouraged creativity and experimentation, empowering users to create unique and personalized artwork.
00:34:28 Early Experiments with Windows and User Interfaces
First Tests of Windows on a Bitmap Display: Alan Kay presents the first experiments with windows, where users could drag them around by one corner, open them at the other corner, and create new windows by going to the upper right-hand corner. These tests were conducted on a bitmap display, and they explored user interface ideas such as manipulating windows.
Initial Group of Kids Interacting with the System: Kay showcases one of the first groups of kids interacting with the system. The display covers were removed, and cardboard boxes were used to protect the children from electrocution. Adele Goldberg, who later led the group after Kay’s departure, was responsible for many of the ideas in teaching the children.
Experiments with Animated Windows: Kay demonstrates the first test of an animated window, which was later abandoned due to insufficient machine speed. This approach involved animating all the text itself and having it occlude the window, creating a smoother transition compared to simply moving an open frame.
Alto Screen and Animation Capabilities: The Alto screen is presented in early 1973, showcasing its powerful capabilities for animating graphics in multiple planes. The Alto could handle complex 2.5D animation effects, making it a suitable machine for experimenting with animation.
Smalltalk and Collaboration with Kids: Kay highlights the integration of Smalltalk with the animated windows and graphics capabilities. This integration led to interesting outcomes when children used the system.
Second Level Class Taught by a 13-Year-Old: Marion, a 13-year-old student, taught a class of 12-year-olds using Smalltalk. Marion had developed one of the first tools in Smalltalk and was eager to share her knowledge. The class was videotaped, providing valuable insights into how children interacted with the system.
00:37:54 Computer Literacy: Access, Creativity, and Genre
Early Examples of Child Programmers in the 1970s: In 1975, a 12-year-old girl created an object-oriented graphics illustrator in Smalltalk 72, demonstrating the accessibility of programming to children. Another example is a 15-year-old boy who developed a sophisticated ham radio illustrator, highlighting the creative potential of programming.
Animators Using Programming for Animation: Animators at PARC in 1974 used programming to create an animation system that allowed them to preview their work before committing to film. An animator used this system to create a squashing ball animation, demonstrating the power of programming for creative expression.
A 12-Year-Old Girl’s Combined Horse and Jockey Animation: A 12-year-old girl combined two animations, a horse and a jockey, into a single fused drawing, showcasing the ability of children to modify existing programs.
Computer Literacy’s Three Key Components: Access literacy: The ability to understand and penetrate material created by others. Creative literacy: The ability to express one’s own intent and ideas through a medium. Genre literacy: The ability to adapt to different genres and styles within a medium.
Understanding the Mind’s Complexity: Difficulty in understanding how the mind works, as exemplified by an optical illusion where the mouth and eyes appear upside down.
00:46:58 Mentalities and Learning: Implications for Education and Creativity
Multiple Mentalities: Alan Kay proposes that our unitary sense of self is composed of multiple mentalities, each with distinct functions and ways of thinking. These mentalities evolved for different reasons and can process information in different ways, often conflicting with each other. Most of these mentalities operate subconsciously, while only a few are accessible to our conscious awareness.
Piaget’s Stages of Cognitive Development: Piaget’s theory suggests that children’s cognitive development progresses through distinct stages: Doing stage: Children focus on physical actions and immediate sensory experiences. Image stage: Children become dominated by visual images, leading to perceptual biases like the “tall, thin glass” experiment. Facts and logic stage: Children develop logical reasoning abilities, moving beyond immediate visual cues.
Brunner’s Experiments on Mentalities: Brunner’s experiments revealed the existence of multiple mentalities by demonstrating that children can switch between visual and symbolic thinking depending on the context. This challenges Piaget’s idea of a unitary cognitive development and suggests that different mentalities can coexist and interact.
Creativity and Mentalities: Studies on creativity, including Jacques Hadamard’s survey of mathematicians and physicists, indicate that many creative individuals rely on imagery and figurative thinking rather than mathematical symbology. This suggests that genius may involve the ability to access and engage with different mentalities, allowing for innovative problem-solving.
Teaching through Appropriate Channels: Kay criticizes traditional education methods that rely heavily on verbal communication, which may not be the most effective channel for teaching certain concepts. He emphasizes the importance of designing user interfaces that directly engage the appropriate mentalities involved in a particular learning task.
The Tennis Lesson Example: Kay presents an example from a television show where a 55-year-old woman with no tennis experience is taught to play tennis in 20 minutes using an unconventional method. The instructor, Tim Galway, acts as the user interface, guiding the woman’s learning process through specific instructions and sensory cues. This example illustrates the effectiveness of teaching directly to the mentality involved in the learning task, bypassing the limitations of traditional verbal instruction.
00:57:58 How to Learn Tennis and User Interface Design from a Tennis Coach
Karen Matthews’ Teaching Style: Focuses on keeping the mind calm and relaxed, allowing the body to perform naturally. Emphasizes watching the ball and enjoying the process of hitting it, rather than trying to control the outcome. Encourages learners to feel the racket as part of their arm and to listen to the sound of the ball on the racket.
The Serve as a Dance: Compares the serve to a dance, emphasizing rhythm and fluidity. Encourages learners to hum the rhythm of the serve to help them stay focused and relaxed.
Learning through Focus and Removing Interference: Learning happens when attention is focused and interference is removed. The talking and commenting mentality can interfere with learning and performance. It is important to find ways to short-circuit this mentality and allow the body to learn naturally.
Beginner’s Mindset and Hitting the Ball: Beginners tend to chase the ball, which reinforces the beginner’s mindset. The goal is to catapult learners into the hitting the ball stage as quickly as possible. Matthews’ method allows learners to hit the ball 98% of the time, even as beginners.
Application in User Interface Design: The principles of focusing on the process and removing interference can be applied to user interface design. The goal is to design interfaces that allow users to act like intermediates from the first time they use it, making the learning process rewarding in itself.
Application in Other Areas: The principles of Karen Matthews’ teaching style can be applied to various areas, including music, cognitive areas, and programming.
Differential Geometry in Drawing Circles: Alan Kay highlights three age groups and their approaches to drawing a circle using Logo. A 5-year-old’s body mentality allows them to intuitively understand the concept of a circle and create it by moving their body. A 10-year-old’s visual mentality leads them to use a compass and geometry to draw a circle. The 15-year-old, stuck in facts and logic, struggles with the abstract concept of x^2 + y^2 = r^2.
Importance of Context and Mentality: Kay emphasizes the power of context and mentality in problem-solving. Choosing the right data structure can simplify algorithms and provide most of the result automatically. Different mentalities, such as kinesthetic, image, and symbolic, offer unique strengths for various tasks.
Kinesthetic, Image, and Symbolic Mentalities: The kinesthetic mentality involves physical movement and knowing where things are in space. The image mentality excels at recognition and picking out objects from a conglomeration. The symbolic mentality allows for abstract reasoning and inferencing.
Smalltalk User Interface: The Smalltalk user interface, designed by Alan Kay and Dan Ingalls, combined these mentalities to create an innovative and intuitive interface. Collapsed windows, multiple applications, and Larry Tesler’s galley editor allowed users to work in the context of their results. Bob Flagle’s multiple views into the same structure enabled users to see the same information through different filters.
Evolution of the Smalltalk Interface: Over time, the Smalltalk interface evolved, incorporating new features such as pop-up menus, position-sensitive scroll arrows, and the ability to execute typed commands. The painting program showcased radio buttons for controlling features and intuitive drawing tools.
01:18:47 Exploration of Human-Centered Computing Technologies
Innovations from PARC Now Available on Powerful Micros: PARC’s innovations, such as the browser structure and debugger, are now accessible on powerful microcomputers like the Apple Macintosh 2.
Robust Error Handling: The Smalltalk system handles errors gracefully, such as when attempting to add three to a piece of text. Instead of crashing, the system provides a notification and allows the user to debug the issue.
Debugging Tools: The debugger in Smalltalk allows users to examine the state of a running process, including the code, parameters, and arguments.
Beyond Childhood: Computers for All: Kay emphasizes the importance of creating computers that are accessible to users of all ages, from children to adults.
Toddler’s Interaction with the Macintosh: Kay shares an anecdote about a 22-month-old girl who interacts with a Macintosh naturally, demonstrating the computer’s user-friendly design.
User Surveys with Young Users: Kay conducted a user survey with the young girl and discovered that she preferred Macintoshes with installed hard disks.
Child’s Proficiency with Macintosh Commands: The girl displayed a remarkable ability to use 70% of the generic window commands found in Macintosh applications, highlighting the computer’s intuitive interface.
Human-Centered Design: Kay emphasizes the importance of studying human beings and involving children in the design process to create user-friendly and accessible technologies.
01:24:45 Intellectual Property Protection in Interface Design
Alan Kay’s Influences and Inspiration: Alan Kay acknowledges the influence of multiple mentality ideas from Jerome Bruner, Tim Galway’s inner game of tennis, Suzuki’s teaching methods, and others on the development of user interface design concepts at Xerox PARC.
Winston Churchill Joke and Conspiracy in User Interface Design: Kay shares a joke about Winston Churchill emphasizing the importance of involving people in a shared conspiracy to gain their support and cooperation. He relates this to the concept of user interface design as a conspiracy that more people should get involved in.
Legal Protection of Fundamental Ideas in Graphics: Kay discusses the tension between protecting inventions and ensuring fair competition. He criticizes the confusion between ideas and inventions in patent law and highlights the complexities surrounding copyright issues. Kay believes that companies should have some protection for their investments in developing new ideas, but he recognizes the need for balance to prevent stifling innovation.
Future of Powerful Micros: Kay expresses concern that the computer industry is too conservative and focused on maintaining existing architectures like the 86,000 and 68,000. He believes that powerful micros have the potential to take us much further in terms of technological advancements, but the industry’s conservatism may hinder progress.
01:28:16 Computers, Education, and Human Potential
Computer Performance and Graphics: The field of computing is constantly evolving, and there is much potential for innovation and progress in the next five to ten years. Real-time 3D graphics at the flight simulator level could be achievable on a desktop within five years if companies invested in its development. The cost-performance of computers improves in jumps and fits, driven by advancements in packaging and the ability to increase the number of transistors on a chip.
Computers in Education: Computers have not been a significant success in direct teaching and computer-aided instruction. A piano can amplify musical impulse, but it can also turn people away from music if they focus solely on the instrument and not the underlying ideas. In education, the curriculum should be based on ideas, not media. Computers can amplify ideas, but they cannot replace them. Bad concepts cannot be fixed with technology.
Technology for the Disabled: There is significant potential for technology to improve the lives of people with disabilities. For example, technology could help the 10 million aphasics in the United States communicate more effectively. User interface design for the disabled requires a deep understanding of first principles and a willingness to rethink assumptions.
Increasing Human Potential: Students today face challenges in retaining their sense of self and their own ideas in a system that often emphasizes hurdles and obstacles. Students interested in science and engineering should recognize that these fields are art forms and require an aesthetic feeling. A solid liberal arts education can provide a strong foundation for developing an aesthetic sense and generating ideas for science and engineering. Aesthetic development is essential for scientists and engineers to connect with their work and find fresh perspectives.
Renewal and Fresh Perspectives: Renewal, not just education, is crucial for continued growth and learning. As people age, it becomes more challenging to see things from a fresh perspective. Engaging in activities like music can help maintain a fresh perspective and generate new ideas.
Abstract
The Evolution of User Interface Design: A Journey from Vision to Reality
Abstract
This article traces the groundbreaking advancements in user interface (UI) design, from early concepts to practical applications. Key figures like Alan Kay, Ivan Sutherland, and Douglas Engelbart played pivotal roles in shaping the field. We explore their contributions, the development of innovative technologies, and the challenges encountered along the way. This journey highlights how user-centric design and visionary thinking have transformed computing into an accessible and integral part of modern life.
The Inception of User Interface Design
The emergence of user interface design as a distinct field began in the early 1960s, with the goal of creating machines that were both understandable and efficient for users. The Link, designed by Wes Clark in 1962, stood out as the first personal computer, featuring a user-friendly design with a display, keyboard, and deck tapes. This era also witnessed the development of foundational principles in UI design. Alan Kay emphasized the significance of the user interface as an intuitive layer over the operating system, enhancing the interaction between user and machine. Douglas Engelbart envisioned a computing environment that enabled collaboration, sharing, and direct communication among users, focusing on interfaces that tapped into the natural human desire for communication and collaboration.
Ivan Sutherland’s Sketchpad: The Foundation of Interactive Computing
1962 marked a milestone with Ivan Sutherland’s Sketchpad, a system that introduced revolutionary concepts in computer graphics. It enabled real-time manipulation of objects, non-procedural programming, and featured the first window-based system. Notable functionalities included flexible manipulation of objects using rubber bands, the creation of master drawings and instances, and the incorporation of object-oriented programming principles. However, due to the computational limitations of the era, Sketchpad operated on a large machine, underscoring the technological constraints of the time.
Douglas Engelbart’s Visionary Contributions
Douglas Engelbart emerged as a visionary pioneer, introducing transformative tools like the mouse, black-on-white display, and hypertext, which fundamentally changed user interactions with computers. His focus on user behavior and ergonomics led to innovative solutions like the lap board and a unique combination of keyboard and mouse. Engelbart’s live demonstration of a collaborative, responsive computing system set a precedent for future UI developments. His system also featured multimodal interaction, incorporating a regular keyboard, a five-finger keyboard, and a mouse, each serving distinct purposes.
Alan Kay’s Dynabook: Envisioning the Future
Alan Kay’s Dynabook concept envisioned a universal tool, akin to a pencil, initially targeted towards children but eventually influencing designs for adults. His work at Xerox PARC led to significant innovations such as high-resolution displays, a font editor, and the integration of handwritten text and drawings. The Dynabook concept, which originated as the Kitty Comp, was intended to be built around a Sony Trinitron display. Xerox PARC’s supportive culture, epitomized by beanbag chairs, fostered creativity and led to the development of a high-resolution display and the first font editor. This environment also facilitated the creation of the Minicom, a significant step in the development of the Dynabook concept.
Early Experiments with Windows and Smalltalk
The development of pioneering windowing systems at Xerox PARC led to features like draggable and resizable windows, laying the groundwork for modern graphical user interfaces (GUIs). Smalltalk, an educational tool, allowed children as young as 13 to create tools, underscoring the language’s accessibility. Sketchpad and Simula, an object-oriented programming language, inspired the development of Smalltalk. The Alto computer, the first workstation, incorporated an improved version of Engelbart’s mouse and set the stage for subsequent workstations. Children played a crucial role in testing the Alto machines, providing insights and feedback. The Alto’s capabilities, such as dynamic animation and hand character recognition, were showcased through animated demonstrations, influencing the design of the Dynabook and other personal computers.
The Role of Mental Models in UI Design
Alan Kay highlighted the importance of catering UIs to different mental models, including kinesthetic, visual, and symbolic, and stressed the significance of context in problem-solving. The 1970s saw children demonstrating computer literacy by creating an object-oriented graphics illustrator and a ham radio illustrator. Animators at PARC used programming to create an animation system, which they employed for creative expression, like producing a squashing ball animation. A notable instance involved a 12-year-old girl who combined two animations into a single drawing, illustrating the creative potential of modifying existing programs. Kay identified three key components of computer literacy: access literacy, creative literacy, and genre literacy. These concepts underlined the importance of understanding the mind’s complexity and adapting teaching methods to suit various mentalities.
Advancements and Challenges in User Interface Design
At PARC, advancements like a browser structure and an intuitive debugger were demonstrated, pushing the boundaries of UI design. The involvement of users, particularly children, in the design process was emphasized, highlighting their valuable insights. The article also delves into the complexities of protecting UI innovations through intellectual property rights. Alan Kay discussed the different approaches to drawing a circle by children of various ages, demonstrating the role of mental models in problem-solving. The Smalltalk user interface, designed by Kay and Dan Ingalls, integrated these mentalities to create an intuitive interface. Over time, this interface evolved to include features like pop-up menus and dynamic animation, enhancing user interaction.
Future Perspectives and Challenges
Looking ahead, significant progress is anticipated in areas like real-time 3D graphics. The article emphasizes the importance of developing an educational curriculum that integrates technology effectively. Alan Kay’s insights on computing, education, and human potential suggest a focus on ideas and concepts rather than specific technologies. He advocates for activities like music to cultivate creativity and new perspectives. The concept of learning through multiple mentalities, as proposed by Kay, aligns with Piaget’s theory of cognitive development and Brunner’s experiments on visual and symbolic thinking. Karen Matthews’ tennis lessons, focusing on process over outcome, serve as a metaphor for UI design, emphasizing the importance of a relaxed and natural approach.
Conclusion
The evolution of user interface design highlights the significance of visionary thinking and user-centric approaches. Innovators like Kay, Engelbart, and Sutherland have not only introduced groundbreaking technologies but also established the philosophical and practical foundations for making computing accessible and intuitive. Their contributions have transformed computers into essential tools, seamlessly integrated into daily life. The challenge ahead lies in balancing technological advancements with thoughtful design, ensuring that user interfaces continue to evolve in ways that enrich and empower users.
Alan Kay's vision for personal workstations emphasized powerful tools for creativity and learning, leading to milestones like Smalltalk and the graphical user interface. His Dynabook concept aimed to provide information access anywhere, inspiring the development of modern portable devices....
User interface design pioneers like Alan Kay and Douglas Engelbart focused on understanding human behavior and creating intuitive interfaces, revolutionizing computing by making technology more user-friendly and accessible. Kay's vision for computing empowers all users, from children to adults, through user-friendly environments and human-centered design....
Alan Kay, a pioneer in object-oriented programming, emphasized the significance of holistic approaches, creativity, architectural integrity, and continuous innovation in programming. Smalltalk's influence extends beyond syntax and libraries, embodying architectural principles and structural integrity, akin to the construction of the Chartres Cathedral....
Computing's evolution reflects human creativity, perception, and adaptability, with promises of future transformation. Computers' impact on human experience will continue to shape the world in profound ways....
Insights from pioneers like Alan Kay and Joe Armstrong reveal the evolution of computing, emphasizing the importance of learning, scaling, and addressing unsolved problems. Computer scientists should think like scientists, considering scaling aspects early and embracing failure to drive innovation....
Alan Kay critiqued Smalltalk's stagnation after commercialization and emphasized its value as a bootstrapping tool for innovative systems design. He introduced Squeak as a platform for future developments and advocated for continuous innovation and exploration in systems design....
PARC's innovations in the 1970s, led by Adele Goldberg, revolutionized human-computer interaction with the Alto's GUI and Smalltalk's intuitive programming language, shaping modern computing and education. These innovations emphasized simplicity, user engagement, and joyful discovery, inspiring future advancements in personal computing and graphical user interfaces....