Intelligent User Interfaces
Dateline: 09/28/99
Some thirty years ago Nicholas Negroponte, the founding director of MIT's Media Lab, mentioned the idea of the conversational computer, a machine that could understand human voice, gestures and body language. Arthur Clarke's novel "2001: A Space Odyssey" introduced probably the most famous conversational machine, HAL 9000. Thirty years have passed and the conversational computer is still very much the unreached goal of AI community. Remember, HAL became operational at the H.A.L. plant in Urbana, Illinois, on the twelfth of January, 1997...
Rejecting the idea that people must adapt to machines, some researchers have made interesting breakthroughs. Probably the most fundamental technique underlying the concept of intelligent interface is natural-language processing, often augmented by voice recognition technology. You can refer to previous articles on Verbots, Klone Server, Artificial Life, Inc., Microsoft Agent and similar "social interfaces" (sometimes called bots, chatterbots, avatars, agents...) for more information on implementing this technologies into modern operating systems. However, Microsoft's Bob™, one of the first and heavily publicized attempts to introduce such interface to a broad audience, was a market failure. The concept still lives in Intelligent Assistants featured as a part of newer versions of Microsoft Office. Yes, I am talking about famous paperclip, friendly dog or a little Einstein that pops up when Word detects that you are having problems with some routine task. People either hate this technology strong enough to remove it completely from the system, or like it and have "conversations" with these characters all the time.
The underlying concept is that animated computer characters can intelligently help users by prompting them with questions and suggestions. It should further adapt to the user, keeping track of problems and offering useful tips. The need for such helpers makes it very clear that our computer systems are simply too complex. It is still unclear whether an intelligent agent should be a graphical character or simply a set of scripted rules. NetSage Corporation, is creating key technologies that are at the heart of Microsoft's Assistants in Office 2000 and General Magic's highly acclaimed voice-based assistant, Portico. Their Sages are intelligent social interfaces that connect people to digital content. To put it simply, these animated software characters reside on user desktop and react with Web servers to assist the users. Sages can take information from the site and present it to users in their preferred manner, or use their personal profile to find the most appropriate and interesting content.
Bob was developed around research showing that working with a computer becomes increasingly intuitive the more it resembles working with another person. With the social interface, users control the computer using more familiar social forms of communication rather than abstract, technical commands. Clifford Nass and Byron Reeves, leading researchers at NetSage, are claiming that interactions with computers, television and new technologies are identical to real social relationships and to the navigation of real physical spaces. The results of numerous psychological studies that led them to the conclusion that people treat computers, television and new media as real people and places. One of their most intriguing conclusions is that the human brain has not evolved quickly enough to assimilate twentieth-century technology. Now it can be easily understood how Joseph Weisenbaum became gravely concerned by how popular his Eliza (first chatterbot simulating rogerian psychiatrist) became and how immersed people could become in a conversation with a computer program.
There are other approaches that are focused on simplifying the process human-computer interaction. Actioneer, Inc. provides improved means for managing and executing actions in various software products and services. For example, Actioneer for Outlook features an intelligent pop-up window where users enter text to capture action items on the fly. Actioneer then automatically interprets the entry and distributes the action item to the appropriate Outlook folder. A to-do item entered into Actioneer, "Call Tim tomorrow at 2 PM," can be automatically distributed to Outlook's Contacts List and Calendar or Task List, without even opening Outlook. The pop-up window is available with a single click, so a user can capture an action item and return to their work very quickly.
There are other, interactive and animated 3D characters designed to express affect, as well as model dynamics of human emotions such as emotional intensity, transitions from one state to another, and the decay of emotional states. One of the most interesting projects in this area that I'm aware of is Bruzard, a platform for computer expression of emotion from MIT Media Lab. Their page on interaction with Affective Computers presents so many interesting projects and concepts, including bio-sensors and a personal wearable computer systems.
Social interfaces, intelligent agents and assistants, affective computers... what's next? The ultimate user interface will me modeled after holodeck from Star Trek series. Several multimodal systems are being developed to allow users to interact with a computer without ever having to touch a keyboard or mouse, not to mention reading those lengthy and boring manuals. Scientists are building virtual worlds which recognize and respond intelligently to user actions. Coupling voice, gesture and context with an expert system creates an interface more powerful than any of these elements alone. Of course, this process is still far from complete, but the goal could be reached within next few years. Mark Lucente, a staff member at IBM's Thomas J. Watson Research Center, collaborating with researchers at MIT, built a DreamSpace system that allows for natural interaction: it "hears" users' voice commands and "sees" their gestures and body positions. Such systems will be commercially available between 2002 and 2007, but the bits and pieces of this technology, like IBM ViaVoice™, are already entering the market.
Recent science fiction often portrays social interfaces. Michael Crighton's novel Disclosure, introduces the social interface taking the form of an angel. Neal Stephenson's Snow Crash features a helpful and knowledgeable librarian as another typical example of future intelligent interface. Multimodal interfaces are even more exciting... What do you think about the "look-and-feel" of tomorrow's UI?
Discuss this and other AI-related issues at Artificial Intelligence Bulletin Board.
Want more timely information and resources related to AI?
Site(s) of interest:
Arthur Clarke's novel "2001: A Space Odyssey" introduced probably the most famous conversational machine, HAL 9000. Thirty years have passed and the conversational computer is still very much the unreached goal of AI community...
Of course, I didn't forget MIT Medial Lab and their Smart Rooms. MIT researchers have already built smart rooms that can recognize who is in the room and can interpret their hand gestures, and smart car interiors that know when drivers are trying to turn, stop, pass, etc., without being told. This project share many similarities with Smart Desks and Smart Clothes projects.
The Cyberdock version of Nicholas Negroponte's bestselling "Being Digital" goes beyond displaying and distributing the book's contents in another medium. A "Cyberdock" is what it sounds like: a "cyber dock" of sorts, featuring the book's text files as a stable casting off point, from which the reader is invited to explore the Internet Sea on one of the many "URL-Boats" tied to the Dock. More interesting material can be found in one of his interviews.
Source of interesting information on intelligent training, knowledge acqusition, cybernetics, conversationa theory...
Research and development company developing and marketing technologies in pattern recognition, artificial intelligence, evolutionary algorithms, signal processing, simulation of natural processes, and related areas.
Introductory paper by Annika Woern.
Another excellent article by Henry Lieberman.
As we move into the next millennium the Information Revolution is gaining momentum. At the core of this revolution is how people create, interact, disseminate, and share information. Carnegie Mellon University's Human-Computer Interaction Institute (HCII) is helping to shape that revolution.
User Modeling and User-Adapted Interaction provides an interdisciplinary forum for the dissemination of new research results on interactive computer systems that can be adapted or adapt themselves to their current users, and on the usage of user models for adaptation purposes.
Microsoft Research's Decision Theory and Adaptive Systems Group - the research on studying Bayesian Networks, a system that creates an interconnected network of the probability of concurrence among various actions and conditions.
Microsoft site focusing on speech recognition and text-to-speech technologies.
Interactive Computing Environments Lab at University of Illinois at Chicago (UIC).
Man and machine interaction group at Universidade Fernando Pessoa, Portugal.
Humans use their hands for many everyday tasks. The study and analysis of goal-directed human hand movement is an interesting area of study in its own right, but it is becoming an increasingly important topic in the field of human-computer interaction (HCI) because of the growing popularity of 2-D graphical user interfaces (GUIs) and more recently 3-D virtual reality systems (VR), where gestural input and full hand pointing are being explored as the input technique of choice for future computer systems and for telerobotic systems used in some advanced manufacturing applications.
"A Brief History of Human Computer Interaction Technology", by Brad A. Myers.
Human-Computer Interaction Resources.
"Steps to take before IUI becomes real", paper by Kristina H��k.
Programs that can act independently will ease the burdens that computers put on people by Pattie Maes.
Paper by Alan Wexelblat and Pattie Maes MIT Media Lab, Software Agents Group
Simulators and environments from Interface Technologies Corporation.
The network software architecture of the Distributed Interactive Virtual Environment, designed to scale to a large number of simultaneous participants, while ensuring maximum interaction at each site.
The Center for Human-Computer Communication is a recently formed research center at OGI within the Department of Computer Science and Engineering.
Rejecting the idea that people must adapt to machines, IBM scientists are enabling computers to listen to our speech, sense our gaze and read our body language. Excelent article from IBM Research Magazine.
The new millennium is upon us, and with it comes an exciting computing future that we can only begin to imagine today. In this ten-part special report, PC Magazine�s experts identify and explain the trends that will most significantly affect the course of 21st-century computing.
Beginning in 1989, members of the MIT Spatial Imaging Group created a new visual medium -- electro-holography -- capable of producing realistic 3-D holographic images in real time. Subsequent research (see the "holovideo timeline") led to computation at interactive rates, full-color images, synthetic images and real-world input, and most recently a scale-up to a 36-MB display system capable of producing images as large as about 100 mm in width, height, and depth.