Information Visualization & Interaction

PARC's pioneering UI accomplishments include the GUI, first commercial mouse, and 3-D visualization/navigation technologies

Today's affordable, high-performance computer graphics have yielded larger displays that can help users process their ever-expanding worlds of digital information.

But simply enlarging a graphic display or workspace won't help users navigate large, complex information environments and diverse content sources. How information is organized, presented, integrated, and controlled directly affects how easily and thoroughly users will analyze and understand it — especially in mission-critical and time-pressured situations.

PARC's approach to information visualization and interaction addresses these needs and helps users:
-	find information faster;
-	"connect the dots" sooner;
-	identify more relevant and context-based information;
-	discover deeper and previously hidden information;
-	gather more thorough evidence;
-	counter cognitive biases;
-	and focus attention in a more targeted manner.

PARC's Approach

• Enables manageable visualizations of massive and diverse content collections through fluid, interactive, and user-controlled rich-media displays — driven by underlying theories of cognition, perception, and visual attention
• Emphasizes HII (human-information interaction) not just HCI (human-computer interaction) — focuses on how users interact with information not only devices
• Harnesses a variety of information-visualization technologies including "focus + context" techniques for maximizing display resources — enables user to process content of interest while simultaneously preserving an overview and context of the content
• Allows platform-sensitive delivery of content — such as screen size where content will be displayed (e.g., is it a mobile phone or large wall?) and provides seam-aware visualization across multiple displays
• Balances important considerations such as visual realism, readability, interactivity, and scalability
• Supports collaborative ACH (analysis of competing hypotheses) — aids careful weighing of alternatives and minimizes common decision-making pitfalls/cognitive biases
• Is informed by user modeling, cognitive task analyses, and instrumentation such as eye trackers — considers both design (how to structure information) and evaluation (how users engage with that information)
• Builds on PARC scientists' deep and broad expertise in UI (user interfaces), HCI, HII, cognitive science, psychology, and artificial intelligence — PARC's pioneering history in UI research includes the GUI (graphical user interface), augmented cognition (through the first commercial mouse), and 3-D visualization and navigation

Applications and Features

The suite of technologies listed below can be integrated into a single workspace and customized for specific needs.

• Book-like display for rapid navigation, skimming, and reading of large documents — as if the user is physically flipping through book pages on-screen
• Text highlighting based on "information scent" — helps users easily find conceptually relevant passages
• Semantic index — reorganizes subject indexes and customizes them according to user interests
• Three-dimensional, "perspective wall" display of all entities extracted from a content source — provides a big-picture overview of the content and helps uncover relationships among highlighted entities across the entire content source
• Focus + context "popout prism" displays — provides a fluid, 3-D overview of a content section where the user is momentarily focusing attention (such as a book chapter and passages adjacent to a paragraph of interest)
• Tree-like display of all extracted entities and degrees of relationship among them — allows user to track relationships and degrees of separation between entities or across entity clusters
• Chronological and degree-of-interest trees that allocate space to the most important information — helps user follow entity changes over a timeline and fluidly hone in on events by degree of interest
• Proximal and collaborative proximal search — finds topically related documents of interest close to where other users, experts, or another community of users are looking
• ACH tools — helps users explicitly capture hypotheses, overcome cognitive biases, and actively search for disconfirming or inconsistent evidence
• Intelligent evidence file and recommender system — gathers and groups user-identified facts of interest, and makes recommendations about what to explore next based on cognitive theories of spreading activation

To learn how content can be pre-processed (e.g., through text mining, entity extraction) before it is visualized, please see Natural Language Processing for Sensemaking. Non-digital content is pre-processed through standard OCR (optical character recognition) and scanning techniques.

Demonstrations and Proof-of-Concept

The following examples were drawn from a hypothetical government intelligence analysis task.

Focus + Context Displays:

Compact representation of entire content source (this example was originally a hard book copy) and at-a-glance overview of color-coded entities of interest [high-resolution image]	3-D "perspective wall" overview display of color-coded entities and their across an entire content source [high-resolution image]	Fluid, context-anchored, 3-D overview of passage where user is momentarily focusing attention [high-resolution image]

Degree-of-Interest and Relationship Trees:

Tree browser to help user follow entity changes over a timeline and manipulate a slider to hone in on events by degree of interest [high-resolution image]	Concentric circle-type display of entity relationships, which can be simplified by choosing fewer degrees-of-relationship among them [high-resolution image]

Other Tools:

3-D physical book-like display for rapid navigation, skimming, and reading of large documents [high-resolution image]	Index organized by user interest with conceptually relevant information highlighted [high-resolution image]	Display of search results from dual search engines, which displays topically close rankings alongside standard rankings [high-resolution image]

Commercial Output

• PARC spun off Inxight Software Inc. to commercialize over 70 PARC patents. The company provides scalable, multilingual information-discovery solutions to customers such as Boeing, Charles Schwab, Deutsche Telekom, Eli Lilly, Factiva, GlaxoSmithKline, Korean Telecom, LexisNexis, MCI, Merrill Lynch, Microsoft, Oracle, Pricewaterhouse Coopers, Reuters, SAP, Yahoo, and various U.S. government agencies. 

• Download the ACH tool

This work has been partially funded by the NIMD (Novel Intelligence from Massive Data) and AQUAINT (Advanced QUestion and Answering for INTelligence) programs in the U.S. Intelligence Community's/ DARPA's Advanced Research and Development Activity center.