Navigating the network of knowledge: Mining quotations from massive-scale digital libraries of books

Scanning books, magazines, and newspapers is widespread because people believe a great deal of the world’s information still resides off-line. In general, after works are scanned they are indexed for search and processed to add links. In this talk I will describe a new approach to automatically add links by mining repeated passages. This technique connects elements that are semantically rich, so strong relations are made. Moreover, link targets point within rather than to the entire work, facilitating navigation. Our system has been run on a digital library of over 1 million books (Google Book Search), has been used by thousands of people, and has generated the world’s largest collection of quotations. I will also present a follow-on project based on the theory that authors copy passages from book to book because these quotations capture an idea particularly well: Jefferson on liberty; Stanton on women’s rights; and Gibson on cyberpunk. These projects suggest that mining quotations for links and ideas are an important mechanism for understanding the knowledge contained in books.

This work is in collaboration with Okan Kolak, Google Research.

Presenter(s)

Bill Schilit is a researcher at Google. Before joining Google, Schilit was principal scientist with Intel's Digital Home Product Group, co-director of Intel Research Seattle, managed personal computing research at Fuji-Xerox (FXPAL), worked on networked systems at AT&T's Bell Labs, and was part of the team that invented ubiquitous computing at PARC from 1992-1995. His interest is ubiquitous information with a focus on the development of personal and mobile technologies supporting knowledge work. Schilit received a PhD in computer science from Columbia University. He is an associate editor in chief of Computer, a member of the IEEE Computer Society and the ACM. Contact him at schilit@computer.org.