At the heart of curation: Kiffets backstory

In their competition for readership and advertising revenue, online news publishers need to differentiate themselves through curation. Read more about the why and the how here — including our step towards a solution for news companies, the Kiffets Social Indexing Engine, which is based on a research project at PARC that I’ve been leading. Be sure to watch this video overview.

Behind the research scenes

This project (as with many things at PARC!) has roots in a trajectory of evolving expertise — spanning early collaborative filtering and later information visualization and sensemaking systems, to social computing today.

After 9/11, PARC developed and deployed information systems for U.S. government intelligence analysts — whom we dubbed “the jet pilots of sensemaking” because their information-seeking and collaboration challenges were so intense. One output of this research is my colleague Peter Pirolli’s book on Information Foraging theory, which has been referred to as “the ultimate source”. In our research, we developed a set of “cognitive amplifiers” for analysts based on artificial intelligence (AI) and collaboration technology.

Intelligence analysts’ situations then are not that different from people’s information needs now: too much too fast or too little too late. So insights from our analyst research have guided development of the Kiffets system, which personalizes news along people’s content needs and passions. As in our systems for intelligence analysts,  the AI and collaboration technology serves as a cognitive amplifier that enables scaling the sheer amount of  information that needs to be collected, filtered, and organized.

Within the intelligence community, some analysts act as curators or guides in sharing their expertise. These individuals help others to catch up and keep up in their areas of expertise. Now with open information on the Web, curation is emerging as a growth opportunity for information consumers and publishers.

The nuances of curation

In a recent graphical analysis of online media consumption, the conclusion was that winning media sites “all show a mix of large collections of content, mixing high quality created content with contributed, and curated content”.

The success of some curating news sites has generated buzz — although different people mean different things by “curation”. For example, the Nieman Lab pointed out important differences between AGGREGATORS (collectors), INDEXERS (auto-clusterers), and CURATORS (manual selection, organization, and commentary).

But, but… these distinctions, though useful, represent trade-offs: at one extreme, minimal human labor; at the other, time-consuming, unscalable human effort. Guess which approach is most expensive for publishers. Now guess which approach is most convenient and cost-effective. They won’t be the same.

So why not design a system that combines these approaches — recognizing the mundane, repetitive steps that can be amplified, as well as the opportunities that can benefit from large scale? Additionally, news companies already know they can compete against the pack by: (1) deepening coverage (investigative reporting or well-written opinion pieces) and (2) extending coverage into long tail areas. What they can’t do is scale these approaches because there are too many niche areas with too small audiences. Kiffets addresses all of the above by drawing on three sources of power for filtering and organizing information: the hard work of the few (curators), the light work of the many (crowd wisdom), and the tireless work of the machines (AI).

Open beta

To deliver the best of this information organized for its users, the Kiffets Social Indexing Engine combines the expertise of human curators with artificial intelligence technology and social input. Kiffets users subscribe to specialized channels on subjects they care about, curated by people that they trust. The curators (users, media companies’ in-house editors, others) tell Kiffets how to select and organize information on a subject. Then Kiffets collects, classifies, and delivers the information automatically to its users according to their interests.

Kiffets is now in open beta release on the Web. Try it out. Also check out and subscribe to the Kiffets Voice blog, which covers some of our thinking on the future of journalism, personalized news, and curation.