Enabling news companies as content curators

There’s a feast and a famine in news today. With the rise of the Real-Time Web, as RWW’s Richard MacManus highlighted, we’re getting too much news too fast, and struggle to filter quality information from noise. At the same time, we struggle to find high-quality, relevant content along our individual long tail interests.

Louis Gray points out that “With tools like Twitter and other networks making it ever easier to hit the publish button, our ability to screen, filter and decide what information is good for us is going to be increasingly tested.” So curator-bloggers like Robert Scoble are trying to “find the best interpretations of the news from [their] favorite sources.” Rishad Tobaccowala notes:”It is not content that is rare. It is not compelling content that is rare. It is time that is rare” — and then asks, “Who can curate, combine and help us discover this content so we can make the most of our time? Who can get us things at the right time not just in real time?”

Enter curation

Curation is one way to deal with this problem. In many ways, it already happens everywhere… Individuals curate by sharing links in Twitter and Facebook.  Organizations curate for their members through newsletters. Crowds curate if you aggregate individual tagging and attention data.

But sharing is not necessarily curating.

Curation at its best produces a structured collection that gives readers more value than the sum of its parts. It requires domain knowledge and strategic thinking to organize topics with a purpose and point of view for the curated collection.

The future of news

News companies already do this for readers with their own content. Newspapers and magazines rely on editors to classify, link, and lay out stories. These editor-curators provide consistency in tone, perspective, and quality.

What’s missing: an effective, scalable way to do this across the Web. As the Recovering Journalist asserts, media publishers are “trapped in their walled gardens, putting together their daily reports only from the sources they pay for: their own reporters, maybe some wire and syndicated copy and photos, and that’s about it.”

This is a huge opportunity. Applying their curation strengths to the Web, editors can package syndicated and user-generated content to add value to their original content. Wouldn’t it be great to have alongside every news story an index with links to analytical articles from different perspectives, the backstory, reference articles, reviews, and opinions?

But it’s too expensive for news companies to do this manually at scale; they need an automatic approach with the precision of human editors.

Enter social indexing

PARC has been working on a research project for the past few years with this concept in mind. You can read about the backstory of this work here. We just launched an open beta of the demonstration site for personal curation and news aggregation powered by the Kiffets Social Indexing Engine. You can try out the beta here or watch this video overview.

A Social Index is a topically organized information collection created by human curators. The curators specify good topics, sources, and examples to create a point of view for the collection (see my Netbooks index or Mark Stefik’s Future of Journalism index as examples.) Kiffets learns the curator’s intent from the examples, creating models for classification that are more sophisticated than keyword- or entity-based patterns. Kiffets then uses the models to classify new content from the sources as well as recommend new sources for the curator.

Kiffets’ understanding of topics evolves over time as curators give feedback. Furthermore, the system automatically finds relationships between topics, articles, and indexes. By making these connections visible, Kiffets encourages readers to discover and explore related content when they are reading articles. This increased engagement creates value both for the reader and the publisher.

Kiffets is not unique in combining human input with algorithms. What is unique, however, is Kiffets’ ability to infer the perspective of human editors so the system can intelligently classify new content according to that point of view. Curators retain control to remove articles that do not fit, but Kiffets does all the heavy lifting.

Technology powers the future of news

The future of news is still being defined. But technology can help news companies adapt to changing times.

Technology already affects how news is collected, how it is curated, and how it is delivered. While the future of news is certainly about its commercial evolution, it is also about the powerful and important role that news organizations play in how society understands itself and the world.

Our Social Indexing Engine is a first step and we invite you to join us in the journey. Contact me, and read more about this technology, our views on curation, and the future of news at the Kiffets Voice blog.