Credder Wants to Create an Equivalent to “Rotten Tomatoes” for News
The startup could build the largest repository of annotated and scored news articles. Some precautions apply.
No one has ever attempted to create a review system for news comparable to what Rotten Tomatoes is to the movie business. Reasons are multiple: assembling a team of reviewers for news is tricky; the appreciation of news material itself is highly subjective; bias is an issue as readers who chose to evaluate a story usually harbors strong opinion about it; the whole process is vulnerable to manipulation and trolling — especially in an increasingly polarized society…
Credder, a startup based North of San Francisco, sits at the opposite of our Deepnews.ai project. While we try to rely exclusively on algorithms to evaluate stories, Credder is all about human scoring.
The platform is currently in private beta. You can register to get an invitation. The public beta will be launched in May.
As explained by its co-founders, Jared Fesler, Chase Palmieri and Austin Walter, the platform works as follow.
An author writes a piece that she’s willing to get reviewed. In the story, there is a link that says, “rate this piece on Credder.” In an ideal world, it could be a badge attached to a Medium or a Wordpress post, or accessible via a browser’s extension.
A click on the review button sends the reader to the Article Rating Page, where the piece has been partially cross-published. The page looks like this:
The Credder team drew its inspiration from Rotten Tomatoes and its “Tomatometer” (the movie review site derives its name from the practice of audiences throwing rotten tomatoes when disapproving of a bad stage performance). Like in RT, reviewers will be split between professionals and the general public.
The movie/TV rating site draws its credibility from the strict requirements imposed to its 4400 reviews:
In order to be vetted as an individual critic, you should meet the following guidelines:
Written
• Consistent output for a minimum of two years.
• Demonstrated film/TV coverage at a publication outside of a self-published website. If you are a self-publishing critic, your reviews should reflect our key values.
Broadcast
• A regular appearance on a major local/national/international TV or radio outlet (either via syndication or network broadcast).
Video
• Consistent output for a minimum of two years.
• A minimum of 30K subscribers on a video publishing platform. Critics reaching underrepresented groups will also be considered on a case-by-case basis.
Podcast
• Podcast critics, please apply as a publication.A publication willing to become an approved reviewer will be subject to the same process. Requirements include at least two years of existence and verified audiences of at least 500K unique monthly visitors for a written publication or at least 200k subscribers for a video site and a significant presence on social media. Podcast publishers should have a minimum 200 ratings on iTunes with a minimum score of 4 stars.
This set of exigences gave Rotten Tomatoes its authority — and its valuation. In 2011, RT and its parent company Flixter were sold to Warner Bros, then resold in 2016 to the ticketing company Fandango, which was in turn acquired by Comcast.
Credder will apply roughly the same rules for what it calls “Professional Journalists” that will constitute the core of its evaluation system.
Building a scoring system is not as simple as it sounds. Among the factors to be taken into consideration:
- The critical mass of reviewers. A newly released blockbuster like First Man will draw 399 reviews from professional and 7500 general public evaluations. (Also, RT has enrolled a small subset of well-known “Top Critics”).
- The weight of the reviews: RT decided that at least 40 to 80 reviewers (depending on how wide is the movie release), expressing at least 75 percent of positive appreciations were required to achieve the best score labeled “Certified Fresh”. The “Fresh” (good) status requires 60 percent positive reviews or more. Below that threshold, the movie or the TV show is considered as “Rotten”.
- The diversity of the reviewers is also a big issue. Theoretically, it should represent a cross-section of the population. Again, Rotten Tomatoes learned the hard way how delicate it was. Last year, research from the Annenberg School of Communication found that 78 percent of the reviews were written by male critics and 82 percent of the evaluations for the 100 highest-grossing movies of 2017 were produced by white people. RT responded quickly by approving 200 additional carefully selected reviewers.
Applying these principles to the news ecosystem isn’t easy.
First, the media industry does not rely on reviews as much as the movie or TV industry does. Regrettably, journalists, editors, or publishers have never been interested in getting a quantified perception of their work, beyond the number of people buying a newspaper or visiting a website. (The industry is even slow to accept daily measurement, preferring the cozy approximation of “monthly uniques”).
Two, the production pipelines have nothing in common. In the United States alone, about 600 movies and 500 scripted TV shows were released last year. By comparison, the news media output is way bigger: at Deepnews.ai for instance, we collected 1100 articles related to the Boeing 737 accident, in just a week, over only 40 publications. This abundant and scattered market structure is a huge problem to overcome.
The good news is the abundance of potential reviewers. On the professional side, Credder will tap in a population of about 86,000 professional journalists employed in US newsrooms of which 38,000 people working in newspapers.
For the system to work, Credder evaluates at 10,000 the critical number of reviews needed, says Chase Palmieri, co-founder (emphasis mine):
Most of the world’s news consumption comes from the top 70 outlets. Each of the top 70 outlets produces on average 20 articles per day. 70 outlets x 20 articles a day = 1,400 articles per day that need to be reviewed. At a minimum, it takes 7 reviews on an article to generate an accurate rating. 7 reviews per article x 1,400 articles per day = 9,800 reviews per day. We’ll round this number up to a clean 10,000 reviews per day.
That means we need 10,000 people reviewing one article per day to cover the vast majority of the world’s daily news. Credder will also have “Super Users” that review more than 1 article per day, making this goal even more manageable.
Seven reviews to ensure a reasonable accuracy might sound small, but as I explained in a previous Monday Note, at Deepnews.ai we deem as reliable an evaluation made by only three qualified readers (in our case, paid journalism students).
One of the most promising features of Credder is the granularity of its review. It took eight months of tests to come up with a two-click process that could yield precise data on the appreciation of an article, as shown here:
Except for the “Trust” grade, each of the five “meta” appreciations is broken down into multiple sub-labels. When using it, the system works just fine and covers nearly all the situations a reader might encounter.
The key question to how to prime the pump: growing an initial base of casual users, recruiting certified journalists and incentivizing them to review articles. Credder will rely on traffic sent by publishers page and platforms who should be happy to insert a badge in exchange on data relevant to their production. “Pros” reviewers will benefit from a boost of their social footprint for each contribution.
The startup also intends to develop a tipping model comparable to Patreon which generates almost $13 million a month in payouts, according to Graphtreon. On the $150 million payouts per year, the platform keeps 5 percent which yields $7~8 million a year in net revenue. That is considered to be an attractive model for Credder.
Aside from this, the startup wants to develop a sponsored-articles system in which an author writing on a specific field will be able to push a piece to a precise segment of readers. The platform will have to build an auction system that writers will use to reach the desired audience.
Credder has still a long way to go to develop a frictionless system that will favor a large and regular stream of reviews. After all, Rotten Tomatoes was created in 1998 by three UC Berkely undergrads, who eventually raised a million dollars. Pretty cheap for such an influential media.
frederic.filloux@mondaynote.com
