Re-engineering News with Technology

Years ago, in college, I went to a presentation by a big internet company, as part of a recruitment event. At the time, I was working at the college newspaper, and the talk was about their “front page”. They said it was the biggest news site at the time, so I was excited.

The bulk of the talk was technical. But the presenter mentioned that one of the biggest challenges was keeping abreast of what they called the “National Enquirer effect”. The problem, as she described, was this. The main goal of the front page is to drive traffic to other properties; and the system was always optimizing both the selection of content on the front page and its ordering based on raw clicks. He said, while no one admits to it, content with the best-clickthrough rate was always “bikini women”, so left alone, algorithms would turn the front page into National Enquirer. Ironically, this means that no one would visit them, over a long enough period. They said they were trying to fix this by some longer term optimizations, but for now, there was essentially a team for each locale that monitored the site, and kept it “clean”.

Couple days ago, I saw a tweet about a NYTimes wedding post. It said “Trevor George asked Morgan Sarner out to dinner 10 nights in a row, and won her heart”. The person, whose retweet I saw, said she’d probably get a restraining order. It was funny, I “liked it” but it seemed odd that NYTimes would promote such creepy behavior. So I clicked on the link. It turns out the groom did not just ask her out 10 days in a row, but took her out as such. It’s a minor difference, between the text in the tweet but the actual content, but it was enough to get that person to retweet a mock of it, and more ironically to get me to click on it.

Ev Williams, the co-founder of Twitter and Medium, likened the algorithms that govern the internet as to a Deus ex machina that provides you with the most extreme of what you want; you think car crashes are interesting? Here’s a pile up! It feels true, and definitely explains the long-winded global nausea.

Looking at it another, though, this is just specific application of the paperclip maximizer. Instead of natural resources of the earth, we are just mining minds. And instead of making more paperclips, we are just making some people in Bay Area richer. I live in the Bay Area, for now, so of course I shouldn’t complain.

But what’s really missing from the debate is how technology has really failed to find a way to attract attention of its readers without sacrificing the content. And while some of it is done automatically, some of it is self inflicted.

There are structural and economic explanations for the problem. Internet first destroyed the newspapers’ monopoly on advertisement. Then the glut of content came, with democratization of publishing tools, further pushing down the value of any individual work. Unbundling of pieces from the newspapers and magazines that carried them reduced the value of a brand, and in turn pieces that make up a bundle too. As social media platforms further flattened all content into same structure, be it from The New York Times or some kids from Macedonia, any semblance of product differentiation.

The number of knobs publishers have is dwindling and their editorial decisions is one of their last levers. When you are competing with so much content, and you don’t control how your content is distributed, your only option is to change your content to fit your distribution channels. Legitimate news organizations have long erected a well between “the church and the state”, or rather “editorial and the advertising” but at least in terms of packaging, the wall no longer exists. The only difference is that it’s not he advertisers that determine your content now, but your distributors.

I saw this first hand too. At Digg, we would casually tell big publishers and famous individual alike that if they worded headlines a specific way, they would get more clicks. Sometimes it worked, sometimes it didn’t. Google has entire guides, mostly technical but with editorial hints, on how to help you get more traffic.  Facebook does it too, but slightly for different reasons. They want people to click on the content, but not too much, so publishers better avoid clickbait titles. And of course, most publishers, especially smaller ones that do not have big subscription revenues or rich patrons to back them, get in line.

This is not a jab at newspapers, although it is that a bit. My real qualm is that we still don’t have a proper way to consume the news where the hook doesn’t dictate the content. We built search engines that can scour the entire web in less than a second, but I still can’t figure out whether a piece of content is worth my time, or is just fluff. I can take a virtual tour across the globe, but I cannot tell what a federal policy change means for me as a resident in California. The primary problem is funding and revenue, but is there a lack of imagination as well?

I also don’t know if the solutions to these problems will exist on the supply side, or the demand side. Probably, it will need to be both. Publishers need ways to authenticate and brand their content, and consumers need reading experiences that respect those. Moreover, consumers need a better way to find and consume content that respects the integrity of it, and not let it be violated for distribution.

There are a lot of attempts to build a new stack for consuming news. Services like Blendle attempt to fix monetization by removing the hurdle of micropayment, and also consolidate subscriptions. Facebook and Google try various things too; AMP is a way to clear up the reading experience (and cynically move more of the content to Google servers), Facebook’s Instant Articles is a more locked-in and heavy-handed way of doing the same. Both Facebook and Google also want to help publishers gain more subscribers, and the subscribed users to have more fluid, integrated experiences on the web with their own platforms.

And of course, publishers, try their hands too. One of my personal favorites is what Axios does, with their telegraphic, lightly structured way of presenting their content. It feels respectful of my time as a reader, cuts through the fluff without sounding too clinical. I wish more publishers experimented with radically different, but still thoughtful ways of producing and presenting content like them.

At that talk I went, they said one of the ideas was to have a fluff lever; slide it to one side and you get practically smut. Slide it all the way to the other, it’s all dreary politics, which wasn’t smut at the time. As far as I know, they never launched it.

Internet has undermined, intentionally or not, the workings of all news organizations. It took over their advertising, their users’ attention, and now a few companies inadvertently are guiding more of the content too. The different responses to this change lie across the political spectrum. What is common, though, is that the problems will not go away, and the economics that govern newspapers will not go back to where they were. But maybe, there are ways to attack this problem with technology, as well as with policy.

I am not sure if that is the answer, but maybe it could be worth trying.

Digg was all about news and nothing else. It didn’t work out.

Couple days ago, I was having lunch with a friend who used to work at Twitter. Eventually, the issue of Fake News came up. I told him, as more of a joke, that Facebook could just solve the Fake News problem by taking the News out of News Feed, and turning it to essentially just a bunch of social update. He retorted, saying that product already existed and it was called Instagram. We both sighed and shrugged and downed a few more drinks.

Now, apparently Facebook is trying that exactly, and of course publishers are freaking out. You can’t really blame them. For many publishers, Facebook is their biggest source of traffic, which they monetize via ads. But you can also not just feel bad for them, because, that is the risk of building your business on someone else’s platform. Just ask Zynga.

My understanding is that Facebook started promoting news sources and publishers more or less as a defense mechanism against Twitter. It might be ancient history now, but there was a time where the fates of these companies weren’t as far apart as they are now. Facebook noticed that Twitter was getting an undue amount of attention from the media folks, with newscasters and individual journalists signing up on in droves and moving the conversation there. Facebook wasn’t a fan, decided to flex its muscles a bit.

I don’t know if that’s true, but it rings true. And I know this, because I used to work at a company that was in the same boat, at the same time. When we launched Digg V4, one if its goals was to cut down the noise of Twitter and just focus on links instead of the mundane status updates. It didn’t work out, and Digg imploded rather spectacularly but the idea was solid. Digg was always at the forefront of many ideas that are common now, such as “liking” things both in and out of Digg’s website and apps. But with Digg V4, it all came crushing down.

To understand all this, you need to go back to 2010, if not earlier. Twitter and Digg were both merely curiosities, largely unknown outside of Silicon Valley. Twitter However, Digg controlled a significant amount of traffic, and getting on its front page could be a huge boost to not just publishers but really any company. Even Dropbox, now a pretty much a household name, attributed a significant amount of their early users to getting on Digg’s front page.

However, Twitter was already gaining momentum. Although the site could barely stand without failwhaling, it was already signing up big time users like Ashton Kutcher, Justin Bieber. But they didn’t really drive traffic to anyone; and most people were using it as a more public stream of consciousness than anything.

So that was one of Digg’s plays with V4; that we’d be the driver of traffic to publishers because we didn’t have any of those pesky “I am eating a cheese sandwich” updates that littered your Twitter timeline that you didn’t know what to do with.

Kevin Rose tweet about Digg V4
Kevin Rose tweet about Digg V4

The why and the how of Digg’s failure is complicated. But largely, it was a perfect storm of technical issues (mostly of our own doing), management mishaps, and of course the Cold War Digg used to wage on its users finally erupting into thermonuclear skirmishes. Digg always had a delicate relationship with its most influential user base; either side never really blinked but with Digg V4,  it all changed.

One of the most controversial changes was making My News, the logged in personalized page the default option, as opposed to the “Top News”, which was The Digg Homepage. With this change, the importance of Top News was significantly reduced since we essentially distributed the logged in page views across thousands of personalized homepages. This was both a way to keep more people logged in, by providing them a better and more engaging homepage, but also to make sure that we had more unique pages where most publishers could get clicks from.

The real controversial change was actually allowing publishers to automatically submit items to Digg by sucking in their RSS feeds. What this meant is that now you could participate in Digg without really participating; we could just suggest your account to new users who would see your content, which Digg would automatically ingest, without you doing ever anything. And the fact that we accidentally, I swear, promoted those items over manually submitted items did not help.

While Digg made a conscious decision to prioritize big publishers, we managed to scare away most of the user base. Without users, of course, the traffic publishers received started dwindling down. But more critically, without eyeballs on Digg itself, the advertisers slowly fled. The rest is history.

Facebook, for what’s its worth, never had a problem with users leaving its service and I doubt they ever will. I don’t know if they would, but they could remove all the links to publishers from News Feed and most users wouldn’t give a damn. The genius of News Feed was never the links, but it was the ability to give living and breathing person on earth their own personalized rumor mill. The outrage articles, especially in the age of Trump is addictive, for sure, but it doesn’t hold a candle to the addictiveness of being able to see a new update from one of your friends.

I am a strong believer in the importance of journalism for a liberal democracy. I would dare not wish for publishers, and journalists to lose their sources of revenue. But at the same time, I can’t imagine that at least the big publishers did not see this coming. No one in their right mind would put all their eggs in someone’s basket. And hey, maybe this is a good thing. Maybe this is the wake up call we all needed.

On quiet

Istanbul is not a quiet place. The streets are filled to the brim with cars, honking. The kid is screaming to his mom, the girlfriend to her boyfriend, the police to the street vendor. It’s not pleasant, but it is Turkey.

However, the real noise is not the people, or the cars, or the ferries. It is the news. Everyone in Turkey is always watching the news. It’s on the background when you are at home, with your parents. It’s blaring at you when you are at the corner store from the TV hung to the corner. It’s shouting at you when you are at bank, from the small radio sitting next to the framed photo of the teller’s daughter. It’s even on at the waiting room at the doctor’s office, because that’s when you really need a pick me up.

And when you are, by some miraculous happenstance out of the earshot of a TV, there’s Twitter. Everyone is always on their phones, and if they are not checking Instagram, they are checking the news on Twitter. It never ends. It wasn’t always that way, I want to say, but for the love of me, I can’t remember when it wasn’t.

It used to be fashionable to call Turkey the “Little America”, largely due to an overzealous adoption of neoliberalism and all the joys and pains that come with it. It used to be a thing, a family tradition, to enjoy the even the most inane of American traditions. Having visited America was a sign of not just wealth, but also a checkmark on the pursuit of a more enlightened world.

Now, slowly it looks America is on its way to become a “Little Turkey” itself, primarily starting from people’s addiction to the news and a constant state of screaming.

Many a words have been said about the 24/7 cable news networks in the US. How the inane, and insane, need to fill up over the hours drives networks to just have talking faces on TV. The current boogeyman for the orange man in the White House is partly responsible, people argue, for him being there. When I was a kid, CNN for me was the night-vision imagery from the first, of seemingly endlessly many, Iraq war. Now it’s a bunch of talking heads, that are always there.

And then, there’s Twitter. And push notifications. Always the push notifications. It used to be different though. When I first moved to US, in 2006, we also had a scandalous president. He didn’t seem to be that coherent, and his policies didn’t earn him many favors in or outside the US. There was some political turmoil, maybe even a war, but it happened on a different timescale. There were other things going on.

One of the first things that America lost when Trump got elected is the quiet, the personal space millions had to themselves. You had a time to yourself to be in love, to be with your friends. There were conversations that never touched on politics. Some things were downstream politics, but most things were not. There was a time, when you could just be angry at your things in your world at your own time. Now, you are required to be angry all the time because of something you didn’t do, don’t have control over and seemingly with no end in sight.

Has it been 6 months since Trump took the office, or 6 years? Is anyone even counting anymore? How would it feel different if this wasn’t just 1/8 (hopefully) into the dumpster fire that’s this administration but we were just halfway there. I am aware that I am speaking from a privileged position here, as a white man with a stable job in a well-paying industry, as opposed to being a minority. Maybe things were always this loud, if you always had to worry about your job, or your livelihood.

But in the objective space I can carve out, I feel that things got worse. And we need to do something about it.

I am not suggesting that people ignore the news or disengage from the public discourse. Or disconnect entirely or at all. I don’t think a democracy works with a fully disengaged public. And it certainly does not, with a public that only is informed about topics that interest them. We all have a responsibility to be informed, including on things that don’t matter to us but to those around us. But it also matters what we each decide to think about, what we need to care about. We built ourselves empires on capturing attention, and we are slowly realizing that our minds cannot keep up with its demands. But, I think we have yet to realize that our minds aren’t also capable of being outraged, all the time. We can’t always be mad, lest we lose our connection with the reality. Everything is political but politics isn’t everything.

One goal of politics is to arrange relationships between big groups of people. Not necessarily divide or unite them, but to establish some sort of structure. A network of roads, where connections happen. It doesn’t care if you run tanks on them, or ice-cream trucks. You can drive away, or run towards someone. But the world is not about those roads. It’s not not about them either, of course -just ask any commuter- but it’s just a part of it.

Somewhere along the way, we need to park our cars, get off our bikes and look around the world as is. The quiet is easily disturbed, but in the end, it’s what makes each of us human, unique and it’s what keeps the society humming along. We can’t always scream, we need to be quiet so that everyone else can have it too.