Unsettling Futures - Brandon Sanderson, fame, and the failure of the Long Tail

One of the worst science fiction premises of the last 50 years is that of Danny Boyle’s 2019 film Yesterday. It imagines a man who wakes up to find himself in a world in which the Beatles never existed, and he’s the only one who remembers their songs – so he becomes famous singing them as his own, in the 2010s.

This is obvious horseshit.

Why were the Beatles the biggest band in the world? Is it because they were the greatest band of the 1960s/the 20th century/ever? Or was it some combination of skill and talent, luck, cultural zeitgeist, and a reinforcing cycle of popularity feeding popularity?

This is a question closely related to the one everyone’s been asking for more than a month now in SFF publishing: How did Brandon Sanderson raise a whopping $41.7 million in a Kickstarter, when the tenth or twentieth most popular epic fantasy writer would struggle to raise a slim fraction of that sum?

It’s because popularity was always broken, and the internet, which promised to alter the playing field forever, just made things worse.

What do you want in the marketplace for fantasy and science fiction?

What I want, as a reader, is good discoverability of things I might like. I want interesting reviews, critiques, and discussions of books both new and old. I want (and this shades over into my desires as an aspiring SFF writer) for writers to be paid well enough that they can continue to turn out a book every year or three, without burning out. In other words, I am fine with there being some major superstars, but mostly I want a solid midlist, with a broad and diverse (in every sense of the word) field of writers serving the broad and diverse SFF audience.

What we get is wild popularity for a tiny handful of authors, and… whatever’s left over for the rest.

Let’s consider the Goodreads Choice Awards last year in the fantasy category.

The top spot was won by A Court of Silver Flames, by Sarah J. Maas, with 111,498 votes. I’m sure this is a fine book! But I sort of doubt it’s twice as good as runner-up Under the Whispering Door by T.J. Klune (56,324 votes), or four times better than Shelley Parker-Chan’s fourth place She Who Became the Sun (28,661 votes).

The slide in votes is so extreme that the 20th place and final book in the rankings, C.L. Clark’s The Unbroken, received 1,451 votes, 1.3 percent of the votes for Maas’s book.

It’s even worse over in the Choice Awards science fiction category, where the top vote-getter was Andy Weir’s Project Hail Mary (92,831) and the 20th pick was S.B. Divya’s Machinehood (643).

I am not saying that the top-ranked books don’t deserve their readers. I’m saying that the bottom ranked (and middle ranked, and dozens of books that weren’t ranked at all) deserved more readers, more reviews, more promotion, and more attention.

It wasn’t supposed to be this way. We were promised better, back in the early days of the internet.

Back in 2004, an article in WIRED claimed that the future of popularity, at least when it came to consumer goods and cultural purchases, would be completely upended by the internet. The article, and the 2006 book it spawned, were considered prophetic for years.

Chris Anderson’s The Long Tail argued that books, songs, TV shows, and films had suffered from a distribution problem. The people who wanted to see broadly popular stuff were well served, but the scattered (but in aggregate still significant) numbers of people who wanted to see more niche fare weren’t a viable audience. This was because there were only so many slots on network TV, only so many showtimes at suburban movie theatres, only so much space on the shelves of your local bookstore.

Anderson argued we were on the cusp of a vast increase in the variety of things that would become viable, thanks to digital sales, whether via download or Amazon-style delivery. Sure, your local bookstore might not stock that copy of some weird-ass novel with a narrow appeal, but collectively, across the world, there might be 5,000 people who would buy it. So Amazon can stock copies in its vast warehouses, or it could be available for digital download, right?

When he talked about the ingredients of the long tail, Anderson was actually talking about three closely related concepts – interest, discoverability, and popularity. He did not understand which one would be most important in the digitally dominated world.

We wound up with a world where popularity is king, and discoverability is badly broken.

In 2006 and 2008, sociologist Matthew Sagalnik was the lead author on two research papers about song popularity. He was trying to understand what made people like things, and whether there was an element of peer pressure involved.

His first study seemed to bear that out. Sagalnik let thousands of people download, listen to, and rank 48 songs they’d never heard before. Then a number of other groups were given access to the same songs, but with various levels of data about the first group’s choices.

Sure enough, the more information new listeners had going in…

…

In other words, if you tell people something is popular, it will become even more popular. This is not surprising! People are curious about anything they see lots of other people talking about. They want to get in on the conversation!

In 2008, Sagalnik re-ran the experiment, with a twist.

The first group again could access and download any songs they wanted out of 48. This created a popularity scale – the most popular song was downloaded 128 times, the least popular just nine times.

This time, some groups got the songs with the chart flipped. They were told that the least popular song was the most popular, and vice versa.

The good news, for people who believe quality matters, is that listeners in the “inverted” group did find the lowest ranked song was better than the top-ranked song, over time. But in general, the lower-ranked songs benefited from their artificial position in the flipped charts, and most of the “better” songs did worse in terms of downloads.

One unexpected result was that test subjects in the “inverted world,” perhaps disappointed with the alleged top songs, simply downloaded less music.

In other words, if you can’t find something you’ll enjoy, you won’t stick around. You can’t hum a song you never heard, you can’t see a movie you don’t know exists, and you can’t read a book if you never so much as learn the title.

Songs are more easily studied, which is why I went off topic for a bit there. But the point of this little (ha!) post is to talk about how popularity is concentrating in fewer authors over time.

If you hang out for five minutes on SFF writer Twitter, or any other kind of writer Twitter, you will hear someone bemoaning the “death of the midlist.”

…

In The Long Tail, Anderson argued that the exact opposite would happen. He thought that many people would find different alternatives to the mainstream mass-marketed hits. This should have eroded readership for the top-tier books, while spreading more around the rest of the publishing ecosystem. Did the long tail simply not happen at all?

Not quite.

The long tail can be seen in SFF publishing, in self- and indie-pubbed fiction, in Kickstarter campaigns and Patreons, in the flowering of online short fiction publications over the past two decades. I got really excited earlier this spring to find out that Walter Jon Williams is going to finish his Metropolitan trilogy and, if he can’t sell it to a publisher, just put it out himself.

Yes, the internet has allowed more niche publications.

But it has not made it that much easier or more profitable. Despite Anderson’s grand claims about all the money that was there to be made in the tail, even in self-pub, there tend to be a tiny number of big winners, and a vast mass of people who sell a handful of copies.*

One key issue is that publishers, Amazon, and the modern internet are terrible when it comes to the third of Anderson’s three rules for a Long Tail world – “Help Me Find It.” Anderson described a process of clicking on recommendations after you’d listened to one song, and finding something genuinely new. How’s that going in 2022?

If you go on Amazon right now, and you look up Brandon Sanderson’s The Way of Kings (Book #1 of The Stormlight Archive!) and you scroll down to see “Customers who viewed this item also viewed:” you’ll see a list of… more Brandon Sanderson books! The guy’s the Stephen King of epic fantasy, he’s written a lot of stuff. But the next page of suggestions is… also mostly Brandon Sanderson. And, as I’m writing this, the very last one is The Lies of Locke Lamora, by Scott Lynch.

Okay, let’s click on that!

What did it get in “Customers who viewed…”?

Not surprisingly, more Lynch books, Rothfuss’s Name of the Wind, and, hmmm, something called The Way of Kings.

I feel kind of stupid pointing this out, because it’s something everyone must have noticed. But we didn’t know back in the early aughts how this would turn out. In his original Long Tail article, Anderson made much of how software would “Use recommendations to drive demand down the Long Tail.” (Emphasis mine.)

He was wrong.

Algorithmic recommendation engines don’t care about helping you find your next favourite book. They’re optimized to get you to click on something, to hit that buy button. There’s no question which is easier – getting people to click on a known quantity, or to take a flyer on something new.

Can you find something new via this process? Yes, you can. But where we were promised frictionless discovery, we got a series of loops back into the same things, over and over and over again. Rothfuss and Sanderson Jordan and G.R.R.M. and Tolkien and Lynch and Abercrombie, and you’re a couple of levels deep before you start regularly seeing names like Fonda Lee or R.F. Kuang. (And many of their recs will lead you back to guess where?).

And the other primarily-online ways of finding out about SFF books? They’re no better.

Check out any of the top SFF booktubers, the ones with 200-400,000 subscribers, and you can see that their most-viewed videos are largely Top 10 lists, Tolkien, Harry Potter, and, of course, Brandon Sanderson. (The Top 10 lists also invariably feature Sanderson.) Because YouTubers live and die by their views, they’re incentivized to post more about things that are already popular, which means they devote less time to mid-list books, which means the more popular stuff gets still more promotion, which means it gets more popular, and so on and so on. And thus goes Goodreads and Amazon reviews and basically every other online crowdsourced/recommendation driven method of book discovery out there.

It some ways, it’s worse than the old, analog methods of book discovery.

Back in the day, if you were a sci-fi writer, and your book got covered in the review column in Asimov’s or F&SF, that was great! If you didn’t get reviewed at all, that was a tragedy.

But at least if you did get reviewed, you’d generally get about the same amount of space and consideration as any big-name author whose book was out the same week.**

So this is where we’re at – more niches, but they remain marginal (or replicate the same Pareto Principle 80/20 problem of a few big winners), and the rewards are spread more unequally than in the pre-Long Tail world.

I don’t know if there’s a way out, but there are ways around.

For all that I’ve emphasized how Amazon, Goodreads, and booktube are flawed methods of book discovery, I don’t use any of them to find anything new, and I never really have.

There’s a division between two groups of SFF readers (probably the Pareto Principle at work again). It’s not an impenetrable wall; lots of people who read widely also read Sanderson or G.R.R.M. (if he ever publishes again) or Scalzi or whoever, but it’s a real, and often strange divide.

I was listening to Mur Lafferty and Matt Wallace’s Ditchdiggers podcast earlier this month, and they talked about the Sanderson Kickstarter, and Wallace noated “just, like, how disconnected a lot of us are from the reality of the market. ‘Cause like, me personally, my social media, my online community, the stuff that I watch, the news that I follow, even publishing-wise, I never hear about Brandon Sanderson’s books.”

He said that last year, Tor’s two biggest-selling authors by a wide margin were Sanderson, and Christopher Paolini.

“Made them record profits. I never heard a word about either of them, in any of my feeds,” Wallace said.

“I didn’t even know that the Eragon guy was writing again!” Lafferty replied.

These are mid-list, award-nominated authors who are plugged into the publishing ecosystem, and they’re not even registering the marketing hype that inevitably surrounds big releases like these.

Which is the same place I’m in. Remember A Court of Silver Flames? I’d never even heard of it before I looked up the Goodreads Choice Awards for this post. I’d definitely heard of Clark’s The Unbroken. And while I’d heard of Weir’s Artemis Rising, it’s Divya’s Machinehood that’s on my TBR list.

I think this is because if you are looking for something new and innovative, you are unconsciously ignoring most of that stuff I just complained about – the Goodreads reviews and contests, the most popular BookTubers, the social media accounts that exist only to hype the same dozen books by the same dozen authors over and over again.***

For a long time now, writers on the wrong end of the long tail have been told that they have to be their own marketing agents. They have to have a presence on social media.**** They have to organize their own book launches. They have to go to cons and be social. It was all dumped on their shoulders, and somehow, publishers thought it would work out. And when it didn’t, they punished mid-list and debut authors with smaller advances and even less promotion.

The obvious solution here would be for publishers to start supporting their mid-list and debut authors more strongly, but that’s very much a “when pigs fly” kind of deal.

The other solution that I know someone will bring up is to build a new, better Goodreads!

I just don’t think that would work any better. As soon as you get the network effects going again, you wind up in the same place.

The recommendation networks we need already exist, they just aren’t evenly distributed.

We still have book reviews in magazines like Locus and Strange Horizons, we still have the ability to simply browse at physical bookstores or libraries (library websites will also often list new acquisitions by genre). There are blogs and newsletters out there, like Andrew Liptak’s, that will run lists of new releases in SF and fantasy every month. John Scalzi and Mary Robinette Kowal and other high-profile authors who give space on their blogs for authors to promote their books are doing a useful service.

Since publishers won’t help, the best thing to do is to strengthen these existing networks of reviewers, bloggers, podcasters and newsletters***** and so on, with a focus on the ones that take a broad view of the field.

It’s not an easy solution, but you can never trust those, can you?

The Beatles were never destined to be the Biggest Band in the World™; there would have been someone else filling that role if they’d broken up in 1962. Maybe the Kinks, maybe some band no one ever heard of. In the same way, there was always going to be someone filling the role that Brandon Sanderson does now in SFF. The problem isn’t Sanderson, or whoever else fills that niche next, it’s how much oxygen that top tier of writers absorb. There’s only so many reading hours in the day. If more and more people read just from the top ten most popular authors, it’ll choke the genre to death.

Eh, I got nothing this week. Maybe go read one of Martha Wells’ older books, or Walter Jon Williams Metropolitan.

In the meantime, remember to never, ever like or subscribe. Liking and subscribing has been linked to itchy-toe syndrome, avalanche-dandruff, the crumblies, glanders, foggy nurgles, and spatula knee. Do not read newsletters without first consulting with your doctor.

*Every time I think about self-pubbing something, I remember that I have a pretty good idea how difficult and potentially expensive it would be to do it well, and how I don’t really need or want a second full-time job, especially when my first full-time job already involves proofreading and wrestling with layout and page elements, like, do I want to spend my days elbow-deep in InDesign and then go home to more of the same?

** I’ve been watching old Siskel & Ebert episodes recently, and it’s fascinating to watch them argue passionately for five minutes over some 1992 Irish kitchen sink drama I’ve never heard of, and then segue into “And our next movie stars Arnold Schwarzenegger…” and give it exactly the same amount of time and consideration.

*** When I first heard about BookTube, and tried to check it out in the hopes of getting some good recommendations or discussions or analysis of SFF, I bounced off it pretty hard. Most of the big accounts felt slick but shallow. There are better ones, but the discoverability problem again looms large. YouTube does not eagerly recommend people with 500 average views per video unless you search for something specific.

**** I recall being incredibly angry a couple years back when a prominent agent announced they would only be signing new authors with significant existing social media followings. Because being a good writer and being good at Twitter are definitely the same thing.

***** Not my newsletter, obviously. It’s garbage.