This bot is trying to write the next Game of Thrones book
I’m a big Game of Thrones fan, and I watch the show religiously, but lately I just haven’t been able to summon the same amount of enthusiasm that I once harboured for HBO’s hit TV series. That’s not to say that the show is bad now, or that I think the showrunners are doing a poor job. Nah, my opinion has more to do with the fact that the novels were the thing I loved first. Yeah, in case you forgot, before the TV series was even conceived of, Martin’s A Song of Ice and Fire series of novels (which began with A Game of Thrones way back in 1996) was what all true-blooded fantasy nerds were obsessed with, and in high school I devoured the Game of Thrones books page-by-page.
For me, the best part of the Game of Thrones TV series was watching how the novel adapted key scenes, and how it brought Martin’s vivid characterisations and detailed fantasy world to life. I enjoyed seeing Westeros play out on the small screen because it was a setting that already felt so familiar to me, and I wanted to see if things played out like they did in print, and swap fan theories about storylines to come. Now that the show has outpaced the narrative of the novels, it just isn’t the same for me.
It’s all the more painful because, as any ‘ASOIAF’ fan knows, the wait between books can often seem interminable. I know Martin’s only human, and that his novels are too dense and complex for him to simply bang out like cheap airport paperbacks, but it seems like we’ve been waiting aeons for him to finish. The fandom has already been pretty vocal, even downright hostile, about Martin’s seemingly-glacial pace. The outcries got so aggressive that fellow fantasy novelist Neil Gaiman felt obliged to curtly inform some of the more abusive fans that “George R.R. Martin is not your b*tch.” Still, it’s easy to see why people have been so frustrated. If only there was a way someone, or something, could write the books for him. An author who never got tired, never needed a break, and who can definitely meet deadlines.
Good thing that there are more than a few computer programmers out there who, upon hearing the words “if only”, reply “why not?” ‘ASOIAF’ fan and software engineer Zack Thoutt eventually got sick of waiting for Martin to finish, and decided to complete the book with the help of an artificial intelligence that he’d designed himself. Thoutt’s bot is what’s known in programming lingo as a “recurrent neural network.”
In (very) simple terms, this means that, instead of a computer simply being taught specific responses to specific inputs one layer at a time, the computer is fed information, and the data from the output of the last problem solved will directly affect the decision-making process used in the next input. It’s the difference between asking two people for help with a problem, one with short-term memory loss, the other without. A non-recurrent neural network is only able to solve one problem at a time, because it’s already forgotten the answer to the last question when someone asks a new one, while a recurrent one is able to solve multiple problems and distinguish between priorities, because it can make judgements based on the problems it ‘remembers’ solving before.
Because of this, recurrent neural networks make good bots because they’re able to make predictions based on probability and refer to their own feedback loop. This meant that, when Thoutt fed his clever little AI all +5000 pages of the ‘ASOIAF’ novels, it was able to crudely analyse Martin’s prose, work out what words and phrases were commonly used, and then attempt to replicate the novels itself. The result: a self-generating software author. Pretty cool, right? Except for one teeny, tiny little snag.
Namely the fact that the bot is an absolutely dreadful writer. The text it has produced simply doesn’t make sense; maybe not to the same extent as ‘50 Shades of Grey’, but it’s still pretty rubbish. See, while the machine was able to produce a reasonable facsimile of Martin’s style - a cursory glance at its haphazard prose reveals that it’s mostly churning out passages of complete non-sequiturs made up of Martinesque phraseology gummed together more or less at random. It’s like the machine is able to regurgitate the style of Martin’s writing, but without being able to juxtapose that with any kind of coherency.
Take this passage as an example: “The next morning he awoke another hundred feet above the winesink beside the temple with a battering wall, like the wall a pack looming ominous. His right leg was solid below. He ended that tight. To show, summer came as well as he had and the intruder sept bowlegged one great younger man all gathered in front of the rocking of iron chains, his foot short of black steel and the cold old shadow. The shield in his hand bobbed, leaving a third of his neck in the dangling water.” I mean, seriously, what in the hell is that supposed to mean? It’s like reading ‘Finnegan’s Wake’ by James Joyce, by which I mean, “completely incomprehensible to the human mind.”
Another absolute gem the bot wrote reads: “It is an effort. Mine uncle had do the same color. She could hardly count by death” and in another it informs us that “Meera kicked the turtle warily at Tyrion’s face, stirred a fallen lantern, as the new giant offered a stick for lollys to the empty hall he’d played down as well.” Even if you take it as experimental, postmodern literature, it’s pretty bizarre. The bot is also unable to remember which characters are where, how relevant they are to the plot, or even whether they are alive or dead. There’s no continuity whatsoever. At one point, Varys (rather casually) poisons Daenerys, and The Hound seems to teleport around the world just to exchange gibberish with other characters. I’ve heard of non-linear storytelling, but this is just insane.
Still, a few of the plotlines it’s managed to generate via the monkeys-at-typewriters approach have actually managed to mirror the series; Jaime leaves Kings Landing, presumably to go fight up north, but the book believes that he will later return to kill Cersei, and Jon rides a dragon at one point. But there are many more twists that make far less sense in context. Sansa, for example, turns out to secretly have been a Baratheon all along, as mentioned above, Varys decides to casually murder Dany, and a tall piratey character called Greenbeard (eloquently described as a “big blind bearded pimple with his fallen body”) shows up to steal the limelight from the rest of the cast.
In a recent interview with Vice on the subject of his creation, Thoutt himself stated: “It’s obviously not perfect. It isn’t building a long-term story and the grammar isn’t perfect. But the network is able to learn the basics of the English language and structure of George R.R. Martin’s style on its own. A perfect model would take everything that has happened in the books into account and not write about characters being alive when they died two books ago."
He added, “The reality, though, is that the model isn't good enough to do that. If the model were that good, authors might be in trouble ... but it makes a lot of mistakes because the technology to train a perfect text generator that can remember complex plots over millions of words doesn't exist yet."
I guess this means that writers won’t be losing their jobs to machines quite yet (phew) and that we’re still going to have to wait a little longer before ‘The Winds of Winter’ hits the shelves. Still I, for one, am going to keep up with the bot’s efforts. If you just treat it like free-form blank verse, it’s actually pretty enjoyable. Check out the five chapters it’s written up so far here.