Newly developed AI system can accurately judge a book by its cover

The tech world sure loves to disrupt conventional wisdom. Its latest victim? The old adage that you should never judge a book by its cover.

With disproving that sentiment in mind, researchers at Japan’s Kyushu University have trained a neural network to be able to predict which genre a book falls into simply by studying its cover.

“The purpose of this work is to determine if machines can learn the meaning behind book covers without textual clues,” researcher and paper co-author Brian Kenji Iwana told Digital Trends. “For this study, we took book cover images and classified them by genre using an artificial neural network. We also look at some of the hidden design rules of the covers found by the network.”

For their dataset, Iwana and colleague Seiichi Uchida used a total of 137,788 book covers for titles available for sale on Amazon. These fell into 20 different categories, and was simplified slightly by only using the primary category a book was listed under, in instances where it fell under multiple genre headings.

Eighty percent of this data was then used to train the four-layer neural network the pair used, thereby leaving 20 percent for validating and testing it.

More than 40 percent of the time, the algorithm was able to place the correct genre within its three best guesses, while it predicted the right genre first guess upward of 20 percent of the time.

Unfortunately, the pair didn’t research how well humans do at the classification task (which is relatively straightforward for a genre like cookery books, but tougher when it comes to broader genres like biographies or memoirs). However, the results of the algorithm show significantly better results than just a random guess.

“The idea came from our previous work with font and document recognition,” Iwana said. “We are particularly interested in pushing the field of machine learning into tasks that traditionally require human feelings, such as impression and design.”

There are multiple possible applications for this research. It could, for instance, be used to help classify digitized books in cases where labelled data is lacking. It could also (creative-minded designers beware!) be used to help find “rules” that more easily visually describe what a book is about — helpful for both machines and bookstore-browsing humans alike.

Longer term, it even opens up the possibility of algorithms being able to generate cover concepts by themselves.

“Our work shows that it’s possible to use machines to learn the relationship between book covers and genre,” Iwana concluded. “This can lead to tools used to help authors design book covers or to automate genre prediction. It’s one step closer to bringing machine learning into the field of design.”