But that is not to say it is perfect by any stretch of the imagination.
“Deep learning has led to some big advances in computer vision, natural language processing, and other areas,” Tommi Jaakkola, a Massachusetts Institute of Technology professor of electrical engineering and computer science, told Digital Trends. “It’s tremendously flexible in terms of learning input/output mappings, but the flexibility and power comes at a cost. That is it that it’s very difficult to work out why it is performing a certain prediction in a particular context.”
This black-boxed lack of transparency would be one thing if deep learning systems were still confined to being lab experiments, but they are not. Today, AI systems are increasingly rolling out into the real world — and that means they need to be available for scrutiny by humans.
“This becomes a real issue in any situation where there are consequences to making a prediction, or actions that are taken on the basis of that prediction,” Jaakkola said.
Fortunately, that is where a new project from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) comes into play. What researchers there have come up with is preliminary work showing that it is possible to train neural networks in such a way that they do not just offer predictions and classifications, but also rationalize their decision.
For the study, the researchers examined neural nets that were trained on textual data. This network was divided into two modules: one which extracted segments of text and scored them on their length and coherence, the second which performed the job of predicting or classifying.
A data set the researchers tested their system on was a group of reviews from a website in which users rated beers. The data the researchers used included both a text review and also a corresponding star review, ranked out of five. With these inputs and outputs, the researchers were able to fine-tune a system which “thought” along the same lines as human reviewers — thereby making its decisions more understandable.
Ultimately, the system’s agreement with human annotations was 96 percent and 95 percent, respectively, when predicting ratings of beer appearance and aroma, and 80 percent when predicting palate.
The research is still in its early stages, but it is an intriguing advance in developing AI systems which make sense to human creators and can justify decisions accordingly.
“The question of justifying predictions will be a prevalent issue across complex AI systems,” Jaakkola said. “They need to be able to communicate with people. Whether the solution is this particular architecture or not remains to be seen. Right now, we’re in the process of revising this work and making it more sophisticated. But it absolutely opens up an area of research that is very important.”
