Statistician raises red flag about reliability of machine learning techniques

Machine learning is everywhere in science and technology: powering facial recognition, picking your recommendations on Netflix, and controlling self-driving cars. But how reliable are machine learning techniques really? A statistician says that the answer is “not very,” arguing that questions of accuracy and reproducability of machine learning have not been fully addressed.

Dr Genevera Allen, associate professor of statistics, computer science, and electrical and computer engineering Rice University in Houston, Texas has discussed this topic at a press briefing and at a scientific conference, the 2019 Annual Meeting of the American Association for the Advancement of Science (AAAS). She warned that researchers in the field of machine learning have spent so much time developing predictive models that they have not devoted enough attention to checking the accuracy of their models, and that the field must develop systems which can assess the accuracy of their own findings.

“The question is, ‘Can we really trust the discoveries that are currently being made using machine-learning techniques applied to large data sets?'” Allen said in a statement. “The answer in many situations is probably, ‘Not without checking,’ but work is underway on next-generation machine-learning systems that will assess the uncertainty and reproducibility of their predictions.”

As an example, recently machine learning has been used to study patients with cancer. To study the disease, scientists use machine learning to identify genetically similar individuals so that drug therapies can then be targeted to these specific genomes. But when comparing across different studies, the clusters identified by machine learning are completely different from each other.

The problem is that machine learning techniques do not have a way to say “I don’t know” or “It’s not clear.” The techniques will generally always produce an answer — in the example of the cancer patients, they will always identify a group in some way — but this answer may not be as certain or accurate as it is believed to be. The techniques are able to find a pattern that exists in the data set, even if only dimly, but the pattern may not hold in the real world.

“There is general recognition of a reproducibility crisis in science right now,” Allen told BBC News. “I would venture to argue that a huge part of that does come from the use of machine learning techniques in science.”

Emerging Tech

A.I.-generated text is supercharging fake news. This is how we fight back

A new A.I. tool is reportedly able to spot passages of text written by algorithm. Here's why similar systems might prove essential in a world of fake news created by smart machines.
Computing

Nvidia’s A.I. Playground lets you edit photos, experience deep learning research

Nvidia is making it easier to access information on deep learning research. It has launched an online space with three demos for image editing, styling, as well as photorealistic image synthesis. 
Computing

Give your discs some extra life by watching DVDs and Blu-rays on Windows 10

Popped a disc into your Windows machine but feel lost without Media Center? You're not alone. But don't fret, with just a few tips you can learn how to watch DVDs and Blu-rays for free in Windows 10 in no time.
Smart Home

Is your Keurig making gross coffee? Might be time for a cleaning

No one likes a dirty, scaled, or smelly Keurig, but how are you supposed to clean them? Before you throw yours out the window, here is a quick guide on cleaning your machine out thoroughly.
Computing

At $99, Nvidia’s Jetson Nano minicomputer seeks to bring robotics to the masses

Nvidia announced a new A.I. computer, the Jetson Nano. This computer comes with an 128-core GPU that Nvidia claims can handle pretty much any A.I. framework you could imagine. At $99, it's an affordable way for A.I. newbies to get involved.
Business

British Airways’ new Club Suite for business class comes with a door

British Airways is going after a bigger slice of the business class market with the imminent launch of the Club Suite. The plush seating option offers a more private space as well as an easier route to the bathroom.
Emerging Tech

The U.S. Army is building a giant VR battlefield to train soldiers virtually

Imagine if the U.S. Army was able to rehearse battlezone scenarios dozens, or even hundreds, or times before settling foot on actual terrain. Thanks to virtual reality, that's now a possibility.
Smart Home

Sony’s Aibo robot dog can now patrol your home for persons of interest

Sony released the all-new Aibo in the U.S. around nine months ago, and since then the robot dog has (hopefully) been melting owners' hearts with its cute looks and clever tricks. Now it has a new one up its sleeve.
Emerging Tech

Inflating smart pills could be a painless alternative to injections

Could an inflating pill containing hidden microneedles replace painful injections? The creators of the RaniPill robotic capsule think so — and they have the human trials to prove it.
Emerging Tech

A silver bullet is being aimed at the drug-resistant superbugs on the ISS

A bacteria which is benign here on Earth can mutate into a drug-resistant superbug once it enters space. Now this problem is being tackled by a team of microbiologists who have found a way to inhibit the spread of bacteria in the ISS.
Emerging Tech

Tombot is the hyper-realistic dog robot that puts Spot to shame

Forget Boston Dynamics’ Spot! When it comes to robot dogs, the folks behind a new Kickstarter campaign have plans to stake their claim as makers of man’s (and woman’s) newest best friend.
Emerging Tech

Researchers gave alligators headphones and ketamine, and all for a good cause

Researchers in Germany and the United States recently gave ketamine and earphones to alligators to monitor how they process sounds. Here's what it reveals about alligator evolution.
Emerging Tech

Cheese tastes different when it listens to Led Zeppelin, Swiss study finds

A funky new study says that exposing cheese to music changes its aroma and flavor. What’s more, the genre of music matters. Researchers from the Bern University of Arts played music to nine, 22-pound wheels of Emmental cheese.
Emerging Tech

Astronomers plan to beam Earth’s greatest hits into deep space, and you can help

A new project from the SETI Institute (search for extraterrestrial intelligence) will give the public the chance to submit compositions to be beamed into space, with the aim of connecting people around the world through music.