Climbing into the black Porsche Panamera, as the snow began to fall heavily outside, the Mate 10 Pro stuck to the windshield caught our attention. It looked like it was there to help with navigation; but we knew better. A cable running from the USB port deep down into the car revealed its true purpose. The phone was going to drive the car, and make sure we didn’t hit something that would suddenly appear in front of us.
Nervous? Wouldn’t you be?
Can a smartphone really power a driverless car? At MWC 2018, we found the answer to this question is yes, provided the phone is adept at harnessing artificial intelligence. Huawei invited us along to a demonstration of Project Road Reader, which despite outward appearances isn’t the announcement of its intention to build an autonomous car; but a very visual and exciting way to show off the capabilities of its Kirin 970 processor and its Neural Processing Unit (NPU) inside the Mate 10 Pro phone.
Demonstrating the power of AI
As a term, “artificial intelligence” has lost much of its meaning recently, due to so many products and services being promoted as having some kind of AI behind it. Huawei launched the Mate 10 Pro phone with the NPU last year and struggled to illustrate the chip’s benefits effectively. We saw it at work in the camera app, and throughout the operating system; but its hard to understand how powerful it really can be. That’s why Huawei fitted a modified Porsche Panamera with a Mate 10 Pro and drove journalists towards fake bikes, balls, and animals.
The Mate 10 Pro’s job was to recognize hazards popping up in front of it, and take appropriate action. Think about that: The same phone you use for Twitter, email, and calling a friend, is also clever enough to see and recognize an object, while moving at speed, then make the decision to carry out a pre-determined action.
This isn’t some beefed up Mate 10 Pro. It’s a standard, out-of-the-box phone.
To make it clear, this isn’t some beefed up Mate 10 Pro. It’s a standard, out-of-the-box phone. The image recognition has the same capabilities too, it has just been taught something new. It has to recognise bikes, balls, and dogs, and not sunsets or snow. No changes have been made at all, outside of an app to control the car being produced. This was made with Huawei’s readily available development kit, and in theory, with the right talent, car, and private race track, anyone could replicate the same experience. However, it’s not recommended, for obvious reasons.
Why the Panamera? Porsche’s car was chosen because it’s not autonomous at the moment. The modified car here is controlled by hydraulics and servos — the driverless part is mechanical rather than electronic or robotic. On the top is a ski box protecting a large camera from the weather — it was actually snowing during our test. Although the Mate 10 Pro’s own cameras could be used, there were concerns about then being obscured by the windscreen, or something falling on it. Safety first.
Taking control
The camera films the road ahead and delivers the information to the Mate 10 Pro by HDMI cable, scanning for objects, which it can recognise at 30 frames per second (fps). This uses the same tech that the camera uses to improve pictures. None of the AI processing happens in the cloud, it’s all done on the device, which is the NPU’s real special ability.
The first run was taken at 5 mph, and three objects were randomly presented in the road ahead of the car, allowing the Mate 10 Pro to capture and recognize them. A total of 1,000 common objects found on or near roads had been programmed into the phone. Once the learning phase was complete, we told the phone what we’d like to do when it encountered one or more of these objects on the next run. For example, brake when it sees the dog, or swerve when it sees a bike.
No bikes, fake cyclists, real journalists, or expensive Panameras were hurt or damaged.
With this done, it was time for the main run, which would be at 30mph — significantly faster, and the true test of the Mate 10 Pro’s ability. Huawei claims it could safely react up to 60 mph, but space constraints prevented us from finding out.
The car set off abruptly, reaching its target speed immediately. What happened next was over in a split second, as the fake bike came into its path, the Mate 10 Pro saw it, and swerved in the direction we had previously told it to. No bikes, fake cyclists, real journalists, or expensive Panameras were hurt or damaged.
Further potential
The fact a phone did this is impressive enough, but not just any phone could have been used in the demonstration. The developers tried out other phones, but the image recognition wasn’t robust enough. The NPU inside the Mate 10 Pro helps the phone recognize 2,000 images per minute, while on some other flagships this number drops down to around 500, or even as low as 80.
Impressively, the demo could be taken a stage further, with the system reacting intelligently to an emergency situation. Anyone knows that if a ball rolls into the road, a child is likely to follow. Standard radar systems on autonomous cars don’t know this, and a computer is required to make those calls. The Mate 10 Pro is capable of doing that job.
What does this mean for us, if Huawei isn’t about to offer a Mate 10 Pro-controlled car? It’s a visual showcase of the NPU’s prowess and AI’s growing ability, in a way people can better understand. Huawei told us there are a lot more things to do with the onboard AI coming soon. While the camera is the most prominent feature for AI at Huawei, there are many other possibilities for the future, some of which may be revealed when the P20 arrives at the end of March, a fact also revealed by CEO Richard Yu when we spoke to him at the show.
No, we won’t be getting a car that’s controlled by a phone in the near future, but there is one out there with enough processing power and smarts to actually do so. That means taking better photos and making my life easier should be comparatively easy by comparison, and that’s very reassuring.
