Google retires Octane JavaScript benchmark, will focus on ‘real web pages’ instead

google retires octane javascript benchmark
Web browsing has become an increasingly intensive computing process over the years, moving from the simple display of static graphics and text to the running of sophisticated client-side applications using languages like JavaScript. Knowing how well a web browser can run today’s dynamic web experience on a given device is an important consideration for many people, and to that end, various benchmarks have been created to help test performance.

However, benchmarks have distinct life cycles, as technology advances and browser developers work around the benchmarks’ limitations. SunSpider was one of the first JavaScript benchmarks, and as its usefulness waned, Google’s Octane was introduced to take its place in 2012. Now Octane, as well, has reached the end of its usefulness and it, too, is being retired.

The reasons for Octane’s demise are fairly complex, and you can check out Google’s announcement at the V8 project blog for all of the details. In simple terms, starting around 2015, most JavaScript engines had optimized their compilers to score well on Octane tests. As developers worked to achieve ever-higher benchmark results, the actual benefits to real-life web page performance became increasingly marginal.

In addition, sites like Facebook, Twitter, and Wikipedia have demonstrated that the Octane benchmark wasn’t accurately reflecting how Google’s V8 JavaScript engine actually worked on real sites. Therefore, Octane wasn’t actually capturing important information on how V8 and other JavaScript engines perform in the modern web environment.

Finally, it became apparent that efforts to gain higher Octane benchmark results were actually having a deleterious effect on performance in real-world scenarios. The combination of Octane’s increasing disconnect with how web sites actually work with developers’ efforts to achieve higher and higher Octane scores meant an increasingly negative impact on how JavaScript engines were designed to perform when it really matters. Even bugs in Octane have been leveraged by developers to gain higher benchmark results.

All benchmarks suffer from similar problems, according to Google, meaning that the very process of making benchmarks to demonstrate performance eventually leads to performance decreases as developers write code that’s optimized for the benchmarks and not the real world. Google’s efforts going forward will be focused on improving performance measurements of browser performance on real web pages as opposed to merely running static test suites.


'World's best gaming processor'? We put Intel's new i9 through the ringer

Intel has launched the first Core i9 for the average gamer. Despite some controversies around its release, it’s the fastest gaming processor we’ve yet tested.

What's the best laptop? We've reviewed a lot of them, and this is our answer

The best laptop should be one that checks all the boxes: Great battery life, beautiful design, and top-notch performance. The laptops we've chosen for our best laptops you can buy do all that — and throw in some extra features while…
Home Theater

Google Chromecast and Chromecast Ultra: Everything you need to know

Google's Chromecast plugs into your TV's HDMI port, allowing you to stream content from your tablet, laptop, or smartphone directly to your TV. Here's what you need to know about all iterations, including the 4K-ready Chromecast Ultra.
Product Review

LG's new V40 has 5 cameras, but ThinQ twice before you buy

The LG V40 ThinQ has five cameras -- three on the back and two on the front. This makes it one of the most versatile camera phones LG has released to date, and it’s creatively fun to use. Read on for more in-depth analysis.

Samsung’s HMD Odyssey Plus gives you a clearer view into the virtual world

Samsung's refreshed HMD Odyssey+ promises to make Windows Mixed Reality experiences better by eliminating pixelated views caused by screen doors. The $500 headset also focuses on comfort this year with ergonomic improvements.

Intel denies rumors that 10nm Cannon Lake CPUs have been canned

Intel's long-in-development and oft-delayed, Cannon Lake 10nm CPU design has reportedly been canceled. Intel is denying the rumor, but if true, it could push back the release of new Intel chips by a long time.

Not to be outdone, Samsung says it’s making a laptop with a foldable display

Samsung announced that it is also working on a dual-screen computer. But rather than using two separate display panels, Samsung said that its novel laptop will come with a large flexible display that can fold when closed.

Free your digital memories, and frame them, with the best photo printers

Printed photos are experiencing a revival at the moment, but you don’t need to go to a special lab. Here’s our favorite options for making quality prints, from pocket-sized printers to wide-format photo printers capable of spitting out…

A new bug in the Windows 10 October 2018 Update could delete your files

The Windows 10 October 2018 Update has been on a rough path and in the latest set of issues, a new bug is impacting native zip file operations, potentially leading to overwritten files in some instances. 

Antivirus software has evolved a lot recently, and we need it more than ever

Everyone says you need it, but really is antivirus software, and how does it work? It depends on who you ask as different digital security companies employ different techniques to combat the latest malware threats.

Nvidia’s new GTX 1060 6GB could counter AMD’s rumored RX 590

Nvidia's GTX 1060 is about to get more powerful for new buyers, as the green team has introduced a new version with GDDR5X memory at its disposal. This could prove competitive with AMD's rumored RX 590.

A canceled education order is increasing hopes for new Macbook model

With Apple's October 30 event fast approaching, rumors continue to surface about new Macs and iPad models. In the latest news, a canceled education order is stoking hopes for a new MacBook model.

Facebook’s reported Rift 2 cancellation caused a rift with Oculus executive

Facebook may be prioritizing virtual reality on mobile experiences, which may have led to the company canceling development of the Oculus Rift 2. That may have been the cause of Oculus co-founder Brendan Iribe quitting.

Apple could show off a new iPad Mini and AirPower charger at October event

Apple's sprung a surprise event on us, even though this year has seen loads of new Apple releases. So what's left to come? Quite a lot as it turns out. Here's what to expect from Apple's October 30 event.