Samsung caught rigging phones to boost benchmark results

Samsung caught rigging phones to boost benchmark results

It has always been my firm belief that benchmarks, specifically mobile benchmarks, were a waste of time and could never offer the same information as actually holding the device in your hands and trying it for yourself. Samsung has helped demonstrate just how true that is after being caught intentionally boosting performance on their hardware in order to deceive popular benchmarking tools.

It was only a matter of time before smartphone manufacturers started trying to optimize their hardware to suit benchmarks. There’s a lot of curious double talk about hardware amongst mobile OEMs right now, some going as far as giving their processors entirely new names to make them sound great to a casual observer. PC manufacturers have been doing it for years, and there’s never really been anything anyone can do to stop them from behaving this way.

With a smartphone, the CPU and GPU benchmarks do little more than complete numbers on a chart or add another line to a graph. You can’t use a CPU or GPU benchmark to tell you which of the smartphones will offer you a better experience in your day to day activities. In theory, you could use GPU benchmarks to tell you which device would give you a better gaming experience if you played a lot of high performance mobile games, but now it would appear that even this is no longer true.

After receiving a tip that some benchmark apps were behaving a little oddly on the octo-core variant of the Galaxy S4, the guys at Anandtech did what they do best and performed a thorough analysis on the phone. According to their research, this variant of the Galaxy S4 contains specific code that boosts the clock speed of the CPU and GPU when specific benchmarking apps are running on the phone.

The app responsible for this behavior, TwDVFSApp.apk, will push the GPU from 480MHz to 532MHz, and as far as anyone can tell that clock speed is only possible on the phone when the specific benchmarks being called out in the app are being run. This list includes AnTuTu, Benchmark Pi, Linpack, and all versions of the Quadrant apps. Each of these are popular, well known Android benchmark apps that can be installed in most cases for free from the Google Play Store.

The frustrating thing about this isn’t that Samsung is rigging their own hardware to get better benchmarks, but that those clock speeds aren’t available to users when playing games. There’s no way to enjoy the Exynos 5 Octa at 532MHz, even though your benchmarking app tells you that’s what you should expect. It’s possible there’s a good reason that users aren’t allowed to use the higher clockspeed. This may be a heat management issue, as every version of the Galaxy S4 has some temperature issues after an extended gameplay session.

Whatever is going on here, though, it’s clear that benchmarks really aren’t to be trusted on mobile hardware any more than they are on traditional computers.