Vellamo: Android Web Benchmark from Qualcomm

 

Earlier today, I talked to Qualcomm’s Sy Choudhury (Director, Product Management)  about Vellamo, a benchmarck aimed at measuring several aspects of web browsing performance: page rendering, Javascript execution, user experience, networking. This project originated from Qualcomm’s desire to improve its own software. As one of the leading chip provider, Qualcomm does contribute quite of software in the Android eco-system, and Vellamo has started as an internal tool to quantify the performance gain in Qualcomm’s own code.

The interesting part is that Vellamo is now available to the public. It is available for download today on the Android Market and should be upgraded every 9 months or so. Let’s take a quick look at the four main performance metrics: Rendering: the rendering represents the phase during which the browser takes all the web page data (HTML, CSS, Javascript) and builds an actual image that you can look at. This is key to web performance as there are a lot of pixels to process on modern smartphones.

Javascript: This is usually more important for Web-apps that execute a bunch of code to make the site more interactive. Most sites aren’t very interactive (News, Wikipedia, Google), but sites like Mint or Google Docs are highly interactive and deserve the title of “web application”. Upcoming web-based games also use Javascript intensely, although today, there aren’t many of them around.

User Experience: This represents things like scrolling speed and page reactivity when users scroll and zoom. The idea is that at some level some page blocks need to be rebuilt, and if they aren’t built fast enough, you may see blank portions of the page, or the scrolling might be jittery. If it is fast enough, the scrolling is fluid.

Networking: Of course, all of the above need data from the network, so it’s extremely important that the networking software layer is well optimized. There are many strategies to improve networking performance, and many of them rely on the smart use of cached data (a local version) so that there is as few network requests as possible.

At the end of the tests, users will be given a score, and Vellamo will even compare the current device with others from an online database. At the moment, the Galaxy Tab 10.1 sits at the top because Honeycomb (Android 3.x) has a lot of browser and Javascript engine optimization. In the end, keep in mind that this is not measuring the speed of the hardware, but of the entire system that enables web browsing (hardware, software, optimizations…).

Obviously, one may ask whether or not Qualcomm’s benchmark would favor its own chips. It’s a reasonable question to ask. Given that the benchmark is an app like any other, Qualcomm’s chip can indeed lose to a competitor (believe me, creating your own benchmark can bite you in the behind later on). Also Qualcomm actually uses it internally, so it’s best of it works properly… Finally, Qualcomm is considering releasing the code to the public, and that is probably the best way to gain legitimacy.

Hubert Nguyen
Categories: Cellphones, Tablets
Tags: benchmark, performance, Qualcomm, SoC