On UA sniffing, browser detection, and Alex's post

Unless you’ve not been paying attention during the past week, you may have come across Alex Russell’s recent treatises on the cost of feature detection and one possible solution1. Alex is one of the smartest folks I’ve ever met, and I’ve always admired his willingness to share his opinion in any forum regardless of the popularity of the thought or the response quality he’d receive. While I can’t say I always agree with his conclusions, I can say that I respect how he arrives at them. And this is why I feel badly when his points of view get misrepresented to the point of confusion.

The beginning

In his first post on the subject, Cutting the interrogation short1, Alex makes several claims:

  1. Feature detection is not the panacea for cross-browser solutions
  2. Some feature detection techniques incur a performance hit that isn’t always reasonable
  3. The set of available features for known browsers is known

I don’t find anything terribly controversial about these claims, and further, I believe them all to be correct and easily verifiable. The second point is actually the key to understanding his position.

The fastest running code is the code that performs the fewest number of operations. As a good programmer, and certainly one that wishes to deliver the best user experience, it is your job to complete any given ask using the fewest number of operations. Feature detection necessarily introduces additional operations to determine the appropriate set of next operations.

While I’ve never been opposed to feature detection such as determining whether a given function or property exists, I’ve openly opposed the type of long and involved feature detection techniques2 employed by some libraries and developers, especially when performed as an upfront evaluation of multiple features, such as those found in Modernizr3. As someone who’s worked on several large-scale, highly trafficked web sites, I’ve always made it a point to avoid this type of methodology for performance reasons.

The proposal

Alex’s proposal for improving the performance of feature detection was to determine and then cache the results of feature tests for known browsers. This would allow libraries to skip passed the long and time-consuming feature detection code when the results are actually already known. The approach requires a certain level of user-agent detection4 to serve up the correct feature detection set.

Now, I’ve been (in)famous for saying in that past that I don’t believe user-agent detection is bad or evil or that it breaks the spirit of the Web or any such thing – I’ve simply stated that it’s a technique you should know in general and understand where and when it’s appropriate to use. I’ll say this again: you use user-agent detection when you want to identify the browser being used. That’s it. Feature detection, on the other hand, is used when you want to determine that a feature is available for use. These are two different techniques with two very different use cases.

The proposal from Alex is to use user-agent detection to load the results of feature tests run in a particular user-agent while leaving regular feature detection for browsers that are unknown entities. Let’s face it, Internet Explorer 6′s feature set is not changing, so if you can accurately detect this browser it makes sense to preload its feature set.

I would also augment Alex’s proposal with the same caution that I have with user-agent sniffing, which is to only identify previous versions of browsers (not current ones). You cannot  be certain that a feature set is frozen for a particular browser until the next version is released. Case in point: Internet Explorer 8 shipped with a native JSON implementation that didn’t match the final ECMAScript 5 specification. This was later fixed in a patch5. At that point in time, Internet Explorer 8 was the most recent release so it would only be reasonable to cache results from Internet Explorer 7 or earlier.

What he didn’t say

Almost as interesting as what Alex did say is what he didn’t say. Mostly because people immediately started hinting that he actually was saying the things that he didn’t say. This is an incredibly frustrating yet unbelievably common occurrence on the web that I’ve also dealt with. Not that Alex needs anyone coming to his rescue, but I do want to outline the things he never said in his posts:

  1. He never said that user-agent detection is better than feature detection
  2. He never said that feature detection is bad and shouldn’t be used
  3. He never said that user-agent detection is awesome and should always be used
  4. He never said his proposal is the only solution

As tends to happen with controversial topics, people have been latching on to one or two sentences in the entire post rather than trying to absorb the larger point.

My opinion

I was asked by a colleague last week what I thought about Alex’s proposal. Since I had only skimmed the two posts, I decided to go back and actually read them carefully. First and foremost, I think Alex accurately outlines the problems with the current feature detection craze, which can be summarized neatly as “all feature detection, all the time” or even more succinctly, “feature detection, always.” He’s correct in pointing out that the feature detection craze doesn’t pay close enough attention to the performance overhead associated with running a bunch of feature tests upfront.

Generally, I like the idea of having pre-built caches of feature tests for older, known browsers such as Internet Explorer 6 and 7. We absolutely know the issues with these browsers and neither the issues nor the browsers are going away anytime soon. I’m less convinced of the need to cache information for other classes of browsers, especially those that update with regular frequency. For instance, I think it would be wasteful to do such caching for Chrome, which auto-updates at such a dizzying pace that I can’t tell you off the top of my head which version I’m running on this computer.

At this point, I’m more in favor of Alex’s proposal than I am against it. I do believe there’s value in caching feature detection results for known entities, however, I believe the number of UAs for which that should be done is small. I would target two sets of browsers: older ones (IE6/IE7) and specific mobile ones. Interestingly, these share the common aspect of running code slower than modern browsers running on desktops. Keeping a small static cache designed to optimize for the worst-performing browsers makes the most sense to me, and then I would only do additional feature tests on an as-needed basis – running the test on the first attempt to use the feature and then caching it dynamically.

I’m sure there’s a sweet spot of cached feature data that can be found by focusing primarily on the outliers, especially ones that are using slower JavaScript engines (IE6) or low-powered devices (mobile) that cause slower-running JavaScript. Of course, as with every theory, this approach would have to be tested out in real world scenarios to figure out the exact savings. Personally, I think it’s worth investigating.

  1. Cutting the interrogation short, by Alex Russell
  2. JavaScript feature testing
  3. Modernizr
  4. Performance innumeracy & false positives, by Alex Russell
  5. An update is available for the native JSON features in Internet Explorer 8

Comments

  1. Dominykas

    Am I the only one thinking, that browser vendors, especially mobile browser vendors need to start standardizing and exposing the list of features they support?

    e.g. if (navigator.features.html.inputTypeDate) { useBrowserCalendar(); } else { renderMyJavascriptCalendar(); }

    Of course, there's still the things like Chrome issue #52845 and the fact that this wouldn't be widely available for the next few years, but I'm not currently planning to change my career or anything - I can wait.

  2. Keven

    Why not take the best of both worlds and keep the feature-detection in your code, for all the good reasons given by Alex Russell, and rely on UA sniffing for well-known browsers which we know the features list, and more heavy process for rare browsers ?

  3. Nicholas C. Zakas

    @Keven - That's exactly my conclusion in the last paragraph. :)

  4. David Bruant

    I've recently written an article on what I've called "lazy feature testing":
    http://longtermlaziness.wor...

    I think that combining the solution of pre-computing test results for old browsers (IE6/7 as you pointed out) and lazy-feature testing could be a good compromise to avoid the performance hit both in old and well-known browser as well as unknown web browsers where lazy-feature testing only perform tests which results are actually used.

  5. Peter Cranstone

    It's the DEVCAP problem - what is the device really capable of supporting and in the same breath what is the browser capable of supporting. Here's 5 user agents - they all came from the same device. So what do you send? You have no idea.

    HTTP_USER_AGENT=[Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16]
    HTTP_USER_AGENT=[Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B367 Safari/531.21.10]
    HTTP_USER_AGENT=[Mozilla/5.0 (Linux; U; Android 2.2; en-us; PC36100 Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1]
    HTTP_USER_AGENT=[Mozilla/5.0 (Linux; U; Android 2.2; en-us; Sprint APA9292KT Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1]
    HTTP_USER_AGENT=[Mozilla/5.0 (Android; Linux armv7l; rv:2.0b11pre) Gecko/20110126 Firefox/4.0b11pre Fennec/4.0b4]

    What's interesting is that the device is an Android phone. Not an iPad or iPhone, but with the ability to spoof the UA I can make it seem like I have an iPad.

    Let's face the current system is completely broken. There's insufficient data to really determine what to send or what feature is really supported.

    Cheers,

    Peter
    5o9 Inc
    Web Tools for Mobile

  6. Christian

    I think it's healthy to challenge established truths, and as such I'm enjoying this debate. However, I have yet to see some actual data supporting the claim that feature testing is so expensive that it actually hurts the user experience. The only data I've seen so far is the two screenshots John David Dalton posted over at http://allyoucanleet.com/

    John's benchmarks show that at least the iPhone and some desktop browsers can handle vast amounts (more than you'd actually use in any typical application) with no sweat. Of course, benchmarks are what they are, and this may not show the whole picture.

    I'd be interested to see similar data from the "feature testing hurts performance" side of the fence. In what actual cases you worked on did feature testing actually hurt the user experience? And simply counting operations isn't a very solid measure.

  7. Christian

    Another thought: The whole premise of this idea assumes that you can actually reliably determine the browser. That's a fairly risky assumption. Is the (presumably minor) performance gain worth the added uncertainty?

  8. Dustin Machi

    @Keven, @David,

    I believe this is exactly what Alex is suggesting and more or less what has been planned for builds with has.js for a while. Using strict matching UA to detect well known browsers and fallback to feature testing, which is already lazily applied in has.js. Unlike Modernizer, which runs the tests up front, has.js executes tests on demand and only once. By using strict matching UA, fingerprints of different browsers can resolve the answers ahead of time (and it needn't be all tests either).

    @Dominykas,

    That is ok to an extent, however feature detection is a bit of a misnomer. While it does detect features, it also detects bugs and other problems that the approach wouldn't work for.

  9. Kyle Simpson

    [What would a Nicholas Zakas post mentioning user-agent detection be without me commenting to dissent?] :)

    I have come to the conclusion that the crux of this argument is not about whether user-agent detection is good or not. Of course it's good if you can, in one test, determine (RELIABLY) the exact user-agent, and in so doing if you know (as Nicholas points out) that the user-agent definitely isn't changing its feature set, so you can cut out unnecessary single-feature tests. Good on you.

    BUT, and this is a HUGE but, with very few exceptions, you can't actually "user-agent detect". You can "user-agent inference" or "user-agent sniff" by looking at some convoluted and spoofable UA string. But this is not, in the same sense of the word as applied to features, a "detect", it's at best an inference.

    Contrast UA string inference/sniffing to say looking at `window.opera`. This property is there for the express purpose of identifying the browser/user-agent to JavaScript undeniably and unspoofably. The object is a special object, who, when called against `Object.prototype.toString()`, returns the unspoofable native "[object Opera]" value. If that's not "user-agent detection", then I don't know what is.

    The fact is, THIS IS "user-agent detection". And I fully support that kind of user-agent detection (for certain purposes). But looking at a ridiculously complex and ever-changing and unreliable free-form string to try and divine what the browser/OS/etc is? That's not even close to the standard of the term "detection".

    Because real "user-agent detection" is so hard to come by, we try to accept by proxy the process of parsing a UA string as detection. And time and time again the community proves this is not viable.

    I say, "down with user-agent inferences" and "up with REAL user-agent detection". And in the meantime, until the browsers give us real user-agent detection, I'll stick with feature-detection as the next best thing. "User-agent inferencing is WAY down that list of acceptable options."

    What we should be demanding of browser vendors is that they give us real user-agent detection. Then we could finally put this whole debate to bed once and for all.

  10. Kristiaan Van den Eynde

    If I recall correctly, Opera tends to (or used to) spoof User Agent strings for various websites.

    What happens if you cache certain test results for, let's say, Firefox and in walks Opera saying "Hello yahoo.com, my name is Firefox!"? Or what happens with any UA spoofing for that matter?

    The only reliable fallback would be to still test if the feature is actually there, which beats the purpose of caching test results in the first place.

  11. Yoav Weiss

    If you exclude the latest stable version of Chrome and FF from detection, then you're basically boosting performance only for old IEs, which is not ideal. If a certain feature changed during the lifetime of the browser (because of a bug), we can simply declare it as unknown, and feature-detect it.
    "REAL user-agent detection” as Kyle says is more reliable, but for many things it is too late. You may have already sent the 100K script to support IE6, or may have sent those excessively large images only to be scaled down on a mobile device.
    What we can do is have a small inline script that adds the "really detected" UA as a small cookie to the page's domain. We can also add the UA as a URL parameter for cross-domain resources. This script should run before any other JS/CSS/images are fetched, to avoid waste of BW. Then we can RELIABLY detect the user agent on the server side and:
    * Send a JSON with it's known features, avoiding feature detection only on those features
    * Avoid sending useless data
    * Feature detect whatever's left

    IMO, this solves the problem also for UA spoofers, and benefits everyone. We could also add a web service to fetch the "known-features" from (for example as part of BrowserScope).

  12. Kyle Simpson

    +1 to Yoav's idea. I think that makes a lot of pragmatic sense.

  13. David Mark

    No, you can't infer anything from the UA string. Worried about sending unnecessary workarounds for old IE versions? Use Conditional Comments. That's what they are there for (and they've worked beautifully for over a decade). Nothing but the old IE browsers will parse them. That's 100% for sure (and I'll take those odds any time). On the contrary, there are plenty of IE spoofers (users and browsers) out there. They'll fool you every time until you realize...

    You don't need to parse the UA string and you don't need to take over loading duties from the browser (which will only slow down loading). In fifteen plus years, I've never had to do either (and my stuff runs on virtually everything). The Dojo guys, on the other hand, mashed up lots of IE sniffing in their code, so now they have to figure out a way to make it appealing today (which is impossible). They've got grand plans of "fast" browser sniffing loaders, but they are simply barking up the wrong tree (again).

Understanding JavaScript Promises E-book Cover

Demystify JavaScript promises with the e-book that explains not just concepts, but also real-world uses of promises.

Download the Free E-book!

The community edition of Understanding JavaScript Promises is a free download that arrives in minutes.