In defense of localStorage

Earlier this week, Chris Heilmann wrote a blog post entitled, There is no simple solution for localStorage1 in which he decried localStorage as slow and encouraged everyone to stop using it. Surprisingly, in a post about performance, there was no mention of what “slow” or “terrible performance” actually meant. Performance can’t be discussed in a vacuum, which is part of what made my reaction to his post one of confusion more than anything else.

What is slow?

So does localStorage have a performance problem? Quite simply, I don’t know. Is storing and retrieving data from localStorage slower than that of a regular, in-memory object? Yes. I wrote a post about this for 2011 Performance advent calendar2. In fact, it’s quite a bit slower when reading data out. My conclusion was that you should try to limit reads by storing multiple pieces of data in the same key. But as with most performance metrics, this really only matters when you’re performing the same operation multiple times in a row. If you’re only ever reading one value or writing one value, you’ll likely never run into a performance issue regardless of the data size or what’s going on with your system.

So localStorage is slower than using an in-memory object. Cars are slower than airplanes. What does that tell us? Not a whole lot.

Pain points

The fact of the matter is that localStorage reads from and writes to disk, which is always slower than an in-memory operation because there’s hardware involved. That’s the first problem. The second problem is the per-origin nature of localStorage. This characteristic means that two browser windows or tabs open to the same origin can both be reading from or writing to the same localStorage at the same time. That, in turn, means the browser needs to be incredibly smart about how it performs each operation. If tab A is writing to localStorage around the same time that tab B is reading from localStorage, which operation should happen first?

Each operation, read and write, then needs to lock localStorage to ensure data integrity. This is actually a big issue. Interacting with files is also dicey since another system process might also be working the same file. Consider a simple write operation:

localStorage.setItem("foo", "bar");

This operation is synchronous, meaning that the UI thread is blocked in the browser until the write is complete. No further JavaScript will execute and no further UI updates drawn. During this one method call, several things happen:

  1. If localStorage is busy with another window or tab, then wait. This is problematic since there’s no way to know how long this will be.
  2. If the file is busy, then wait. The files may be scanned by antivirus, be included in a backup job, etc., and may therefore be unavailable until such operations complete. This is also problematic because it’s unpredictable.
  3. Open the file for writing.
  4. Seek the right spot to write to the file.
  5. Write to the file.
  6. Close the file.

Only after all of that completes can the browser continue on to execute other statements. So really, the issue isn’t that localStorage is slow, it’s that localStorage must necessarily block on each operation to maintain data integrity.

Compared to cookies

The closest comparable API for JavaScript is the cookie API (though calling document.cookie an API is incredibly generous). Cookies are also name-value pairs, albeit with some additional metadata, which uses files as storage and must be synchronized across browser windows and tabs. I was surprised that Chris didn’t compare localStorage to cookies since the API was clearly meant to move us from storing client-only data in cookies to storing it in localStorage. It’s no accident that the localStorage API looks a lot like various cookie APIs.

When I created a benchmark3 to test localStorage against cookies for reading and writing, the results were quite interesting. Internet Explorer, Chrome, and Safari (including iOS), reading cookies was slower than reading from  localStorage and writing to cookies was much slower than writing to localStorage. Firefox and Opera exhibit the same performance characteristics on writes as the others (with cookies being slower), but reading from a cookie is faster. So in many cases across browsers, localStorage is actually a performance improvement over using cookies with JavaScript.

APIs

The reason localStorage is popular is partly due to its simplicity. That simplicity is by design and was first designed and implemented by browser vendors, which is why it seems strange that a browser vendor would now lead the charge against an API it had a hand in creating. Yes, humans create browsers and humans can make mistakes, but I don’t think the design of localStorage is a mistake.

As I was reading over Chris’ plea to look for alternatives, my engineer brain kept repeating to myself, “this is an implementation issue, not an interface issue”. Firefox is choosing to preload the localStorage data to improve read performance later, but that’s an implementation issue. Likewise, the need to read and write synchronously is an implementation issue – many forget that Internet Explorer 8′s implementation of localStorage actually wrote asynchronously. That an implementation-specific detail. Why not make all writes happen asynchronously and just keep a copy of the data in memory so it can always be read correctly regardless of the write state?

I’m not saying that this is necessarily an easy problem to solve; what I am saying is that the API design works well for developers, and for that reason it’s worth looking at the implementation details to figure out if there’s an efficient way to hide the warts of the system from web developers.

The proposed alternative, IndexedDB, is perhaps one of the worst API designs I’ve ever seen. To read and write a single piece of data requires way too many lines of code, ensuring that the API won’t be used by most developers until someone comes up with a library to abstract away the horribleness. I understand the rationale behind providing such a low-level, asynchronous API (I was even part of the discussions held at Mozilla around web databases), but this absolutely stinks of browser developers creating an API that’s easy to implement rather than creating an API that’s easy to consume. This is the opposite of how good APIs are made. IndexedDB will never be a replacement for localStorage, it’s just too complicated for most uses.

Non-blocking localStorage

As discussed previously, the real issue is that localStorage blocks on reads and writes, and the amount of time it blocks can’t be determined ahead of time. If this turns out to be a concern for you (after benchmarking, of course), then the solution is to use a non-blocking  localStorage mechanism. When you hear the term “non-blocking” these days, you should immediately be thinking about Web Workers.

In the near future, I believe that client-side architectures that perform I/O should perform all of that I/O in Web Workers. That means all of your localStorage, XMLHttpRequest, Web Socket, etc., I/O should be done inside of a worker. Basically, you should be able to do something like this:

var worker = new Worker("io.js"); 

worker.postMessage({ 
    type: "write", 
    src: "localStorage", 
    key: "foo", 
    value: "bar"  
}); 

worker.postMessage({ 
    type: "read", 
    src: "localStorage", 
    key: "foo" 
}); 

worker.onmessage = function(event) { 
    alert(event.data.value); 
};

All of the reading and writing would be done off of the UI thread, so blocking really doesn’t matter. I know I’m not alone in thinking this is the way of the future as the IndexedDB spec has a whole section on synchronous APIs available in workers4. Having synchronous APIs for IndexedDB makes it less horrible to deal with, but you need to use them in a worker. This hasn’t been implemented by all browsers yet, but should be coming soon. Add to that the concept of shared workers, web workers that are shared amongst all tabs with pages from the same origin, and you have a great recipe for resolving a lot of I/O issues.

Workers currently have access to XMLHttpRequest, Web Sockets, File Readers, and the such…and yet no access to localStorage. Why? This is really the solution to the problem: don’t throw away a great API because in some cases it will cause issues. Instead, make it available in workers so that we have an option for moving the reading/writing off of the UI thread.

Note: It’s possible that the cross-domain localStorage approach I wrote about previously5 might provide some non-blocking benefits. The cross-frame postMessage() API is asynchronous, but I’ve not figured out a good way to test if the containing page freezes if an iframe from the same domain accesses localStorage .

Conclusion

Asking web developers to give up localStorage is ridiculous. Are there problems with the API? Yes, indeed there are. Are they bad enough to abandon using it altogether? Absolutely not. Claims of terrible performance haven’t been substantiated. Despite the complaints of browser developers as to the technical difficulties, there are no good alternatives to localStorage. We could always go back to using cookies, but as the previous benchmark shows, that doesn’t necessarily guarantee better performance. And IndexedDB is a non-starter because the API is too complex for most use cases.

So to Mozilla, and the other browser vendors out there, you’re a victim of your own success. You wanted to create an API that could be used in place of cookies for storing client-side data, and you created something great. The interface is friendly to web developers and that’s why it’s had such rapid adoption. You’re all quite smart, capable people and I’m sure you can come up with better ways to implement the API than what we have today. And also, make localStorage accessible in Web Workers, please.

Update (8 March 2012): Fixed typos and added shared worker reference.

Footnotes

  1. There is no simple solution for localStorage by Chris Heilmann

  2. localStorage Read Performance by Nicholas C. Zakas

  3. localStorage vs. cookies by Nicholas C. Zakas

  4. Indexed Database – Synchronous APIs

  5. Learning from XAuth: Cross-Domain localStorage by Nicholas C. Zakas

Comments

  1. Mikeal Rogers

    There are other problems with localStorage.

    * Size limits tend to be very low, even in a web view in an iOS app.
    * Writing to the same localStorage from multiple tabs causes bad things to happen, especially if you're using an API like lawnchair or couchie that have higher order ideas about what the "database" is and their own consistency plan.

    I also don't understand this aversion to using non-blocking APIs. The reason people aren't using IndexedDB yet are:

    * It's not on a single mobile browser.
    * It's still changing and not consistent across desktop browsers.
    * It's ridiculously complicated and verbose (being callback based has little to do with it.)

    The WebWorker API sync API for IDB seems stupid. The WebWorker API you have here for localStorage makes sense since there is no native non-blocking API, but it's just insane to me that you would spawn an isolate just so that you can write in a blocking manor within the isolate when you have access to fully working non-blocking API already in your main thread.

  2. Nicholas C. Zakas

    This size limits aren't arbitrary problem, that could easily be an issue for any client-side storage solution. Writing from multiple tabs is, as I mentioned, and implementation problem more than an API problem.

    The problem with trying to push an asynchronous API is that no one will use it when there is a workable synchronous API available. And as I said, I'm not sure that being completely synchronous or completely asynchronous is the answer. There is a nice middle ground in there. Why can't we consider all reads a d writes being performed in a single JavaScript job to be a transaction that's only committed once that job is finished?

    And the IndexedDB API suffers from more than lack of support. It's just all-around too complex for most web developers to bother with. Especially if there's an option of cookies or localStorage available.

  3. Arjen

    fyi: You're missing the word 'think', I think, in "but I don’t the design of localStorage is a mistake."

  4. Daniel

    For what it is, I think localStorage works just fine, I've had lots of "smaller" excellent use cases where it simply a good replacement for cookies.

    It's a lightweight tool, with a very easy-to-use API, and that comes with a few drawbacks. IT WORKS JUST FINE.

    If your trying to do something it wasn't designed for, find another tool for your problem. Maybe IndexedDB isn't quite mature yet, but it will be, some day.

  5. Curly Brace

    Thanks for the post, Nicholas. It brings an order to the sudden anti-localStorage campaign.

    Just a small correction needed in this sentence in the first paragraph under API subtitle: "but I don’t the design of localStorage is a mistake".

  6. Yotam Praag

    I agree with NCZ on this post.

    After reading Chris Heilemans post I was asking the same question as to why a browser vendor would ask developers to stay away from a perfectly good API?
    Put aside the "bad performance" part and this is a great API to use, simple quick and correctly solves the inherent problems cookies have.

    You would expect a post from Mozilla to explain that there is a performance problem and they are trying to fix it. Not the other way around.
    It's obvious that this is an implementation problem in the browser. We're not talking about vendor prefixes which have two (three?) sides to blame.

    A simple process such as storing textual data in key/value pairs should not cause the developer to think about performance unless the data is ridiculously large, in which case you should turn to a solution that accommodates large data blocks like IDB.

  7. ludo

    Hi,
    Thanks for this great article (and the previous, and future ones)
    Just wanted to make you notice a typo.
    In the last sentence of the first paragraph of 'API' section. A verb is missing. (I think it is 'think')

    "...but I don’t __ the design of localStorage is a mistake."

    Please moderate and do not show this comment :]

  8. DaveC

    The Web Worker route was precisely what I was thinking whilst reading Chris' article - I did start to test whether localStorage was available inside a Worker but didn't really have time and didn't finish.

    I have to agree with you, this is an implementation issue - the fact that developers are trying to use it to store images or other binary files, simply points to what localStorage should be capable of doing.

  9. Aaron Rankin

    Nicholas,

    Nice job of spelling out the theory (memory I/O vs. disk I/O) in black and white. I'd go one step further and first point mention that computationally, both an associative array and localStorage are (I think) O(1), so they look the same on the surface. So the matter of I/O is really what bites you.

    And to your real-world points about disk slowness, I'd add virtual memory. Many computer users are naive enough to run their machines in a perpetual memory swapping party, because they leave lots of apps open, never reboot and don't have great hardware to begin with. With localStorage, you're contending for disk I/O with that. It's just one more guy in line for the dwindling keg at an overstuffed house party.

  10. Roland Bouman

    Hi!

    Interesting to hear another point of view in this recent discussion

    I agree that indexedDB is a terrible API, and that localstorage API should not simply be discarded. I also agree that claims that it is slow should be substantiated, and I presume it very possible that some of the current perceived problems are due to implementation rather than API.

    However, I don't agree with this:

    "If this turns out to be a concern for you (after benchmarking, of course), then the solution is to use a non-blocking localStorage mechanism. When you hear the term “non-blocking” these days, you should immediately be thinking about Web Workers.

    In the near future, I believe that client-side architectures that perform I/O should perform all of that I/O in Web Workers. That means all of your localStorage, XMLHttpRequest, Web Socket, etc., I/O should be done inside of a worker."

    There is an inherent difference between asynchronous XMLHttpRequests and asynchronous storage. When you do multiple HTTP requests to the same URL, most devs will agree that they cannot have any expectations when the requests will actually be handled, received by the host, and in what order responses will be received. But if we were to have the same low level of expectation of a storage medium, something is going wrong. You can't just have different tabs spawn a bunch of workers that independently write to the same shared storage and expect the results to be reliable.

    If I do a write "now", and read that key back "later", I should retrieve it, provided no other process removed it. With workers, there is not going to be "now" and "later", there is only going to be "eventually". If we cannot rely that we will be able to read back our writes, our storage is next to useless. There are ways to make storage io non blocking, but simply wrapping everything in a worker is not the solution. There has to be some mechanism that either provides locking, or versioning, and some sort of conflict resolution in the more relaxed scenarios.

    Simply having everybody write and read independently and asynchronously is not a solution to any problem.

  11. Joe Larson

    I've been able to make serious performance improvements in several apps using localStorage (for instance, newly updated http://unicodinator.com). Your car vs. airplane comparison is apt: going back to the server is like sending it via snail mail. Using cookies for this is like... riding a fast reindeer? Cookies just aren't designed for this use.

  12. Joe Larson

    ..also, the issue with different tabs and windows is, I believe, something that's important to consider but not usually an issue for most apps, which generally are going to be retrieving and storing stuff due to user interaction, which can only be happening in one tab/window at a time.

    Of course if your doing loading in the background, etc, this could get in the way -- but fine, then you figure out how to deal with that. I'd imagine for most apps it's not a real concern.

  13. Nicholas C. Zakas

    @Roland - There is more to this approach than what I covered here, and I'll do a followup blog post on the technique discussing the ins and outs. There is a synchronicity issue with the hacked together example I use in this post, but that doesn't mean the underlying approach is incorrect. In my future world, the I/O happens in a shared worker, which becomes an I/O hub for your application. Every instance in every tab hits the same shared worker for I/O, which actually ensures that everyone is getting up-to-date information and no one is out of sync. Then the worker can manage expectations about data consistency. I rarely simply write to or read from localStorage, I typically use that as a backing store for some in-memory data structure. That way, I'm always reading/writing with memory and only periodically caching to disk.

  14. John Allsopp

    Hi Nicholas,

    we posted something similar on the same day.

    In my case I focussed on the assertion that performance is an issue. Yes, in theory, with a synchronous API, LS could have performance impacts. But I wanted to know in practice whether it did. So it took about 5 mintues to write a test page. Not sure why all those suggesting performance is an issue didn't actually do some research? Anyhoo. The results are that 10K writes to localStorage (different key) in a single tight for loop takes between 50ms and 2s (depending on browser) on my reasonably modern MacBook Pro, and isalso O(100ms) on iOS devices (iPad 1, iPhone 4G).

    Results here

    http://www.webdirections.or...

    So, I don't think for most developers' purposes, peformance of LS is an issue at all.

    I 10% agree that most of the concerns expressed are in fact implementation details for browser develoeprs, who in my experience (knowing several personally very well) are very smart people, who know a a lot and care a great deal about issues like performance, real and perceived.

    Thanks for pouring some oil on these suddenly choppy waters

    john

  15. Nicholas C. Zakas

    I saw your post this morning when MozHacks tweeted both of ours. :) Thanks for doing your benchmark, it's quite useful to have that data handy. The question I always ask with performance is this: how frequently are you doing the "slow" operation? Doing 10,000 writes to localStorage is probably something that never happens (I've currently seen a max of 20 keys stored on certain sites). So given that 10,000 writes can happen in a max of 2s, that means you're looking at 100ms to write 20 keys serially. In practice, most sites don't write all of their keys serially, so you really want to talk 5ms or lower per write as a max, which isn't too bad when spread out.

  16. Marco Campos

    as i have first tought when i saw the article saying it is bad, i tought about why does i/o blocking realy matters? as i'm not thinking in huge stuff to store, the same blocking problems happens when i make a big query in a database on the server-side (saying the database work like localStorage, doing all this i/o), of course it needs to be server-proportion big to be a problem.

    i think one of the solutions to the implementation is to think as a database with caching and aging of data, it can have relevant data up on the browser start or on the page first load in this execution and be aging the data acording to witch site it cames from to know if it should be loaded.

  17. Paul Bakaus

    Hi Nicolas,

    thanks for writing this. I couldn't agree more and posted a follow up that will likely annoy some of my browser vendor friends...but that's the whole point: http://paulbakaus.com/2012/...

    Looking forward to catch up with you in SF again soon!

    Cheers,
    Paul

  18. sylvain

    Why don't vendors create a non-blocking way to access local storage? They should run their local storage read/write implementations in RAM so that developers don't have to worry about performance, and then work their own queuing mechanism that reads the entire session (max 5MB anyway) on session start and then writes to the hard drive as needed.

  19. Constantine Vesna

    ... it seems that Mozilla devs are afraid of situation when every site will be using 5MB of their quota to the full, and thus consuming gigabytes of drive-space. "Slowness" was just a scare-crow, to somehow defer that moment - but you, Nicolas, just ruined it ;)

    I think Steve Souders gave one really insightful comment on topic - "localStorage is bad, when its used as cache":
    http://hacks.mozilla.org/20...

    - browser vendors should improve caching,
    - async set/get with call back (via new methods, or via extra flag, or by specifying callback) should be added.
    - 'last accessed on' timestamp should be somehow visible to user, when he will eventually want to purge his localStorage (this can also be done automatically, after, say, 90 days)

  20. ChrisB

    > [Daniel[] For what it is, I think localStorage works just fine, I’ve had lots of “smaller” excellent use cases where it simply a good replacement for cookies.

    That last part hits the nail on the head: As a replacement for cookies, localStorage is excellent. Local data like UI states etc. can stay local - no need any more to roundtrip it to the server and back on each (or at least many) HTTP Request.

    If you're using it for "bigger" chunks of data - Chris Heilmann wrote about storing images into it - it might just be the wrong tool for the job. You wouldn't (want to) store image data in cookies either, would you?

    > [Nicholas] This size limits aren’t arbitrary problem, that could easily be an issue for any client-side storage solution.

    Let's not forget, "storage space" regarding cookies is also limited. As said, you would not normally squeeze whole images into them ...
    What amount of data you can store in cookies, you should easily be able to put into localStorage, too.

    Expiry is/might be a problem though.
    That could be handled either on API side (expire date as with cookies) - but that would make this lovely simple API more complex again; or implementation side - let me as browser user decide, after which interval of time of not visiting a website again localStorage data is to be discarded.

    Roundup: Use localStorage as an excellent replacement for cookies and be happy with it; don't expect it to handle (much) bigger use cases and thus make it an I/O / file system / database replacement.

  21. MarcelS

    @ChrisB, 100% agree! After I read Chris' article, I thought to myself "Why would you do this sort of stuff with localStorage anyway, just wait for IndexDB?". I think it's a great solution to managing user preferences in a web app, which is how I am using it. Stateful boolean values that don't suffer from the problems of cookies = excellent. It could use a little more fine tuning as lots of commenters have pointed out, but generally great API.

  22. richtaur

    We shipped an HTML5 game that was writing to localStorage every tick (an issue we fixed later), but we never noticed any slowness in Chrome. The localStorage hate sounds assumptive to me without any real-world tests.

Understanding JavaScript Promises E-book Cover

Demystify JavaScript promises with the e-book that explains not just concepts, but also real-world uses of promises.

Download the Free E-book!

The community edition of Understanding JavaScript Promises is a free download that arrives in minutes.