Tag Archives: dropbox

Dropbox: How to REALLY not run a public beta

Man, am I having a bad week. No sooner had I been burned by beta testing Mendeley, I get absolutely toasted by trying out Dropbox. The goal of Dropbox, in case you haven’t heard of them, is to allow one to keep a folder of files transparently synchronized across multiple computers (and the web). In theory, all your computers will have the same set of files, transparently maintained by the Dropbox daemon (background program) running locally. Awesome, right? No more treking around an external drive, no more juggling multiple versions of files when you need to work on something on two or more platforms. It also handles folder sharing among users.

Fantastic. Awesome. Terrible. Sync is one of those killer applications that usually ends up killing the user, like a hand gun being passed around at a frat party. The result is often a solution worse than the problem, with data corruption and inconsistent data between locations a common failure mode. People have gotten it right recently, however. Apple has done a very good job with MobileMe, at least in terms of sync reliability. I have my complaints about them, but they’ve never messed up my data, even after more than a year and probably over 1000 synchronizations.

I was, therefore, perhaps a bit too unwary in trusting my data to Dropbox. I also figured that if they are already charging people they must have the bugs worked out, right? This is people’s data we’re talking about. No company is going to take control of your data with a product that is still buggy, right? Right?

Wrongo. It took me only two weeks of using Dropbox to find out I was grossly mistaken. Yesterday I moved a few large folders around on my linux machine, and the result was hopeless corruption of my Dropbox file system, with the server basically throwing its hands up and locking itself in the bathroom (folder “rejected by server”). Note that my problem wasn’t caused by conflicting edits made simultaneously to the same data on different devices (the typical difficulty with synchronization). Dropbox failed spectacularly just because I made multiple changes to ONE of my local copies and it got hopelessly confused. (Fortunately, I was able to restore everything from a backup on another computer, so you can stop sending cards and letters. I appreciate the sympathy, however.)

Talking with tech support, and looking at the forums, this is clearly a known issue that many users are having. A known issue that results in database corruption if you have the audacity to do something insane like move folders around! And they don’t mention this in the FAQ, let alone bright red flashing letters on their web page. Did I mention they are accepting legal tender for this product?

It seems to me their fundamental sync architecture is flawed (it apparently doesn’t record file operations in a way that is guaranteed to preserve the transformation of the file system from one state to another). I wonder if they don’t warn against this in their FAQ because they don’t want their VC funders (who are surprisingly big names) to know they are in over their heads, or if they are so far in over their heads they don’t know they have a problem. To do file sync, as far as I can tell, you basically have to be able to hook into all possible I/O operations on the disk and make sure you record every single change, in order, so that those operations can be “replayed” on the remote copy. I can’t think of another way to guarantee consistency. Maybe the folks at Dropbox found a way to avoid this complication. Maybe they were wrong. I’m not saying I’d be able to do better, and I know it’s a notoriously hard problem, but I’d hope that if I couldn’t solve it I’d at least know that I hadn’t. And I certainly hope I wouldn’t look for funding and customers before I’d solved it.

Looking at the Dropbox staff list, I should’ve been more careful. Its CEO and CTO seem like great guys, but they also look like they just started shaving last week. Their CTO, and well over half their staff, are very young, very recent MIT dropouts. With all the new humanities course requirements, I guess you can’t trust MIT undergrads with your data until they’ve gotten at least an MS. Either that, or MIT must cover some pretty important material senior year in Course 6. The fact that the first several iterations of their Mac OS X client didn’t even synchronize all possible parts of a file (despite not informing the user of this) should’ve been a red flag that Dropbox was not being run with a whole lot of discipline or adult supervision.

Am I just writing this to complain? Of course not. I would never do that! I’m writing this with the hope that my experience may prevent at least one other person from wasting their time with Dropbox, or losing their data. I’m also writing about this because my experience with Dropbox, as well as Mendeley, bring up interesting questions about VC technical vetting, a topic which I will discuss in my next post.