The Exodus from Del.icio.us to Pinboard
Three days ago Maciej Ceglowski was running a small better-del.icio.us bookmarking service (for those who wanted (a) speed, (b) protection against linkrot, and (c) not to be embarrassed by what they had bookmarked) with perhaps 5K users and perhaps 5M bookmarks.
Then disgruntled ex-Yahoo employees leaked that Yahoo was "sunsetting" delicious, and John Gruber wrote:
Daring Fireball: Now’s Probably a Good Time to Link to Pinboard Again: I’ve previously linked to and recommended Pinboard, Maciej Ceglowski’s “antisocial” bookmarking service. It’s not a copy of Delicious, but rather more like a re-imagining of Delicious. If you use Delicious and regret its apparently imminent demise, Pinboard is probably what you want. Delicious-like features and interface (including a clone of its API), and Delicious importing. And, the sooner you sign up for Pinboard, the less you pay.
Selections from Maciej's twitter feed from that moment on:
it's nearly 2011. Why do I still have to ask my computer's permission to unplug an external drive?
@niels_k I understand the technical issue, but it's 2011. We have the technology
@cdoussin welcome aboard!
Apparently delicious is done: http://yfrog.com/f/h3z89p/ This sucks. Yahoo finishes the process of turning gold into lead.
@mlednor we should be able to grow by a factor of ten or so before there would need to be changes. I will pause signups if necessary.
@patbrumfield you should be able to import all your delicious stuff
we'll set up some tools to make migration easier for people from delicious (like being able to find people by their del username)
@zyrcster no, but I'll hack something up to make that possible now that everyone is migrating
if we weren't a paid service we'd be down by now...
@nathansmith mixed feelings, really. Also high stress as I try to manage the traffic
mirroring from twitter and other outside services is going to be off for a few hours so that things stay fast for everyone. Crazy traffic
people balking at the signup fee - please think about why delicious is being shut down
@Panicnfreakout we haven't crashed
RT @kb: Hey, y’all. Happy you’ve found a del alternative, but I’m trying to bookmark shit here. Ease up on the @pinboardIN servers, maybe?
@mormolyke I think I'll pass on taking scaling advice from @twitter :-)
@devrandom send us email if it doesn't complete soon. The poor importer is working like crazy this afternoon
there are worse things than being DDOS'ed by people trying to give you money
Sorry for the delay in imports - we're adding bookmarks as fast as we can consistent with not slowing down the site
we have added just short of one million bookmarks today
@anildash ha, thanks Anil! I will reveal myself to be a Yahoo! skunkworks operative in due time
@balaji_dutt yes sorry about that, we had to revive our poor mail server and it took a while to get outbound mail back up
@KenBavier yes a bunch of background services are disabled so the servers don't crash. Will bring them back up ASAP
@mark_philpot heh, thanks! I'm very proud that we kept the servers up and fast today through what was basically a redirect from Yahoo
I love you guys but I also can't wait for you all to go to sleep
@thanland thanks man! it's been an interesting adventure in systems administration
@kevadamson yeah we're actually Yahoo Bookmarks with a crappy stylesheet as disguise
@kopischke thanks, I really appreciate it. Astonished we stayed up
Import queue is still about eight hours long right now
@gfmorris thanks man! luckily I am well taken care of, and this is a fun way to pull an all-nighter
@martijnvdven it's going to be a few more hours before we start crawling outside services again
@digitalbdesigns my "office" is my desk and I am eating a bleary-eyed bowl of cereal after a luxurious 90 minutes of sleep
Alright, we have got the import queue down to four hours from eight. Continuing to attack it. Thanks to everyone for being so patient!
@JimRoepcke we're not set up yet for this high volume of writes (5 million bookmarks in a day). Otherwise the server is actually doing fine
Good luck today, @zootool - I feel your pain!
Added 6.7 million bookmarks in the last 24 hours. That's more than in the entire preceding year
okay, import has caught up and things are back to normal. Thanks to everyone for their patience, and welcome
The funny thing is, our import is really pretty peppy - just not when ten thousand people try it at once
@zenom_ thanks, I appreciate it. It was a nervous 24 hours over here
@ichilton mysql is perfectly fine for the kind of write traffic we're ever likely to get. The problem was my own bad code around it
@andydavies I'll do a technical post-mortem for curious people in a few days
Now that I've gotten some sleep I feel like I can use sudo again
And the status update:
Welcome! (Pinboard Blog): As you may have seen on Twitter, the last 30 hours or so have been hectic. Our new users brought with them over seven million bookmarks, more than we had collected over the entire lifetime of the service. Traffic to the site was over a hundred times normal for the best part of a day.
My priority during this surge in traffic was to keep the site up and responsive, so that new users could take a look around without locking our grizzled veterans out of their bookmarks. This meant suspending a lot of normal background tasks like search, archiving, and polling outside feeds, so that the database could concentrate on adding bookmarks. It also meant keeping a very long import queue - at times over eight hours long - in order not to bog down the site.
Longtime users of Pinboard know how much pride Peter and I have taken in being the fastest, lightest bookmarking service around. Now that we've digested that big meal of bookmarks, I hope our new users will get to see this side of the site as well. It's amazing how peppy things can get when you're not saving 78 bookmarks per second.
With the traffic calming down, I've been able to start bringing background services back up. Tag clouds should finish generating later tonight; search will be up-to-date in about a half hour, and tomorrow morning we'll start crawling links again for our archival users. I've started the various feed crawlers and users should start seeing items from Twitter, Instapaper, Google Reader, Read It Later and Delicious appear in their feeds within a few hours as well.
In the future we'll have a status page so that users have a clearer idea of what's going on behind the scenes. And we'll make a number of changes to the signup process based on what we noticed yesterday.
If there are any remaining signup issues, bad imports, or plain old bug reports, please email us at [email protected] and we'll do our best to help.
And now I'm going to sleep.
Happy bookmarking!
—maciej on December 18, 2010
