Researching Link Rot

This week I'll be running a little experiment in link rot, in preparation for an upcoming conference talk. I'm interested in quantifying long it takes for a typical link to go offline, and if this rate is steady or changing with time. Pinboard now has enough bookmarks (about 100 million) to make this information interesting.

This research is important because we don't have a lot of data for how link rot affects stuff people actually care about. Presumably things that you've bookmarked are more important to you than some random URL off the street.

To run the experiment, I am going to be drawing a few thousand links at random from the entire pool of Pinboard bookmarks. This will include private bookmarks, which make up about half the Pinboard collection. I'll use a combination of scripts and my own weary hands to figure out what proportion of links still point to the original material saved. The URLs I look at will not be associated with your username, and no one except me will look at them.

I will publish some aggregate information about what I find, and use it to seek glory, and persuade people to sign up for archiving. But I won't release anything that could lead back to specific users or links.

If you are uncomfortable with this research and wish to opt out, please email me with your username, and I'll keep your bookmarks out of the pool. If you have questions, ask me on Twitter or email me privately, and I'll be happy to answer them.

As a reminder, the site's privacy policy is here.

—maciej on August 09, 2014