Pinboard - bookmarking for introverts Public weblog for Pinboard en-us Sunsetting Delicious Since the birth of the site, Pinboard has always offered a Delicious sync feature. You could enter your Delicious username on the settings page, and the site would periodically poll your public Delicious feed and add any new bookmarks it found.

Because Delicious appears to be in a terminal coma, and because working around bugs in their RSS feed has historically consumed a lot of development time, I am going to turn Delicious sync off effective October 1, 2014. If you want to keep hooking the services together, you will still be able to do it through an outside service like IFTTT or Zapier.

Of course you will still be able to import all your Delicious bookmarks, as well as export Pinboard bookmarks in a format that Delicious can (sometimes) read. The only thing going away is the automatic sync. ]]> (Maciej Ceglowski) Thu, 21 Aug 2014 17:09:27 -0800 c1224bdf92489e9f38541e2f0d55730b966fa26f Researching Link Rot This week I'll be running a little experiment in link rot, in preparation for an upcoming conference talk. I'm interested in quantifying long it takes for a typical link to go offline, and if this rate is steady or changing with time. Pinboard now has enough bookmarks (about 100 million) to make this information interesting.

This research is important because we don't have a lot of data for how link rot affects stuff people actually care about. Presumably things that you've bookmarked are more important to you than some random URL off the street.

To run the experiment, I am going to be drawing a few thousand links at random from the entire pool of Pinboard bookmarks. This will include private bookmarks, which make up about half the Pinboard collection. I'll use a combination of scripts and my own weary hands to figure out what proportion of links still point to the original material saved. The URLs I look at will not be associated with your username, and no one except me will look at them.

I will publish some aggregate information about what I find, and use it to seek glory, and persuade people to sign up for archiving. But I won't release anything that could lead back to specific users or links.

If you are uncomfortable with this research and wish to opt out, please email me with your username, and I'll keep your bookmarks out of the pool. If you have questions, ask me on Twitter or email me privately, and I'll be happy to answer them.

As a reminder, the site's privacy policy is here.

Thank you! ]]> (Maciej Ceglowski) Sat, 09 Aug 2014 13:33:23 -0800 cec818bf660aadabd8fee44a676b682db07a2f79 Pinboard Turns Five Today marks five years since I launched the website that my mom still refers to as 'the other bedbugs'. Happy business birthday to me!

Any site that aspires to be an archive starts life with a credibility problem. The Internet is strewn with the corpses (or in some cases, zombies) of sites that once promised to save your links forever. As people keep discovering, building a bookmarking site is easy, but making a business of bookmarking is hard. Like one of those leathery, spiny plants that is able to thrive in the desert where everything else dies, I have tried to find ways to adapt to this hostile business environment. And I have feasted on the flesh of my rivals!

I raise this brimming skull to the awesome group of users and fellow-travelers who have made it possible.

It's my tradition to post updated statistics about the site:

2010 2011 2012 2013 2014
bookmarks 3.5 M 27 M 53 M 76 M 97 M
tags 11 M 76 M 135 M 178 M 212 M
active users 2.8 K 16 K 23 K 23 K 24 K
bytes archived 200 G 3.0 T 5.9 T 8.8 T 14.2 T
downtime 6 h 29 h 22 h 12 h* some?
unique URLs 2.5 M 16 M 32 M 48 M 63 M

The biggest surprise (to me) is how predictable Pinboard has been over the past three years. Users come and go, like on every site, but the number of active users stays roughly the same. And the site makes roughly the same amount of money (around $200K) every year.

If you've ever run a small website, you'll recognize how weird this is. Typically everything in a small project—traffic, user count, revenue—is spiky. You spend a long time treading water and then big events happen that dominate everything else. This was true for the first two years I ran the site, but since then, things have settled down remarkably.

I regret that I totally forgot to keep downtime stats this year. There wasn't a lot of it, but I should probably track it better so I can brag about it next year, unless it goes up, in which case I will never mention it again.

Now back to some beard-stroking:

I see my role much like a small-town praire banker in the 1880's. My job is to project an aura of calm, solvency, and permanence in an industry where none of those adjectives applies. People are justifiably risk-averse when it comes to their bookmarks, and they are looking for stability. This means several things at once:

On the most basic level, the site just has to work.

On the design level, it means not futzing with stuff unnecessarily, except for bug fixes and basic improvements. Luckily there is so much work to do on Pinboard that I am immune to the temptations of a redesign. If there is a feature (or bug) you love in 2014, chances are excellent it will still be there, like a cherished friend, years from now when your trembling and aged hands go to make that final click.

Finally, there is stability on the business level. This means persuading people (including myself) that I am going to stick around, and then actually earning enough money to do that.

The money part turns out to be easy. People will pay for a decent service. As long as you stay small and don't forget to have revenue, you too can build a bookmarking website. There is plenty of room to specialize!

My strategy of pre-emptively antagonizing anyone who might possibly have an interest in acquiring or funding the site has worked wonderfully. In five years, I haven't received a single email from an investor or potential acquirer. The closest I came was a few months ago, when the new Delicious owners reached out to me about providing "vision", but I think they were just unfamiliar with my oeuvre. They learned quickly.

So the biggest risk in a project like this remains burnout.

Avoiding burnout is difficult to write about, because the basic premise is obnoxious. Burnout is a rich man's game. Rice farmers don't get burned out and spend long afternoons thinking about whether to switch to sorghum. Most people don't have the luxury of thinking about their lives in those terms. But at the rarefied socioeconomic heights of computerland, it's true that if you run a popular project by yourself for a long time, there's a high risk that it will wear you out.

It's not the fact of working on just one project that's the problem. This dude, for example, has spent much of his life building a Boeing 777 out of manila folders. Another guy (always dudes!) is slowly excavating his basement with toy trucks.

What burns you out is the constant strain of being responsible for a lot of other people's stuff.

The good news is, as you get older, you gain perspective. Perspective helps alleviate burnout.

The bad news is, you gain perspective by having incredibly shitty things happen to you and the people you love. Nature has made it so that perspective is only delivered in bulk quantities. A railcar of perspective arrives and dumps itself on your lawn when all you needed was a microgram. This is a grossly inefficient aspect of the human condition, but I'm sure bright minds in Silicon Valley are working on a fix.

Perspective does not make you immune to burnout. It just makes burnout less scary. I've gone through a few episodes since starting Pinboard, and I'm sure there will be more to come. People have been very understanding about my occasional need to flee the Internet. I find that the longer I run the site, the more resistant I become to the idea of ever giving it up, even if I need to take the occasional break. It is pleasant to work on something that people draw benefit from. It is especially pleasant to work on something lasting. And I enjoy the looking-glass aspect of our industry, where running a mildly profitable small business makes me a crazy maverick not afraid to break all the rules.

Most of all, I'm gratified that people have been patient and considerate over the whole lifetime of the project. There has been a lot of goodwill sent my way that makes my job vastly easier. Thank you to all the people who have used the site over the years, and the many people who have helped me build it and keep it running. To my competitors: I will crush you! To everyone else: you're wonderful! Upgrade!

]]> (Maciej Ceglowski) Wed, 09 Jul 2014 19:38:05 -0800 ff946c0f1f9628df4c598f2059474dfe9d6accf4
Cloudy Snake Oil I have nothing against Amazon S3, but I was astonished today to find this claim on their FAQ page:

Amazon S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years. In addition, Amazon S3 is designed to sustain the concurrent loss of data in two facilities.

This is an impressive number, but it's utterly dishonest to make such claims. It implies that there is a less than one-in-one-hundred-billion chance that Amazon will abruptly go out of business, or that a rogue employee will cause massive data loss, or an unexpected bug will result in massive data loss, or a defect in storage media will cause millions of devices to fail silently, or a large solar flare will destroy equipment across three data centers, or that a comet impact will destory three data centers, or that a nuclear exchange will destroy three data centers.

I think these events are all incredibly unlikely, but none of them is one-in-a-hundred-billion unlikely. Yet here is Amazon not only making that argument, but implying that you can safely use S3, a service that launched in 2006, for another ten million years.

Rare events are rare! That's why promises past five or six nines of reliability are functionally meaningless. At that point the "unknown unknowns" must overwhelm any certainty you have about what you think your system is doing.

The risks you failed to model will become obvious in retrospect, and make for an entertaining post-mortem, but that won't get anybody's data back.

Promises like Amazon's should serve as a kind of anti-marketing, suggesting that the company has not thought seriously about the limits of risk assessment and planning.

I suggest the following rule of thumb: if you can't count the number of nines in the reliability claim at a glance, it's specious.

Of course this rant is available in book form, phrased better than I have here. But it's worth repeating at every opportunity. ]]> (Maciej Ceglowski) Sat, 19 Apr 2014 15:27:13 -0800 e1a85c14c3b4334bce18c6bd6f1ddd1390030000 Heartbleed and Pinboard By now you may have heard about the heartbleed bug, which affected many websites that use encryption. This is the Spanish Flu of security bugs—it hit almost everyone and took a disproportionate toll on the healthiest, those sites that followed security best practices.

Servers affected by the bug (including the Pinboard site and API) could be tricked into sending private information that happened to be in memory. This included authentication cookies, passwords, secret API tokens, and any data you posted to the site. There is also evidence that the bug could expose a site's private key, which would mean anybody eavesdropping on a Pinboard connection could decipher it. The bug was live from the spring of 2012 until I patched the servers on Monday night.

Worst of all, there's no way to tell from logs if anyone's data was exposed. It's possible that no one looked at any Pinboard data; it's also possible that the site was completely compromised.

This morning, I issued a new TLS certificate for the site, with a new private key. Now that the servers are trustworthy, please do these two things:

  1. Change your Pinboard password. Go to and have at it.

  2. Reset your API token. On that same page (, click the reset button. You'll need to update any outside services and apps that use the API token to authenticate. (Remember never to share your Pinboard password with any third party, no matter how nicely they ask. Outside sites should be able to get all the access they need using only the API token.)

In layman's terms, the bug was the equivalent of asking a stranger "hey, what's up?" and having them tell you their most private thoughts, going on about their divorce, sharing their credit card info, whatever was on their mind at the time. You could keep asking "what's up" as often as you wanted, and hear new things each time. Worst of all, the stranger would have no recollection that it had happened.

Of course, I heard about heartbleed before it was cool. The servers were patched by around 7 PM on Monday night, California time, before half the Internet started casually playing with Python scripts that exposed the bug.

So only truly malicious people could have seen your Pinboard secrets. Hooray!

In awful times like these, it's good to stop and reflect on the timeless wisdom of the Pinboard security page:

"Please do not store truly sensitive information in your Pinboard account."

I don't want anyone getting shot because I used the wrong Linux distro.

This is terrible! Good luck out there! Please feel free to email me if you have questions, or concerns, or would just like to kvetch. ]]> (Maciej Ceglowski) Wed, 09 Apr 2014 10:02:51 -0800 e5bfafb9ea75d1a5ae6798e9e2b5366caebe6286 Planned Maintenance March 22 (Maciej Ceglowski) Wed, 19 Mar 2014 09:32:30 -0800 243ce0fb85dcadbfa9556b1fb7ce6333c0492236 Give Lavabit Money Ladar Levison is raising money for legal defense after shutting down Lavabit, the encrypted email service he's been running for ten years.

Levison's problem is that he's barred from talking about what the government told him to do. But from circumstantial evidence, it appears he was being forced to installing monitoring equipment on his servers.

Levison has already taken a big risk by shutting the service down. Not only has he shuttered a project, but he risks prosecution for implicitly revealing the request for surveillance. And he's in the impossible position of trying to mount a legal defense without being allowed to talk about the case.

If you have been at all bothered by the scope of government surveillance on the Internet, please donate to Levison's fund. Even if you can only give a couple of dollars, it's important that we show up in large numbers, not just to support Lavabit, but to send a signal to the next small company that finds itself debating whether to fight a gag order, or publish a national security letter. They need to know we'll have their back.

Even if Lavabit fails in its appeal, the process will create a paper trail that may prove useful to future efforts at reform. We have to pick at every chink in the armor of secrecy.

If we don't support Lavabit, we'll send a signal of a different kind. A wealthy industry, one capable of throwing millions of dollars at the most nebulous of business ideas, will not put its money where its mouth is when it comes to defending the personal liberties it so vociferously advocates on message boards and in blog posts.

For my part, I'm pledging the next five days of Pinboard receipts to the Lavabit legal defense fund. If you've thought of joining Pinboard, or upgrading your account, you can do so now with the knowledge that all the money will go to Lavabit.

Please join me in donating whatever you can afford. Levison is currently $19,000 of the way to a $40,000 goal, but his costs will mount rapidly if the case makes it to higher appellate courts. If you're not comfortable with the site, there's a direct PayPal link you can use to donate. ]]> (Maciej Ceglowski) Tue, 01 Oct 2013 18:53:44 -0800 3fe0cf03783fdc9eccc0cee24c68024d857794a0 XOXO Talk Notes Last week I had the honor of presenting at XOXO, a wonderful conference organized up in Portland by Andy Baio and Andy McMillan. I've posted a text version of my talk here.

I had mixed feelings about speaking. The two Andys are wonderful people, and they had assembled a stellar list of attendees.

But that list made me a little uneasy. XOXO was an event full of establishment figures (myself included) preaching an alternative gospel. This led to some strangely dissonant moments, like an online billionaire exhorting us to build a better web, which he had presumably forgotten to do earlier. The audience was similarly packed with fossils from the the early Yahoo, antedeluvian Odeo, and pre-Cambrian blog eras.

There needs to be a web equivalent to the Salon des Refusés, where young punk kids with no money can come, make everything we've ever done look lame, and then roast us in our own food trucks.

XOXO has the right spirit for that, but the wrong butts in the seats. If it happens next year, maybe the selection rule can change to help the audience match the message.

I realize I risk sounding ungrateful saying so. But part of the strength of this conference, and why I hope it continues in future years, is the organizers' unusual willingness to listen, and their sincere committment to making the event wonderful. It can be thankless work organizing an event of this size, but I sure do hope they keep it going. ]]> (Maciej Ceglowski) Fri, 27 Sep 2013 19:41:39 -0800 1607cd73738fdce3e8ce6562ca301d73991942d5 An Interview With Alex Bayley of Growstuff

Last year I chose Alex "Skud" Bayley as one of the six winners of the Pinboard Co-Prosperity Cloud, a prestigious $37 investment in six businesses that wanted to join me in thumbing their noses at Silicon Valley flapdoodle and just build something cool and mildly profitable on their own.

Alex's effort, Growstuff, is a community site for food gardeners. I think gardening is a terrific metaphor for maintaining a solo web project; there are dozens of small tasks that need frequent attention, or the site soon becomes unmanageable and withers. So I was curious to see what a community website around gardening would look like, and whether Skud could divert some of the energy from her industrious users to make her development efforts a little easier.

Skud had committed to open sourcing the site and making the development and planning process for it transparent, decisions that chilled me to the very marrow, but which she considered crucial to the success of her project. Reasoning that I had the most to learn from people who took a radically different approach, I forked over the $37 and sat back to see what would happen.

Several weeks ago, I got to meet Skud here in San Francisco, ate fancy little sandwiches with her, and got to hear a lot about her website. Now that Growstuff has launched, Skud has kindly agreed to share some of our conversation as an email interview:

MC: Congratulations on your public launch! What's next for the project in the months to come?

AB: Thank you! Our public launch was just the beginning. Our initial goal was to build something with basic functionality, that you could use to track your garden and which would also the potential of the site in terms of aggregating data and making it available both on the website and via an API. That's what we launched in June. It's only a small part of what we hope to build, though. We have heaps more to do, both in terms of user-facing functionality and in terms of our open data and API.

In July, I surveyed and interviewed a number of our members and we developed a roadmap for what's next. We had so many things on our development wishlist that we had to whittle them down somehow. In the end we came up with a list of things we want to work on that we think is achievable and that'll make the most people happy. It's posted on our wiki, but in short we want to improve the crop database; build ways to track your seed stash, harvests, and wishlist; work on a "1.0" API (as opposed to the current "version 0" we have); and a bunch of other little stuff.

MC: How did you spend my $37 investment?

AB: At that time, I was running Growstuff as a hobby project, and hadn't incorporated or anything. My only real expenses were web hosting, so it went to that. Which I guess is pretty much what you expected. Or another way of looking at it is that it paid for my groceries. When you have a hobby project and just one personal bank account, it's all a bit unclear. Since then I've incorporated Growstuff and the expenses have increased. I actually posted about our first few months' finances on the Growstuff blog. I found incredibly useful to see other small tech businesses' financial spreadsheets and breakdowns when I was just starting out, so I thought I'd pay it forward.

MC: How many garderners do you have going now? Is it following the usual pattern of a few very heavy users, and a larger number of occasional ones?

AB: Around 400 members at present. And yes, the usual activity distribution seems to apply, though the numbers are small at present and there is a pretty strong/active community of early adopters, so it's heavier on the "active" end than I'd expect to see when we have 400,000 members.

MC: You have a fairly extensive roadmap on the site, and it seems like you're amending it based on user feedback. What's the biggest divergence so far between stuff you planned and what has actually happened?

AB: Very little, really! We just passed our one year anniversary of the original idea of Growstuff, and I marked the occasion by revisiting the blog post I made back in July 2012. What I described there is basically what we've built (and are building). I think the vision of "Ravelry for food gardens" was pretty clear in the first place. It's only in the details that things have changed: we realised we needed to have a hierarchy of crops to note all the varieties of different things, we didn't manage to get international/alternative names for crops into the system as quickly as we would have liked, etc. But the overall bones of it are still what was originally talked about.

MC: When we spoke in San Francisco, it sounded like the open source part of the project had been a success, with a variety of people committing code. What's your secret to getting and keeping active contributors?

AB: Yup! We have over 100 people on our "discuss" mailing list where most of the dev work takes place. A lot of people join us because we advertise that we do pair programming and welcome non-experts. I suspect people stick around because it's a warm-fuzzy sort of project: you feel like you're doing something good and the people are friendly and appreciative. Plus a lot of people seem to just want something like Growstuff to exist, either for themselves or for people close to them. I hear a lot of people saying that their family members would be really into a site like Growstuff, and that seems to give them a really grounded sense of building something useful.

MC: Money! How do you make it, and is it enough to live on?

AS: Through paid memberships, which get you special features on the site (i.e. "freemium" model). At present these aren't enough to live on, but I'm also involved in an Australian government program for small businesses, which pays me about $1100 a month to work on this for 1 year. Between that and savings I'm doing okay for now, while Growstuff memberships are paying Growstuff's other expenses (including hiring a graphic designer, getting a lawyer to review our TOS, and stuff like that). I'll really need to start paying myself sometime in the next year, though, so we're going to have to build membership and make sure the paid accounts are compelling enough to get lots of people to subscribe and support the site and, ultimately, pay my rent.

MC: Is it true that Australia has a bizarre, non-cyclical pattern to its growing seasons, or is that just Yankee propaganda?

AB: Sort of true! There are at least four things that make Australian weather seem strange to northern-hemisphere people. The first, of course, is that the southern hemisphere is offset by 6 months due to the earth's axial tilt: our side of the world is closest to the sun around December-January. That sounds obvious, but wasn't so for the first white colonists here, who took a while to get used to it.

The second thing is that even in the temperate parts of Australia (including Melbourne, where I am) the seasons don't follow summer-autumn-winter-spring in quite the same way that eg. Europe does. For instance, summer for us is incredibly dry and hot, which means that it's *not* a peak growing time, unless you use a lot of artificial irrigation. Our lawns, for instance, tend to go through a growth spurt as the weather cools down. Here's a more realistic chart of seasonal trends in my area.

Then, thirdly, lots of Australia is in the tropics, which means that instead of having "summer" and "winter" it's just hot all the time and you have "wet" and "dry" seasons. And finally, Australia is very strongly affected by the El Niño/La Niña weather patterns, which have to do with slow-moving currents in the Pacific Ocean and involve multi-year fluctuations. So we often have a few dry years followed by a few wet years. When you hear about droughts and/or floods in Australia, it's often to do with this. So, all in all, it's complicated. And don't even get me started on our soil quality, which adds a whole new level of complexity for Australian gardeners trying to deal with growing traditions that were developed in Northern Europe. Ugh.

MC: Does the fact that gardening is a seasonal activity affect the rhythm of your work on the site?

AB: Not much, so far, though I expect that when our membership gets bigger we'll see distinct seasonality in our membership and activity. Northern hemisphere spring/summer is likely to be the busiest time.

MC: You've worked in the thick of things in the Bay Area, as well as back home. Would you advise people following your footsteps to get as far away from computerland as possible, or make an effort to be where other coders are?

AB: This is such a complicated question for me. I've had a very up-and-down experience in the tech industry, and the Bay Area had some of the highest "ups" and the lowest "downs". For me, being around heaps of incredibly smart people was great, but being around people intent on making bucketloads money at the expense of other peoples' wellbeing was horrible. I had mixed feelings about leaving, but I think it was the right choice. However, getting away from the Bay Area doesn't mean I got away from coders, and "computerland", if it's anywhere, is everywhere. I'm still working with coders every day, through in-person and remote pairing sessions and online discussions, but now I'm working with a different set of coders, mostly outside the Bay Area bubble.

As far as what I'd advise others: think hard about your values and act according to them. If your values match those of the Bay Area tech community, and you're able to do so, then by all means work there. If they don't, then don't.

MC: What can we do to encourage other people to start small, self-sustaining businesses?

AB: It seems like we have pretty strong informal networks to support each other, and I've had heaps of advice from you and from other indie tech founders. At the same time, I've found more formal groups (mailing lists, industry associations, meetups, etc) to be largely useless. I have no idea why this is, but I wish we could fix it.

I mentioned I've been involved in an Australian government program for small businesses. They weren't very tech-savvy, but as long as you ignored the tech side of things, they actually provided a lot of the information and training I needed when it comes to running a business: taxes, incorporation, planning, marketing, all that stuff. I'd love to see more of this.

Apart from that: I think we just have to keep doing what we're doing, and keep talking about why (and how) we're doing it. I think of indie tech businesses as being like farmers markets: there are lots of reasons to be unhappy with the systems of industrial food (or tech) production, but we can provide a meaningful alternative, and I think we have a growing community of people who appreciate that.

MC: Thank you so much for your time!

If you have a garden, big or small, aspire to have one, or just want to hack on some code in hopes of scoring some local vegetables, go check out Growstuff!

]]> (Maciej Ceglowski) Mon, 26 Aug 2013 18:08:39 -0800 b12e93ad4aef08a73b22871f360cdf1c025cc32c