The Five Stages of Hosting

As a proud VPS survivor, I thought it might be fun to write up five common options for hosting a web business, ranked in decreasing order of 'cloudiness'. People who aren't interested in this kind of minutia would be wise to pull the rip cord right here.

1. The Monastery

You run your site on an 'application platform' like Heroku, Azure, or Google App Engine. You design your application around whatever metaphors and APIs the service lays out, and in return you are veiled from all the mysteries of implementation. You never interact with the computer directly, but upload your code to the platform with the proper incantations and it runs. The orders vary in strictness, with GAE requiring that you purify yourself of all worldly design habits before writing your app, Azure insisting you renounce the demon Unix, and Heroku somewhat more welcoming to the fallen. On all three platforms, when your application needs more resources, you press the 'more resources' button. A lot of fancy sandboxing, monitoring and administration tools come standard, and it's very easy to deploy and test different versions of your app.

Good: Skilled beardos toiling behind the scenes make sure everything Just Works. As long as your application obeys the commandments, you can have faith that it will endure any trial. Prototypes are fast and easy to build, and all you have to do to turn them into production services is add money. You don't have to spend any time worrying about backups, load balancing, configuration, hardware, or anything except your app.

Bad: You have to design your app in a very specific way, and learn the holy texts by heart. If you decide to convert to another service, or become an apostate and run things yourself, you'll find yourself having to rebuild from the foundations. Of course, being a programmer it will be more tempting to just create your own application platform instead. All this abstraction is expensive, and it can be hard to estimate cost ahead of time. It's also easy to rack up significant fees if you don't pay attention. You are vulnerable to sudden changes in the pricing model.

2. The Dorm Room

You lease a virtual server from a company like Linode, prgmr, Slicehost or Amazon. You get to pretend like you have your own computer, and nobody can tell you what to do, man! In reality you inhabit a small slice of a giant shared resource, but the hosting company does a good job of making it feel like a standalone machine. You're assigned a certain amount of memory, disk, and CPU, and can do whatever you want within those bounds. You can put up your Bob Marley posters and even build a loft!

Good: You are isolated from a lot of real responsibility. If a disk goes bad, people will fix it and you may not even know anything happened. For small servers, the cost of virtual hosting is pretty low. Pricing is straightforward and transparent, and you can usually add resources just by asking. If there is downtime, it's likely to be brief, as your virtual machine can move to another server. Most of the time you won't notice the neighbors.

Bad: Sometimes you will REALLY notice the neighbors, and can't do anything about it. I/O performance in particular can be awful. Your operating system will lie to you about performance because it lives in the Matrix and can't see all the way down to the hardware. And if you test the boundaries, you'll discover you can't actually do whatever you want. Deviate too far from expected behavior (by churning through millions of files, for example) and the R.A. will come knocking.

The larger tiers of virtual hosting are expensive compared to equivalent dedicated servers. There's often no way to choose the ratio of CPU to disk to RAM.

3. The Apartment

You lease a dedicated server from a company like Leaseweb or ServerBeach or a thousand others. This is just like (2), except now you really do have full control over an actual, physical computer that is not shared with anyone. You get a wider choice of operating systems and hardware, and you can custom-order all kinds of extras for your server. The machine still belongs to the hosting company, but you pay by the month or year for exclusive use.

There is a subtype of this arrangement known as "managed" hosting, where for additional money the hosting company takes care of general housekeeping and holds your hand if you get scared. Think of it as renting a basement apartment from your parents.

Good: Extensive configuration options. If any hardware breaks, the hosting company will fix it. It's easier to diagnose and characterize performance problems. Frequently you'll get an 'out of band' console, so you can reboot or monitor your server from the outside. Sustained and random access I/O speed is much higher and performance in general is predictable and consistent.

Bad: The cost of rental will quickly exceed the amortized cost of just buying hardware, particularly if you require a lot of RAM or hard disk space. It's hard to evaluate the quality of tech support until you actually need it. Pricing is opaque and highly variable. If something breaks, it can be quite a while before the host can replace it.

4. The Condo

You buy your own computer and rent space for it in a colocation facility. This is just like (3), except now the machine sitting in the data center physically belongs to you. You buy it, you set it up, you run it, and it's your job to fix it if it breaks. The hosting company provides electricity, cooling, physical security, and some minimal "remote hands" service if you need someone to press a button or look at your blinkenlights. But ask not for whom the pager beeps — for sysadmin, it beeps for thee.

Good: You can fit your hardware to the exact needs of your application. If you use it long enough, you will save substantial money over rental. You can run your own routers, switches, and storage appliances - whatever fits in the rack. You have total control over everything.

Bad: You have total control over everything. Hardware is hard. You pay a lot of money up front. If something goes wrong, expect to either travel or FedEx heavy packages back and forth while under great stress. Pricing is even more opaque and variable than with rental, and you may have trouble finding good options if you need less than a full rack (room for about a dozen big servers).

5. The Stately Manor

Your own datacenter.

Good: No need to take hosting advice from blog posts.

Bad: God help you.

—maciej on January 29, 2012

A Note on Recent Downtime

The site was offline for about four hours in the early morning of January 2, California time. Unlike some previous outages, this one was pure PEBKAC and entirely my fault.

As many of you know, I spent some time in December configuring and installing a big physical server in Fremont. This was supposed to be ready at the start of the month, but the need to replace two pieces of hardware[*] pushed the date foward. In late November, I asked my current web hosts in Holland to extend my contract by a month, which they happily agreed to. That server rents on a three-month cycle, and when I paid my invoice, I filed it away in my brain as such. But looking back at my correspondence it's clear I asked for and received an extension until Dec 31.

I noticed this discrepancy on the evening of December 30, and sent an email to support asking for a further extension. In the meantime, I worked to prepare the Fremont server so it could take over. On the evening of Jan 1, the site in Holland was still up, and I decided to wait another day before hitting the big red button. I didn't want to start such a major change in the evening, when I was tired and apt to make mistakes. And I held out hope that my request for an extension had gone through.

Of course I was wrong about that, and my web hosts pulled the plug early in the morning on the 2nd. Bookmarks and archives were not affected, but I neglected to do a final sync of notes (notes in pinboard are saved as files). This meant about 20 users who created or edited notes between December 31 and Jan 2 lost those notes. In most cases, these were short snippets of text, some of them duplicated in the bookmark description, but that's no excuse for losing the stuff. I've emailed everyone affected to explain the situation, and given them a free year of archiving.

Any kind of data loss on this site is unacceptable, but it's especially bad when it's due to completely preventable operator error. I'll be making some changes to make sure I can't repeat this kind of mistake:

• First, I've written out all important service deadlines in the little notebook where I write my daily work notes. This is a low-tech but effective way to get me to pay attention to stuff.

• Second, I've worked out a formal backup policy for everything that lives on the filesystem (logs, configuration files, notes, uploaded files, feeds from other services etc). Now they will be backed up automatically with the same thoroughness as the database and user archives.

• Third, I'm working on better automating server configuration and setup. Though the site was back up quickly, it took me most of the day to get services like search and feed import working smoothly. This should be something I can do with one command.

• Finally, I'm going to spiff up the status page so that it shows the actual status of specific services. I have this stuff in my admin console, and there's no reason not to share it with users.

I appreciate everyone's patience during the outage, and apologize again to those users whose notes I lost. Downtime has been rare on Pinboard, and I'll do everything I can to keep it that way.

On the plus side, Pinboard is now running on a really nice piece of hardware hosted in California. For those of us in the US, this should mean a much faster site. Users in Europe may see a little more latency, but hopefully offset by better performance.

I'll post about the new setup in a few days, once I've had time to make the important changes outlined here.

* For the curious, a SATA card that turned out to freeze after about 12 hours, and a pair of SSDs I had purchased for the database, but which I realized would soon be too small at the current rate of growth.

—maciej on January 03, 2012

What I Did in 2011

At the end of last year, I began keeping a little journal to help me keep track of my work. Looking back, here's a summary of what I did on Pinboard in 2011 (other than field support emails), month by month:

January

Recovering from the December onslaught. Made RSS feeds available in JSON format. Localized site to Japanese, added a tag completion pulldown, enabled linkrolls, wrote FAQ.

February

German localization, first steps to integrating with Twitter via OAuth. Lots of backend work moving to a proper message queue from MySQL.

March

Further struggles with new Twitter backend. French localization. Introduced the popular+wikipedia page. Made tweets searchable. Did an interview with the Babbage blog. API documentation. Added tag intersection support to API. Fixed pagination issues for tag intersections. Got a test version of a Firefox plugin working.

April

Localized the account page. Added cloud of related tags to user+tag pages, with a clickable link for making tag intersections. Initial Safari extension. Introduced "Save Tab Set" feature. Ordered and set up a storage appliance for archiving old database backups. Added Readability support. Released Chrome extension.

May

Added keyboard shortcuts. Released Firefox plugin. Added autocomplete pulldown to search box. Moved URL crawling infrastructure onto message queue.

June

Got rid of Urchin cruft on incoming URLs. Dealt with FBI raid on colo and ensuing drama. Interview with Wired.

July

Accidentally followed a link to tvtropes.org.

August

Moved site off of DigitalOne (may their name be cursed) onto LeaseWeb servers. Spent time semi-automating server setup. Made improvemetns to fulltext search.

September

Migrated stored user passwords to a safer format (from salted sha1 to bcrypt). General cleanup of authentication code. Rewrote cookies to make them revokable. Set up a staging server. A generous pal performed a security audit. Ordered components for my first physical server. Interview with Time magazine.

October

Dealt with fallout from Delicious relaunch. Struggled against some overzealous bots aggressively crawling the site. Feature to grey out bookmarks you already have when viewing other users' pages. Tag subscriptions. Time zones! Made user+tag pages sortable by title, added preference for number of bookmarks displayed per page. Public networks. Option to sort tags in pulldown by popularity. Major cleanup of inline edit form and copy-to-mine. Greeted fandom and worked on their wish list of features.

November

Public profiles and search. Big javascript overhaul. Added importers from Firefox JSON format and Google Reader. Wrote scripts for tarring up user archives in human-readable form so people can download them. Removed Google Reader support (RIP, Google Reader, I'll mourn you til I join you)

December

Configured new server and set it up in Fremont colo. Major database cleanup. Lots of work on new tag editing interface. Added cookie support to crawler. Wrote this list.

—maciej on December 26, 2011

A Chrome Mystery

Update: @rotron pointed me at this Chrome bug report, which seems to be the culprit. In their last update, Google auto-installed a YouTube app for everyone who didn't have any apps installed. This app will break bookmarklets on any YouTube page. The workaround is to delete the YouTube app (open a new tab, right click youtube, delete) and restart Chrome.

A number of users have been reporting an intermittent bug when using a Pinboard bookmarklet in Chrome. They click on the bookmarklet, and all that shows up is a window with 'about:blank' in the URL field.

I've been able to reproduce this reliably on my own machine by visiting YouTube and clicking the bookmarklet. But some other people with what seems to be an identical version of Chrome (16.0.912.63 on OS 10.7 with no extensions installed ) don't see the bug.

Since the bookmarklet is kind of complicated, I reduced the problem to this test case:

1. Drag this to your bookmarks bar: dragme

2. Go to www.youtube.com.

3. Click the bookmarklet

Here's what I see after about thirty seconds:

My question to the Internets is: can anyone reproduce this behavior? And as a chrome novice, how do I go about chasing this bug?

I know from trying this on URLs I control that the new window runs all the expected HTTP requests for the target page before it hangs.

Any help you can give me (@Pinboard on Twitter or support@pinboard.in) will be most welcome.

—maciej on December 22, 2011

Don't Be A Free User

Were you a big Gowalla fan? Did you like Dodgeball? Did you think Trunk.ly (gasp!) was better than Pinboard? Did you make a lot of contributions to Nextstop? Do you miss Aardvark and EtherPad? Did "I Want Sandy" change your life?

These projects are all very different, but the dynamic is the same. Someone builds a cool, free product, it gets popular, and that popularity attracts a buyer. The new owner shuts the product down and the founders issue a glowing press release about how excited they are about synergies going forward. They are never heard from again.

Whether or not this is done in good faith, in practice this kind of 'exit event' is a pump-and-dump scheme. The very popularity that attracts a buyer also makes the project financially unsustainable. The owners cash out, the acquirer gets some good engineers, and the users get screwed.

To avoid this problem, avoid mom-and-pop projects that don't take your money! You might call this the anti-free-software movement.

If every additional user is putting money in the developers' pockets, then you're less likely to see the site disappear overnight. If every new user is costing the developers money, and the site is really taking off, then get ready to read about those synergies.

To illustrate, I have prepared this handy chart:

What if a little site you love doesn't have a business model? Yell at the developers! Explain that you are tired of good projects folding and are willing to pay cash American dollar to prevent that from happening. It doesn't take prohibitive per-user revenue to put a project in the black. It just requires a number greater than zero.

I love free software and could not have built my site without it. But free web services are not like free software. If your free software project suddenly gets popular, you gain resources: testers, developers and people willing to pitch in. If your free website takes off, you lose resources. Your time is spent firefighting and your money all goes to the nice people at Linode.

So stop getting caught off guard when your favorite project sells out! “They were getting so popular, why did they have to shut it down?” Because it's hard to resist a big payday when you are rapidly heading into debt. And because it's culturally acceptable to leave your user base high and dry if you get a good offer, citing self-inflicted financial hardship.

Like a service? Make them charge you or show you ads. If they won't do it, clone them and do it yourself. Soon you'll be the only game in town!

DISCLAIMER: I run a paid bookmarking site. Every morning I wake up and dive into my vault of golden coins.

—maciej on December 06, 2011

The Social Graph is Neither

I first came across the phrase social graph in 2007, in an essay by Brad Fitzpatrick, though I'd be curious to know if it goes back further.

The idea of representing relationships between people as networks is old, but this was the first time I had thought about treating the connections between all living people as one big object that you could manipulate with a computer.

At the time he wrote, Fitzpatrick had two points to make. The first was that it made no sense for every social website to try and recreate the same web of relationships, over and over, by making people send each other follow requests. The second was that this relationship data should not be proprietary, but a common resource that rival services could build on as a foundation.

Fitzpatrick subsequently went to work for Google, and his Utopian vision of open standards and open data became subsumed in a rivalry between Google and Facebook. Both companies now offer their version of a social graph API, and Google (which is trying to catch up) has taken up the banner of open standards and data portability.

This rivalry has brought the phrase 'social graph' into wider use. Last week Forbes even went to the extent of calling the social graph an exploitable resource comprarable to crude oil, with riches to those who figure out how to mine it and refine it.

I think this is a fascinating metaphor. If the social graph is crude oil, doesn't that make our friends and colleagues the little animals that get crushed and buried underground?

But right now I would like to take issue with the underlying concept, which I think has two flaws:

I. It's not a graph

The idea of the social graph is that each person is a dot in a kind of grand connect-the-dots game, the various relationships between us forming the lines. Some of these lines may point in one direction (Ted works for Sylvia), while others are reciprocal (Bob is my neighbor). All of them taken together form a mathematical structure called the social graph. Facebook even has a pretty picture!.

We nerds love graphs because they are easy to represent in a computer and there is a vast literature on how to do useful things with them. When you ask Google for directions from Detroit to Redwood City, for example, you're interacting with a graph that represents the US road network. The same principle applies any time a site tells you people who bought object X might also be interested in book Y.

In order to model something as a graph, you have to have a clear definition of what its nodes and edges represent. In most social sites, this does not pose a problem. The nodes are users, while edges means something like 'accepted a connection request from', or 'followed', or 'exchanged email with', depending on where you are.

The way you interpret this is another matter - does clicking 'follow' imply you're friends with someone in real life? But at least what the data model represents is unambiguous.

But when you start talking about building a social graph that transcends any specific implementation, you quickly find yourself in the weeds. Is accepting someone's invitation on LinkedIn the same kind of connection as mutually following them on Twitter? Can we define some generic connections like 'fan of' or 'follower' and re-use them for multiple sites? Does it matter that you can see who your followers are on site X but not on site Y?

One way to solve this comparison problem is with standards. Before pooling your data in the social graph, you first map it to a common vocabulary. Google, for example, uses XFN as part of their Social Graph API. This defines a set of about twenty allowed relationships. (Facebook has a much more austere set: close_friends, acquaintances, restricted, and the weaselly user_created).

But these common relationships turn out to be kind of slippery. To use XFN as my example, how do I decide if my cubicle mate is a friend, acquaintance or just a contact? And if I call him my friend, should I interpret that in the northern California sense, or in in some kind of universal sense of friendship?

In the old country, for example, we have two kinds of 'friendship' (distinguished by whether you address one another with the informal pronoun) and going from one status to the other is a pretty big deal; you have to drink a toast with your arms all in a pretzel and it's considered a huge faux pas to suggest it before both people feel ready. But at least it's not ambiguous!

And of course sex complicates things even more. Will it get me in hot water to have a crush on someone but have a different person as my muse? Does spouse imply sweetheart, or do I have to explicilty declare that (perhaps on our 20th anniversary)? And should restrainingOrder be an edge or a node in this data model?

There's also the matter of things that XFN doesn't allow you to describe. There's no nemesis or rival, since the standards writers wanted to exclude negativity. The gender-dependent second e on fiancé(e) panicked the spec writers, so they left that relationship out. Neither will they allow you to declare an ex-spouse or an ex-colleague.

And then there's the question of how to describe the more complicated relationships that human beings have. Maybe my friend Bill is a little abrasive if he starts drinking, but wonderful with kids - how do I mark that? Dawn and I go out sometimes to kvetch over coffee, but I can't really tell if she and I would stay friends if we didn't work together. I'd like to be better friends with Pat. Alex is my AA sponsor. Just how many kinds of edges are in this thing?

And speaking of booze, how come there's a field for declaring I'm an alcoholic (opensocial.Enum.Drinker.HEAVILY) but no way to tell people I smoke pot? Why are the only genders male and female? Have the people who designed this protocol really never made the twenty mile drive to San Francisco?

What happens to dead people in the social graph? Facebook keeps profiles around for a while in memoriam, so we probably shouldn't just purge dead contacts from the social graph immediately. But we certainly don't want them haunting us on LinkedIn - maybe there should be a second, Elysian social graph where we can put those nodes to await us?

You can call this nitpicking, but this stuff matters! This is supposed to be a canonical representation of human relationships. But it only takes five minutes of reading the existing standards to see that they're completely inadequate.

Here the Ghost of Abstractions Past materializes in a flurry of angle brackets, and says in a sepulchral whisper:

“How about we let people define arbitrary relationships between nodes…”

(subject,verb,object)

“Maybe even in XML…”

<Person "john">
    <likesToShareRecipesWith "susan" />
</Person>

“Of course, we'll need namespaces…”

	
  <ns:Person rdf:about="http://www.example.org/#john">
    <ns:likesToShareRecipesWith 
     rdf:resource="http://www.example.org/#susan" />
  </ns:Person>

And RDF rises lurching out of the grave to infect the brains of another generation of young developers.

But even if we go ahead and build the Semantic Web, 2004 edition, and populate it with information about all our connections to other people, it still won't be expressive enough.

One big sticking point is privacy. Do I really want to find out that my pastor and I share the same dominatrix? If not, then who is going to be in charge of maintaining all the access control lists for every node and edge so that some information is not shared? You can either have a decentralized, communally owned social graph (like Fitzpatrick envisioned) or good privacy controls, but not the two together.

There's another fundamental problem in that a graph is a static thing, with no concept of time. Real life relationships are a shared history, but in the social graph they're just a single connection. My friend from ten years ago has the same relationship to me as the friend I dined with yesterday. You're left with forcing people (or their software) to maintain lists like 'Recent Contacts' because there is no place in the model to fit this information.

"No problem," says Poindexter. "We'll add a time series of state transitions and exponentially decaying edge weights, model group dynamics as directional flows, and pass a context object in with each query..." and around we go.

This obsession with modeling has led us into a social version of the Uncanny Valley, that weird phenomenon from computer graphics where the more faithfully you try to represent something human, the creepier it becomes. As the model becomes more expressive, we really start to notice the places where it fails.

Personally, I think finding an adequate data model for the totality of interpersonal connections is an AI-hard problem. But even if you disagree, it's clear that a plain old graph is not going to cut it.

II. It's Not Social

The social graph project has roots in something called Friend of a Friend, or FOAF (disclaimer: I worked on a rival project called LOAF, and you may feel free to ascribe everything I say here to purest bitterness).

The idea of FOAF was that everyone would create little XML snippets that represented their interests. For example, if you liked burgers and had a huge crush on your neighbor Matt, you could generate an RDF file that said so and stick it in to your Geocities page.

The problem FOAF ran headlong into was that declaring relationships explicitly is a social act. Documenting my huge crush on Matt in an XML snippet might faithfully reflect the state of the world, but it also broadcasts a strong signal about me to others, and above all to Matt. The essence of a crush is that it's furtive, so by declaring it in this open (but weirdly passive) way I've turned it into something different and now, dammit, I have to go back and edit my FOAF file again.

This is a ridiculous example (though it comes up with strange regularity in the docs), but we run into its milder manifestations all the time. Your best friend from high school surfaces and sends a friend request. Do you just click accept, or do you send a little message? Or do you ignore him because you don't want to deal with the awkward situation? Declaring connections is about as much fun as trying to whittle people from a guest list, with the added stress that social networking is too new for us to have shared social conventions around it.

OkCupid was one of the first social sites to understand that every visible action sent a signal. While other dating sites nagged you to upgrade to an expensive 'Gold' status, which branded you as a foreveralone pariah, OKCupid took pains to make sure people had stuff to do on the site that was unrelated to dating. A popular activity was building and taking personality quizzes. The quiz feature removed some of the stigma from hanging out on the site (I'm just a cool guy having some fun with these quizzes!) while creating a whole new avenue for meeting people.

Social graph proponents seem uninterested in the signaling problem. Leaving aside the technical issues of how to implemented, how does cutting ties actually work socially? Is there any way to be discreet, for example, or have connections naturally degrade over time? In real life, all relationships fade naturally if you don't maintain them, but right now social networks preserve ties in amber until we explicitly break them. Is my sister going to resent me if I finally defriend her annoying husband? Can I unfollow my ex now, or is that going to make her think I'm still hung up on her?

There's no way to take a time-out from our social life and describe it to a computer without social consequences. At the very least, the fact that I have an exquisitely maintained and categorized contact list telegraphs the fact that I'm the kind of schlub who would spend hours gardening a contact list, instead of going out and being an awesome guy. The social graph wants to turn us back into third graders, laboriously spelling out just who is our fifth-best-friend. But there's a reason we stopped doing that kind of thing in third grade!

You might almost think that the whole scheme had been cooked up by a bunch of hyperintelligent but hopelessly socially naive people, and you would not be wrong. Asking computer nerds to design social software is a little bit like hiring a Mormon bartender. Our industry abounds in people for whom social interaction has always been more of a puzzle to be reverse-engineered than a good time to be had, and the result is these vaguely Martian protocols.

But let's say an inspired mathlete proves me wrong. There's a brilliant hack that fixes all the issues I've raised and we go ahead and build the Platonic social graph. What can you actually do with it?

Well, one thing we've seen is that machine-readable lists of friends make it much easier to launch social sites. Letting a thousand startups bloom is one of the big justifications in Fitzpatrick's essay. But is removing this friction a good thing? It is admittedly annoying to have to re-follow people every time you sign up for something, but it also forces the authors to make the site appealing enough to get us over that hurdle. We're already starting to see apps whose first act is to suction down our contact list and spam our various accounts with invites without bothering to woo us at all. I can't imagine having open API access to the social graph is going to improve that.

In other domains, a big graph would be good for recommendations, but friendship is not transitive. There's just no way to tell if you'll get along with someone in my social circle, no matter how many friends we have in common.

But one thing you can do is mine a huge amount of information about my friends and infer things about their interests, income, social status and tastes. And then maybe you can use that information to bring them valuable news and offers, or help them digitally engage with their favorite brands.

Imagine the U.S. Census as conducted by direct marketers - that's the social graph.

Social networks exist to sell you crap. The icky feeling you get when your friend starts to talk to you about Amway, or when you spot someone passing out business cards at a birthday party, is the entire driving force behind a site like Facebook.

Because their collection methods are kind of primitive, these sites have to coax you into doing as much of your social interaction as possible while logged in, so they can see it. It's as if an ad agency built a nationwide chain of pubs and night clubs in the hopes that people would spend all their time there, rigging the place with microphones and cameras to keep abreast of the latest trends (and staffing it, of course, with that Mormon bartender).

We're used to talking about how disturbing this in the context of privacy, but it's worth pointing out how weirdly unsocial it is, too. How are you supposed to feel at home when you know a place is full of one-way mirrors?

We have a name for the kind of person who collects a detailed, permanent dossier on everyone they interact with, with the intent of using it to manipulate others for personal advantage - we call that person a sociopath. And both Google and Facebook have gone deep into stalker territory with their attempts to track our every action. Even if you have faith in their good intentions, you feel misgivings about stepping into the elaborate shrine they've built to document your entire online life.

Open data advocates tell us the answer is to reclaim this obsessive dossier for ourselves, so we can decide where to store it. But this misses the point of how stifling it is to have such a permanent record in the first place. Who does that kind of thing and calls it social?

III What, then, is to be done?

The funny thing is, no one's really hiding the secret of how to make awesome online communities. Give people something cool to do and a way to talk to each other, moderate a little bit, and your job is done. Games like Eve Online or WoW have developed entire economies on top of what's basically a message board. MetaFilter, Reddit, LiveJournal and SA all started with a couple of buttons and a textfield and have produced some fascinating subcultures. And maybe the purest (!) example is 4chan, a Lord of the Flies community that invents all the stuff you end up sharing elsewhere: image macros, copypasta, rage comics, the lolrus. The data model for 4chan is three fields long - image, timestamp, text.

Now tell me one bit of original culture that's ever come out of Facebook.

Right now the social networking sites occupy a similar position to CompuServe, Prodigy, or AOL in the mid 90's. At that time each company was trying to figure out how to become a mass-market gateway to the Internet. Looking back now, their early attempts look ridiculous and doomed to failure, for we have seen the Web, and we have tasted of the blogroll and the lolcat and found that they were good.

But at the time no one knew what it would feel like to have a big global network. We were all waiting for the Information Superhighway to arrive in our TV set, and meanwhile these big sites were trying to design an online experience from the ground up. Thank God we left ourselves the freedom to blunder into the series of fortuitous decisions that gave us the Web.

My hope is that whatever replaces Facebook and Google+ will look equally inevitable, and that our kids will think we were complete rubes for ever having thrown a sheep or clicked a +1 button. It's just a matter of waiting things out, and leaving ourselves enough freedom to find some interesting, organic, and human ways to bring our social lives online.

—maciej on November 08, 2011

Import from Google Reader

I've made it possible to import all your shared or starred articles from Google Reader into Pinboard. You can find step-by-step instructions for this on the import page,

—maciej on November 05, 2011

Creeping Socialism

I've added the ability to create a public profile on the site. This lets you specify your contact info on various other websites and provide a little information about yourself. You can choose to have your profile visible just to Pinboard users or to the entire Internet.

Here's what mine looks like:

The purpose of these profiles is to let people find one another on the site and build (or re-build) their networks. Naturally, like all social features on Pinboard, this one is strictly opt-in. If you'd like to create your own profile, go to settings -> privacy and click the friendly checkbox. You'll see a new 'profile' tab appear in your navigation bar.

And if you're curious, you can find a full list of users with profiles on the pinboard.in/directory page. I've been getting a big kick out of the variety of people and responses showing up there.

—maciej on November 03, 2011

Show URLs and Alphabetized Tags

I added two new bookmark display options this weekend:

• Show URLs will display the full URL of every bookmark underneath the title.

• Alphabetize Tags will always display your bookmark tags in that order, rather than the order they were entered.

To make things a little cleaner, I've broken these and related options out into a 'Bookmark Display' subsection on the settings page.

—maciej on October 31, 2011

Pinboard On Time.com

This week Pinboard was profiled on Time.com, in an article about foreveralone entrepreneurs, along with more glamorous fellow-traveler, Instapaper. I really like the stock photo of ferocious productivity at the top of the article. You must picture me off frame to the right, squinting and rubbing my temples. (You know it's a stock photo when the laptop's not a Mac).

The article doesn't say much I haven't already written about here, except for some details about total revenue. You guys have given me a Google salary this year and I am extremely grateful for that!

—maciej on October 27, 2011