Incubate All the Things

The Process

Everything about how the W3C works is codified in a document called "The Process". It's a really important document, but a very dull one. However, if you want to change the way that the W3C actually works, then ultimately, you have to modify it, which must be done in accordance with... surprise: The Process.

Generally speaking, the Terrible, Awful, Horrible, No Good, Very Bad Job of maintaining this document is the domain of the Advisory Board (AB). It takes a long time and it is painful process that involves lots of debate with lots of people about all sorts of arcane silliness.

Traditionally, these changes and discussion about them had happened more or less behind closed doors until this list was started in 2011. Kudos to the W3C for opening it up. Now, pretty much anyone in the whole world can follow, read, join or participate in the discussion, etc.

If a process change occurs in the forrest... Well, you might see where I'm going with this: In the ~5 years that it's been open, only 43 brave souls on the entire planet have signed up. Far fewer have actually participated at all and only a tiny few are involved more than a few times. You might wonder why that is, because it sounds kind of crazy. Let me answer by way of analogy: If someone forced you to choose between following W3C process changes and eating drywall, I'd recommend getting a bib and a whole lot of your favorite condoment. It is that boring and arcane.

Right now there is a conversation happening there which is 50+ replies deep and spans several different email threads. Most of the posts are about this size of this article and the topic is incubation.

Old Fiction: Why we need incubation

If you ask "who writes the specs" the answer is inevitably "some standard development organization". This often leads to the wrong mental image. To paraphrase a friend who will remain nameless just incase they regret their particularly colorful choice of words:

So many people come to standards work with stars in their eyes and say, "Well, I just have to get it in the standard and it will be real" and "we're doing R&D here" and... I just want to find ways to kill those people [because] this is really about consolidation and building a platform that we can agree upon and then building value on top of that [and markets in which they can fail].

That's a big problem historically with no easy seeming answers deployed so far. Or, rather, it's several big problems. Let me explain...

Once you're on a working group you can propose anything and very frequently it is difficult for the average developer to tell the difference between "Just some shit than Brian wants to do" and "Something that has a really good shot at being widely implemented soon". There's also no formal recognition of some very real realities.

First, that they operate generally by willing cooperation of interested parties and very few actually fund development of the software or control the codebases required to make these things a reality. We can say "that sucks" or "it shouldn't be that way" but, we're lying if we simply continue to pretend that that "isn't so". It is. Second, each of those organizations that do fund or control a codebase have different interests and priorites.

This mix of things leads to, as you might imagine, a lot of contention in the working groups themselves between all parties.

Worse still was the idea that browsers would ship "experimental" support by simply prefixing it. Developers didn't understand that these were experimental and that their intent was to have some way for people to actually use it and give feedback about how to make it better and then kill it off in favor of the "real" thing - so that actually couldn't happen. Instead, this led to even more contention - accusations that vendors were trying to create a scenario where the only viable option was to use precisely what they'd proposed.

All of this, as you can probably guess, leads to it taking a very long time to get both concensus and interoperable implementations widely deployed to where the average developer can actually try to use them, because there is some promise of actually getting some work done with it. That is, after all, the point. It is also, all too often the case that very frequently initial enthusiasm is quickly dampened as developers come to find that it isn't quite what they were hoping for and a number of things that they wanted aren't there, and that getting that means going back to the drawing board.

This is frequently the stage where a number of people attempt to enter the process, now that we have mostly open mailing lists. To do so, they have to find which arcane list to join and figure that out. Once joining they are often met with a number of unfortunate side-effects - the realization that this is noisy and complicated and takes a really long time."Super-groups" like CSS have tons of proposals at various stages and covering a staggering variety of concerns. It is nearly economically impossible for a developer to be involved in the real formation of a spec. These incentives mean that an awful lot of conversation that happens in the working groups surrounds trying to "get it right" for developers: What will they prefer, what will they understand, what use cases will they have. Which, of course, everyone has an opinion on. As you may predict, this makes things take even longer.

If all of this sounds kind of deeply intertwined and seems like it creates a kind of self-magnifying cascade of poo, I agree. Thus, the desire to change it and the proposal for ways we might do better.

Incubation: Useful speculation

Put very simply, incubation is based on the idea that until a certain point, things are really just useful speculation and that finding better ways to identify it as such and work within that idea may be very advantageous. If we're really fortunate, instead of the cascade of bad, we can build a virtuous cycle of good. This is the aim of the Web Incubator Community Group (WICG).

Instead of entering the conversation as a something that looks like more than it is in a working group then, ideas begin in incubation. Incubation happens out in the open, and not on a mailing list where you have to subscribe to everything but in a forum where you can choose smaller, more focused topics to participate in and engage in those. They don't just come from working group members either because they are, just that: Ideas.

No more vendors shipping with a prefix - experiments are experimental, they can't leak out into the open by being confused with more than they are.

This lowers the barrier to entry and focuses the discussion. Working group members too can benefit from this as it turns out that not even all members are really interested in all things. Let's just move the noise in to an appropriately noisy place and be clear about what it is until we're reasonably certain that it stands a really good chance of becomming more.

Incubation goes hand in hand with the Extensible Web Manifesto. For a lot of things, we can build prollyfills. If you're unfamilliar with that term, there are some new efforts to re-brand those "speculative polyfills" - that is, something experimental which is intended to give you a very good idea what working with that standard will be like, but actually works today in all browers. The idea here is to short-circut the conversation allowing developers to weigh in themselves at a far earlier stage and with a better value proposition.

How much earlier? Well, the "real lifetime" of a lot of ideas that have made it to "widely deployed standard" status is a decade or more. People were talking about what would become flexbox, for example, in at least 2004 (very possibly earlier). Most of the things in Selectors Level 4 (and some still not in 4) were already in a draft in 1998. But you couldn't use any of them. This meant that taking the time to follow the conversations and understand them and comment on them along the way was impractical - very impractical. It was intimidating too for anyone who wanted to discuss, and frustrating because chances are very good that there were long conversations and topics years ago that you really should have a look at before commenting, but how would you even know that. The mailing lists make this hard.

By comparison, I have seen people turn around a functional approximation in incubation upon request in just a day or two. Not all of them, of course, but it is frequently faster, the number of people who can contribute is much greater, and it is clearly nothing more than "potentially useful speculation" on a particular/limited topic. Generally speaking, topics are much easier to deal with as topics than a collection of searchable emails.

Fears

Change is difficult and make no mistake, including incubation as an even relatively formal concept in W3C is a change. It's natural then that some people have fears. Working groups have been the exclusive way to work for a really long time. Why should something not happen in a working group if members of the working group think it should start there and most mailiing lists, like www-style, are technically forums open to the public? How doesn't this hollow out working groups and make them nothing more than paper-pushers as things move through the process?

The interesting thing is, it's actually a smaller change in practice than you might think. Currently, the vast majority of specs have a single primary editor and sometimes two or three collaborators. They work outside of any working group and answer most of the questions, and have deep discussion and come back with an answer. It's only at this point that the rest of the working group often talks about it much at all. There are also exceptionally noisy conversations along the way between other folks on IRC or slack or lots of other venues to help them hammer out ideas which they then bring back to wider audiences. They move the noise, because that's just how information works. In many cases then, what incubation provides is, realistically, mostly a move of the venue and the admission of how serious/likely it is to become a standard.

Working Groups and Incubation are not mutually exclusive ideas, they are complimentary ideas. The aim of incubation is not to replace working groups, as much as to simply to help setup healthier scenarios that provide checks and balances earlier on and allow ideas to succeed or to improve or fork in ways that working groups currently make confusing and difficult. It's to help balance out the idea of 'powers' and recognize realities. It's even to just let ideas fail and move on.

Is it the only way to achieve these goals? Probably not, and it's not fully fleshed out or perfect - but I think it's the best start on an answer that we have so far.

Cool Story Bro

So why do you care, and what am I asking of you? Well, it's my opinion that this is your Web, as much as it is any W3C member orgs Web. Ultimately, it is developers who have to live with this stuff. It's developers who advocate and adopt and occasionally complain about or even occasionally outright reject standards. You are, in the end, a very important piece of the puzzle and I don't know that anyone is really specifically asking you what you think. I don't think that you should have to wade through all of the discussions but, somehow, you should be heard. Your voice matters, if you want it to. But noone else is gonna do it, if we want to be involved, we've got to be involved. I think that incubation is an important part of that, but I'm just one guy. So I'm asking you to start a public discussion, share your perspective - tweet, blog, share. Or, if you're uninterested with changing anything, feel free to ignore. Here's my own perspective: