Note from the author...
My posts frequently (like this one) have a 'theme' and tend to use a number of images for visual flourish. Personally, I like it that way, I find it more engaging and I prefer for people to read it that way. However, for users on a metered or slow connection, downloading unnecessary images is, well, unnecessary, potentially costly and kind of rude. Just to be polite to my users, I offer the ability for you to opt out of 'optional' images if the total size of viewing the page would exceed a budget I have currently defined as 200k...
The Future Web Wants You
A few weeks ago I gave a talk in Pittsburgh, PA at Code and Supply. It was recorded if you would rather watch the video (the actual talk is ~40 minutes, the video captures some of the Q&A afterward as well), but I'm more comfortable writing and I thought it might be worth a stab at a companion piece that tries to make the same points/arguments in blog-post-size, so - here it is.
In the mid-1990's, I didn't really know much about standards, but it seemed that they obviously existed. So, when I heard that there was going to be a "standards organization" setup to bring together all sorts of powerful tech interests and that it would be led by Tim Berners-Lee (creator of the Web) himself, it didn't take much more to win my confidence.
I suppose I imagined something between Michelangelo's Creation panel on the Sistine Chapel Ceiling and Raphael's School of Athens. Somewhere, perhaps high in a tower in a proverbial Mount Olympus where the Gods of programming - wise and benevolent beings would debate and design - and the outcome would be an elegant, simple and beautiful solution. They would show us the One True Way. Specifications might have been handed down on tablets of stone as far as I imagined. The future was bright. They would lead me to the promised land.
My imagination of what a W3C meeting was like..
By around 2009, I guess you could say that my outlook had "matured".
I was jaded, yes.
What had happened? I decided to begin to try following standards a little more "from the inside" and I learned a lot. I talk more about it in the video, but here is the most important takeaway I can give you: There is no standard for standards.
That is: We really don't know what we're doing. Standards are a really "young" idea. In the roughly 100 years we've been trying to deal with them you can sum up a brief history something like this:
- Countries established national standards organizations - here in the US, ANSI.
- National Standards really weren't good enough for some things, so we got an international standards organization: ISO.
- ISO tweaked how they approached things a few times along the way, but when it came to networks and software, they were kind of abysmal. After a decade of working on the OSI 7 Layer Model, Vint Cerf and some others left and created the IETF. We got the internet. The IETF works very differently from ISO/ANSI.
- When Tim Berners-Lee came along he could have taken things to ISO, or the IETF - and in fact, he did choose the later. Some things were standardized there, others languished and never actually reached what you could call, in IETF terms, a standard. After some mulling, the W3C was created. It works differently than ANSI, ISO or the IETF.
- When Internet Explorer began reverse engineering JavaScript and Netscape wanted to standardize it, they could have taken it to any of the above. Instead, they took it to ECMA - a body previously dedicated to the manufacturing standards for computers in Europe. Why? Because historical events led them to believe that Microsoft would wield less powerful influence in this venue and that ECMA would be more fair to the creators. It works differently than all of the above.
- After a period in which much of the world (including major players like Microsoft who controlled 95% of the browser market share at the time) decided that perhaps HTML wasn't the future we wanted after all and spent a decade trying to influence a different possible future in the W3C, a group defected and created the WHATWG which - again - works very differently than all of the above. The WHATWG was spun up in 2004, the first draft of HTML was published in 2008.
Along the way we've seen features that were disappointing (AppCache) things that aren't quite interoperable (IndexedDB/WebSQL), things that failed to materialize (native dialogs, the document outline) and battles over control of the "really official standard" as well as what that even means. In late 2014, it reached W3C status that we might call 'standard' - however, there's still a lot that doesn't work in all browsers - HTML input types support, for example. So it would be foolish to say that process really "worked well" in total either.
It's not blasphemous to suggest that we can do better.
The interesting point here is that the reason there are many venues is simple: That the ones that came before them weren't working well and that each of these has tried to adapt to get better.
Lessons Learned
In a nutshell: We've moved around a lot of variables a lot of times, trying to figure this out - but the one thing we haven't figured out how to tap into is developers. This is strange because ultimately, it is developers who decide the fate of it all. Over the years standards have come to say "we have businesses, we have academia, we have government."
Bring your army, we have developers.
Yay. That's great. But, the truth is: we have developers. Developers are like the Hulk, their potential power is nearly limitless, it's just untapped! If you want to win the day - you need the Hulk on your side.
Think about it. Microsoft quite literally "owned" the browser market and disbanded the team. When work continued on HTML, it created what might have been an impossible impasse. There was no obvious way to get there from here.
What happened? Polyfills. Remy Sharp coined the term and developers stepped up and filled the gap, providing a way forward.
When virtually every major tech company on earth was focused on "how on earth can we imagine a new, better 'Web' based on XML?" - when billions of dollars in R&D had been spent over a decade and everyone was desperately trying to figure it out, developers said "JSON: I choose you!". Guess who carried the day?
JSON: I choose you!
It's not that standards bodies are "bad" at making standards. The problem, at its core, is how we approach/view standards and how we set up the right economics.
Fixing the economics
The Extensible Web Manifesto Logo by the great Bruce Lawson
Around 2010, a lot of people began talking about what was wrong with Web Standards and how we might fix it. This led to, not a new standards body, but a joint statement of core principles by people involved at many levels: The Extensible Web Manifesto.
Since it was published in 2013 it has become a statement of "core principles" for all of the major standards bodies involved with the Web.
The Extensible Web Manifesto is a short document, but it comes from considerably more detailed discussions and a bigger vision. It's a vision that says that the economics are broken. Failure isn't avoidable, it's inevitable. Experiments are necessary in order to get there from here.
Early electrical appliances plugged into light sockets!
As my friend Matt Griffin explains well both in his A List Apart Article The Future of the Web and his documentary on the Web The Future is Next - you can't do it right until you've done it wrong.
History, both of the Web and physical standards proves out that an evolutionary result is inevitable. When homes were first electrified, for example, it was for the purpose of artificial light. There weren't outlets - there was nothing to plug in. Companies were battling over lights. The result? Early appliance inventors stepped up and filled the gap - they made cords that screwed into light sockets and birthed a whole new industry!
The Extensible Web Manifesto simply argues that while we're busy arguing about light bulbs, the really amazing stuff is what you can do given electricity - and that we'll very likely miss it. It's unanticipatable. We will try, and we will fail. All failures aren't dead ends though. Service Workers, for example, are the result of many failed experiments.
Some failure persists and only looks like failure for a time - the sum of the DNA however ultimately provide new possibilities far far beyond any of our plans. If you were busy trying to "design" the perfect canine you'd never come up with a maned wolf. Chances are you've never seen a maned wolf since they only evolved in a certain environment in South America. But they are amazing and kind of a testiment to the power of evolution to create something that survives. Ultimately, we need things that survive in all of the environments, even the ones we aren't thinking of - and to do that we need to be adaptable.
The maned wolf is real, and it is awesome.
So, experiments and failure to reach "standard" are actually good things - that's how we can get better by exploring the edges and learning. But the original Web plan made it the norm that experimentations ship with browsers - out in the open, and usually very high level. That led to serious problems of miscommunication and frustration and interoperability challenges.
Polyfills showed us a different way forward though by mixing what little DNA we had exposed to us to fill the gaps. If you could polyfill a feature because a few browsers didn't support it, you could just as easily fill it before any browser supported it. Instead of proposing something that only works in a single browser - why not use the power of the Web to propose something that works in all browsers. A prollyfill (will it become a standard? I dunno, prolly something like it). Given lower level DNA, we can experiment. The Extensible Web Manifesto calls this DNA "Fundamental Primitives" and it encourages standards to focus majority efforts on them. Sometimes this may mean introducing new ones, but there's already a lot of rich DNA already locked away within the existing higher level APIs of the platform. Exposing it means we have more raw materials and can prollyfill more and better experiments. Beneath the existing features are all sorts of things that deal with network fetching, streaming, parsing, caching, layout, painting and so on. Each of these is being currently being 'excavated'.
The huge shift in economics that this could create is amazing.
In the mid-2000's, a lot of people wanted something like flexbox. It's only now, in 2016, that we can really begin to get broad feedback from developers who are largely just starting to see what they can use it for. In all likelihood, they will find some faults and have some better ideas. But if we could have given developers flexbox in a fashion that at least many of them could use to accomplish real things - that's a good incentive to be involved. The feedback loop could be tighted up considerably and it's possible because even if it fails to become a standard, it still works to accomplish something.
Wait a minute. Hold the presses. Think about that for a moment: Why do developers want to learn about standards? To feel smart? Shit no. Developers want standards because they have work to get done. A standard way is portable. "Being standard" means it's had a lot of eyeballs and ultimately it winds up being "free". But if they can't use a standard, they'll use a library. Why? Because things have to get done. Libraries have many of the same benefits, but not all. A lot of people ask me "why didn't we just standardize library X". The answer is generally simple: No library has been proposed as a standard, in a fashion compatible with standardization. They're usually too big, there are IP issues, and at the end of the day lots of people feel like they didn't get a say. But... If a proposal is delivered like a prollyfill that works everywhere, it's roughly that - only in the right form!
What we need then is a way to incubate ideas, build prolyfills and somehow get lots of eyeballs, use and participation. We need to see what sticks, and what can be better. We need ideas to fail safely without breaking the economics of participation or breaking the Web. And we need standards to act more like dictionary editors than startups. I explain in the presentation that, in fact, most of their successes have been this and that the idea is not at all radical, but I'll spare you that here.
A million voices cried out...
Meetup.com has 4 million people signed up who call themselves Web developers. How can we involve even thousands of them the way that we do standards now? The answer is simple: We can't. Discussing things on mailing lists while we wait forever for consensus and implementation doesn't scale. An incubator would need people helping cream rise to the top - it needs networked communication. Just as in networking, not all noise needs to be in all places.
Chapters.io is the answer (or at least the first attempt at an answer) to that problem. Chapters is an effort to pair people who are involved with standards with meetups about the Web who can help them find, try, and discuss things that are in incubation (or proposed for incubation). They provide a "safe" space for noisy and potentially less formal discussion. Ideas can be collected, summarized and championed.
This is not a far flung dream: It is happening. The Extensible Web has helped shape ideas like the Web Incubator Community Group (WICG) which provides just such an outlet for incubation and the Houdini Task Force. Browser makers and standardistas are making proposals there, and we're figuring out how to incubate them. Good ideas find champions and move forward. The WICG also provides a Discourse instance where developers can subscribe to and participate in a way a lot more plausible than a mailing list. Very recently, the jQuery Foundation announced that the standards team will be helping to establish, manage and champion chapters.
So what do you say? Are you in? The future Web wants you.
Don't be silent: Join the rebellion and help us organize a chapters.io near you. Tweet interest to me @briankardell or open an issue like this with the jQuery standards team and we'll see if we can help you get something started! If you are in Pittsburgh, PA - join us!
Very special thanks to my friends the great Bruce Lawson and (the also great) Greg Rewis for proofreading this piece.