WordPress, Forking, and the Road to 4.0

Today at WordCamp San Francisco, Matt Mullenweg presented the plan for the upcoming 3.7 and 3.8 releases of WordPress.  In a nutshell, they will be small, developed in parallel, and (assuming all goes to plan) both released by the end of 2013.

This is hugely exciting!

The plan for these two releases makes sense and will help keep our code both robust and rapidly developed.  It also paves the way for what I think would be the ideal plan for WordPress 3.9 and 4.0.

An Existing Model

When jQuery planned for a recent release, they forked their own project.  Instead of releasing just one product, the team simultaneously released two versions: 1.9 and 2.0.

Both branches 1 of the project implement the same API, but the 2.X branch contains several speed improvements: namely, dropping compatibility support with older browsers and removing deprecated functionality:

You asked for it, you got it: jQuery 2.0 has arrived!

As promised, this version leaves behind the older Internet Explorer 6, 7, and 8 browsers. In return it is smaller, faster, and can be used in JavaScript environments where the code needed for old-IE compatibility often causes problems of its own.

Developers now have a clear choice when building their projects – maintain support for older browsers if necessary, or upgrade project requirements and leverage a faster framework.

WordPress

There has been a lot of talk about forking WordPress in recent months.  I’m suggesting a similar, albeit slightly different approach.

WordPress doesn’t need any more schisms in the community.  Forking the project to create a separate, discrete application would be a huge disservice to developers, clients, customers, and every variation of stakeholder in between.

Instead, I think we should follow jQuery’s example and release two, separate, complimentary branches of WordPress at the same time.

Background

At the WordPress Community Summit last October, Matt presented a fairly damning slide detailing WordPress’ system performance over the history of our various releases.  Despite every claim that new releases of WordPress are faster than their predecessors, only one release actually lived up to that claim.

Every subsequent release is faster not necessarily due to optimized code, but due to the fact that the underlying hardware continues to advance.

WordPress today is a robust application framework.  But to use that application framework, we have to drag along the collective bloat of 50+ releases worth of backwards compatibility, huge swaths of deprecated (or poorly designed) functionality, and workarounds for features lacking in older versions of PHP.

Dropping much of this cruft – while still maintaining the current API – would be a huge gain.

The Road Ahead

After WordPress 3.8 is out, I propose we work in two groups:

  1. The collection of teams is responsible for building the 3.9 API
  2. A team to walk through the code base and upgrade where possible

When we release 3.9 and 4.0, the APIs should be identical.  Any code written to run on WordPress 3.9 (with no deprecated notices, of course) will also run on 4.0.  The differences between the versions would include;

  • Upgrading the minimum PHP version requirement (allowing the use of namespaces, closures, and possibly traits)
  • Removing all functionality deprecated before version 3.9
  • Refactoring bottlenecks in code (i.e. $wpdb) for performance

The Future

When WordPress 3.9 and 4.0 are released, we will keep minor releases in sync (possible through 3.10 and 4.1 and beyond).  This will allow developers to build code utilizing newer PHP features by making the 4.0 release of WordPress a requirement.  It will also allow end-users to keep using WordPress on an un-upgraded server.

However, the dual release will also put pressure on developers, clients, and hosts to begin upgrading their infrastructure to support newer requirements – but without making the upgrade a hard requirement.

Why?

I write code for a living.  Most of my projects are deployed on servers that play well with newer APIs or boxes which I can control myself.  As a result, I can already use advanced PHP features in my projects (and leverage certain performance benefits at the same time).

I also give code away for fun – but that code has to function back to the minimum system requirements for WordPress.  I want to present a clear path for upgrading these requirements, one that’s beneficial to both developers and our customers.

I think a 3.9/4.0 dual release is the way to do that – and I’m more than happy to help make this a reality.  What do you think?

Notes:

  1. Since both versions of jQuery are maintained by the same team, it’s not a true fork.

Comments

  1. says

    With the rate of change in both hardware and software, maintaining backward compatibility too far into the past makes no sense. I like the idea of branching (rather than forking) to maximize future-proofing while accommodating backward compatibility. End-of-life-ing older branches is a way to limit how far into the past support needs to be ongoing.

    Discussion on this topic needs to keep going so thanks for the post!

    • says

      I’m not looking to maintain full backwards compatibility. That would be pretty much impossible. Just making it so that any code that works with the 3.9 API (excluding deprecated functionality) would also work with a 4.0 API. Basically, I’m opting to break forwards compatibility in order to move things ahead.

      At the same time, this idea is fraught with complications anyway because some of the API changes I’d like to make could potentially break far too much to be reasonable.

  2. says

    What I’d love to see is removing a lot of the legacy support and moving it to a plugin, i.e. a “compatibility pack” like Microsoft did with Office 2007. They released a small program that would allow you to load and edit items made with 2007 with your previous version. Not sure how the implementation would work, but it seems like that would cover a lot of ground.

    • says

      Considering the new goal to make future WordPress development plugin-based (ala MP6), I think this is a fantastic idea. Keep legacy/deprecated code in a compatibility plugin instead while we continue to add new, experimental functionality in plugins as well. Supporting older themes/plugins becomes a matter of installing a plugin. Awesome idea!

    • says

      I love this idea. I think this would also make the core really light, and we can make a lot of the suggestions that Matt talked about (if he had to re-do WordPress all over).

    • Bryan Petty says

      It seems like a great idea at first, but this really isn’t any different than how WordPress has been moving deprecated functions into wp-includes/deprecated.php, and still bundling it. As long as you still expect those deprecated methods to continue to work even if you moved them to a plugin and installed that plugin in the latest version of WordPress that they were removed from, any new features in that version of WordPress *still* need to be written in a way that they don’t break that old functionality or deprecated methods. Otherwise that compatibility plugin is actually completely useless.

      So in reality, it doesn’t make any sense to remove those deprecated methods or other API from core still. They still need to be maintained exactly the same way they would need to be in a plugin.

      Has anyone ever mass-removed deprecated.php from numerous WordPress installations to see how well things still worked? Keep in mind that there’s also still lots more deprecated API scattered throughout other files too, sometimes they can’t be moved in situations where they are expected to be loaded at specific times (and that stuff couldn’t be moved to a plugin either).

      • says

        My thought was that any potential compatibility pack would function similar to the jQuery Migrate plugin. It wouldn’t be required, but if you were for some reason using deprecated functionality that couldn’t be upgraded to work with the newer APIs, you could use the compatibility pack instead.

  3. says

    I’m not sure that forking is the way to go. It isn’t all that long ago that WordPress simply stopped supporting older versions pf PHP, with adequate warning that it was going to happen. If the customer base has no problems upgrading their PHP to keep in line with code advances, how much harder can upgrading a browser be?

    The ideal is for WordPress to be fast, efficient, and bring all the benefits that we already know and love. Working on two branches of WordPress code is not going to help matters. Taking people into account who do not upgrade their browsers should not be a major consideration at all. If they upgrade WordPress, why not their browser? Added to which, many of the browser upgrades are automatic these days, certainly with Firefox.

    • KleinBloemetje says

      @Viktoria,
      I think you should consider three types of users, each with their own upgrading scheme:
      1. WordPress hosting providers: the people running the hosting services for the WP sites who need to upgrade supporting tools like PHP and MySQL.
      2. WordPress site developers: the people creating new sites to be deployed on hosting services who need to upgrade the WP core, themes and plugins.
      3. WordPress site visitors: the people visiting the sites build on WP who should upgrade the browsers to be able to keep visiting the new sites.

      All three groups have their own ideas on upgrading the tools they use.

    • says

      Dropping support for PHP4 was a long, bumping, incredibly painful process. It started with developers labelling their code as “this requires PHP5″ and forcing their own clients to upgrade.

      When it comes to users, though, they aren’t often the ones who handle the upgrade – it’s up to the host. The majority of self-hosted WordPress sites live on shared hosting servers where the blogger has 0 control over the environment. Upgrading PHP simply is not an option for them.

      WordPress won’t drop support for PHP 5.2 (the current minimum requirement) until the number of sites/hosts/blogs using it drops to a tiny percentage of the whole. At the same time, hosts have no reason to upgrade beyond it since there isn’t any pressure on them from the vendor (WordPress) to do so.

      My proposal to branch is intended to apply such pressure to upgrade, while not making it an immediate, strict requirement.

  4. says

    I think it could be done in another way: by creating “compatibility plugins”. With that -possibly gigantic- plugin and a warning system, one can install it on demand and use deprecated stuff and draw mammoths on their walls, hehe.

    But yeah, branching would be a smart option, too. Mozilla did it a long time ago (even before switching to a rapid-release cycle) and they gradually stopped maintaining 2.x versions and moved on with 3.x versions. I really hope that we can see WordPress doing it.

    • says

      You’re not the first to recommend a compatibility plugin. I think that might be a fair compromise to the branching idea. I have some thoughts in mind for how to make that work, too.

  5. says

    I like your idea of branching instead of forking. WordPress uses SVN, but I know that Git-based projects often branch to add features or make major changes. It would be so much easier on plugin developers to be able to use the features offered by newer versions of PHP (such as lamba functions and namespaces!).

    • says

      The three things I really want to use in WordPress are:

      1. Namespaces
      2. Closures (lambda functions)
      3. Array literals and dereferencing

      In much of my code, I can use them already because the project I work on are hosted privately (i.e. the code is GPL, but doesn’t live in the .org repository). Array literals are easy to work with and dereferencing makes array work in PHP more similar to JavaScript (I argue this is a good thing).

      Closures are fantastic for callback functions – not necessarily for hooks and filters, due to the fact that they can’t be removed. But I would much rather use a closure than pass the function body as a string to create_function when I really do need lambda-style functionality.

      Namespaces are a huge win for unit testing. The way WordPress uses global variables will still get in the way (but could potentially be refactored a few further versions down the road), but “global” functionality is easier to work with. Let’s say an object in the \Posts namespace references get_permalink(). First, it will check to see if \Posts\get_permalink() exists, then it will default to the global \get_permalink(). Mocking API functionality becomes substantially easier when the code we’re trying to test is namespaced!

      • says

        I don’t have numbers on it, but the 3 features you name are more “quality of life” features vs. performance improvements. I’m not sure if branching/etc is worth it unless there is a nice performance boost to go along with it. It would be interesting to test some things to see how feasible it is to “clean up” bits of the WP core around backwards compatibility and what kind of performance boost you would get.

        • says

          The three features I mentioned specifically were top-of-mind things I do now in client code that I can’t do in plugins/themes for general release. The work arounds to not use this functionality in those cases does detract somewhat significantly from performance.

          How much upgrading PHP requirements and refactoring code would speed up WordPress core, though, would take some significant research. It would be an interesting thing to do, but without even the possibility of branching and/or just upgrading WP’s minimum requirements, it’s not even worth it right now.

  6. says

    Great thoughts here. As an alternative to a compatibility plugin, another approach would be to fill up deprecated.php with wrappers for the old API to translate them into the new.

    By the way, what are we talking about in terms of a new API? Are we talking about reforming the naming conventions for classes and functions, providing consistency and proper namespacing and eliminating global variables?

  7. says

    I think the opportunity to “break” backwards compatibility is definitely worthwhile at the 10 year mark. I know that Joomla! did this though and they didn’t do it very elegantly, so there is some history there on how NOT to do it. I think using the upgrade of the PHP version is a perfect time to do it.

  8. says

    This is kind of funny to me. The discussion is essentially, how can WordPress start to include the latest in quality code features and standards, without loosing the advantage they have gained by leaving in bad, old, depreciated code so that people won’t jump to fresh platforms. It is kind of like the life cycle of consoles. WordPress has been able to layer on top of itself for ten years and now has to consider having a next generation because of the effect of bloat on performance. It sure would be nice to see some real competition from a fresh cms instead of a bunch of coders trying to infuse WordPress with all the new hotness.

    How about a cms that is a bit more democratic and decentralized like linux with no one official brand hovering over it. That way when you contribute to the open source brand you aren’t shoving money in one mans pocket. Imagine it were Microsoft or google, Do you really want to contribute code to them for free when they are making billions and it gains you no official status? I’m not sure why Matt is any different. Can you find a directory of wordpress developers on WordPress.com? Or at least a directory of devs that contributed to core. We need a true open source cms so that all implementations are about quality of services and aren’t just the “official” brand.

    I have always felt that when I tell someone about WordPress that they are going to leave and go to WordPress.com where automatic gets a chance to sell them, and see them on something that is possibly contrary to what I suggested. That doesn’t feel like an open community.

    Now I feel like a troll, but I honestly get frustrated when someone like Eric, who is clearly a talented developer takes all of that knowledge and energy and tries to use it for WordPress. It feels like taking something that is better, and is not WordPress, tacking it on, and just handing it over to WordPress.com. When there are so many other people out there doing great stuff with cms’s that deserve to have part of the WordPress market, and deserve to have great developers contributing to their projects.

    I hear Matt is jumping back, I didn’t read to many details, but I applaud him for doing it because I think automatic should be funding 100% of core dev as long as WordPress.com exists and is picking off the low hanging fruit that could be sustaining other members of the community. And him getting back in shows that he might recognize that he should be building his own commercial product.

    On topic. +1 for a compatibility pack that can be discarded or toggled off to increase performance. It seems to make sense to move to migrating all depreciated functions to separate files where you can toggle off and test different depreciated code. It might make sense to have a separate set of depreciations for each version moving forward. So that if you need something from an earlier version you can set it to that version and all following version and gradually move that up to drop functionality. I’m don’t no what the right balance is for having the code be maintainable. I think that it might be nice to have WordPress move to be fully unit tested with old browser compatibility before dropping that compatibility.

    I think that WordPress has an obligation to stay backwards compatible until such time as a very small percentage of people are effected by removing depreciated functionality. I think that it is that backwards compatibility that has allowed it to be the leading cms. And that it should continue to be saddled with that responsibility and not foist off the responsibility for getting it current and keeping it competitive onto everyone that sold it and bought into it because of the free and easy mentality. So whatever is done I think that it should definitely be done slowly. I don’t care about the required php version because I think it is the responsibility of hosts to have a current offering. I do care about all of the code that I wrote continuing to function after a WordPress upgrade. To me that is part of the definition of WordPress.

    That is the core tenant of the WordPress empire just like Microsoft doesn’t make you write new win32 apps everytime a new version of windows comes out. If you want to latest and greatest your time, money, and energy should go to support people who develop that, not to propping up increasingly bloated mass market products. There are already more technically advanced ways to develop websites. Maybe stop trying to make WordPress something it is not and just use those :).

    • says

      ??? This sounds like a diatribe from someone with an axe to grind. For the most part it feels like the orginal topic got hijacked in order to provide a soapbox. It’s hard to take the poster seriously though, even when he does bring the topic back to development, since he doesn’t seem to know the difference between “deprecated” and “depreciated.”

    • says

      “Essentially” the argument is on how to continue to improve WordPress and push forward innovation and development while not leaving stakeholders in the dust. Several people are hesitant to upgrade WordPress (or its underlying dependencies – PHP, MySql, Nginx/Apache, Linux, etc) for specific, often valid reasons. Larger enterprise clients want to make sure the bugs are worked out of a new release before adapting. Government clients are concerned about the potential of newly-released security vulnerabilities. The fact is, WordPress releases often enough that maintaining backwards compatibility – despite the effort to maintain forward momentum – is hugely important.

      As for “democratic and decentralized,” I think you have a gross misunderstanding of how the WordPress community works. First of all, I don’t work for Matt, yet I contribute to WordPress and the ongoing development of WordPress pays my bills. Secondly, the lead developers on the 3.6 release weren’t affiliated with Matt or Automattic either. You can find a directory of all devs who contributed to WordPress in every copy of the software that you download from WordPress.org. My team – again, not affiliated with Matt or Automattic – currently has 10 people (out of ~30) listed on the Credits page.

      Again, clients going to WordPress.com is a misunderstanding of what WordPress is and who is involved. For many people, WordPress.com is just fine. It’s free, hosted in parallel with an incredible CDN, spread across multiple data centers for speed, and often follows the development of WordPress trunk. If you don’t own (or plan to own) a server and want to get started with WordPress, it’s one of the best options with the lowest barrier to entry. Also, if you want to migrate off of WordPress.com, their team offers ways to help you manage the migration.

      Why should Automattic fund 100% of WordPress core development when they aren’t the sole beneficiary of the product? Yes, they own WordPress.com, one of the largest single installations anywhere. But they aren’t the only host, they aren’t the only development team, and again they aren’t the only company that contributes to core.

      As for why I contribute – WordPress pays my bills. I’m employed full time to work on WordPress and related projects. As a result, my success depends largely on the continued progress and success of WordPress – there is a major incentive there for me to continue to contribute. At the same time, I also contribute to other open source projects and I’m involved in those communities as well.

      On topic, the idea of a “compatibility pack” is still in progress. During his State of the Word presentation, Matt talked largely about the future of WordPress lying in plugin development. My hope is that testing 3.7 or 3.8 or 3.9 could be as easy as installing a plugin. When you’re sure it works, upgrade and disable the plugin. Likewise, deprecated functionality could either be included in a core/canonical plugin installed alongside WordPress core (ala Akismet) and either kept around or disabled by the site maintainer.

      I care a bit about PHP versions because of the features I want to use in code. When writing client code for servers over which I have control, I can safely use functionality defined in PHP 5.3, 5.4, and 5.5. When writing code for public release – or for core – however, I’m limited to 5.2 compatibility. This means no namespaces, closures, late static binding, traits. There are workarounds here for sure, but the code is much less elegant and, in some cases, a bit less performant.

      Considering I began my career as a Windows developer, I know quite a bit about writing code for the Win32 API. And for the various .Net framework versions. And for the newer WinRT platform as well. No, Microsoft doesn’t force you to keep writing for the Win32 API – but that API still works. Even on Windows 8. While Microsoft continues to introduce new tools to aid in future development, they have maintained a huge amount of backwards compatibility at the same time – WordPress largely follows the same model, and I would argue this is why both products have been so successful.

      WordPress is leading the market because it’s stable, open, and well-supported. With new players joining the fold every day, it’s more challenging for WordPress to keep its edge. Faster development cycles is one way to do just that. Moving core development to plugins, as Matt has proposed for the 3.8 development cycle, is a solid way to keep that faster development cycle moving. I’m proposing a way to make that plugin-based-development idea work a bit more smoothly in parallel with core APIs.

      You say this is “taking something that is better, and is not WordPress, tacking it on, and just handing it over to WordPress.com.” I say this is taking WordPress, making it better, and handing it over to the community so everyone can benefit. That is the definition of WordPress.

  9. Bryan Petty says

    Sorry for the late entry, this post was on my radar for when I got a moment to gather my thoughts since I’m very interested in how we can finally tear out much of this functionality without breaking too much at once.

    One reason I’m really interested in this too is because from my viewpoint, there’s really one specific aspect of WordPress that is standing in the way of this happening, and that’s plugins (and themes, but really it’s plugins mostly). In fact, that’s exactly what you’re worried about breaking with any of the strategies suggested here when you say “this idea is fraught with complications anyway because some of the API changes I’d like to make could potentially break far too much to be reasonable”.

    So I had given this a lot of thought myself almost a year ago, and put together my own proposal for how we can achieve what you want to do here, except I was approaching the issue of safely, and gradually disabling those same plugins likely to break through an API versioning system modeled after Chrome extensions instead while removing that deprecated API at the same time:
    http://ibaku.wordpress.com/2012/11/08/wordpress-plugin-and-theme-api-manifest-versioning/

    Most of that inspiration came directly from Matt Mullenweg back when he mentioned that he’s like WordPress to work like Chrome in his 2012 State of the Word. Ironically, he mentioned Chrome twice this time in his 2013 State of the Word.

  10. says

    What I personally dreamed about is having one more branch of WP, based on it’s purpose. In fact, it already has 2 – regular WP and multisite mode.In my perfect world it would be great to have 2×2 branches – regular one and multisite, both of them can have WP “Blog” mode and WP “CMS” mode. In “Blog” mode you get everything you need for running a blog, no extra CMS features – posts, pages, media, default taxonomies, widgets etc. In “CMS” mode you get CMS-oriented features and more general defaults – pages, media, menus, custom settings page PLUS some essential CMS features we already have with plugins – custom post types builder, custom metaboxes builder, custom layout builder, custom taxonomies, better file management, custom user roles etc. As a matter of fact, WordPress is used as a CMS. And pure bloggers are looking for less heavy and simpler alternatives – like Ghost. That’s one of the reasons new post formats failed. It’s a feature we actually don’t need. The team would better spend all that time on smth CMS-oriented. But having this branch-based approach – a simpler, cleaner blog mode for pure bloggers along with a feature-packed CMS mode would make everyone’s life easier.

    Just my thoughts on the topic.

    And I completely agree on dropping legacy support for many old versions.

    • says

      I disagree with maintaining use case forks like that. We actually used to maintain separate forks of WordPress for single sites and multisite (remember WordPress MU?). Keeping code in-sync across the two was a maintainability nightmare, which is why we merged things back in WordPress 3.0 to consolidate our effort.

      Also, post formats didn’t fail. If anything, they’re picking up speed as development continues to move forward. Most new themes support them by default, and the new UI features that were being developed for 3.6 (and were pulled in order to meet a release deadline) will make it back in to core in the future. Failure would mean no one is using them … and that’s just not the case.

  11. Yngve says

    Any news on when folders will be introduced in media manager? Dont want to let this important feature be left to plugins.

Trackbacks

Leave a Reply