Distilling market noise into market sense

VisionMobile is the leading research company in the app economy. Our Developer Economics research program tracks developer experiences across platforms, revenues, apps, tools, APIs, segments and regions, via the largest, most global developer surveys.

[Report] A new way of measuring Openness, from Android to WebKit: The Open Governance Index [Updated]

[Much has been said about open source projects – and open source platforms are now powering an ever-increasing share of the mobile market. But what is “open” and how can you measure openness? As part of our new research report (free download), VisionMobile Research Partner Liz Laffan introduces the Open Governance Index – a new approach to measuring the “openness” of software projects, from Android to WebKit]

Update: We have been amazed by the amount of interest to our Open Governance Index (OGI) report that we published just over two weeks ago. Our report was covered in mainstream media across Wired, ZDnet, PCPro, Gizmodo, ARS Technica, BGR, Zeit Online and ReadWrite Mobile. Our intention was to start a debate around ‘what’ openness is, ‘how’ it can be measured and ‘why’ it is important – and we certainly got the ball rolling!

Open Governance Index cover

Openness = governance

We at VisionMobile have been researching, investigating and helping to educate the industry about open source for the past five years.  In this time open source software has been transformed from geekware to business as usual. Much has been written and debated regarding open source licenses – from the early days of the GPL license to the modern days of the Android platform.

Despite the widespread use of open source, from Android to WebKit, there is one very important aspect that has been neglected: openness and how to measure it.

Openness goes far beyond the open source license terms and into what is termed Governance. While licenses determine the rights to use, copy and modify, governance determines the right to gain visibility, to influence and to create derivatives of a project, whether in the form of spin-offs, applications or devices. And while licenses apply to the source code, governance applies to the project or platform.  More importantly, the governance model describes the control points used in an open source project like Android, Qt or WebKit, and is a key determinant in the success or failure of a platform.

VisionMobile - Licensing vs. Governance Models

The governance model used by an open source project encapsulates all the hard questions. Who decides on the project roadmap? How transparent are the decision-making processes? Can anyone follow the discussions and meetings taking place in the community? Can anyone create derivatives based on the project? What compliance requirements are there for creating derivative spin-offs, applications or devices, and how are these requirements enforced?  It is governance that determines who has influence and control over the project or platform – beyond what is legally required in the open source license.

In today’s world of commercially-led mobile open source projects, it is not enough to understand the open source license used by a project. It is the governance model that makes the difference between an “open” and a “closed” project.

Measuring openness

Our research (free copy of full report here) showcases eight mobile open source projects: Android, MeeGo, Linux, Qt, WebKit, Mozilla, Eclipse and Symbian.  We selected these projects based on breadth of coverage; we picked both successful (Android) and unsuccessful projects (Symbian); both single-sponsor (Qt) and multi-sponsor projects (Eclipse); and both projects based on meritocracy (Linux) and membership status (Eclipse).

All of these are open source projects, whether platforms (Android, MeeGo, Qt, Symbian) or engines (Linux kernel, WebKit) or multi-project initiatives with a single, uniform governance. We appreciate that these projects are unique in many ways but they are all ultimately open source projects and to that extent our governance measures can be applied to them all equally. For example all of these projects have decision-making groups and processes that are directly comparable. In the Open Governance Index we attempted to document who these decision-makers are, how they operate, what processes are used to determine project decisions and how easily is to influence these project decisions.

Our research, carried out over a six-month period, included analysis of these popular open source projects, through discussions with community leaders, project representatives, academics and open source scholars. This research was partially funded by webinos, an EU-funded project under the EU FP7 programme, aiming to deliver a platform for web applications across mobile, PC, home media (TV) and in-car devices.

We quantified governance by introducing the Open Governance Index, a measure of open source project “openness”. The Index comprises thirteen metrics across the four areas of governance:

1. Access: availability of the latest source code, developer support mechanisms, public roadmap, and transparency of decision-making
2. Development: the ability of developers to influence the content and direction of the project
3. Derivatives: the ability for developers to create and distribute derivatives of the source code in the form of spin-off projects, handsets or applications.
4. Community: a community structure that does not discriminate between developers

The Open Governance Index quantifies a project’s openness, in terms of transparency, decision-making, reuse and community structure.

Does openness warrant success?

But what is it that makes an open source project successful? Why do some projects become an immediate success, while others barely get off the ground before crashing and burning? We know that just like commercial ventures, open source projects have different cultures and drivers – but we do believe that you should be able to measure the way that open source projects interact with the community of users and contributors that they build up around themselves.

Our research suggests that platforms that are most open will be most successful in the long-term. Eclipse, Linux, WebKit and Mozilla each testify to this.  In terms of openness, Eclipse is by far the most open platform across access, development, derivatives and community attributes of governance.  It is closely followed by Linux and WebKit, and then Mozilla, MeeGo, Symbian and Qt. Seven of the eight platforms reviewed fell within 30 percentage points of each other in the Open Governance Index.

Moreover, our research identified certain attributes that successful open source projects have.  These attributes are timely access to source code, strong developer tools, process transparency, accessibility to contributing code, and accessibility to becoming a committer.  Equal and fair treatment of developers – “meritocracy” – has become the norm, and is expected by developers with regard to their involvement in open source projects.

The Android Paradox

We found Android to be the most “closed” open source project. In the Open Governance Index, Android scores low with regard to timely access to source code in that the platform does not provide source code to all developers at the same time; it clearly prioritises access to specific developer groups or organisations and has acknowledged this with the delayed release of Honeycomb. Additionally Android scores low with regard to access to developer support mechanisms, publicly available roadmap, transparent decision-making processes, transparency of code contributions process, accessibility to become a committer (in that external parties cannot ‘commit’ code to the project) and constraints regarding go-to-market channels.

Android ranks as the most closed project, with an Open Governance Index of 23%, yet at the same time is one of the most successful projects in the history of open source. Is Android proof that open governance is not needed to warrant success in an open source project?

Android’s success may have little to do with the open source licensing of its public codebase. Android would not have risen to its current ubiquity were it not for Google’s financial muscle and famed engineering team. More importantly, Google has made Android available at zero cost, since Google’s core business is not software or search, but driving eyeballs to ads. As is now well understood, Google’s strategy has been to subsidise Android such that it can deliver cheap handsets and low-cost wireless Internet access in order to drive more eyeballs to Google’s ad inventory.

Equally importantly, Android would not have risen were it not for the billions of dollars that OEMs and network operators poured into Android in order to compete with Apple’s iconic devices. As Stephen Elop, Nokia’s CEO, said in June,2011, “Apple created the conditions necessary for Android”.

Moreover, our findings suggest that Android would be successful regardless of whether it is an open source project or not, to the extent that the vast majority of developers working on the project (the platform itself) are actually Google employees.

 Evolving the Open Governance Index

Having published the report, we aim to continue the discussion on governance, to refine our criteria even further and to make the OGI measure as meaningful as possible for the open source community. One of the first suggestions has been with regard to having a time dimension to the criteria i.e. does openness change over time. Mature open source projects such as Eclipse, Linux and WebKit that have stood the test of time, score quite highly with regard to openness of governance. But this has not always been the case. For example consider the following. Apple forked KHTML to create WebKit in the early 2000’s, releasing the first WebKit open source project in 2005 but with reviewer and commit rights restricted to Apple personnel only which effectively sidelined the KDE community. In 2007 however Apple reversed this decision allowing allow non-Apple developers to have full commit access to the WebKit source code version control system. This shows that openness can and does change over the project lifecycle.

Our vision for the Open Governance Index is to for it to be a robust, and as much as is possible, an objective measure of Governance for open source projects. We believe that this is necessary such that users and contributors to open source projects, including commercial entities, understand the means by which they can, or cannot, influence the direction and content of the project.

Download the full report for an in-depth analysis of the openness of Android, MeeGo, Linux, Qt, WebKit, Mozilla, Eclipse and Symbian. Drop us a line and tell us what you think.

– Liz

Addendum – Is copyleft more or less open?

We awarded a higher score to those licenses that are permissive and not copyleft licenses. Firstly it should be noted that all the licenses used by the eight mobile open source projects are Open Source Initiative (OSI) approved and meet the Open Source Initiative Definition, which provides for free redistribution of source code, access to source code and ability to create derived works amongst other requirements. We believe that the OSI is the appropriate arbiter of the appropriate Open Source License definition and all of the licenses used by the open source projects researched in this report meet this definition of being ‘open’.

However we also believe that from a commercial viewpoint there is still some concern about using code that is under a copyleft license – our experience of working with mobile software development organisations confirms this. Our findings suggest that organisations will be more comfortable using permissive licenses which do not mandate copyleft requirements and we reflect this in our criteria and scoring. We are happy to continue debating these findings further with the community. For example it has been suggested that the problem here is not with copyleft licenses but with the business model used by those organisations. Be that as it may, our experience is that this concern is still a valid one being expressed by many organisations, especially in the mobile device domain.

Finally we had a methodology typo which unfortunately survived the proof reading: assigning a bonus to “copyright assignment”. We fully acknowledge that copyright assignment is unnecessary – indeed we state this in our analysis of Qt whereby we acknowledge copyright assignment as inappropriate and a heavy-handed requirement.

[Liz Laffan is a Research Partner at VisionMobile. Liz has been working in the telecoms and mobile industry for over 20 years, with large telco organisations, start-up technology ventures, software development and licensing firms.  Liz’s interests lie in open source software governance and licensing and in particular how best can commercial organisations interact with open source projects.  She can be reached at liz [at] visionmobile.com]

  • Great article and report. A year ago I moved from the mobile industry to the sustainability consulting sector, where I now help global companies become resilient to the growing business risks imposed by climate change, resource depletion, energy security and explosive population growth.

    In my new field, measurement of openness and governance are critical, and have been the subject of much debate for years, and it's striking to see the parallels in your new report: not just in terms of the language used, but also in terms of the underlying reasons why measurement is so important.

    Software platforms have such a profound influence on how individuals engage – not just with the web but with brands, products and services, and ultimately each other – that I can see this becoming an area of much debate.

    • Liz

      Hi Geoff, Thanks for your comments. Our intention with this Report is to start the discussion around openness and governance for the benefit of all engaged parties in this industry, but of course if our work can be translated and useful to other industries that is also a bonus. Liz

  • http://www.youtube.com/watch?v=M3VjJSQ29w8

    Kevin Kelly has – since 1998 – had the best opinion on what's open enough 😉

    (Sorry for the video quality and the weird guy presenting!)

    • Andreas Constantinou

      I think I recognise that image on the opening slide 😉

  • Patrick

    Great article. With some important projects refusing contributions, I wish governance would be advertised in the same way as the license.

    Qt is a good example of project for which you can access the source but you have no way to contribute to the project. Nokia marketing talks a lot about "open governance" but the contributions are always rejected or ignored if they are not trivial.

    Android go further and does not even pretend that one can contribute.

    I wish this kind of projects would be separated from projects like Linux, WebKit and Mozilla. The governance factor is important when one wants to choose which technologies to use in a project.

    • Liz

      Hi Patrick, We agree that governance is a critical element when determining 'which' open source project to work with – not all open source projects are equal in terms of their acceptance of external contributions and this is an important factor for consideration. Hopefully our work will encourage projects to actually be more explicit about their governance terms, at least we have got the ball rolling in this regard. Liz

  • freeweaver

    This is a good article and a good attempt at making the boundaries of 'openness' clear. Unfortunately, the word open in the context of software has no single definition. Its exactly this obscurity that companies such as Google know well and take advantage of. By saying something is open, they are really saying nothing at all. For instance, Microsoft have just announced that they will be adopting an "open surface" cloud software model. This is Orwellian double speak if ever I heard it, don't you think? what is an open surface?

    Also, the "success" of a project is not at all based on the popularity of the results, just as Einstein's theory of relativity did not require popularity to be a success.

    The only thing that matters when it comes to free and open source projects is the free part – free as in freedom. Without this, it does not matter how "open" the source is, if you cannot change it, does it?

    • Liz

      Hi freeweaver, we attempt to bring some clarity and structure to 'what' openness is or at least should be in our Report and whilst this is a first attempt at such definition, it is at least a start in the process. I'm not sure what your saying with regard to success and popularity of results? Whilst 'free as in freedom' matters a great deal in all open source projects, we believe that openness is also critical, if you cannot contribute or if your contributions are ignored then you cannot change the project either. Liz

  • Alan

    Nokia talks a lot about open governance as something they want to do – obviously they are not quite there yet (and with the current progress…). But at least there's hope that Qt will get real open governance. Android seems to have no interest.

    • Liz

      Hi Alan, We understand that Nokia do have plans for more open governance of Qt so let's wait and see what happens there. I would agree with you regarding Android, thus far this is not a priority for them. Liz

  • While I appreciate that attempt to describe openness in a more useful manner for comparative purposes I just don't see the point of it here. The article reads like a hit piece – an attempt to discredit Android on the basis of accessibility and control of the source code and direction of the operating system, not an unbiased article on how a new measure might be applied to all projects. Perhaps this was not your intention, but in that case, focusing almost entirely on how Android stacks up to other mostly-non-operating-system projects does a disservice to your intent. This approach on the whole encourages people to dismiss preemptively the report itself devaluing the investment of time, money, and effort put into it.

    • I mean can you honestly tell me that you cannot see that the likes of Florian Mueller, and many mainstream publications are simply going to run with the headline, and not with the content and theme of the report as a way to continue their efforts of discrediting and casting fear, uncertainty, and doubt over Android as an anti-competitive tactic?

    • Liz

      Hi Anthony, The Article and the Report, which I'm not sure that you have read, goes into depth around the 13 criteria that we chose against which to measure open source projects. We also included Symbian which is an operating system. We explain in the Report why we picked the Projects that we did and as they are all open source projects we then identified criteria that could be used as a measure across all open source project. What became apparent to us very early on was how not-open Android is relative to these other Projects and that is what is highlighted in our Report and Article. What is also very apparent is how successful Android is and so we believe that we must address the fact of Android's success in the face of the non-openness of the Project. Liz

  • John D

    It would be interesting to see how these projects have fared over time. Was Eclipse as open in 2001 in its early days. Is there an argument that Google needs Android kept close to them as it is essential they want it to go in the direction they need as a business. Over time and once it is established things may look very different. A key dimension left out of this analysis is time. It would be good to see it repeated year on year.

    • Shawn W

      I think this is an important point. As I recall, early in the life of Eclipse when the core was progressing rapidly, very little of the work was done outside of IBM. As the tool matured and stabilized, more work shifted to plugins provided by many, many different people, and the core work shifted to be more focused around support for third-party work.

      Android definitely supports lots of third-party apps, but I think the core system is still in that "rapid innovation" phase, where Google engineers are making extensive and deep changes and don't want to slow down in the ways that would be required to make it more inclusive. I suspect that as the platform matures this will change.

      • Liz

        Hi Shawn W, as per my reply to John D. I would also add that Android is of course a relatively new open source project and our findings indicate that mature successful projects are more open than Android is now so we could conclude that over time Android will also become more open. No doubt when we do the next version of the OGI we will be able to test this assumption more precisely. Liz

        • Ryan Grant

          You couldn't be more wrong. Indeed, Android is becoming more closed. Android 3 is closed source, despite the continuing "open source" hype Google have gathered around it. Before you disagree – go try to find the source code and build it yourself.

    • Liz

      Hi John D, You raise a very interesting point. What we did notice was that projects have evolved over time. For example, WebKit. It wasn’t until 2005 that Apple launched the full WebKit open source project, but restricted reviewer and commit rights to Apple personnel only, which effectively sidelined the KDE developer community contribution to the project. Eventually, in late 2007, Apple responded by updating the WebKit committer and reviewer policy to allow non-Apple developers to have full commit access to the WebKit source code version control system. Therefore we do find that Projects become more open over time, as a result of developer community pressure and also the Project being more at ease with accepting external contributions. We will certainly be taking the time dimension into account for future releases of the OGI Report. Liz

  • Cellar

    I'm too lazy to read the thrice-belinked report, but, well, my experience with symbian was that the parts that were "opened up" were missing the important bits, such as a development environment (only available on windows, something I don't even have available, not to mention licensing) or, say, hardware support. The latter, in the case of symbian, means that the hundred million plus handsets already out there are not usable to try your hackery on. You're reduced to a not-even-alpha quality port to the beagle board. Nothing wrong with beagle boards, they make a fine platform, but given the choice I'd not waste effort with symbian on them.

    It doesn't stop there, of course. After closing up all links to documentation went dead, as the entire developer site was replaced by a page pointing at nokia. Following the link promised all the documentation again, even wikis and stuff ("ain't we cool eh") except that all those links were still dead with a "real soon now" notice months after the switchback. And then there was the "we're not 'open source', we're only 'open' as in 'open for business'" statement from them. While outside the scope of your endeavour, it makes oh so painfully clear what their real attitude was. And yes, critique did get silenced. How's that for open?

    The point is, of course, that it's very, very hard to measure objectively just how open is open without something like the GPL to enforce a specific definition of open, in its case full access to source code (and even then…).

    • Liz

      Hi Cellar, I appreciate your specific frustrations with Symbian in terms of its openness. The purpose of our Report, which I would recommend that you read, is to attempt to define some criteria against which 'openness' can be measured. We are trying to create a 'standard' for openness such that it is clear to the developer community and others how 'open' projects are. We agree with you that access to source code is of itself not sufficient for a project to be 'open' in terms of 'open for contributions' and 'open to influence'. Liz

  • Rick

    Just because a project does not solicit/condone outside contributions, this has nothing to do with openness. This governance thing is a separate issue.

    Android has been forked, primarily in an attempt to reduce the non-open aspects that are tied to most hardware. The project is called CyanogenMod. It's installed on my phone, it works very well.

    On the other hand, Google is guilty of being less open than it should be. Android is largely based on GNU/Linux components whose licences (GPL/LGPL) require you to post source for improvements that you distribute. Google is less than diligent in this matter.

    • Liz

      Hi Rick, We believe that acceptance of external contributions is actually an important factor with regard to openness of open source projects in that how can it be open if it is 'closed' to contributions, especially if there is no explanation given as to the contributions acceptance process? We appreciate that there are many forks of Android and we measure this also in our criteria in terms of being able to create 'derivatives' of that project. I cannot comment specifically regarding Google's compliance with the obligations of the GPL/LGPL but I expect that if this is the case then this would be something that FOSS will take up with Google if necessary. Liz

  • dimav

    where I can found 'free' and 'indepnden' symbain build for run it on my (or any other) mobile phone? no. I can not find this. but symbian have "58%" of 'free' instead 28% for android….

  • Jose_X

    >> Our findings suggest that organisations will be more comfortable using permissive licenses which do not mandate copyleft requirements and we reflect this in our criteria and scoring. We are happy to continue debating these findings further with the community.

    Let me contribute to this debate.

    I don't believe it is accurate to consider the GPL to be less open.

    The GPL offers an "attack" on trade secrets. Whether some firms like that or not, it remains true the GPL promotes greater openness than the other "liberal" licenses. If you view consumers, firms with open business models, and hobbyists as part of your intended audience, then the more accurate definition of openness should put the GPL out in front or on an equal footing.

    I am looking to judging businesses (not just software) on openness, and a firm's willingness to open up trade secrets is part of how open they are or are not. Many businesses, for example, small businesses created by former laid off workers (ie, people not ordinary engaged in business and looking for any help possible), don't have the resources of giants, so an open approach is a great help to them. As concerns software, a license like the GPL offers a safety not offered by licenses that can be exploited more readily by large proprietary competitors.

    Another reason to welcome openness would be when maximizing profits and control is not the main goal. Increasing collaboration and the safety net that is a large body of open information (eg, for running a business) are valuable and promote independence as well. Note that much of what we learn in school is effectively open (and firms are stronger for it). Much of our legal system is open. We live in largely open societies in many ways. Too much pulled away into proprietary subdomains (like closed firms) hurts the competitiveness and other areas of a society. Openness is valuable to many.

    So, regardless of the number of corporations which might get nervous over the GPL, we should still label it properly. Accuracy should be valued by anyone thinking about the future. Many groups will in time be looking for open models and options (either to adopt them in their business or to patron them as consumers).

    My opinions are reflected above, of course.

  • Tom Richter

    Try backing up your apps from a non-rooted Android device! – It doesn't work!
    Try getting those same APK files downloaded from the web – doesn't work, even for the "free" ones! So you are at the mercy of others if you ever need to restore your Android device. Google makes great efforts to keep our fingers off the executable apps files. You may look at everything, but it may be taken away from you at any time.

    My take as an end user, looking at the applications:
    – Linux = you control the source code
    – Windows = you control the binaries (i.e. you can keep copies of the Installers or setup media)
    – Android = you do not control any of the above; but Google controls your data.

    BTW did you know that if sync is ON, Google even has your WIFI encryption keys?



Distilling market noise into market sense

What gets desktop developers out of bed in the morning?

desktop developer segmentation

Despite all the hype around the death of the PC and the enormous amount of media attention focused on mobile,…

Continue reading ...

A proven model for targeting IoT developers


  What if you could identify a handful of developer personas, or segments, each with a very distinct set of…

Continue reading ...

1000 skills: Amazon Alexa as a metaphor for the IoT developer community


Alexa is a centerpiece of Amazon’s Smart Home push, and quickly growing to become one of the most promising attempts…

Continue reading ...


Research on the app economy and developer ecosystems

Developer Segmentation 2014


The Developer Segmentation Q3 2014 report is the most sophisticated study of developer segments to date. The report delivers a…

Continue reading ...

App Profits and Costs


This research report examines the critical success factors for a profitable app, and how business and technology choices, such as…

Continue reading ...

Developer Segmentation 2013

Developer Segmentation 2013

The Developer Segmentation 2013 report delivers a needs-based segmentation model that actually works, with extensive profiling of the eight principle…

Continue reading ...