Announcing a new kind of open source organization

Google has deep roots in open source. We're proud of our 20 years of contributions and community collaboration. The scale and tenure of Google’s open source participation has taught us what works well, what doesn’t, and where the corner cases are that challenge projects.

One of the places we’ve historically seen projects stumble is in managing their trademarks—their project’s name and logo. How project trademarks are used is different from how their code is used, as trademarks are a method of quality assurance. This includes the assurance that the code in question has an open source license. When trademarks are properly managed, project maintainers can define their identity, provide assurances to downstream users of the quality of their offering, and give others in the community certainty about the free and fair use of the brand.

In collaboration with academic leaders, independent contributors, and SADA Systems, today we are announcing the Open Usage Commons, an organization focused on extending the philosophy and definition of open source to project trademarks. The mission of the Open Usage Commons is to help open source projects assert and manage their project identity through programs specific to trademark management and conformance testing. Creating a neutral, independent ownership for these trademarks gives contributors and consumers peace of mind regarding their use of project names in a fair and transparent way.

Understanding and managing trademarks is critical for the long-term sustainability of projects, particularly with the increasing number of enterprise products based on open source. Trademarks sit at the juncture of the rule of law and the philosophy of open source, a complicated space; for this reason, we consider it to be the next challenge for open source, one we want to help with.

To get the Open Usage Commons started, Google has contributed initial funding, and the trademarks of Angular, a web application framework for mobile and desktop; Gerrit, web-based team code-collaboration tool; and Istio, an open platform to connect, manage, and secure microservices, will be joining the Open Usage Commons. If you use a trademark of one of the projects currently, you can continue to use those marks, following any current guidance from the project. As the Open Usage Commons is focused on trademark management, the contributor communities and technical roadmaps of these projects are not changed by joining the Commons, although we hope this new model encourages anyone who has stood on the sidelines until now to participate in these projects.

As the Open Usage Commons board wrote in their announcement, this is uncharted territory, and the Commons intends to “walk before they run,” so you can expect more information and activity from the organization in the coming months.

Learn more about the role of trademarks in open source and the Open Usage Commons at openusage.org.

By Chris DiBona, Director, Open Source at Google

Eclipse Day returns to Google

At Google, one of the tools we use in building external and internal products is Eclipse. In addition to that, we also release Eclipse-based tools. To celebrate this and say thank you to the developer community, we’ll be hosting Eclipse Day at the Googleplex on the Mountain View, CA campus on December 18th, 2013.

Eclipse Day is a great opportunity for both Eclipse users and contributors to network and share ideas. This year we have sessions that cover Orion, the Eclipse M2M tools, BIRT, Gerrit, CDT, Dart, Hudson, performance tuning in Eclipse, and scaling Eclipse to work with Google’s massive code base. During the one day conference, Eclipse projects and Eclipse-based products created here at Google will also be highlighted.

In previous years some of the most popular sessions have been the Eclipse Ignite talks: 5-minute, 20-slide presentations by attendees sharing what they are doing with Eclipse.

A big thank you to everyone at the Eclipse Foundation for assembling this great event. We are happy to welcome the Eclipse community to our campus and are always looking for ways to make this conference better. Please share your ideas and let us know your thoughts about this year’s program.

Pre-registration, which includes a $40 contribution to the Eclipse Foundation, is required for attendance. You may pre-register until December 17th, 2013, 2pm PST.

Hope to see you there!

By Alex Ruiz, Android Development Tools

Gerrit Hackathon #3, May 7-11, 2012

A great thanks goes out to Google for hosting the 3rd Gerrit Code Review Hackathon earlier this month at their Mountain View campus, it was well attended and very productive.

After quickly deciding to focus on adding a plugin framework to Gerrit during the hackathon, we discussed a few areas which could really benefit from plugins (authentication, hook, events, UI) and we identified new SSH commands as a possible quick prototype objective. After discovering some minor pitfalls of intense collaboration using Gerrit, we realized that we could make good use of the new Gerrit builtin rebase button. The rebase button did indeed come in handy for many of us to avoid overwriting each other’s changes with rebases, but we quickly noticed that simply having the owner and submitters allowed to rebase would not be adequate. This led to the proposal to upload a new rebase permission, which has been backported to be in the 2.4 release since that is the first release with the new rebase feature.

There were other minor Gerrit UI quirks discussed and fixes proposed for many of these during the following days:
• user links now link to the owner:user search instead of to the user’s dashboard, most of us found  that this is what we really want most of the time.
• Fixing the lack of reviewer visibility in the case of draft reviews
• set-account ssh command was also merged.

Now back to plugins; we completed our first prototype for SSH commands early Tuesday and we were on our way to a plugin framework.  We decided to break plugin concepts into 2 main types: plugins with full access to Gerrit’s internals (and bound to a specific Gerrit version), and extensions which would only be bound to light stable interfaces. Plugins and extensions both got their own classloaders, but extensions were isolated from Gerrit classes. Helloworld sample plugins and repositories were created on gerrit.googlesource.com for several of these allowing users to host plugins alongside Gerrit as distinct projects.

Shawn Pearce created two jar files to compile plugins and extensions against, the plugin jar is one massive jar with Gerrit and all its dependencies in it. This large jar file should make it possible for plugin developers to create plugins quickly without having to download the many dependencies independently using maven. The extensions jar file consists mainly of the new fairly sparse top level gerrit-extensions-api package.

Other plugin accomplishments:
• plugins got their own Guice injectors
• HttpServlet plugin support in Gerrit
• pegdown can be used by plugin developers to create lightweight documentation for their plugins
• plugin command to be able to operate on and view plugins in a live Gerrit instance

With an initial plugin infrastructure in place, initial rework of the Gerrit core started by expressing part of it in terms of plugin: refactoring the replication as plug-in and introducing the ability to create aliases in order to use plugins commands as surrogates of existing commands. We added the ability to capture JSON/RPC and SSH actions to generate audit-trails and, later in the week, the ability to add extra custom commit validation through plugins.

Additional plugins have been created to provide long awaited functionalities in Gerrit, such as the delete project command and an online server viewer accessible through a web link. As the number of plugins is expected to grow quickly it was proposed that we define a simple but published process to allow people to search through existing plugins and host in Gerrit contributed plugins and their descriptive wikis.

The final day we focused on how topic branches might be approached in Gerrit. We decided we did not want to add support for change labels to the database since we are attempting to migrate away from the db. To put this data into git notes, it is necessary to have an indexing system and we concluded it would need to be done (or attempted) with Lucene. This could then pave the path for greater use of Lucene in Gerrit.

Scalability was naturally on many of our minds. Git maintainer Junio C Hamano joined our discussions of the many hacks being implemented to deal with the git protocol's ref advertisement performance impacts.  Android Gerrit users seem to push many of the original git use case assumptions beyond their original intents, especially when it comes to ref counts.  We were delighted to get Junio's approval to propose non backwards compatible changes to the git protocol to make our use cases more efficient.

We also spent some time outlining an incremental approach to making open source Gerrit clusterable (the version running for android-review and gerrit-review is already clustered but uses much of Google's proprietary technologies such as GFS and BigTable). Several incremental steps were outlined on how to move Gerrit in that direction. The first step which is needed is to support a very simple master/master installation of Gerrit where both (or all if more than 2) masters share a common filesystem backend (likely a high end NFS server) and a common db. Two issues were identified here which need to be resolved before this is possible: Cache coherency and Submit conflict resolution.

So while this is by no means a full run down of the week, it should give you some insights into the progress which we made during this week and also into some of the plans for future Gerrit directions. With the beginnings of a solid plugin infrastructure now in place, we hope that the Gerrit community will pick up the task of providing shared plugins for common tasks and host these new plugins alongside the gerrit project at the gerrit-review site.

Some fun stats (external contributions counted also):
• New changes created during the 5 days of the Hackathon: 115
• New patchsets uploaded during the 5 days of the Hackathon: 353
• Changes touched during the 5 days of the Hackathon:143
• Changes submitted during the 5 days of the Hackathon: 63
• Changes submitted during the week after the Hackathon (many carryovers): 43

By Martin Fick, Qualcomm Innovation Center, Gerrit project submitter

This has been cross posted from the Repo and Gerrit Discussion List

GitTogether 2011

It has become an autumn tradition to host GitTogether, two days of unconference where Git developers, enthusiasts, and users gather at Google’s headquarters in Mountain View, California to share ideas on future directions of Git and to discuss the tools that support the larger ecosystem. This year was no exception. More than 55 people showed up for GitTogether 2011, and one thing that is notable is that at many of the attendees’ $DAY_JOB, the combination of Git on the desktop with Gerrit Code Review on the server side has become the standard set-up to run development work.

Photo by Brandon Casey

Google’s Dave Borowitz gave a talk about the recent addition of Git support to Google Code Project Hosting. Google also recently moved the Git server that hosts the Android open source community from kernel.org to an in-house Git server, which was the topic of Shawn Pearce’s talk. Junio C Hamano briefly described additional workflow elements being designed to help the kernel developers ensure the authenticity of the patch-flow after the recent break-in incident at kernel.org.

Since last year’s GitTogether, submodule support has been heavily enhanced. Jens Lehmann and Heiko Voigt, two Git developers who have been primarily working on the submodule support, gave an overview of where we are and what the future entails. The general direction is to make it easier to run various operations from the top-level superproject and have it go into submodules recursively when the project and the user choose to. There has been interest in using the native submodule support to update the implementation of “repo” wrapper used in Android and other projects. It seems that we are getting closer.

Support for large blobs that would not fit in the memory has been always lacking in Git. There recently has been a lot of work in the native support (e.g. storing them straight to the object store without having to read and hold the whole thing in core, checking out from the object store to the working tree without having to hold the whole thing in core, etc.). There are a few third-party tools and approaches with their own pros-and-cons, but it was generally agreed that adding a split-object encoding like Avery Pennarun’s “bup” tools uses would be the right way to help support object transfer between repositories to advance the native support of large objects in Git further.

The animated diff in Roberto Tyley’s Agit (git running on Android) demo was cool and received warm applause.

In addition to the technical discussions, we had a chance to get to know each other socially, enjoying lunches in the conference room (unlike the previous years, the group has grown too large to visit Google cafes without disturbing other Googlers) and dinners at some local restaurants. True to the tradition, there was a “patch of questionable value” to add different hash algorithm to Git in case SHA-1 hash collisions are found.

By Junio C Hamano, Open Source Programs

Shawn Pearce on FLOSS Weekly

Recently, the Google Open Source Programs Office’s Shawn Pearce appeared on FLOSS Weekly for the show’s 118th episode. FLOSS Weekly was started by Leo Laporte and Open Source Programs Manager Chris DiBona in 2006, and in 2009 it was nominated for a Podcast Award in the Technology/Science category.

Shawn is the creator of JGit, a reimplementation git in Java, and he spoke to hosts Randal Schwartz and Randi Harper on version control systems, including Gerrit Code Review, JGit, EGit, and Git.

By Ellen Ko, Open Source Team

Gerrit and Repo, the Android Source Management Tools

By Jeff Bailey, Open Source Team

A couple weeks ago, we announced the Android open source release. Beside it, we silently released the tools that we wrote in order to make handling a large multi-repository project manageable in git. If you had a chance to look through the Android open source website, you'll notice references to a tool called repo. Why did we write this? With approximately 8.5 million lines of code (not including things like the Linux Kernel!), keeping this all in one git tree would've been problematic for a few reasons:

* We want to delineate access control based on location in the tree.
* We want to be able to make some components replaceable at a later date.
* We needed trivial overlays for OEMs and other projects who either aren't ready or aren't able to embrace open source.
* We don't want our most technical people to spend their time as patch monkeys.

The repo tool uses an XML-based manifest file describing where the upstream repositories are, and how to merge them into a single working checkout. repo will recurse across all the git subtrees and handle uploads, pulls, and other needed items. repo has built-in knowledge of topic branches and makes working with them an essential part of the workflow.

The gerrit code review tool is based off of rietveld. Gerrit is itself split into two components: Half that runs on Google App Engine to provide front-end web service, and half that runs on a machine to handle attempted merges into the "upstream" branch, and the various code review branches. When we integrate the auto-builders into the system, that will also be handled by Gerrit.

We have a workflow diagram that shows how code gets into the system for Android. If you're looking to switch to git, but don't want to lose the ability for multiple people to commit into an upstream tree, this is one solution for you to consider. Interested? Find us at [email protected]