Committer's Guide

Some types of fixes have "blanket approval" from the Documentation Engineering Team <[email protected]>, allowing any committer to fix those categories of problems on any part of the doc tree. These fixes do not need approval or review from a doc committer if the author doesn’t have a doc commit bit.

Blanket approval applies to these types of fixes:

Over the years, some implicit approvals were granted in the doc tree. This list shows the most common cases:

Before any commit, a build test is necessary; see the 'Overview' and 'The FreeBSD Documentation Build Process' sections of the FreeBSD Documentation Project Primer for New Contributors for more details.

If you want to download FreeBSD, compile it from sources, and generally keep up to date that way, this primer is for you. It covers getting the sources, updating the sources, bisecting and touches briefly on how to cope with a few local changes. It covers the basics, and tries to give good pointers to more in-depth treatment for when the reader finds the basics insufficient. Other sections of this guide cover more advanced topics related to contributing to the project.

The goal of this section is to highlight those bits of Git needed to track sources. They assume a basic understanding of Git. There are many primers for Git on the web, but the Git Book provides one of the better treatments.

This section describes the read-write access for committers to push the commits from developers or contributors.

First step: cloning a tree. This downloads the entire tree. There are two ways to download. Most people will want to do a deep clone of the repository. However, there are times when you may wish to do a shallow clone.

In Git, git checkout checks out both branches and specific versions. Git’s versions are the long hashes rather than a sequential number.

When you checkout a specific version, just specify the hash you want on the command line (the git log command can help you decide which hash you might want):

and you have that checked out. You will be greeted with a message similar to the following:

where the last line is generated from the hash you are checking out and the first line of the commit message from that revision. The hash can be abbreviated to the shortest unique length. Git itself is inconsistent about how many digits it displays.

Sometimes, things go wrong. The last version worked, but the one you just updated to does not. A developer may ask you to bisect the problem to track down which commit caused the regression.

Git makes bisecting changes easy with a powerful git bisect command. Here’s a brief outline of how to use it. For more information, you can view https://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination or https://git-scm.com/docs/git-bisect for more details. The man git-bisect page is good at describing what can go wrong, what to do when versions won’t build, when you want to use terms other than 'good' and 'bad', etc, none of which will be covered here.

git bisect start --first-parent will start the bisection process. Next, you need to tell a range to go through. git bisect good XXXXXX will tell it the working version and git bisect bad XXXXX will tell it the bad version. The bad version will almost always be HEAD (a special tag for what you have checked out). The good version will be the last one you checked out. The --first-parent argument is necessary so that subsequent git bisect commands do not try to check out a vendor branch which lacks the full FreeBSD source tree.

Set the 'good' version first, then set the bad (though the order doesn’t matter). When you set the bad version, it will give you some statistics on the process:

You would then build/install that version. If it’s good you’d type git bisect good otherwise git bisect bad. If the version doesn’t compile, type git bisect skip. You will get a similar message to the above after each step. When you are done, report the bad version to the developer (or fix the bug yourself and send a patch). git bisect reset will end the process and return you back to where you started (usually tip of main). Again, the git-bisect manual (linked above) is a good resource for when things go wrong or for unusual cases.

Git knows how to sign commits, tags, and pushes. When you sign a Git commit or a tag, you can prove that the code you submitted came from you and wasn’t altered while you were transferring it. You also can prove that you submitted the code and not someone else.

A more in-depth documentation on signing commits and tags can be found in the Git Tools - Signing Your Work chapter of the Git’s book.

The rationale behind signing pushes can be found in the commit that introduced the feature.

The best way is to simply tell Git you always want to sign commits, tags, and pushes. You can do this by setting a few configuration variables:

The ports tree operates the same way. The branch names are different and the repositories are in different locations.

The cgit repository web interface for use with web browsers is at https://cgit.FreeBSD.org/ports/ . The production Git repository is at https://git.FreeBSD.org/ports.git and at ssh://[email protected]/ports.git (or [email protected]:ports.git).

There is also a mirror on GitHub, see External mirrors for an overview. The latest branch is main. The quarterly branches are named yyyyQn for year 'yyyy' and quarter 'n'.

This section addresses tracking local changes. If you have no local changes you can skip this section.

One item that is important for all of them: all changes are local until pushed. Unlike Subversion, Git uses a distributed model. For users, for most things, there is very little difference. However, if you have local changes, you can use the same tool to manage them as you use to pull in changes from FreeBSD. All changes that you have not pushed are local and can easily be modified (git rebase, discussed below does this).

MFC workflow can be summarized as git cherry-pick -x plus git commit --amend to adjust the commit message. For multiple commits, use git rebase -i to squash them together and edit the commit message.

For MFC commits, for example a vendor import, you would need to specify one parent for cherry-pick purposes. Normally, that would be the "first parent" of the branch you are cherry-picking from, so:

If things go wrong, you’ll either need to abort the cherry-pick with git cherry-pick --abort or fix it up and do a git cherry-pick --continue.

Once the cherry-pick is finished, push with git push. If you get an error due to losing the commit race, use git pull --rebase and try to push again.

MFCs to branches that require approval require a bit more care. The process is the same for either a typical merge or an exceptional direct commit.

If you forget to to add the Approved by: line, you can do a git commit --amend to edit the commit message before you push the change.

If the push fails due to losing the commit race, rebase and try again:

Once the MFC is complete, you can delete the temporary branch:

Vendor imports are the only thing in the tree that creates a merge commit in the main branch. Cherry picking merge commits into stable/XX presents an additional difficulty because there are two parents for a merge commit. Generally, you’ll want the first parent’s diff since that’s the diff to main (though there may be some exceptions).

is typically what you want. This will tell cherry-pick to apply the correct diff.

There are some, hopefully, rare cases where it’s possible that the main branch was merged backwards by the conversion script. Should that be the case (and we’ve not found any yet), you’d change the above to -m 2 to pickup the proper parent. Just do:

to do that. The --abort will cleanup the failed first attempt.

If you do a MFC, and it goes horribly wrong and you want to start over, then the easiest way is to use git reset --hard like so:

though if you have some revs you want to keep, and others you don’t, using git rebase -i is better.

When committing source commits to stable and releng branches, we have the following goals:

With Subversion, we used the following practices to achieve these goals:

With Git, we will need to use different strategies to achieve the same goals. This document aims to define best practices when merging source commits using Git that achieve these goals. In general, we aim to use Git’s native support to achieve these goals rather than enforcing practices built on Subversion’s model.

One general note: due to technical differences with Git, we will not be using Git "merge commits" (created via git merge) in stable or releng branches. Instead, when this document refers to "merge commits", it means a commit originally made to main that is replicated or "landed" to a stable branch, or a commit from a stable branch that is replicated to a releng branch with some variation of git cherry-pick.

Git provides some built-in support for this via the git cherry and git log --cherry commands. These commands compare the raw diffs of commits (but not other metadata such as log messages) to determine if two commits are identical. This works well when each commit from main is landed as a single commit to a stable branch, but it falls over if multiple commits from main are squashed together as a single commit to a stable branch. The project makes extensive use of git cherry-pick -x with all lines preserved to work around these difficulties and is working on automated tooling to take advantage of this.

All vendor branches and tags start with vendor/. These branches and tags are visible by default.

We will explore an example for updating NetBSD’s mtree that is in our tree. The vendor branch for this is vendor/NetBSD/mtree.

The vendor trees usually have only the subset of the third-party software that is appropriate to FreeBSD. These trees are usually tiny in comparison to the FreeBSD tree. Git worktrees are thus quite small and fast and the preferred method to use. Make sure that whatever directory you choose below (the ../mtree) does not currently exist.

Prepare a full, clean tree of the vendor sources. Import everything but merge only what is needed.

This example assumes the NetBSD source is checked out from their GitHub mirror in ~/git/NetBSD. Note that "upstream" might have added or removed files, so we want to make sure deletions are propagated as well. net/rsync is commonly installed, so I’ll use that.

It is critical to verify that the source code you are importing comes from a trustworthy source. Many open-source projects use cryptographic signatures to sign code changes, git tags, and/or source code tarballs. Always verify these signatures, and use isolation mechanisms like jails, chroot, in combination with a dedicated, non-privileged user account that is different from the one you regularly use (see the Updating the FreeBSD source tree section below for more details), until you are confident that the source code you are importing looks safe. Following the upstream development and occasionally reviewing the upstream code changes can greatly help in improving code quality and benefit everyone involved. It is also a good idea to examine the git diff results before importing them into the vendor area.

Always run the git diff and git status commands and examine the results carefully. When in doubt, it is useful to do a git annotate on the vendor branch or the upstream git repository to see who and why a change was made.

In the example above we used -m to illustrate, but you should compose a proper message in an editor (using a commit message template).

It is also important to create an annotated tag using git tag -a, otherwise the push will be rejected. Only annotated tags are allowed to be pushed. The annotated tag gives you a chance to enter a commit message. Enter the version you are importing, along with any salient new features or fixes in that version.

At this point you can push the import to vendor into our repo.

--follow-tags tells git push to also push tags associated with the locally committed revision.

Now you need to update the mtree in FreeBSD. The sources live in contrib/mtree since it is upstream software.

From time to time, we may have to make changes to the contributed code to better satisfy FreeBSD’s needs. Whenever possible, please try to contribute the local changes back to the upstream projects, this helps them to better support FreeBSD, and also saves your time for future conflict resolutions when importing updates.

This would generate a subtree merge commit of contrib/mtree against the local vendor/NetBSD/mtree branch. Examine the diff from the merge result and the contents of the upstream branch. If the merge reduced our local changes to more trivial difference like blank line or indenting changes, try amending the local changes to reduce diff against upstream, or try to contribute the remaining changes back to the upstream project. If there were conflicts, you would need to fix them before committing. Include details about the changes being merged in the merge commit message.

Some open-source software includes a configure script that generates files used to define how the code is built; usually, these generated files like config.h should be updated as part of the import process. When doing this, always keep in mind that these scripts are executable code running under the current user’s credentials. This process should always be run in an isolated environment, ideally inside a jail that does not have network access, and with an unprivileged account; or, at minimum, a dedicated account that is different from the user account you normally use for everyday purposes or for pushing to the FreeBSD source code repository. This minimizes the risk of encountering bugs that can cause data loss or, in worse cases, maliciously planted code. Using an isolated jail also prevents the configure scripts from detecting locally installed software packages, which may lead to unexpected results.

When testing your changes, run them in a chroot or jailed environment, or even within a virtual machine first, especially for kernel or library modifications. This approach helps prevent adverse interactions with your working environment. It can be particularly beneficial for changes to libraries that many base system components use, among others.

Because the current policy recommends against using merges, if the upstream FreeBSD main moved forward before you get a chance to push, you would have to redo the merge.

Regular git rebase or git pull --rebase doesn’t know how to rebase a merge commit as a merge commit, so instead of that you would have to recreate the commit.

The following steps should be taken to easily recreate the merge commit as if git rebase --merge-commits worked properly:

The commands one would use, following the above example of mtree, would be like so (the # starts a comment to help link commands to descriptions above):

Note: if things go wrong with the commit, you can reset the YYY branch by reissuing the checkout command that created it with -B to start over:

Once you think you have a set of changes that are good, you can push it to a fork off GitHub or GitLab for others to review. One nice thing about Git is that it allows you to publish rough drafts of your work for others to review. While Phabricator is good for content review, publishing the updated vendor branch and merge commits lets others check the details as they will eventually appear in the repository.

After review, when you are sure it is a good change, you can push it to the FreeBSD repo:

Note: I used XXX and YYY to make it obvious they are terrible names and should not leave your machine. If you use such names for other work, then you’ll need to pick different names, or risk losing the other work. There is nothing magic about these names. Upstream will not allow you to push them, but never the less, please pay attention to the exact commands above. Some commands use syntax that differs only slightly from typical uses and that different behavior is critical to this recipe working.

If you’ve tried to do the push in the previous section and it fails, then you should do the following to 'redo' things. This sequence keeps the commit with the commit message always at XXX~1 to make committing easier.

Then go check it out as above and push as above when ready.

At this point, you have a new repo, where all new commits will go on the vendor/glorbnitz branch.

Git experts can also do this right in their FreeBSD clone, using git checkout --orphan vendor/glorbnitz if they are more comfortable with that.

Since this is a new import, you can just cp the sources in, or use tar or even rsync as shown above. And we will add everything, assuming no dot files.

At this point, you should have a pristine copy of glorbnitz ready to commit.

As above, I used -m for simplicity, but you should likely create a commit message that explains what a Glorb is and why you’d use a Nitz to get it. Not everybody will know so, for your actual commit, you should follow the commit log message section instead of emulating the brief style used here.

Now you need to import the branch into our repository.

Note the vendor/glorbnitz branch is in the repo. At this point the /some/where/glorbnitz can be deleted, if you like. It was only a means to an end.

Steps from here on out are much the same as they are in the case of updating a vendor branch, though without the updating the vendor branch step.

By 'good' we mean:

Here 'good' means:

Q: I’d like to mirror the entire Git repository, how do I do that?

A: If all you want to do is mirror, then

will do the trick. However, there are two disadvantages to this if you want to use it for anything other than a mirror you’ll reclone.

First, this is a 'bare repository' which has the repository database, but no checked out worktree. This is great for mirroring, but terrible for day to day work. There’s a number of ways around this with git worktree:

But if you aren’t using your mirror for further local clones, then it’s a poor match.

The second disadvantage is that Git normally rewrites the refs (branch name, tags, etc) from upstream so that your local refs can evolve independently of upstream. This means that you’ll lose changes if you are committing to this repository on anything other than private project branches.

Q: So what can I do instead?

A: Well, you can stuff all of the upstream repository’s refs into a private namespace in your local repository. Git clones everything via a 'refspec' and the default refspec is:

which says just fetch the branch refs.

However, the FreeBSD repository has a number of other things in it. To see those, you can add explicit refspecs for each ref namespace, or you can fetch everything. To setup your repository to do that:

which will put everything in the upstream repository into your local repository’s refs/freebsd/ namespace. Please note, that this also grabs all the unconverted vendor branches and the number of refs associated with them is quite large.

You’ll need to refer to these 'refs' with their full name because they aren’t in and of Git’s regular namespaces.

would look at the log for the vendor branch for zlib starting at 1.2.10.

For those willing to send e-mail messages through the FreeBSD.org infrastructure, follow the instructions below:

The in-tree compiler is currently Clang 15.x. Currently, GCC 12 and Clang 12, 13, 14 and 15 are tested in the github and project’s CI jenkins jobs. Work is underway to get the tree ready for Clang 16. The oldest project supported branch has Clang 12, so the bootstrap portions of the build must work for Clang major versions 12 to 15.

Tier 1 platforms are the most mature FreeBSD platforms. They are supported by the security officer, release engineering, and Ports Management Team. Tier 1 architectures are expected to be Production Quality with respect to all aspects of the FreeBSD operating system, including installation and development environments.

The FreeBSD Project provides the following guarantees to consumers of Tier 1 platforms:

To maintain maturity of Tier 1 platforms, the FreeBSD Project will maintain the following resources to support development:

Collectively, developers are required to provide the following to maintain the Tier 1 status of a platform:

Tier 2 platforms are functional, but less mature FreeBSD platforms. They are not supported by the security officer, release engineering, and Ports Management Team.

Tier 2 platforms may be Tier 1 platform candidates that are still under active development. Architectures reaching end of life may also be moved from Tier 1 status to Tier 2 status as the availability of resources to continue to maintain the system in a Production Quality state diminishes. Well-supported niche architectures may also be Tier 2.

The FreeBSD Project provides the following guarantees to consumers of Tier 2 platforms:

To maintain maturity of Tier 2 platforms, the FreeBSD Project will maintain the following resources to support development:

Collectively, developers are required to provide the following to maintain the Tier 2 status of a platform:

Tier 3 platforms have at least partial FreeBSD support. They are not supported by the security officer, release engineering, and Ports Management Team.

Tier 3 platforms are architectures in the early stages of development, for non-mainstream hardware platforms, or which are considered legacy systems unlikely to see broad future use. Initial support for Tier 3 platforms may exist in a separate repository rather than the main source repository.

The FreeBSD Project provides no guarantees to consumers of Tier 3 platforms and is not committed to maintaining resources to support development. Tier 3 platforms may not always be buildable, nor are any kernel or userland ABIs considered stable.

Other platforms are not supported in any form by the project. The project previously described these as Tier 4 systems.

After a platform transitions to unsupported, all support for the platform is removed from the source, ports and documentation trees. Note that ports support should remain as long as the platform is supported in a branch supported by ports.

Adding a port to the tree is relatively simple. Once the port is ready to be added, as explained later here, you need to add the port’s directory entry in the category’s Makefile. In this Makefile, ports are listed in alphabetical order and added to the SUBDIR variable, like this:

Once the port and its category’s Makefile are ready, the new port can be committed:

Check the port, preferably to make sure it compiles and packages correctly.

The Porters Handbook’s Testing Chapter contains more detailed instructions. See the Portclippy / Portfmt and the poudriere sections.

You do not necessarily have to eliminate all warnings but make sure you have fixed the simple ones.

If the port came from a submitter who has not contributed to the Project before, add that person’s name to the Additional Contributors section of the FreeBSD Contributors List.

Close the PR if the port came in as a PR. To close a PR, change the state to Issue Resolved and the resolution as Fixed.

First, please read the section about repository copies. Before you remove the port, you have to verify there are no other ports depending on it.

Alternatively, you can use the rmport script, from ports/Tools/scripts. This script was written by Vasil Dimov <[email protected]>. When sending questions about this script to the FreeBSD ports mailing list, please also CC Chris Rees <[email protected]>, the current maintainer.

A “ports freeze” was a restricted state the ports tree was put in before a release. It was used to ensure a higher quality for the packages shipped with a release. It usually lasted a couple of weeks. During that time, build problems were fixed, and the release packages were built. This practice is no longer used, as the packages for the releases are built from the current stable, quarterly branch.

For more information on how to merge commits to the quarterly branch, see What is the procedure to request authorization for merging a commit to the quarterly branch?.

As of November 30, 2020, there is no need to seek explicit approval to commit to the quarterly branch.

Merging commits to the quarterly branch (a process we call MFH for a historical reason) is very similar to MFC’ing a commit in the src repository, so basically:

where $HASH is the hash of the commit you want to copy over to the quarterly branch. The -x parameter ensures the hash $HASH of the main branch is included in the new commit message of the quarterly branch.

Please see Proposing a New Category in the Porter’s Handbook. Once that procedure has been followed and the PR has been assigned to the Ports Management Team <[email protected]>, it is their decision whether or not to approve it. If they do, it is their responsibility to:

This is much simpler than a physical category. Only a few modifications are needed:

Blanket approval for most ports applies to these types of fixes:

The packages are built multiple times each week. If a port fails, the maintainer will receive an email from [email protected].

Reports for all the package builds (official, experimental, and non-regression) are aggregated at pkg-status.FreeBSD.org.

No. The file can either be generated by running make index, or a pre-generated version can be downloaded with make fetchindex.

Any file directly under ports/, or any file under a subdirectory that starts with an uppercase letter (Mk/, Tools/, etc.). In particular, the Ports Management Team <[email protected]> is very protective of ports/Mk/bsd.port*.mk so do not commit changes to those files unless you want to face their wrath.

When the checksum for a distribution file is updated due to the author updating the file without changing the port revision, the commit message includes a summary of the relevant diffs between the original and new distfile to ensure that the distfile has not been corrupted or maliciously altered. If the current version of the port has been in the ports tree for a while, a copy of the old distfile will usually be available on the ftp servers; otherwise the author or maintainer should be contacted to find out why the distfile has changed.

An exp-run must be completed before patches with a significant ports impact are committed. The patch can be against the ports tree or the base system.

Full package builds will be done with the patches provided by the submitter, and the submitter is required to fix detected problems (fallout) before commit.