Linux Shared Libraries course, Munich, Germany, 20 July 2017

I've scheduled a public instance of my "Building and Using Shared Libraries on Linux" course to take place in Munich, Germany on 20 July 2017.  This one-day course provides a thorough introduction to building and using shared libraries. covering topics such as: the basics of creating, installing, and using shared libraries; shared library versioning and naming conventions; the role of the dynamic linker; run-time symbol resolution; controlling symbol visibility; symbol versioning; preloading shared libraries; and dynamically loaded libraries (dlopen). The course format is a mixture of theory and practical.

The course is aimed at programmers who create and use shared libraries. Systems administrators who are managing and troubleshooting applications that use shared libraries will also find the course useful.

You can find out more about the course (such as expected background and course pricing) at http://man7.org/training/shlib/ and see a detailed course outline at
http://man7.org/training/shlib/shlib_course_outline.html.

Cgroups/namespaces/seccomp/capabilities course

There are still some places available on my "Linux Security and Isolation APIs" that will take place in Munich, Germany on 17-19 July 2017.  This three-day course provides a deep understanding of the low-level Linux features (set-UID/set-GID programs, capabilities, namespaces, cgroups, and seccomp) used to implement privileged applications and build container, virtualization, and sandboxing technologies. The course format is a mixture of theory and practical.

The course is aimed at designers and programmers building privileged applications, container applications, and sandboxing applications. Systems administrators who are managing such applications are also likely to find the course of benefit.

You can find out more about the course (such as expected background and course pricing) at
http://man7.org/training/sec_isol_apis/
and see a detailed course outline at
http://man7.org/training/sec_isol_apis/sec_isol_apis_course_outline.html

man-pages-4.11 is released

I've released man-pages-4.11. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from over 30 contributors. It includes more than 300 commits changing over 100 pages. The changes include the addition of 5 pages, significant rewriting of 1 other page, and enhancements to many other pages.

Among the more significant changes in man-pages-4.11 are the following:

• Two new pages, userfaultfd(2) and ioctl_userfaultfd(2), written by me and Mike Rapoport, document the userfaultfd() system call that was added in Linux 4.3, along with associated ioctl() operations that can be performed on the file descriptor returned by that system call.
• A new statx(2) man page, written by David Howells, documents the statx() system call that was added in the just-released Linux 4.11 kernel.
• A new pthread_atfork(3) manual page documents the pthread_atfork() library function.
• The slabinfo(5) has been heavily updated to reflect current kernel details.

Linux Security and Isolation APIs course in Munich (17-19 July 2017)

I've scheduled the first public instance of my "Linux Security and Isolation APIs" course to take place in Munich, Germany on 17-19 July 2017. (I've already run the course a few times very successfully in non-public settings.) This three-day course provides a deep understanding of the low-level Linux features (set-UID/set-GID programs, capabilities, namespaces, cgroups, and seccomp) used to build container, virtualization, and sandboxing technologies. The course format is a mixture of theory and practical.

The course is aimed at designers and programmers building privileged applications, container applications, and sandboxing applications. Systems administrators who are managing such applications are also likely to find the course of benefit.

You can find out more about the course (such as expected background and course pricing) at
http://man7.org/training/sec_isol_apis/
and see a detailed course outline at
http://man7.org/training/sec_isol_apis/sec_isol_apis_course_outline.html

man-pages-4.10 is released

I've released man-pages-4.10. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from over 40 contributors. This release sees a large number of changes: over 600 commits changing around 160 pages. The changes include the addition of 11 pages, significant rewrites of 3 other pages, and enhancements to many other pages.

Among the more significant changes in man-pages-4.10 are the following:

• The pages—add_key(2), keyctl(2), and request_key(2)—describing the system calls for the kernel key-management facility have been substantially revised and extended. The keyctl(2) page consequently saw a tenfold increase in size. I did much of the work here, with a lot of help from Eugene Syromyatnikov.
• In cooperation with David Howells, the maintainer of the libkeyutils package (and the developer of the kernel key management facility), a number of pages in the libkeyutils package were moved to the man-pages project. The rationale for this change is that these pages describe kernel interfaces, and so man-pages is more reasonably their home. During the migration, many of these pages were also substantially enhanced. The migrated pages are: keyrings(7), persistent-keyring(7), process-keyring(7), session-keyring(7), thread-keyring(7), user-keyring(7), and user-session-keyring(7). Thanks to David Howells and Eugene Syromyatnikov for a lot of assistance with reworking the pages.
• I've added a new ioctl_iflags(2) page which describes inode flags (the attributes manipulated by the chattr(1) command) and the ioctl() operations for working with those flags.
• The details on the ioctl() operations that can be used with namespaces have been moved from the namespaces(7) page into a new ioctl_ns(2) page
• I've written a getentropy(3) page, which describes the new getentropy() function added in glibc version 2.25. This function, layered on top of the getrandom(2) system call, enables the caller to obtain bytes of randomness.
• The discussion of async-signal-safety has been moved out of the signal(7) manual page into a new new signal-safety(7) page. Along the way, some details have been added to the page, including discussion of a few glibc deviations from the POSIX standard.

Next Linux/UNIX System Programming course in Munich: 15-19 May, 2017

I've scheduled another 5-day Linux/UNIX System Programming course to take place in Munich, Germany, for the week of 15-19 May 2017.

The course is intended for programmers developing system-level, embedded, or network applications for Linux and UNIX systems, or programmers porting such applications from other operating systems (e.g., Windows) to Linux or UNIX. The course is based on my book, The Linux Programming Interface (TLPI), and covers topics such as low-level file I/O; signals and timers; creating processes and executing programs; POSIX threads programming; interprocess communication (pipes, FIFOs, message queues, semaphores, shared memory), and network programming (sockets).
     
The course has a lecture+lab format, and devotes substantial time to working on some carefully chosen programming exercises that put the "theory" into practice. Students receive printed and electronic copies of TLPI, along with a 600-page course book that includes all slides and exercises presented in the course. A reading knowledge of C is assumed; no previous system programming experience is needed.

Some useful links for anyone interested in the course:

• course overview (includes sample course materials, course dates and locations, and prices); 
• course topic list; and 
• information about the trainer (i.e., me).
• 2-page PDF containing key information about the course

Questions about the course? Email me via [email protected].

man-pages-4.09 is released

I've released man-pages-4.09. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from 44 contributors. This is one of the more substantial releases in recent times, with more than 500 commits changing around 190 pages. The changes include the addition of eight new pages and significant enhancements or rewrites to many existing pages.

Among the more significant changes in man-pages-4.09 are the following:

• A new pkey_alloc(2) page, written  by Dave Hansen, documents the pkey_alloc() and pkey_free() system calls added in Linux 4.9.
• Updates to the mprotect(2) page by Dave Hansen document the pkey_mprotect() system call added in Linux 4.9.
• A new pkeys(7) page, written  by Dave Hansen, provides an overview of the Memory Protection Keys feature added in Linux 4.9.
• A new pthread_getattr_default_np(3) page, written by me, documents the pthread_getattr_default_np(3) and pthread_setattr_default_np(3) library functions.
• A new strfromd(3) page, written by Wainer dos Santos Moschetta, documents the strfromd(3), strfromf(3), and strfroml(3) library functions added in the upcoming GNU C Library 2.25 release.
• A new fuse(4) page, written by Keno Fischer, partially documents the /dev/fuse device.
• A new tmpfs(5) page, written by me, provides an overview of the tmpfs filesystem.
• A new random(7) page, written by me, with help from Nikos Mavrogiannopoulos, Laurent Georget, and Ted T'so, provides an overview of the interfaces for obtaining randomness.
• A new sock_diag(7) page, written by Pavel Emelyanov and Dmitry V. Levin, documents the NETLINK_SOCK_DIAG interface.
• Updates to the namespaces(7) page, written by me, document the NS_GET_USERNS and NS_GET_PARENT ioctl() operations (added In Linux 4.9) that can be used for introspecting namespace relationships (discovering hierarchical relationships of user and PID namespaces, and the relationships of non-user namespaces with their associated user namespaces).
• Updates to the sched(7) page, written by me, document the autogroup feature added to the CFS scheduler in Linux 2.6.38 and describe how it renders the nice value ineffective in many use cases.
• Substantial updates to the elf(5) page, by Mike Frysinger, document ELF notes.
• Yet another slew of updates to the perf_event_open(2) page were provided by Vince Weaver.

In addition to the above, substantial changes were also made to the close(2), getpriority(2), nice(2), timer_create(2), timerfd_create(2), random(4), and proc(5) pages.

man-pages-4.08 is released

I've released man-pages-4.08. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from around 40 contributors. The release includes changes to nearly 200 man pages. Among the more significant changes in man-pages-4.08 are the following:

• Eugene Syromyatnikov made substantial updates to the quotactl(2) page, including adding information about project quotas and a number of XFS-specific subcommands.
• I made substantial rewrites, additions, and corrections in the cgroups(7) page.
• I added a new bswap(3) page documenting the bswap_16(), bswap_32(), and bswap_64() library functions.
• A large number of updates to the proc(5) added documentation for several files and for additional fields in already documented files.
• I updated the discussion of capability requirements in many pages to clarify cases where a capability is required in the user namespace where the process resides (rather than in the initial user namespace).

man-pages-4.07 is released

I've released man-pages-4.07. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from around 50 contributors. The release includes changes to over 140 man pages. Among the more significant changes in man-pages-4.07 are the following:

• Two new pages by Darrick Wong document ioctl() operations that he added to the kernel: ioctl_ficlonerange(2) documents the FICLONE and FICLONERANGE ioctls, and ioctl_fideduperange(2) documents the FIDEDUPERANGE ioctl.
• I've written a new page, mount_namespaces(7), which documents mount namespaces and the shared subtrees feature.
• I've written a new page, nextup(3), which documents the new nextup() and nextdown() functions that will appear in the upcoming glibc 2.24 release.
• I've added documentation for the MS_SHARED, MS_PRIVATE, MS_SLAVE, MS_UNBINDABLE, and MS_REC flags to the mount(2) page. In addition, I've substantially restructured, clarified, and tweaked the existing text in this manual page.
• I (with help from Kees Cook, Jann Horn, Eric W. Biederman, and Stephen Smalley) have added documentation of so-called "ptrace access mode checks" to the ptrace(2) page. These kinds of checks are performed in a number of system calls and other interfaces (e.g., accesses to various /proc files), and many other manual pages have been updated in this release to note the kind of check performed and add a reference to the ptrace(2) page.
• I've made various changes in the proc(5), most notably documenting various ptrace access mode checks, and also describing various new fields that have been added to the /proc/PID/status file in recent kernel releases.
• I added a description of the effect of capabilities within a user namespace to the user_namespaces(7) page.
• I substantially rewrote the getitimer(2) page, and in the process changed the license to one that is unambiguously free.

man-pages-4.06 is released

I've released man-pages-4.06. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from around 20 contributors. The release includes changes to just over 40 man pages. Among the more significant changes in man-pages-4.06 are the following:

• A new cgroups(7) page documents container groups, and includes some information about cgroups version 2, which was officially released in Linux 4.5.
• A new cgroups_namespaces(7) page documents cgroup namespaces, which will shortly be released with Linux 4.6. Corresponding changes have also been made in the clone(2), setns(2), and unshare(2) pages.
• The readv(2) page adds documentation of the preadv2() and pwritev2() system calls added in Linux 4.6.

man-pages-4.05 is released

I've released man-pages-4.05. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports, reviews, and comments from more than 70 contributors. The release includes changes to more than 400 man pages. Among the more significant changes in man-pages-4.05 are the following:

• A new copy_file_range(2) page, contributed by kernel developer Anna Schumaker, documents the copy_file_range() system call added in Linux 4.5.
• The personality(2) page has been greatly expanded, to add descriptions of personality domains.
• The fmemopen(3) page has been split into two, with a new open_memstream(3) page documenting just the latter function. At the same time, the description of fmemopen(3) (which was completely rewritten in glibc 2.22) was substantially revised and enhanced. 
• A new ntp_gettime(3) page documents ntp_gettime(3) and ntp_gettimex(3).
• A new posix_spawn(3) page, cowritten long ago by Bill O Gallmeister and me, documents posix_spawn(3) and posix_spawnp(3).
• A new lirc(4) page by Alec Leamas documents the lirc device driver.
• The adjtimex(2) manual page has been substantially revised and adds documentation of the ntp_adjtime(3) library function.
• The feature test macro (FTM) requirements in all manual pages have been updated, corrected, and in some cases simplified:
• Where appropriate, requirements for the use of _DEFAULT_SOURCE (new in glibc 2.19) have been added.
• The feature test macros _BSD_SOURCE and _SVID_SOURCE are now obsolete, and the man pages now indicate that these macros are only used with glibc 2.19 and earlier.
• Since certain FTM values automatically imply that other FTMs will be set to certain values, the FTM descriptions in many pages have been simplified.
• Mention of some obsolete FTMs (e.g., _POSIX_SOURCE and _XOPEN_SOURCE_EXTENDED) has been removed from the man pages.
• The feature_test_macros(7) page has been updated in line with the above changes.

man-pages-4.04 is released

I've released man-pages-4.04. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports,and  comments from more than 30 contributors. As well as a large number of minor fixes to nearly 90 man pages, the more significant changes in man-pages-4.04 include the following:

• The futex(2) man page was long in a sorry state, with many aspects of the system call undocumented in the man page. After more than a year working off and on (and more than 250 commits!), I've completed a major rewrite and expansion of the page (which is now nearly 6 times as long as it used to be). Some of the information was drawn from Ulrich Drepper's highly informative paper, Futexes are tricky. A lot more information came from reading kernel and user-space source code, doing some experimenting (the revised man page even includes an example program!), and significant input from a number of reviewers (special thanks here to Thomas Gleixner and Torvald Riegel).
• A new membarrier(2) man page was added, contributed by Mathieu Desnoyers, the developer of the new system call (added in Linux 4.3).
• Eric Munson added documentation of the new mlock2() system call and the new mlockall() MCL_ONFAULT flag to the mlock(2) man page.

man-pages-4.03 is released

I've released man-pages-4.03. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release is relatively small, but nevertheless nearly 40 people contributed patches, bug reports,and comments. The more significant changes in man-pages-4.03 include the following:

• Vince Weaver has once again been very active on the perf_event_open(2) page, making many updates related to recent kernel changes.
• Documentation of the ambient capabilities feature that was added in Linux 4.3 has been added to the capabilities(7) and prctl(2) pages.

man-pages-4.02 is released

I've released man-pages-4.02. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports,and  comments from around 15 contributors. As well as a large number of minor fixes to nearly 400 man pages, the more significant changes in man-pages-4.02 include the following:

• A new dlinfo(3) page documents the library function of the same name.
• The dlopen(3) page, which, in a single page, documented rather too many functions in the dlopen API, has been split into several smaller pages: dlopen(3), dlerror(3), dladdr(3), and dlsym(3).
• Documentation of the dladdr1(3) library function has been added to the (new) dladdr(3) man page.
• Documentation of the dlmopen(3) library function has been added to the dlopen(3) man page. In the process of writing this documentation, Carlos O'Donnel and I realized that the API has essentially been unusably broken since its first appearance in glibc 2.3.4 (somewhat more than 10 years ago!), as described in the BUGS section of the man page--yet another lesson in the consequences of adding an API without documentation or a clear specification of expected behavior. Together, we're working on fixing the interface, which is to say that we've been trying to determine what would be sane expected behavior, which I've been documenting and testing against against the current Solaris behavior while Carlos does the heavy lifting to fix the glibc dlmopen(3) implementation. With luck, things will be fixed in glibc 2.23 (which should be released in early 2016). 
• Daniel Borkmann has made a number of updates to the bpf(2) page that was added in the last man-pages release.

man-pages-4.01 is released

I've released man-pages-4.01. The release tarball is available on kernel.org. The browsable online pages can be found on man7.org. The Git repository for man-pages is available on kernel.org.

This release resulted from patches, bug reports,and  comments from nearly 50 contributors. As well as a large number of minor fixes to over 100 man pages, the more significant changes in man-pages-4.01 include the following:

• A new bpf(2) man page documents the bpf() system call. This page was primarily written by Alexei Starovoitov, the author of the system call.
• The queue(3) page has been updated (by reimporting and editing a recent FreeBSD version of the man page) to include a number of functions that were formerly undocumented.
• Thread-safety information was added to more than 50 man pages, bringing the total number of pages that carry this information to over 500.
• Marko Myllynen provided substantial updates for the locale(5) man page, so that it now provides near-complete documentation of the format of locale definition files.