|
|
Subscribe / Log in / New account

5.3 Merge window, part 2

By Jonathan Corbet
July 22, 2019
At the end of the 5.3 merge window, 12,608 non-merge changesets had been pulled into the mainline repository. Nearly 6,000 of those were pulled after the first-half summary was written. As expected, there was still a lot of material yet to be merged for this development cycle.

Some of the more significant changes merged in the second half of the merge window are:

Architecture-specific

  • User-mode Linux now has a "time-travel mode" that can speed up or slow down time to help with debugging of timing-related code.
  • The RISC-V architecture now has huge-page support.

Core kernel

  • The io_uring mechanism has gained support for asynchronous sendmsg() and recvmsg() operations.
  • Kernel probes ("kprobes") have gained the ability to dereference pointers into user space. This can be used, for example, to look inside structures passed into system calls. Some documentation can be found in this commit. It is also now possible to set kprobes at boot time.
  • There is a new configuration option (PREEMPT_RT) to select realtime preemption. The code to implement that selection is not yet upstream, but this is a signal that said code is coming soon.

Filesystems and block layer

  • The ubifs filesystem has gained support for zstd compression and verification of offline-signed filesystem images.
  • The Ceph filesystem now supports security labels on files and, thus, now works with SELinux.
  • There is a new mount option (nconnect=) for NFSv4 filesystems that allows the administrator to request that multiple TCP connections be established to the server. Traffic is then load-balanced across those connections.

Hardware support

  • Graphics: Samsung S6E63M0 RGB/SPI panels, OSD OSD101T2587-53TS DSI 1920x1200 video mode panels, ST-Ericsson multichannel display engines, and AMD NAVI GPUs.
  • Miscellaneous: NXP IMX8MN pin controllers, Qualcomm SM8150 pin controllers, Cypress HyperBus-connected devices, ROHM BD70528 power-management ICs, Rockchip RK809 and RK817 power-management ICs Cirrus Logic CS47L15, CS42L92, CS47L92 and CS47L93 codecs, MediaTek MT7621/MT7628 I2C controllers, SiLabs 5341 and 5340 clock generators, and Apple SPI keyboards and trackpads.

Security-related

  • There are two new command-line options, init_on_alloc= and init_on_free=, that will cause memory allocations to be initialized to zero at allocation and free time, respectively. They improve security at some runtime cost, and are expected to be enabled on Android and ChromeOS systems. See this commit for details.
  • There is a new event filtering mechanism for perf events under KVM on the x86 architecture; it allows an administrator to specify which types of events can (or cannot) be monitored by guests. Some details can be found in this commit.

Virtualization and containers

  • The new virtio-iommu driver emulates an I/O memory-management unit on top of the virtio mechanism.
  • There is a new virtio-pmem driver that emulates persistent memory devices in the guest.

Internal kernel changes

  • Kernel makefiles can now contain a header-test-y declaration specifying header files that should be able to be compiled on their own; this assertion will be tested at build time if the HEADER_TEST configuration variable is set. See this commit for details. The plan is to test all kernel headers this way, but many have to be excluded in current kernels.
  • The conversion of kernel documentation to the ReStructured Text format has jumped forward with this massive patch set taking care of most of the remaining plain-text files. Now that the formatting issues are mostly taken care of, all that is left is the trivial task of integrating the documentation and updating it to match the current state of the kernel.
  • The venerable floppy driver has been marked as orphaned, since the last remaining maintainer no longer has a working drive.

Now comes the task of stabilizing all of that work. If the usual schedule holds — and it almost always does these days — the final 5.3 release can be expected on September 8 or 15.

Index entries for this article
KernelReleases/5.3


to post comments

5.3 Merge window, part 2

Posted Jul 22, 2019 18:11 UTC (Mon) by mtaht (subscriber, #11087) [Link] (3 responses)

Time travel mode looks potentially *wonderful* for finding bugs in long running daemons. There was a dhcpd6 once that only happened after 51 days of uptime, and I've hit numerous other bugs like that in my career.

+10!

5.3 Merge window, part 2

Posted Jul 22, 2019 19:25 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Yes, this is super-nice.

You can also do that with VirtualBox (by specifying the "warp factor" via --warp-pct) but I'm not aware of any other virtualization environment that can do this.

5.3 Merge window, part 2

Posted Jul 23, 2019 20:13 UTC (Tue) by johill (subscriber, #25196) [Link]

I looked in qemu, and it sort of has this - if you emulate fully instead of using kvm, which is not feasible.

I think it *could* be done in qemu, remove the TSC, simulate an HPET that has the desired properties and use a virt-aware guest to do the idle handling? But it's tricky and probably requires touching not only qemu and the guest kernel, but quite possibly also the host kernel. Though then I wonder how virtualbox does it, maybe I should look at that.

Anyway, UML is good for me and actually simplifies my use case, so that's it for now. Seeing how much interest this has garnered, maybe somebody else will pick it up elsewhere :-)

Btw, I broke it in a build fix at the last minute, so you need https://patchwork.ozlabs.org/patch/1134739/ to actually use it.

For wifi tests in wpa_s/hostapd it cut down the time required from ~45m to ~7m on my machine, but I run with more kernel debug options now (which wasn't even possible before) in ~10m.

Next up is to add vhost-user support to UML (WIP) and then add a virtio-clock/idle driver to UML that uses it, so I can run multiple things (or instances of UML machines) on the same virtual time...

5.3 Merge window, part 2

Posted Jul 23, 2019 19:23 UTC (Tue) by re:fi.64 (subscriber, #132628) [Link]

Have you seen Mozilla's rr debugger as well? https://rr-project.org/

5.3 Merge window, part 2

Posted Jul 22, 2019 18:43 UTC (Mon) by smurf (subscriber, #17840) [Link] (1 responses)

Ah. Init to zero. Why not init to 0xdeadbeef1337babe, or something along these lines? that could at least find us some bugs instead of papering them over. (Yes I know, that's not helpful for Android users – but it'd be helpful for kernel developers.)

5.3 Merge window, part 2

Posted Jul 22, 2019 20:13 UTC (Mon) by yann.morin.1998 (guest, #54333) [Link]

Hello,

> Why not init to 0xdeadbeef1337babe [...]

I think the following, from the referenced commit, does already address your concerns:

> If either SLUB poisoning or page poisoning is enabled, those options take
> precedence over init_on_alloc and init_on_free: initialization is only
> applied to unpoisoned allocations.

Regards,
Yann E. MORIN.

5.3 Merge window, part 2

Posted Jul 23, 2019 8:27 UTC (Tue) by unixbhaskar (guest, #44758) [Link] (2 responses)

"There is a new mount option (nconnect=) for NFSv4 filesystems that allows the administrator to request that multiple TCP connections be established to the server. Traffic is then load-balanced across those connections. "

This is certainly getting over the lagging issue.

5.3 Merge window, part 2

Posted Jul 23, 2019 20:56 UTC (Tue) by Sesse (subscriber, #53779) [Link] (1 responses)

Or making it much worse by inducing bufferbloat.

5.3 Merge window, part 2

Posted Jul 24, 2019 0:43 UTC (Wed) by neilbrown (subscriber, #359) [Link]

> Or making it much worse by inducing bufferbloat.

Certainly possible. We would love to find a way to have the kernel auto-configure this number, but no brilliant ideas have come forward.

When you have parallel hardware (multiple cables, multiple DMA engines, multiple offload engines, multiple CPU cores), having multiple TCP connections can really help.
When you don't (or don't have enough), then multiple connections defeat congestion control and bloat buffers.

Determining the capabilities of the hardware (which could change) is, as yet, an unsolved challenge.


Copyright © 2019, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds