4.18 Merge window, part 1

By Jonathan Corbet
June 8, 2018

As of this writing, 7,515 non-merge changesets have been pulled into the mainline repository for the 4.18 merge window. Things are clearly off to a strong start. The changes pulled this time around include more than the usual number of interesting new features; read on for the details.

Architecture-specific

The 32-bit ARM architecture has gained fixes for Spectre variants 1 and 2.
32-Bit x86 systems now have a just-in-time compiler for eBPF programs.

Core kernel

There is a new polling interface for use with asynchronous I/O.
The new no5lvl command-line parameter turns off five-level paging even if the kernel and the hardware support it. This is essentially a "chicken bit" that can turn off this new feature if it creates problems.
The power domain performance levels patch set has been merged. This code extends the power-management subsystem to be able to run the entire system (including peripheral devices) according to the needed power/performance balance.
Trace markers (described briefly in this article) can now be used to fire triggers for actions like histogram generation. See this documentation patch for details.
The control-group memory controller supports a new memory.min parameter. Like the existing memory.low, it is meant to ensure that the group has a minimum amount of RAM available to it, but it is meant to provide a stronger guarantee even when no reclaimable memory exists. This commit includes the documentation for this new parameter.

Filesystems

There have been a few Btrfs improvements this time around. An empty subvolume can now be deleted with rmdir(); no special capabilities are required. The new FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR ioctl() commands can be used to manipulate various file attributes (the append-only and immutable flags, for example). There is also a new set of ioctl() commands to allow unprivileged users to look up subvolume information.
It is now possible for users running as root within a user namespace to mount filesystems, even if they lack privilege outside of the namespace. The filesystem type itself must be marked to allow this type of mount; the only filesystem so marked at this point is the filesystems in user space (FUSE) module, which has seen a number of changes to enable this functionality
The fscrypt module, used for encryption of F2FS and ext4 filesystems, has gained support for the Speck128 and Speck256 ciphers. Speck is somewhat controversial (the US NSA seems a little too enthusiastic about promoting it), but it does enable encryption on the lowest-end devices. Ted Ts'o said that it's unlikely to be enabled on higher-end devices at all. "This is really intended for 'The Next Billion Users'; phones like Android Go that was disclosed at the 2017 Google I/O conference, where the unsubsidized price is well under $100 USD".

Hardware support

Audio: Realtek RT1305/RT1306 amplifiers, Realtek RT5668B codecs, Mediatek MT6351 audio codecs, Analog Devices SSM2305 speaker amplifiers, Atmel I2S controllers, and Tempo Semiconductor TSCS454 codecs.
Graphics: NVIDIA Volta GPUs, Xen paravirtualized front-end displays, Allwinner A31 MIPI-DSI controllers, Thine THC63LVD1024 LVDS decoder bridges, Cadence DPI/DSI bridges, Samsung Exynos image scalers, Broadcom V3D 3.x (and newer) GPUs, NXP TDA9950/TDA998X HDMI CEC engines, and AMD Vega 20 GPUs.
Media: Video devices using I2C transports, Sharp QM1D1B0004 tuners, Cadence CSI-2 RX/TX controllers, OmniVision OV7251 sensors, Renesas R-Car MIPI CSI-2 receivers, and Sony IMX258 sensors. The (170 KLOC) atomisp driver has been removed from the staging tree due to a lack of progress.
Miscellaneous: ChipOne icn8505 touchscreens, Crane CR0014114 LED boards, Spreadtrum Communications SC27xx LED controllers, TI LM3601x LED controllers, Lattice MachXO2 SPI FPGA controllers, Rave SP EEPROM controllers, IBM virtual management channel adapters, Rockchip PCIe endpoint controllers, and STMicroelectronics STM32 inter-processor communication controllers.
Networking: Texas Instruments DP83TC822 PHYs and Microsemi Ocelot Ethernet switches.
Pin control: Actions Semi S900 pin controllers, Renesas R8A77470 and R8A77990 pin controllers, and Allwinner H6 R_PIO pin controllers.
USB: Richtek RT1711H Type-C USB controllers, Aspeed vHub virtual hubs, HiSilicon STB xHCI host controllers, Atheros AR71XX/9XXX USB PHYs, and MediaTek XS-PHY transceivers.

Miscellaneous

The crypto subsystem now supports the Zstandard compression algorithm and the AEGIS and MORUS [PDF] encryption algorithms.
The /proc interface for IPMI statistics has been removed; that information is still available in sysfs.
The (scrupulously undocumented) BPF type format mechanism provides a metadata format for the description of the data types used by BPF programs. Its initial use is for the pretty-printing of values in BPF maps.

Networking

The TCP protocol now supports zero-copy receive operations under some conditions.
The AF_XDP subsystem has been merged; it allows zero-copy networking under the control of one or more BPF programs loaded from user space. This commit contains a sample AF_XDP application.
The core bpfilter mechanism has been merged. It is not truly functional for packet filtering at this point, but the infrastructure is now there to build on. That infrastructure includes a reworked user-mode blob helper mechanism that is likely to see use well beyond bpfilter.
The in-kernel TLS protocol implementation has gained support for offloading that protocol support into suitably capable hardware. The Mellanox mlx5 driver now supports TLS offloading.
The TCP protocol supports selective acknowledgment (SACK) compression; its purpose is to limit the number of SACK packets sent when the network is already overloaded.
It is now possible to attach a BPF program to a socket and have it run on sendmsg() calls; that program can do things like rewrite the IP addresses in the outgoing packet.

Internal kernel changes

As part of the new AIO polling mechanism, the interface to the poll() method has changed. The new function is:
```
    __poll_t	(*poll_mask) (struct socket *sock, __poll_t events);
```
Many internal poll() implementations have been converted to this interface. To be able to support AIO polling, drivers should also implement the new get_poll_head() method, which returns the wait queue used for polling.
The qspinlock implementation has been improved to eliminate potential starvation problems.
There is a new __kernel_timespec structure:
```
    struct __kernel_timespec {
	__kernel_time64_t       tv_sec;                 /* seconds */
	long long               tv_nsec;                /* nanoseconds */
    };
```
Its purpose is to facilitate the creation of year-2038-safe system calls on 32-bit systems by making the internal time representation be the same for both the 32-bit and 64-bit versions. Various implementations of system calls with year-2038 problems (nanosleep(), for example) have been updated to use this new type.
The Sys V interprocess communication system calls have seen some work to make them year-2038 safe.
The kernel configuration language has grown a new macro definition subsystem; it is intended to facilitate moving various build-time tests from the makefiles into the Kconfig files. See Documentation.kbuild/kconfig-macro-language for details on how it works.
A number of the improvements to struct page discussed at LSFMM 2018 have been merged.

By the normal schedule, the 4.18 merge window should continue through June 17. The second half is likely to be somewhat slower than the first, though, since Linus Torvalds has indicated that he will be traveling during that time. If all goes to schedule, the final 4.18 release can be expected on August 5 or 12.

Index entries for this article
Kernel	Releases/4.18

4.18 Merge window, part 1

Posted Jun 8, 2018 23:25 UTC (Fri) by fratti (guest, #105722) [Link] (3 responses)

>Graphics: NVIDIA Volta GPUs, [...] and AMD Vega 20 GPUs.

Did a double take there because I was pretty sure both of these were unannounced, but actually at least Vega 20 seems to have been announced a few days ago.

Also, what's NVIDIA desktop stuff doing in the kernel? Is this just glue code for their binary blobs and PCI identifier stuff?

4.18 Merge window, part 1

Posted Jun 9, 2018 0:00 UTC (Sat) by knan (subscriber, #3940) [Link] (1 responses)

Volta has pro variants released. Titan V and Quadro GV100.

And this is for the reverse-engineered Nouveau opensource driver.

4.18 Merge window, part 1

Posted Jun 9, 2018 0:02 UTC (Sat) by fratti (guest, #105722) [Link]

Oh, neat, didn't know that. Nouveau is pretty quick.

4.18 Merge window, part 1

Posted Jun 9, 2018 0:04 UTC (Sat) by Kamilion (subscriber, #42576) [Link]

>Also, what's NVIDIA desktop stuff doing in the kernel? Is this just glue code for their binary blobs and PCI identifier stuff?

DRM console and modesetting, I'd figure.

Mounting filestytems as root in a user namespace

Posted Jun 9, 2018 2:15 UTC (Sat) by ebiederm (subscriber, #35028) [Link]

Fuse is not the first filesystem just the first filesystem with backing store.

The proc, sysfs, tmpfs, ramfs, devpts and mqueue filesystems have been mountable for a while.

Fuse was made possible by vfs work to handle xattrs, device nodes and other filesystem features that could lead to privilege escalation if they retained their usual filesystem handling.

2038 Fixes

Posted Jun 9, 2018 3:58 UTC (Sat) by songmaster (subscriber, #1748) [Link] (2 responses)

Why is __kernel_timespec::tv_nsec defined as a long long? Doesn’t that make it 64 bits (on current architectures), which is twice as wide as is needed to store the number of nanoseconds in a second? Even a signed int32_t can store 2e9, so what will the extra 32 bits ever be used for?

2038 Fixes

Posted Jun 9, 2018 7:51 UTC (Sat) by arnd (subscriber, #8866) [Link]

Two reasons:

- For comparison mode on 64 bit architectures, we want the same binary layout for structures with a timespec between existing 64 bit and new 32 bit user space.

- When copying out timestamps to user space, it's better to avoid structures with padding, this helps prevent data leaks.

Note that both inside of the kernel (timespec64) and in user space, tv_nsec is 'long', only the interface between the two uses an explicit 64 bit type. Most new machines these days are 64 bit (even for embedded), so tv_nsec is already excessively long as mandated by both posix and c99.

2038 Fixes

Posted Jun 14, 2018 17:46 UTC (Thu) by flussence (guest, #85566) [Link]

I suspect in the long term those bits will end up being abused as flags for something or other, by software authors sharing the opinion that they're pointless.

4.18 Merge window, part 1

Posted Jun 9, 2018 8:03 UTC (Sat) by seanyoung (subscriber, #28711) [Link] (2 responses)

Also BPF can do IR decoding now. This should remove the need to run lircd for IR protocols that the kernel IR decoders don't support, once the user-space tooling is complete.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/...

(Obviously I'm biases as I wrote the code).

IR decoding

Posted Jun 9, 2018 13:00 UTC (Sat) by corbet (editor, #1) [Link] (1 responses)

Argh, I'm not sure how I forgot to put that in. It's an interesting development that's been on my radar for a bit...

IR decoding

Posted Jun 11, 2018 18:03 UTC (Mon) by admalledd (subscriber, #95347) [Link]

Chances of an article about it sometime in the future? This is an interesting edge of the kernel that isn't noticed much until someone needs it, and a interesting bpf use as well...

4.18 Merge window, part 1

Posted Jun 11, 2018 3:38 UTC (Mon) by jcm (subscriber, #18262) [Link] (1 responses)

btw, technically it's "arm" or "Arm Architecture" now. They did away with the meaning of the acronym. Many people (myself included) strongly oppose and disagree with that, but it's what happened.

4.18 Merge window, part 1

Posted Jun 12, 2018 6:28 UTC (Tue) by marcH (subscriber, #57642) [Link]

> btw, technically it's "arm" or "Arm Architecture" now. They did away with the meaning of the acronym.

Yeah, like people will care or even notice.

Speck isn't "controversial", it's highly suspicious.

Posted Jun 15, 2018 11:06 UTC (Fri) by nettings (subscriber, #429) [Link] (6 responses)

The mail linked to in the article (well worth a read) is from one of the experts on the ISO standards panel if I understand correctly, and the ISO decided to shoot it down. Not just because of its provenance, but because of a long string of general fishiness that my lack of crypto knowledge doesn't let me understand in detail, but it's out there, both in Tomer Ashur's mail and other news outlets.

More on-the-record commentary from him as a spokesperson for ISO: https://www.cbronline.com/news/nsa-ciphers-iso
And another quite damning statement from another member of the ISO board: https://boingboing.net/2018/04/24/fool-me-once.html

In my opinion it's a shame to see this adopted by Linux until the severe allegations have been addressed. In crypto (particularly when it originates from a repeat offender), "innocent until proven guilty" is really the wrong approach, no matter how wise it is in jurisprudence.

Speck isn't "controversial", it's highly suspicious.

Posted Jun 15, 2018 17:31 UTC (Fri) by zyzzyva (guest, #107472) [Link] (5 responses)

I caution against reading too much into the ciphers themselves, as opposed to the interaction between the ISO and NSA. NSA has traditionally had the best cryptographers and has escaped some of the academic norms; now they are being held to them more strictly, and due to the NSA's past actions and the political environment even being expected to far exceed the norms.

I think that is clear if you actually read the academic literature, including third-party cryptanalysis but also the Simon+Speck team's own papers. The designers' latest paper is actually quite comprehensive and detailed in comparison to what you find from other cryptographers, and certainly *much* more than what you'd find from the NSA in the past. Yes, it does have parts that still need clarification, and naturally the NSA isn't going to provide details of their own cryptanalysis other than saying that it found no weaknesses and used in part some publicly known techniques -- so there's that. But the designers actually do make it clear how they claim to have set the security margins: they added AES-128's security margin of ~30% from the limiting attacks in the standard attack model, which were (and still are) differential attacks. Note that Dr. Ashur's slightly improved attacks using RX cryptanalysis are only valid in the known-key distinguishing attack model for certain weak key classes, so they're not directly comparable to "real" attacks like the differential attacks. I caution against relying too much on one person's opinion, especially if they have in the past stated preconceived political views, which might compromise their impartiality. Claims of a 15% security margin don't appear supported by the literature; the real margin in the standard attack model still appears to be 25-30%, similar to AES-128's.

Of course, some cryptographers (understandably) prefer larger security margins, as more of a buffer against future cryptanalytic advances. But there is a real-world cost to adding more rounds, and Speck is specifically designed to be a "lightweight" cipher, usable on devices where more conservative designs are not possible.

Speck is also an *extremely* simple ARX cipher, with a tiny parameter space. So it's actually quite unclear how it might be able to have any sort of intentional weakness, and I don't think any claim of a weakness actually appears in the literature. Meanwhile the NSA has approved Speck128/256 for encrypting the U.S. government's own data; so in that context it would be seem quite odd for it to have a known weakness.

Note also that due to the restrictions of where they work, the Simon+Speck team isn't generally able to respond publicly to criticism outside of their official publications. I agree it's problematic and it would be much better if they were fully open and more involved with the community, but just know that this means you don't always get their side of the story.

(Not to mention that Speck was added to Linux's crypto API as an alternative for no encryption, on very low-end devices without AES hardware support, where using AES is not feasible. It's disappointing to see people seriously argue for *no encryption* over Speck, when there aren't any currently feasible options for the use case -- see the prior posts in the thread for details. Even if it turns out that Speck is weaker than thought, for some users it still provides a massive security benefit over the status quo of no encryption. As a full disclosure, I am the person who wrote the patches.)

Speck isn't "controversial", it's highly suspicious.

Posted Jun 15, 2018 18:56 UTC (Fri) by zyzzyva (guest, #107472) [Link] (3 responses)

Sorry, my last paragraph should say "when there aren't any *other* currently feasible options for the use case". Speck128-XTS is *almost* still too slow, but is believed to be fast enough especially given the benefit of encryption over no encryption, and is among the fastest options considered. Unfortunately, the faster options had problems like being stream ciphers and therefore unsuitable for length-preserving disk encryption; being hand-rolled or very new constructions lacking proper cryptanalysis or publications; being actually cryptographically broken; and/or not having an effective key length in line with modern ciphers. In my personal opinion, it would be unethical to use no encryption, or a known insecure option like a stream cipher with nonce reuse, just to avoid the (mostly non-technical) "controversy" around Speck. Hopefully there will be other feasible options in the future, but for now Speck128-XTS seems the strongest contender from a technical perspective.

Speck isn't "controversial", it's highly suspicious.

Posted Jun 17, 2018 21:30 UTC (Sun) by nettings (subscriber, #429) [Link] (2 responses)

Thanks for your reply. It's certainly valid to argue for less-than-ideal crypto rather than none at all. But will users always be aware of it? Ok, if I want to prevent random people from prying into my phone, this might be ok. But some people will have more fundamental (or even existential) crypto needs, and they might not always know which compromise they are making. With no encryption at all, at least you know what you have, and act accordingly.

As to your comment about "activism", well, yeah, of course that comes into it, and justifiably so. Taking research from an institution with a track record of subverting random number generators and lobbying them into NIST standards for public use certainly should be questioned.
Looking at its lack of respect for the civil rights of US citizens is already quite sobering. For a non-US citizen like me, should I consider it anything other than a hostile force? I mean, our very own BKA certainly doesn't make me proud, but at least I don't find their stuff in the Linux kernel.

I cannot validate your technical argument for lack of knowledge, but I do value it. However, unlike you, I'm in the same position as most users: I have to trust crypto gurus. Now the lesson I've learned is to look for open-source implementations of algorithms developed in the open and vetted openly by experts who disclose their methods. You are basically saying, trust me, in this particular case it's ok to get into this nice uncle's car and take the ice cream. Yeah, one kid was never seen again a few years back, but uncle has seen the error of his ways. That is undoing many of the education efforts preached for years by Schneier and others, and goes against the only procedure laypersons have for chosing trustworthy cryptography.
If that's activism, call me an activist.

Speck isn't "controversial", it's highly suspicious.

Posted Jun 17, 2018 21:33 UTC (Sun) by nettings (subscriber, #429) [Link]

To stretch the ice cream analogy just a bit more: "Of course, only follow this advice if Mom or Dad can't buy you one".

Speck isn't "controversial", it's highly suspicious.

Posted Jun 18, 2018 2:41 UTC (Mon) by zyzzyva (guest, #107472) [Link]

Actually, today Android doesn't proactively inform you whether your phone is even encrypted or not, though there are ways for users to find out. Likewise, there are also ways for more technical users to check the type of encryption, e.g. full-disk or file-based and the cipher used. FWIW, I also recently updated fs/crypto/ to log the cipher it's using to the kernel log. Personally I'd support making this information more visible to non-technical users; just understand that this isn't a new problem, and having more allowed ciphers just adds another dimension to it. And at some point a simplification clearly has to be made, e.g. web browsers don't make you click through a pop-up showing all the crypto algorithms used in the certificate and TLS connection every time you visit a HTTPS web site, just in case you personally object to any of them. Rather, experts have already made a determination of how secure each algorithm is.

I believe that crypto primitives should never be taken at face value, and the designers should never be trusted. That's why we chose a cipher that, among other things, has undergone extensive independent cryptanalysis (at least in comparison to other AES alternatives), yet still has a security margin similar to AES; has a very simple and straightforward design that is in line with other ARX ciphers found in the literature; where the designers have released extensive design rationale that actually *exceeds* the quality of what is normally found in the literature (if you disagree, I encourage you to actually read the papers and compare them to others); that is naturally resistent to timing attacks; and that supports secure block and key sizes.

Yes, Speck is an NSA-designed algorithm, which given their dual mission I too view as a negative. But I disagree that NSA-designed algorithms should be removed from consideration entirely on principle, any more than you would refuse to consider e.g. Chinese algorithms due to not trusting China, or refuse to use HTTPS, Bitcoin, etc. because their security is dependent on the NSA-designed SHA-256 algorithm. Remember that the NSA's mission also includes "information assurance", they've long had some of the world's best cryptographers, and they've been involved in crypto standards for decades (both designing algorithms, like SHA-1 and SHA-2, and reviewing and strengthening algorithms), with a positive role in all but one case which was actually a very obvious backdoor using public key cryptography. Note that it's impossible for Speck to contain a public key backdoor, as its parameter space is far too tiny. Therefore, any weakness in it can be exploited by anyone, not just the designers. And given that the NSA has approved Speck128/256 for use in U.S. National Security Systems (even up to "Top Secret" stuff, IIUC), it would be *very* awkward for them to know it has a weakness.

I know the field of cryptography has always had a political element, but I really hope that we can retain at least a scrap of technical merit, and not start having to choose algorithms based purely on things like the where the designers work and live; or, even worse, being unable to use crypto at all because all the viable crypto choices are too politically contentious, yet "no crypto" is somehow okay.

BTW, the actual biggest weakness of the encryption used on phones is probably the user's PIN, as it's usually short enough to be brute-forced regardless of which cipher is used; see e.g. https://blog.malwarebytes.com/security-world/2018/03/gray.... I find it silly that people worry about a certain nation-state adversary breaking a *block cipher* (despite pretty solid evidence they can't), when said adversary, and even some less capable adversaries, can almost certainly already "break" the cryptosystem anyway even if the cipher is AES. Of course, this still doesn't make encryption useless; it's still good to make attacks as difficult and costly as possible, and raise the bar to exclude as many adversaries as possible.

Speck isn't "controversial", it's highly suspicious.

Posted Jul 27, 2018 1:42 UTC (Fri) by ras (subscriber, #33059) [Link]

> Speck is also an *extremely* simple ARX cipher, with a tiny parameter space.

It's hard to overstate that point. The meat of it is 7 lines of C, with no ancillary tables like S boxes. There is nowhere to hide. It gives me the same feeling of beauty I got from comparing Sha2 to Keccak. It makes all previous attempts look like they were made someone has throwing obscure code and magic numbers at the wall, stopping when the output passed randomness tests.

SPECK comments

Posted Jun 16, 2018 8:52 UTC (Sat) by zdzichu (subscriber, #17118) [Link] (3 responses)

> Ted Ts'o said that it's unlikely to be enabled on higher-end devices at all. "This is really intended for 'The Next Billion Users'; phones like Android Go that was disclosed at the 2017 Google I/O conference, where the unsubsidized price is well under $100 USD".

I may be reading this wrong, but what the implication here?
“It may be compromised, but people who can only afford low-end phone don't deserve security/privacy”?
or
“Most kernel developers can buy higher-end device and won't be affected by this, so don't complain”?

SPECK comments

Posted Jun 16, 2018 12:12 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

> I may be reading this wrong, but what the implication here?

I read it as, compromises have to be made to push out products at this price point.

SPECK comments

Posted Jun 16, 2018 18:00 UTC (Sat) by zyzzyva (guest, #107472) [Link] (1 responses)

> I read it as, compromises have to be made to push out products at this price point.

That's correct; currently most phones in that price range are unencrypted because AES isn't nearly fast enough, due to the lack of CPU support (ARM CE). Speck-XTS allows getting encryption to those devices now, without waiting years. The belief is that encryption is for everyone, not just people who can afford mid/high-end phones. So Speck is replacing no encryption, not AES.

Also I strongly disagree that Speck doesn't provide any "security/privacy". While I did write in the Kconfig help that "Speck may not be as secure as AES, and should only be used on systems where AES is not fast enough", that's not truly supported by the current academic literature, despite ~70 cryptanalysis papers on Simon and Speck, and even taking into account a certain very vocal person's claims. In fact, currently the best attacks on AES are actually *better* than the best attacks on Speck. I included that conservative statement primarily out of caution, since Speck is a newer algorithm with fewer years of study, so there is probably a greater chance of improved attacks being published on Speck, in comparison to AES; and also to a lesser extent, an understanding that people are using Simon and Speck as an outlet for political activism, so I don't want people unfamiliar with the situation to choose Speck and potentially run into said activism unless they have no other option.

SPECK comments

Posted Jun 17, 2018 23:20 UTC (Sun) by flussence (guest, #85566) [Link]

>In fact, currently the best attacks on AES are actually *better* than the best attacks on Speck.
It would be a strange situation indeed if the opposite were already true.