Tracking issue for fs::Metadata extensions on Windows based on handle information

[alexcrichton]

This is a tracking issue for APIs added in #62980, namely the following Windows-specific APIs:

impl MetadataExt for Metadata {
    fn volume_serial_number(&self) -> Option<u32>;
    fn number_of_links(&self) -> Option<u32>;
    fn file_index(&self) -> Option<u64>;
}

The motivation of these accessors are that the standard library learns about them in its call to GetFileInformationByHandle but doesn't expose it from the standard library, which means consumers that want to learn this either need to reimplement the pieces in the standard library or execute a second syscall. Concrete motivation for these fields arose during CraneStation/wasi-common#42.

These fields all return Option which is a bit odd, but there's a few way of learning about a Metadata on Windows. When learned through a DirEntry these fields aren't available but when learned through a fs::metadata call or File::metadata these fields are available. It's unfortunately not as uniform as Unix, but we don't have a ton of options as well.

[mqudsi]

More context for future readers: the Option is specifically a "limitation" of the Windows API, as it uses a "fast mode" to get info on files when enumerating directories that avoids looking up each directory member in the filesystem index (allocation table/file table) and instead uses information cached in the directory object to return the objects faster with less IO and fewer system calls (once per directory instead of once per file).

This isn't the first time I've come across certain result types that perhaps can be expressed in a more semantically rich fashion than a bare Option<T>, which does not really provide a reason for why the value is not available. When you're the one creating the Option, you know why it's a None and not a Some, but downstream consumers of your code aren't privy to that distinction.

The Windows 10 API team ran into a similar issue, where the basic "dir entry" entry type/"trait" IStorageFile (a tuple of path and expanded metadata, plus functions to act on that) proved to be too much for certain needs such as quickly enumerating directories, and so a change was made to allow for partially initialized StorageFile objects, significantly speeding things up (~by an order of magnitude).

Rather than returning a vague None for fields that simply haven't been initialized, it would really help the APIs be more self-documenting (after all, isn't one of the biggest merits of such a strongly typed language?) if there were some sort of tristate value that could be used here instead, along the lines of Some(T)/NotYet/None, although it wouldn't be as easy to elide entirely from the compiled result (I won't say impossible!).

Getting dangerously close to the topic of asynchronizing std, that is basically a static snapshot of a Future<T>, which would not be a bad fit here at all as the value is initially not available but there is a well-defined method of obtaining it, although I cringe significantly at the idea that it would be necessary to create a full-blown Future just to indicate that the field has not been computed.

[kubkon]

I was wondering if anyone could offer some advice here, as to what needs to happen to get this functionality stabilised?

[Artoria2e5]

I am a bit concerned about the fact that file_index is just a u64. On NTFS and ReFS the type is now more properly a u128, but a different API GetFileInformationByHandleEx is needed to get the info.

[brmmm3]

Any progress here? Is there any chance to get this into stable branch anytime soon?

[blarfoon]

Up, I would also really need this feature to be stabilized if possible

[ChrisDenton]

It's been three years and not much movement so probably not soon unless someone takes an interest in pushing it forward. I think this is still pending figuring out some issues:

• file_index should really be 128 bits (e.g. for ReFs) but currently we only get a 64 bit ID. Maybe it would be good enough to call it file_id_64 and add sufficient warnings to the documentation. Maybe not.
• all of these are currently only got as an implementation detail (which could change) and then not even in all situation (e.g. directory traversal)

Perhaps another option is to have a Windows only ExtendedMetadata struct that guarantees a range of information is available (using extra calls to the OS if necessary)? I don't know, this would require some figuring out and seeing how people would make use of it.

[Chaoses-Ib]

Just a tip: there is now a crate named file-id for getting file ids.

Its FileId is defined as follows, HighRes is preferred to LowRes when available (Windows Server 2012+):

pub enum FileId {
    Inode {
        device_id: u64,
        inode_number: u64,
    },
    LowRes {
        volume_serial_number: u32,
        file_index: u64,
    },
    HighRes {
        volume_serial_number: u64,
        file_id: u128,
    },
}