PEP 718 – Subscriptable functions
- Author:
- James Hilton-Balfe <gobot1234yt at gmail.com>
- Sponsor:
- Guido van Rossum <guido at python.org>
- Discussions-To:
- Discourse thread
- Status:
- Draft
- Type:
- Standards Track
- Topic:
- Typing
- Created:
- 23-Jun-2023
- Python-Version:
- 3.13
- Post-History:
- 24-Jun-2023
Abstract
This PEP proposes making function objects subscriptable for typing purposes. Doing so gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient. It also brings functions in line with regular classes in their ability to be subscriptable.
Motivation
Unknown Types
Currently, it is not possible to infer the type parameters to generic functions in certain situations:
def make_list[T](*args: T) -> list[T]: ...
reveal_type(make_list()) # type checker cannot infer a meaningful type for T
Making instances of FunctionType
subscriptable would allow for this constructor to
be typed:
reveal_type(make_list[int]()) # type is list[int]
Currently you have to use an assignment to provide a precise type:
x: list[int] = make_list()
reveal_type(x) # type is list[int]
but this code is unnecessarily verbose taking up multiple lines for a simple function call.
Similarly, T
in this example cannot currently be meaningfully inferred, so x
is
untyped without an extra assignment:
def factory[T](func: Callable[[T], Any]) -> Foo[T]: ...
reveal_type(factory(lambda x: "Hello World" * x))
If function objects were subscriptable, however, a more specific type could be given:
reveal_type(factory[int](lambda x: "Hello World" * x)) # type is Foo[int]
Undecidable Inference
There are even cases where subclass relations make type inference impossible. However, if you can specialise the function type checkers can infer a meaningful type.
def foo[T](x: Sequence[T] | T) -> list[T]: ...
reveal_type(foo[bytes](b"hello"))
Currently, type checkers do not consistently synthesise a type here.
Unsolvable Type Parameters
Currently, with unspecialised literals, it is not possible to determine a type for situations similar to:
def foo[T](x: list[T]) -> T: ...
reveal_type(foo([])) # type checker cannot infer T (yet again)
reveal_type(foo[int]([])) # type is int
It is also useful to be able to specify in cases in which a certain type must be passed to a function beforehand:
words = ["hello", "world"]
foo[int](words) # Invalid: list[str] is incompatible with list[int]
Allowing subscription makes functions and methods consistent with generic classes where they weren’t already. Whilst all of the proposed changes can be implemented using callable generic classes, syntactic sugar would be highly welcome.
Due to this, specialising the function and using it as a new factory is fine
make_int_list = make_list[int]
reveal_type(make_int_list()) # type is list[int]
Monomorphisation and Reification
This proposal also opens the door to monomorphisation and reified types.
This would allow for a functionality which anecdotally has been requested many times.
Please note this feature is not being proposed by the PEP, but may be implemented in the future.
The syntax for such a feature may look something like:
def foo[T]():
return T.__value__
assert foo[int]() is int
Rationale
Function objects in this PEP is used to refer to FunctionType
, MethodType
,
BuiltinFunctionType
, BuiltinMethodType
and MethodWrapperType
.
For MethodType
you should be able to write:
class Foo:
def make_list[T](self, *args: T) -> list[T]: ...
Foo().make_list[int]()
and have it work similarly to a FunctionType
.
For BuiltinFunctionType
, so builtin generic functions (e.g. max
and min
)
work like ones defined in Python. Built-in functions should behave as much like
functions implemented in Python as possible.
BuiltinMethodType
is the same type as BuiltinFunctionType
.
MethodWrapperType
(e.g. the type of object().__str__
) is useful for
generic magic methods.
Specification
Function objects should implement __getitem__
to allow for subscription at runtime
and return an instance of types.GenericAlias
with __origin__
set as the
callable and __args__
as the types passed.
Type checkers should support subscripting functions and understand that the parameters passed to the function subscription should follow the same rules as a generic callable class.
Setting __orig_class__
Currently, __orig_class__
is an attribute set in GenericAlias.__call__
to the
instance of the GenericAlias
that created the called class e.g.
class Foo[T]: ...
assert Foo[int]().__orig_class__ == Foo[int]
Currently, __orig_class__
is unconditionally set; however, to avoid potential
erasure on any created instances, this attribute should not be set if __origin__
is
an instance of any function object.
The following code snippet would fail at runtime without this change as
__orig_class__
would be bar[str]
and not Foo[int]
.
def bar[U]():
return Foo[int]()
assert bar[str]().__orig_class__ == Foo[int]
Interactions with @typing.overload
Overloaded functions should work much the same as already, since they have no effect on the runtime type. The only change is that more situations will be decidable and the behaviour/overload can be specified by the developer rather than leaving it to ordering of overloads/unions.
Backwards Compatibility
Currently these classes are not subclassable and so there are no backwards
compatibility concerns with regards to classes already implementing
__getitem__
.
Reference Implementation
The runtime changes proposed can be found here https://github.com/Gobot1234/cpython/tree/function-subscript
Acknowledgements
Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido for some motivating examples.
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0718.rst
Last modified: 2024-02-17 03:29:08 GMT