Repository navigation
Conversation
|
|
|
|
||
|
|
||
| filter = array_filter | ||
| take = array_take |
There was a problem hiding this comment.
TODO: redefine take rather than alias to exclude Expression
Co-authored-by: Dan Redding <[email protected]>
Co-authored-by: Dan Redding <[email protected]>
Co-authored-by: Dan Redding <[email protected]>
Co-authored-by: Dan Redding <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Add type annotation stubs for array, table, tensor, builder, memory, device, config, and types modules. Includes type-ignore annotations in related tests.
Drop the TypeVar/Generic parameters from the DataType subclasses, Field, Array, ChunkedArray and the tabular classes. Where a property can only return one of a few types (dictionary index type, run end type, time units) it now returns a union or Literal instead. Most of these parameters weren't buying much yet: scalar.pyi and lib.pyi are still placeholders, so e.g. Array[Scalar[...]] collapsed to Any anyway, and several annotations were being flagged by mypy and pyright (TypeVars used only once, type arguments on subclasses that weren't actually generic). Plain classes are a lot easier to read and review, and we can bring parameters back later where they clearly pay off. This also removes the stub-only _BasicDataType and _HasPrecisionScale helpers, which have no runtime counterpart, plus the IntoArray and SingleOrList aliases.
Co-authored-by: tadeja <[email protected]>
This reverts commit cf459f4.
Co-authored-by: Rok Mihevc <[email protected]>
|
Given there were changes in the data types PR you might want to rebase or merge them here @AlenkaF. |
Annotate pyarrow.compute, pyarrow._compute, pyarrow._acero and pyarrow._compute_docstrings, following the approach of apacheGH-55264: plain classes and unions instead of generics, in the flat pyarrow-stubs/ layout. - Array arguments accept any Array or ChunkedArray, since pa.array() and most of the API return plain Array instances. The return types name the kind of array where the function determines it. The scalar classes are placeholders (Any) until scalar.pyi is annotated. - The signatures match the runtime, as checked by stubtest. - pyarrow.compute and pyarrow._compute are type checked instead of being treated as Any. - Add type annotation tests for the compute and Acero stubs.
e8779e3 to
300602b
Compare
|
Oh, should have squashed, sorry. |
| @@ -0,0 +1,18 @@ | |||
| # Licensed to the Apache Software Foundation (ASF) under one | |||
There was a problem hiding this comment.
Is this file needed or we can remove it?
There was a problem hiding this comment.
I don't remember what this is, I suppose remove.
| from pyarrow import lib | ||
|
|
||
| ArrayLike: TypeAlias = Any | ||
| ScalarLike: TypeAlias = Any |
There was a problem hiding this comment.
Note to myself: when #55592 is merged, update the ScalarLike alias and make sure to narrow return types that use this alias.
| @@ -25,6 +25,7 @@ from pyarrow.lib import Array, ChunkedArray | |||
| from pyarrow import lib | |||
|
|
|||
| ArrayLike: TypeAlias = Any | |||
There was a problem hiding this comment.
Note to myself: define a separate private _ArrayLike type alias for compute stubs only (after comment #55264 (comment) is resolved).
| @@ -15,11 +15,532 @@ | |||
| # specific language governing permissions and limitations | |||
| # under the License. | |||
|
|
|||
There was a problem hiding this comment.
It was a bit hard to review _compute.pyi next to _compute.pyx as the ordering of the classes and methods is different between files. In the stubs it seems the order is meant to be alphabetical but here and there that is not true, for example SkewOptions is placed between VarianceOptions and WeekOptions.
Can I reorder the stubs so they match the code ordering in _compute.pyx? Or at least order alphabetically correctly?
There was a problem hiding this comment.
Please reorder this as you'd like! Right now it's just you on this PR, tag us when you're ready. SkewOptions was likely added later and was not sorted.
There was a problem hiding this comment.
Cool! I have reordered files locally together with rebase on latest #55264 (squashed) and plan to push tomorrow. Will tag u when this is ready for review (I have to get through compute.pyi which will take some time).
Rationale for this change
Closes: #49194
What changes are included in this PR?
Adds type annotation stubs for compute, _compute, _compute_docstring, and _acero modules. Includes source fixes in compute.py and acero.py, and type-ignore annotations in related tests.
Are these changes tested?
Yes, by CI.
Are there any user-facing changes?
Compute module functions will be type annotated.