Skip to content

Use _construct_direct in UxDataArray.to_xarray() and UxDataset.to_xarray() - #1860

Merged
Sevans711 merged 13 commits into
mainfrom
sevans/to_xarray-_construct_direct
Nov 18, 2026
Merged

Sevans711 merged 13 commits into
mainfrom
sevans/to_xarray-_construct_direct

Conversation

@Sevans711

@Sevans711 Sevans711 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1794

Overview

Uses xr.Dataset._construct_direct() and xr.DataArray._construct_direct() to speed up to_xarray() calls, instead of passing inputs into xr.Dataset() and xr.DataArray() which go through all of xarray's initialization validation checks. Every UxDataset / UxDataArray is already a valid xr.Dataset / xr.DataArray, so there's no need to repeat these validation checks.

This probably isn't a particularly hot path because these methods don't tend to take that long compared to other operations. But, the validation checks scale with the number of coordinates and variables. For example, ASV benchmarks are currently showing ~300x speedup (from ~100ms to ~0.3ms) for a dataset with 100 data_vars and 10 coords, and ~3000x speedup (from 7.6s to 2.7ms) for a dataset with 1000 data_vars and 100 coords.

This PR adds a lot of regression tests to ensure the new behavior matches the "naive" implementation of using xr.Dataset() or xr.DataArray() directly. Iterated with Claude to help generate tests I might not have thought of, and debug various little issues I found along the way, but I also tried to limit the tests to a minimum set that remains human-readable and (hopefully) easy enough to read and review.

PR Checklist

General

  • An issue is created and linked
  • Added appropriate labels (if your uxarray repo permissions allow it)
  • Filled out Overview and Expected Usage (if applicable) sections

Testing & Benchmarking

  • There is adequate test coverage of changes from this PR (add new tests if needed)
  • If this PR could affect performance, ran ASV benchmarks and confirmed they show expected behavior (add a new benchmark if necessary)

Documentation and Examples

  • [N/A] Docstrings updated with any function changes, and included in all new functions
  • [N/A] User (public) functions added to docs/api.rst; internal (private) function names start with an underscore (_)
  • [N/A] If touched any notebook files, cleared the output of all cells before committing
  • [N/A] If added new notebook files, put into appropriate directories and referenced in appropriate files

AI Disclosure

AI Usage: GitHub Copilot's inline code suggestions, plus lots of iteration with Claude on the implementation and test details.

  • I have tested and take responsibility for all AI-generated content in my PR.

@Sevans711 Sevans711 added scalability Related to scalability & performance efforts run-benchmark Run ASV benchmark workflow labels Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

ASV Benchmarking

Benchmark Comparison Results

Benchmarks that have improved:

Change Before [440eda7] After [1347ee7] Ratio Benchmark (Parameter)
- 555±3ms 491±2ms 0.89 mpas_ocean.NeighborhoodReduce.time_neighborhood_reduce('120km', 'mean')
- 589±2ms 524±1ms 0.89 mpas_ocean.NeighborhoodReduce.time_neighborhood_reduce('120km', 'median')
- 17.6±0.9ms 15.9±0.02ms 0.9 mpas_ocean.NeighborhoodReduce.time_neighborhood_reduce('480km', 'mean')
- 17.9±0.5ms 16.2±0.01ms 0.9 mpas_ocean.NeighborhoodReduce.time_neighborhood_reduce('480km', 'median')
- 374±8μs 94.8±1μs 0.25 to_xarray.UxDataArrayToXarray.time_to_xarray(10)
- 60.4±0.5ms 168±6μs 0 to_xarray.UxDatasetToXarray.time_to_xarray((100, 10))
- 5.92±0.02s 1.52±0.02ms 0 to_xarray.UxDatasetToXarray.time_to_xarray((1000, 100))
- 65.9±0.7ms 365±10μs 0.01 to_xarray.UxDatasetToXarray.time_to_xarray_healpix((100, 10))
- 6.26±0.02s 3.38±0.04ms 0 to_xarray.UxDatasetToXarray.time_to_xarray_healpix((1000, 100))

Benchmarks that have stayed the same:

Change Before [440eda7] After [1347ee7] Ratio Benchmark (Parameter)
942±30μs 953±20μs 1.01 connectivity.Connectivity.time_edge_face('120km')
460±10μs 469±10μs 1.02 connectivity.Connectivity.time_edge_face('480km')
6.11±0.08ms 6.21±0.1ms 1.02 connectivity.Connectivity.time_edge_node('120km')
1.24±0.05ms 1.25±0.05ms 1.01 connectivity.Connectivity.time_edge_node('480km')
61.6±4μs 64.2±7μs 1.04 connectivity.Connectivity.time_face_edge('120km')
63.2±10μs 61.3±4μs 0.97 connectivity.Connectivity.time_face_edge('480km')
965±10μs 955±20μs 0.99 connectivity.Connectivity.time_face_face('120km')
399±30μs 400±30μs 1.00 connectivity.Connectivity.time_face_face('480km')
65.1±10μs 67.6±6μs 1.04 connectivity.Connectivity.time_face_node('120km')
58.1±8μs 57.1±10μs 0.98 connectivity.Connectivity.time_face_node('480km')
554±70μs 609±40μs 1.10 connectivity.Connectivity.time_n_nodes_per_face('120km')
390±20μs 391±20μs 1.00 connectivity.Connectivity.time_n_nodes_per_face('480km')
1.36±0.06ms 1.35±0.02ms 0.99 connectivity.Connectivity.time_node_edge('120km')
471±9μs 470±20μs 1.00 connectivity.Connectivity.time_node_edge('480km')
84.1±1ms 84.7±2ms 1.01 connectivity.Connectivity.time_node_face('120km')
5.36±0.08ms 5.43±0.06ms 1.01 connectivity.Connectivity.time_node_face('480km')
1.42M 1.42M 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_edge_face('120km')
105k 105k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_edge_face('480km')
6.48M 6.48M 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_edge_node('120km')
419k 419k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_edge_node('480km')
2.05k 2.05k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_edge('120km')
2.05k 2.05k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_edge('480km')
1.6M 1.6M 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_face('120km')
101k 101k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_face('480km')
2.18k 2.18k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_node('120km')
2.18k 2.18k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_face_node('480km')
240k 240k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_n_nodes_per_face('120km')
26k 26k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_n_nodes_per_face('480km')
1.9M 1.9M 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_node_edge('120km')
127k 127k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_node_edge('480km')
11.9M 11.9M 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_node_face('120km')
747k 747k 1.00 connectivity.ConnectivityTracemalloc.track_peakmem_node_face('480km')
9.02±0.05ms 8.89±0.07ms 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
2.59±0.2ms 2.53±0.03ms 0.98 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
11.0±0.5ms 10.9±8ms 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
1.31±0.04ms 1.34±0.05ms 1.02 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
57.3k 57.3k 1.00 face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
12.3k 12.3k 1.00 face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
123k 123k 1.00 face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
128 128 1.00 face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.27M 1.27M 1.00 face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
50.1k 50.1k 1.00 face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
1.48M 1.48M 1.00 face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
712 712 1.00 face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.73M 1.73M 1.00 face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
1.71M 1.71M 1.00 face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
1.87M 1.87M 1.00 face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
35.3k 35.3k 1.00 face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.12±0.03μs 1.14±0.02μs 1.02 geometry_kernels.AccucrossKernels.time_accucross
2.58±0.03μs 2.64±0.03μs 1.02 geometry_kernels.AccucrossKernels.time_accucross_pair
418±20ns 375±20ns ~0.90 geometry_kernels.EFTPrimitives.time_acc_sqrt_re
409±60ns 507±50ns ~1.24 geometry_kernels.EFTPrimitives.time_diff_of_products
331±60ns 341±30ns 1.03 geometry_kernels.EFTPrimitives.time_two_prod
376±30ns 323±30ns ~0.86 geometry_kernels.EFTPrimitives.time_two_sum
711±10ns 706±30ns 0.99 geometry_kernels.GCAConstLatIntersection.time_accux_constlat_kernel
736±20ns 752±20ns 1.02 geometry_kernels.GCAConstLatIntersection.time_gca_const_lat_intersection
841±20ns 827±30ns 0.98 geometry_kernels.GCAConstLatIntersection.time_try_gca_const_lat_intersection
446±5ns 421±5ns 0.94 geometry_kernels.GCAGCAIntersection.time_accux_gca_kernel
611±4ns 606±9ns 0.99 geometry_kernels.GCAGCAIntersection.time_gca_gca_intersection
661±9ns 676±9ns 1.02 geometry_kernels.GCAGCAIntersection.time_try_gca_gca_intersection
21.6±0.3μs 21.6±0.5μs 1.00 geometry_kernels.OrientPredicates.time_on_minor_arc
21.4±0.2μs 21.6±0.3μs 1.01 geometry_kernels.OrientPredicates.time_orient3d_on_sphere
2.63±0.02ms 2.59±0.02ms 0.98 geometry_samebody.SameBodyConstLat.time_accux_dispatch
692±9μs 672±6μs 0.97 geometry_samebody.SameBodyConstLat.time_accux_kernel
2.54±0ms 2.50±0ms 0.98 geometry_samebody.SameBodyConstLat.time_fp64_dispatch
88.1±3μs 83.3±0.2μs 0.95 geometry_samebody.SameBodyConstLat.time_fp64_kernel
33.8±1ms 33.1±0.08ms 0.98 geometry_samebody_gcagca.SameBodyGcaGca.time_accux_dispatch
3.88±0.01ms 3.71±0.01ms 0.95 geometry_samebody_gcagca.SameBodyGcaGca.time_accux_kernel
30.8±0.1ms 30.8±0.5ms 1.00 geometry_samebody_gcagca.SameBodyGcaGca.time_fp64_dispatch
401±4μs 409±4μs 1.02 geometry_samebody_gcagca.SameBodyGcaGca.time_fp64_kernel
839±9ms 826±7ms 0.98 import.Imports.timeraw_import_uxarray
1 1 1.00 lazy_grid_construction.LazyGridConstruction.track_computes_isel
2 2 1.00 lazy_grid_construction.LazyGridConstruction.track_computes_open_grid_chunked
53.3±2ms 53.6±2ms 1.01 lazy_grid_construction.OpenGridChunked.time_open_grid('120km')
76.2±2ms 75.9±1ms 1.00 lazy_grid_construction.OpenGridChunked.time_open_grid('480km')
1.99M 2.01M 1.01 lazy_grid_construction.OpenGridChunked.track_peakmem_open_grid('120km')
2.23M 2.11M 0.94 lazy_grid_construction.OpenGridChunked.track_peakmem_open_grid('480km')
1.62±0.2ms 1.75±0.06ms 1.08 mpas_ocean.AzimuthalAverage.time_azimuthal_average('120km')
581±7μs 559±9μs 0.96 mpas_ocean.AzimuthalAverage.time_azimuthal_average('480km')
2.43±0.02ms 2.40±0.02ms 0.99 mpas_ocean.CheckNorm.time_check_norm('120km')
1.93±0.05ms 1.91±0.01ms 0.99 mpas_ocean.CheckNorm.time_check_norm('480km')
1.12±0.01ms 1.13±0.03ms 1.00 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('120km')
464±20μs 465±6μs 1.00 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('480km')
705±5μs 726±10μs 1.03 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('120km')
512±5μs 525±9μs 1.03 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('480km')
2.65±0.04ms 2.69±0.06ms 1.01 mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('120km')
1.90±0.02ms 1.94±0.02ms 1.02 mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('480km')
74.9±0.3ms 75.3±0.2ms 1.01 mpas_ocean.ConstructFaceLatLon.time_welzl('120km')
6.40±0.01ms 6.66±0.1ms 1.04 mpas_ocean.ConstructFaceLatLon.time_welzl('480km')
13.3±0.04ms 13.2±0.02ms 1.00 mpas_ocean.ConstructTreeStructures.time_ball_tree('120km')
790±50μs 820±20μs 1.04 mpas_ocean.ConstructTreeStructures.time_ball_tree('480km')
11.4±0.02ms 11.5±0.06ms 1.00 mpas_ocean.ConstructTreeStructures.time_kd_tree('120km')
638±20μs 621±8μs 0.97 mpas_ocean.ConstructTreeStructures.time_kd_tree('480km')
585±5ms 585±4ms 1.00 mpas_ocean.CrossSections.time_const_lat('120km', 1)
298±3ms 292±0.5ms 0.98 mpas_ocean.CrossSections.time_const_lat('120km', 2)
153±2ms 153±2ms 1.00 mpas_ocean.CrossSections.time_const_lat('120km', 4)
504±4ms 505±5ms 1.00 mpas_ocean.CrossSections.time_const_lat('480km', 1)
255±2ms 250±0.7ms 0.98 mpas_ocean.CrossSections.time_const_lat('480km', 2)
131±1ms 134±2ms 1.03 mpas_ocean.CrossSections.time_const_lat('480km', 4)
28.6±0.4ms 28.8±0.2ms 1.00 mpas_ocean.DualMesh.time_dual_mesh_construction('120km')
3.32±0.03ms 3.40±0.09ms 1.02 mpas_ocean.DualMesh.time_dual_mesh_construction('480km')
6.48±0.2ms 6.27±0.1ms 0.97 mpas_ocean.FaceAreas.time_face_areas('120km')
2.08±0.1ms 2.79±0.7ms ~1.34 mpas_ocean.FaceAreas.time_face_areas('480km')
229k 229k 1.00 mpas_ocean.FaceAreas.track_nbytes_face_areas('120km')
14.3k 14.3k 1.00 mpas_ocean.FaceAreas.track_nbytes_face_areas('480km')
2.12M 2.12M 1.00 mpas_ocean.FaceAreas.track_peakmem_face_areas('120km')
687k 687k 1.00 mpas_ocean.FaceAreas.track_peakmem_face_areas('480km')
44.6±0.5ms 46.2±0.4ms 1.04 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', 'exclude')
125±0.9ms 126±2ms 1.01 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', 'split')
4.50±0.2ms 4.61±0.2ms 1.02 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', 'exclude')
23.2±0.5ms 23.9±0.5ms 1.03 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', 'split')
9.03±0.7ms 9.43±0.2ms 1.05 mpas_ocean.Gradient.time_gradient('120km')
1.41±0.01ms 1.35±0.02ms 0.96 mpas_ocean.Gradient.time_gradient('480km')
457k 457k 1.00 mpas_ocean.Gradient.track_nbytes_gradient('120km')
28.7k 28.7k 1.00 mpas_ocean.Gradient.track_nbytes_gradient('480km')
3.44M 3.44M 1.00 mpas_ocean.Gradient.track_peakmem_gradient('120km')
226k 226k 1.00 mpas_ocean.Gradient.track_peakmem_gradient('480km')
445±10μs 432±10μs 0.97 mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('120km')
160±9μs 153±0.7μs 0.96 mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('480km')
110±6μs 126±20μs ~1.15 mpas_ocean.Integrate.time_integrate('120km')
86.5±0.8μs 92.7±1μs 1.07 mpas_ocean.Integrate.time_integrate('480km')
18.4M 18.4M 1.00 mpas_ocean.Integrate.track_nbytes_integrate('120km')
1.2M 1.2M 1.00 mpas_ocean.Integrate.track_nbytes_integrate('480km')
80.3±0.8ms 82.2±1ms 1.02 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'exclude')
80.1±0.6ms 80.7±1ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'include')
79.3±0.6ms 80.9±0.6ms 1.02 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'split')
5.74±0.2ms 5.72±0.05ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'exclude')
5.62±0.06ms 5.87±0.7ms 1.04 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'include')
5.61±0.04ms 5.67±0.06ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'split')
174±2ms 168±0.5ms 0.97 mpas_ocean.NeighborhoodBuild.time_build('120km', 1.0)
365±2ms 362±1ms 0.99 mpas_ocean.NeighborhoodBuild.time_build('120km', 5.0)
9.63±0.6ms 9.24±0.03ms 0.96 mpas_ocean.NeighborhoodBuild.time_build('480km', 1.0)
11.7±0.1ms 11.3±0.1ms 0.97 mpas_ocean.NeighborhoodBuild.time_build('480km', 5.0)
169±0.5ms 166±1ms 0.99 mpas_ocean.NeighborhoodBuild.time_query_radius('120km', 1.0)
358±2ms 356±2ms 0.99 mpas_ocean.NeighborhoodBuild.time_query_radius('120km', 5.0)
9.22±0.2ms 9.04±0.03ms 0.98 mpas_ocean.NeighborhoodBuild.time_query_radius('480km', 1.0)
11.5±0.02ms 11.6±0.1ms 1.01 mpas_ocean.NeighborhoodBuild.time_query_radius('480km', 5.0)
612.76 612.76 1.00 mpas_ocean.NeighborhoodBuild.track_mean_neighbors('120km', 15.0)
74.17 74.17 1.00 mpas_ocean.NeighborhoodBuild.track_mean_neighbors('120km', 5.0)
37.29 37.29 1.00 mpas_ocean.NeighborhoodBuild.track_mean_neighbors('480km', 15.0)
6.57 6.57 1.00 mpas_ocean.NeighborhoodBuild.track_mean_neighbors('480km', 5.0)
141M 141M 1.00 mpas_ocean.NeighborhoodBuild.track_nbytes_neighbors('120km', 15.0)
17.4M 17.4M 1.00 mpas_ocean.NeighborhoodBuild.track_nbytes_neighbors('120km', 5.0)
563k 563k 1.00 mpas_ocean.NeighborhoodBuild.track_nbytes_neighbors('480km', 15.0)
123k 123k 1.00 mpas_ocean.NeighborhoodBuild.track_nbytes_neighbors('480km', 5.0)
145M 145M 1.00 mpas_ocean.NeighborhoodBuild.track_peakmem_build('120km', 15.0)
21.5M 21.5M 1.00 mpas_ocean.NeighborhoodBuild.track_peakmem_build('120km', 5.0)
824k 824k 1.00 mpas_ocean.NeighborhoodBuild.track_peakmem_build('480km', 15.0)
384k 384k 1.00 mpas_ocean.NeighborhoodBuild.track_peakmem_build('480km', 5.0)
26.3±0.9ms 25.9±0.4ms 0.98 mpas_ocean.NeighborhoodDask.time_mean('120km', 'grid_chunks')
11.3±0.1ms 11.3±0.04ms 1.00 mpas_ocean.NeighborhoodDask.time_mean('120km', 'numpy')
23.8±0.2ms 24.5±0.7ms 1.03 mpas_ocean.NeighborhoodDask.time_mean('120km', 'time_chunks')
5.85±0.03ms 5.89±0.03ms 1.01 mpas_ocean.NeighborhoodDask.time_mean('480km', 'grid_chunks')
339±4μs 349±4μs 1.03 mpas_ocean.NeighborhoodDask.time_mean('480km', 'numpy')
4.10±0.07ms 4.09±0.05ms 1.00 mpas_ocean.NeighborhoodDask.time_mean('480km', 'time_chunks')
5.9M 5.82M 0.99 mpas_ocean.NeighborhoodDask.track_peakmem_mean('120km', 'grid_chunks')
2.75M 2.75M 1.00 mpas_ocean.NeighborhoodDask.track_peakmem_mean('120km', 'numpy')
5.68M 5.68M 1.00 mpas_ocean.NeighborhoodDask.track_peakmem_mean('120km', 'time_chunks')
744k 755k 1.01 mpas_ocean.NeighborhoodDask.track_peakmem_mean('480km', 'grid_chunks')
177k 177k 1.00 mpas_ocean.NeighborhoodDask.track_peakmem_mean('480km', 'numpy')
528k 528k 1.00 mpas_ocean.NeighborhoodDask.track_peakmem_mean('480km', 'time_chunks')
4.77±0.01s 4.35±0.01s 0.91 mpas_ocean.NeighborhoodReduce.time_dataset_reduce('120km', 'mean')
4.89±0.02s 4.48±0.01s 0.92 mpas_ocean.NeighborhoodReduce.time_dataset_reduce('120km', 'median')
125±1ms 117±0.4ms 0.93 mpas_ocean.NeighborhoodReduce.time_dataset_reduce('480km', 'mean')
127±1ms 118±0.6ms 0.93 mpas_ocean.NeighborhoodReduce.time_dataset_reduce('480km', 'median')
4.51±0.07ms 4.47±0.02ms 0.99 mpas_ocean.NeighborhoodReduce.time_reduce('120km', 'mean')
37.2±0.09ms 36.7±0.05ms 0.99 mpas_ocean.NeighborhoodReduce.time_reduce('120km', 'median')
189±4μs 197±10μs 1.04 mpas_ocean.NeighborhoodReduce.time_reduce('480km', 'mean')
468±10μs 465±20μs 0.99 mpas_ocean.NeighborhoodReduce.time_reduce('480km', 'median')
234k 234k 1.00 mpas_ocean.NeighborhoodReduce.track_peakmem_reduce('120km', 'mean')
235k 235k 1.00 mpas_ocean.NeighborhoodReduce.track_peakmem_reduce('120km', 'median')
19.5k 19.5k 1.00 mpas_ocean.NeighborhoodReduce.track_peakmem_reduce('480km', 'mean')
19.6k 19.6k 1.00 mpas_ocean.NeighborhoodReduce.track_peakmem_reduce('480km', 'median')
357±5μs 365±10μs 1.02 mpas_ocean.PointInPolygon.time_face_search_lonlat('120km')
343±30μs 354±10μs 1.03 mpas_ocean.PointInPolygon.time_face_search_lonlat('480km')
345±20μs 330±20μs 0.96 mpas_ocean.PointInPolygon.time_face_search_xyz('120km')
339±20μs 277±10μs ~0.82 mpas_ocean.PointInPolygon.time_face_search_xyz('480km')
62.1±0.6ms 61.4±0.6ms 0.99 mpas_ocean.RemapDownsample.time_bilinear_remapping
11.1±0.2ms 11.0±0.1ms 0.98 mpas_ocean.RemapDownsample.time_inverse_distance_weighted_remapping
9.88±0.1ms 9.51±0.2ms 0.96 mpas_ocean.RemapDownsample.time_nearest_neighbor_remapping
446±1ms 436±2ms 0.98 mpas_ocean.RemapUpsample.time_bilinear_remapping
14.6±0.04ms 14.9±0.1ms 1.02 mpas_ocean.RemapUpsample.time_inverse_distance_weighted_remapping
4.26±0.2ms 4.33±0.06ms 1.02 mpas_ocean.ZonalAverage.time_zonal_average('120km')
2.37±0.02ms 2.40±0.04ms 1.01 mpas_ocean.ZonalAverage.time_zonal_average('480km')
1.0517688861898458 1.034383084980083 0.98 nogil_scaling.GILScaling.track_gil_scaling
1.20±0.02ms 1.22±0.02ms 1.01 open_dataset.OpenDataset.time_open_dataset('dataset', 'dataset')
2.27±0.04ms 2.26±0.03ms 1.00 open_dataset.OpenDataset.time_open_dataset('dataset', 'file')
4.86±0.1ms 4.82±0.1ms 0.99 open_dataset.OpenDataset.time_open_dataset('file', 'dataset')
5.74±0.07ms 5.66±0.02ms 0.99 open_dataset.OpenDataset.time_open_dataset('file', 'file')
197±10μs 195±3μs 0.99 open_dataset.OpenDataset.time_open_dataset('grid', 'dataset')
1.19±0.02ms 1.20±0.1ms 1.01 open_dataset.OpenDataset.time_open_dataset('grid', 'file')
1.95±0.01s 1.97±0.02s 1.01 plots.PlotContour.time_contour_projected
296±4ms 294±2ms 0.99 plots.PlotHealpix.time_default_plot(6)
2.76±0.01s 2.77±0.01s 1.00 plots.PlotHealpix.time_default_plot(8)
28M 28M 1.00 plots.PlotHealpix.track_peakmem_default_plot(6)
443M 443M 1.00 plots.PlotHealpix.track_peakmem_default_plot(8)
4.54±0.05ms 4.48±0.03ms 0.99 quad_hexagon.QuadHexagon.time_open_dataset
3.82±0.05ms 3.84±0.08ms 1.00 quad_hexagon.QuadHexagon.time_open_grid
408 408 1.00 quad_hexagon.QuadHexagon.track_nbytes_open_dataset
392 392 1.00 quad_hexagon.QuadHexagon.track_nbytes_open_grid
71.3k 71.3k 1.00 quad_hexagon.QuadHexagon.track_peakmem_open_dataset
71.1k 71.1k 1.00 quad_hexagon.QuadHexagon.track_peakmem_open_grid
5.61±0.2ms 6.08±6ms 1.08 subset.BoundingBox.time_bounding_box('120km')
3.22±0.06ms 3.24±0.1ms 1.01 subset.BoundingBox.time_bounding_box('480km')
2.38M 2.38M 1.00 subset.BoundingBox.track_peakmem_bounding_box('120km')
203k 203k 1.00 subset.BoundingBox.track_peakmem_bounding_box('480km')
3.87±0.1ms 4.07±8ms 1.05 subset.BoundingBoxHealpix.time_bounding_box(6)
14.7±1ms 14.5±0.1ms 0.99 subset.BoundingBoxHealpix.time_bounding_box(8)
2.18M 2.18M 1.00 subset.BoundingBoxHealpix.track_peakmem_bounding_box(6)
13.2M 13.2M 1.00 subset.BoundingBoxHealpix.track_peakmem_bounding_box(8)
505±2ms 506±4ms 1.00 to_raster.ToRaster.time_to_raster((10000.0, 10.0))
1.10±0.01s 1.09±0.01s 0.99 to_raster.ToRaster.time_to_raster((10000.0, 100.0))
1.27±0.01s 1.26±0.01s 0.99 to_raster.ToRaster.time_to_raster((100000.0, 100.0))
270M 270M 1.00 to_raster.ToRaster.track_peakmem((10000.0, 10.0))
680M 680M 1.00 to_raster.ToRaster.track_peakmem((10000.0, 100.0))
765M 765M 1.00 to_raster.ToRaster.track_peakmem((100000.0, 100.0))

Benchmarks that have got worse:

Change Before [440eda7] After [1347ee7] Ratio Benchmark (Parameter)
+ 304M 390M 1.28 face_bounds.FaceBoundsColdStartRss.track_peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
+ 305M 390M 1.28 face_bounds.FaceBoundsColdStartRss.track_peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
+ 307M 390M 1.27 face_bounds.FaceBoundsColdStartRss.track_peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
+ 305M 390M 1.28 face_bounds.FaceBoundsColdStartRss.track_peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
+ 278M 388M 1.4 import.Imports.track_peakmem_import_uxarray
+ 328M 390M 1.19 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('120km', 1)
+ 328M 390M 1.19 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('120km', 2)
+ 328M 390M 1.19 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('120km', 4)
+ 308M 390M 1.27 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('480km', 1)
+ 308M 390M 1.27 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('480km', 2)
+ 308M 390M 1.27 mpas_ocean.CrossSectionsPeakMem.track_peakmem_const_lat('480km', 4)
+ 328M 390M 1.19 mpas_ocean.GradientColdStartRss.track_peakmem_gradient('120km')
+ 301M 390M 1.3 mpas_ocean.GradientColdStartRss.track_peakmem_gradient('480km')
+ 5.98±0.04ms 6.66±0.06ms 1.11 mpas_ocean.RemapUpsample.time_nearest_neighbor_remapping
+ 328M 390M 1.19 mpas_ocean.ZonalAveragePeakMem.track_peakmem_zonal_average('120km')
+ 309M 390M 1.26 mpas_ocean.ZonalAveragePeakMem.track_peakmem_zonal_average('480km')

@Sevans711
Sevans711 marked this pull request as ready for review October 29, 2026 19:50

@dylannelson dylannelson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and not seeing any blockers. Only one small thing came up when the PR was tested using different versions. Not blocking though since it could be addressed later if it actually shows up. Thanks!

Comment thread test/core/test_dataarray.py Resolved

@cmdupuis3 cmdupuis3 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the ds._close suggestion, you can also consider adding this unit test:

    @pytest.mark.parametrize("grid_format", ["UGRID", "HEALPix"])
    def test_close_not_shared(self, grid_format):
        """Closing the result never closes the original; the old path gave ``_close=None``."""
        uxds = self.get_uxds()
        closed = []
        uxds.set_close(lambda: closed.append(True))  # UxDataset.__init__ drops _close, so set one
        result = uxds.to_xarray(grid_format)
        assert result._close is None
        result.close()
        assert not closed

Comment thread uxarray/core/dataset.py Resolved
Comment thread benchmarks/to_xarray.py Resolved
Comment thread uxarray/core/dataset.py Resolved
Comment thread uxarray/core/dataset.py Resolved
@Sevans711

Copy link
Copy Markdown
Collaborator Author

@cmdupuis3 Thank you for the review! I incorporated some of your suggestions directly (github automatically marked the corresponding threads as resolved), and the rest of them I incorporated with some slight refactoring. I believe everything has been fixed; if you see any other issues please let me know!

@Sevans711
Sevans711 requested a review from cmdupuis3 November 18, 2026 19:56
@Sevans711
Sevans711 merged commit 86ab978 into main Nov 18, 2026
22 checks passed
@Sevans711
Sevans711 deleted the sevans/to_xarray-_construct_direct branch November 18, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-benchmark Run ASV benchmark workflow scalability Related to scalability & performance efforts

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UxDataset.to_xarray() and UxDataArray.to_xarray() should use _construct_direct

3 participants