Skip to content

Fix the OpenCL CPU compiler crash in the complex dpnp.minmax kernel - #3192

Open
antonwolfy wants to merge 1 commit into
masterfrom
fix-minmax-cpu-compiler-crash
Open

antonwolfy wants to merge 1 commit into
masterfrom
fix-minmax-cpu-compiler-crash

Conversation

@antonwolfy

@antonwolfy antonwolfy commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes an intermittent crash of the OpenCL CPU compiler in the complex dpnp.minmax kernel, which has been failing the Generate coverage job since dpnp.minmax landed in #3135 (e.g. this run). pytest dies at about 59% of the dpnp tests in TestMinMax::test_size, with a SIGABRT inside libintelocl.so or a silent SIGALRM.

Complex types can't use sycl::reduce_over_group with dpnp's min/max operators, so they take CustomMinMaxGroupReducer. It reduced the minimum and the maximum with two custom_reduce_over_group calls on the same local memory, with a group_barrier between them. In the coverage build (-O1 -g1 plus device-side profiling for minmax.cpp), building this kernel corrupts the compiler's heap in ReduceCrossBarrierValuesPass (malloc(): unsorted double linked list corrupted in collectDependencies). It is the same pass and the same kind of corruption as the radix sort crash in #3191 (CMPLRLLVM-79305).

The barriers here are needed: work-items read local memory that others wrote, so they can't become fences. This PR instead folds both extrema in one tree over a local memory of twice the work-group size, with one barrier per step for both. That gives the kernel the barrier structure of dpnp.max on complex, which the coverage job builds without crashing. The local memory is at most 64 KB for complex128 on a CPU (work-group capped at 2048) and 16 KB on a GPU (capped at 512).

On a Panther Lake host with the coverage build and the 2026.2.0 OpenCL CPU runtime, the complex cases of TestMinMax::test_size on opencl:cpu crashed in 5 of 20 runs before the change and in none of 20 after it, in alternating runs (17 of 48 vs 0 of 50 counting all runs). The Release build doesn't crash before the change either (0 of 70). The minmax tests of test_statistics.py, test_sycl_queue.py and test_usm_type.py pass on opencl:cpu and level_zero:gpu. The existing complex test_size cases are the reproducer. No changelog entry, since dpnp.minmax is not released yet.

The change also speeds up the complex reduction. On a Panther Lake H host (Arc B390), with a Release build where only _tensor_reductions_impl differs, the median time per call with the fix divided by master over 9 alternating rounds (lower is faster):

Case opencl:cpu level_zero:gpu
complex64, geomean of 7 shapes 0.50 0.98
complex128, geomean of 7 shapes 0.37 0.98
complex64 / complex128 (1000000, 16), axis=1 0.17 / 0.17 0.90 / 0.99
complex64 / complex128 (10000000,), axis=None 0.52 / 0.51 0.99 / 1.00
complex64 / complex128 (4096, 4096), axis=0 0.86 / 0.86 1.00 / 1.00
float32 (unchanged kernel), geomean of 7 shapes 1.01 1.02

The shapes are 1-D of 1000, 100000 and 10000000 elements with axis=None, (4096, 4096) with axis=0 and axis=1, (16, 1000000) and (1000000, 16) with axis=1. All results match NumPy.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • Have you added documentation for your changes, if necessary?
  • Have you added your changes to the changelog?

The complex `dpnp.minmax` kernel reduced the minimum and the maximum with two `custom_reduce_over_group` calls separated by a barrier. In the coverage build, the OpenCL CPU compiler intermittently corrupts its heap in `ReduceCrossBarrierValuesPass` while building this kernel (CMPLRLLVM-79305), which kills the coverage job. Fold both extrema in one tree over a local memory of twice the work-group size, with one barrier per step for both.
@github-actions

Copy link
Copy Markdown
Contributor

View rendered docs @ https://intelpython.github.io/dpnp/pull/3192/index.html

@github-actions

Copy link
Copy Markdown
Contributor

Array API standard conformance tests for dpnp=0.22.0dev2=np2py314h8d9cdd5_7 ran successfully.
Passed: 1491
Failed: 0
Skipped: 6

@coveralls

Copy link
Copy Markdown
Collaborator

Coverage Status

coverage: 84.269%. remained the same — fix-minmax-cpu-compiler-crash into master

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants