Repository navigation
Fix the OpenCL CPU compiler crash in the one-work-group radix sort #3191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -412,3 +412,33 @@ def test_radix_argsort_size_1_axis(): | |
| x4 = dpt.reshape(dpt.arange(10, dtype="i1"), (1, 10)) | ||
| r4 = dpt.argsort(x4, axis=0, kind="radixsort") | ||
| assert dpt.asnumpy(r4 == 0).all() | ||
|
|
||
|
|
||
| # rows of every size class of the one-work-group radix sort above 256; on the | ||
| # CPU device int8 argsort came out unstable and some data types crashed the | ||
| # compiler at block sizes of 16 and more | ||
| @pytest.mark.parametrize("n", [257, 1025, 2049, 4096, 8193, 16384]) | ||
| @pytest.mark.parametrize("dtype", ["?", "i1", "u1", "i2", "i4", "i8"]) | ||
|
Comment on lines
+420
to
+421
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This misses two size classes: 513..1024 and 4097..8192 (2049 and 4096 share one, 8193 and 16384 another). On master, int8 rows of 8192 crash the CPU compiler too.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Both additions catch crashes on master that the current sizes miss. I extended this test locally to 513, 1024, 4097 and 8192 for every dtype here, plus
The stability check could also use data with few distinct values. In a standalone copy of the kernel, int8 argsort of 3-value data without the work-around was wrong in 34-52 % of runs at 1024-16384 on AVX2 CPUs. I haven't checked whether the random data here catches that. |
||
| @pytest.mark.parametrize("descending", [False, True]) | ||
| def test_radix_sort_one_work_group(n, dtype, descending): | ||
| q = get_queue_or_skip() | ||
| skip_if_dtype_not_supported(dtype, q) | ||
|
|
||
| rng = np.random.default_rng(42) | ||
| if dtype == "?": | ||
| x_np = rng.integers(0, 2, size=(2, n)).astype(dtype) | ||
| else: | ||
| info = np.iinfo(dtype) | ||
| x_np = rng.integers(info.min, info.max, size=(2, n), dtype=dtype) | ||
| x = dpt.asarray(x_np, sycl_queue=q) | ||
|
|
||
| # stable in either order: ties keep the order of the input | ||
| _, ranks = np.unique(x_np, return_inverse=True) | ||
| ranks = ranks.reshape(x_np.shape) | ||
| expected_inds = np.argsort(-ranks if descending else ranks, kind="stable") | ||
| expected = np.take_along_axis(x_np, expected_inds, axis=-1) | ||
|
|
||
| s = dpt.sort(x, descending=descending, kind="radixsort") | ||
| assert_array_equal(dpt.asnumpy(s), expected) | ||
| inds = dpt.argsort(x, descending=descending, kind="radixsort") | ||
| assert_array_equal(dpt.asnumpy(inds), expected_inds) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it the same as CMPLRLLVM-79305?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's the same bug. CMPLRLLVM-79305 was filed from this kernel (int8 argsort, n=12320, so wg 512 and block 32, sub-group 8 on PTL-H), with the crash in
sycl-kernel-reduce-cross-barrier-values(copyAndReplaceUses), and its reproducer has these two per-update barriers. Could you name CMPLRLLVM-79305 in this comment too, so the work-around can be reverted once it is fixed?