Repository navigation
Fix Quantity merging across concatenation semantics - #1979
emanueldellipaoli01-a11y wants to merge 1 commit into
Conversation
zm711
left a comment
There was a problem hiding this comment.
I'll leave @apdavison to comment on this. But for a PR like this could you point at an actual problem that is happening between quantities and Neo? I don't see any issues linked and I see no MREs in your description. This is quite a number of code changes in core bits of Neo so I think we will need a bit more motivation for the benefit of this before we review this.
Thanks for pointing this out. There's actually already an issue for this: #1978. I've updated the PR description to link to it and added some more context about the compatibility issues with the upcoming changes in python-quantities#293. The quantities maintainers confirmed that testing the proposed concatenation changes against Neo's core tests leads to failures in I also mentioned the existing bug in I've added regression tests for these cases, as well as the related IO behaviour. Thanks again for taking the time to review this. Hopefully the extra context makes it clearer why these changes are needed. |
|
Hi @apdavison, I saw Zach's comment and updated the PR description with a link to #1978 and a more detailed explanation of the The core tests I ran pass locally (401 passed, 9 skipped). The two IO test files report 2 passed and 17 skipped. The The Read the Docs build also failed, but the log says it was terminated due to a timeout after 30 minutes and 50 seconds. I'm not sure what caused it. I'd appreciate your thoughts on the changes whenever you have time. Thanks! |
Fixes #1978.
This PR addresses a compatibility issue between Neo and the upcoming changes to
quantitiesin [python-quantities PR #293](python-quantities/python-quantities#293).1. Concatenation of Quantity arrays
Currently,
np.concatenate,np.hstack,np.vstack,np.stackandnp.appendcan discard the units of Quantity arrays.Neo has several operations that rely on this behaviour by explicitly rescaling quantities and reattaching their units after concatenation.
For example:
The upcoming change in
quantitiesmakes these NumPy functions preserve units and automatically rescale compatible quantities.Consequently, the existing Neo implementation would attach the units twice, producing incorrect dimensionality (e.g.
s**2instead ofs).This incompatibility was confirmed when running Neo's core tests against the proposed changes in python-quantities PR #293. The following tests failed:
test_Epoch_mergetest_Event_mergeTestSpikeTrainList::test_multiplexedThe solution is to explicitly concatenate the magnitudes and reattach the reference units, ensuring compatibility with both behaviours of
quantities.2. Incorrect unit conversion in SpikeTrain annotations
There is also an existing issue in
SpikeTrain.merge().The result of
other_ann.rescale(self_ann.units)is currently discarded. Consequently, annotations expressed in different compatible units can be concatenated without proper conversion.For example, merging an annotation containing
1 swith another containing500 msmay produce[1, 500] sinstead of the correct[1, 0.5] s.This PR fixes that behaviour and handles
UncertainQuantityannotations while preserving their uncertainty information.3. Regression tests
New regression tests cover:
The goal is to ensure that Neo produces correct results independently of the concatenation semantics provided by different versions of
quantities.