Repository navigation
Conversation
JackieTien97
left a comment
There was a problem hiding this comment.
Thanks for the optimization. I checked out 3fba980 and ran the PR's tests locally (FileLoaderUtilsTest, TimeSeriesMetadataCacheTest, AlignedSeriesScanBatchTest); all pass. I also added a throwaway probe test (not pushed) that writes a real aligned device with 700 columns (three measurement leaf groups) and compares getDeviceTimeSeriesMetadata against the per-column get(): all 701 series (time column plus 700 fields) matched, including statistics and chunk-metadata counts. So I found no wrong-result issue. The comments below are mostly about performance, diagnostics and metrics.
Main points:
- [P1] The batch path only counts as a cache hit when every selected sensor is in the LRU. If any selected column is absent from a file and that absence can't be cached as a placeholder (
ignoreNotExists=trueormay_cache_nonexist_series=false), every query re-reads the whole device. The per-column path doesn't (TimeSeriesMetadataCacheL362). - [P1] The missing-device error message describes the opposite inconsistency (L491).
- [P2] Metadata IO size and load counts are no longer comparable with the per-column path (L483, L370).
- [P2] The I/O saving only exists when the selected columns span more than one measurement leaf group (more than 256 series by default), and the gate doesn't reflect that (
FileLoaderUtilsL447). Could the same batching be done inside the existingget()miss path instead of a parallel path (L342)? - The PR description is out of date. It still describes a configurable
device_metadata_batch_scan_ratio(default 0.8), anIoTDBConfigchange, anignoreNotExistsgate andreadDeviceMetadata. The code hard-codes 0.6 with no switch, has no such gate, and usesgetDeviceTimeseriesMetadata. Could you update it and add benchmark numbers for a wide device (e.g. 800 columns, cold and warm cache) compared with master? That would make it easier to judge the 0.6 threshold.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved batch-path issues can trigger unnecessary device reads and absent-device I/O errors, while regression tests leave sparse all-null rows and multi-leaf field values unverified.
5 open issues
Add sparse all-null-row coverage for batched scans · New Skip device reads when the requested time column is absent · New Avoid whole-device reads for small remaining cache misses · New Include cold cache probes in disk lookup timing · New Assert values across all leaf groups in the multi-leaf test · New
What changed in this PR
This PR reduces repeated metadata-index traversal in DataNode wide-table queries by loading a device’s metadata in one batch when most field columns are selected.
Changes:
- Adds cache- and bloom-filter-aware batch loading with per-series query statistics.
- Gates batching by table width and a 60% projection threshold, excluding DEBUG and offset-addressed scans.
- Shares aligned metadata assembly, adds localized missing-device errors, and introduces regression tests.
| File | Description |
|---|---|
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/read/reader/series/AlignedSeriesScanBatchTest.java |
Adds a real-file wide-device batch scan regression. |
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/buffer/TimeSeriesMetadataCacheTest.java |
Tests batch loading, caching, missing devices, and metadata equivalence. |
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/execution/operator/source/FileLoaderUtilsTest.java |
Tests batch eligibility boundaries and exclusions. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/buffer/TimeSeriesMetadataCache.java |
Implements cache-aware device metadata loading and statistics accounting. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/parameter/SeriesScanOptions.java |
Carries the table’s total field-column count. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/DataNodeTableOperatorGenerator.java |
Supplies field counts for eligible table scans. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/ExternalTsFileSeriesScanUtil.java |
Keeps external offset-addressed scans on the per-column path. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/FileLoaderUtils.java |
Selects the batch path and shares aligned metadata assembly. |
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/AlignedSeriesScanUtil.java |
Passes the field count into metadata loading. |
iotdb-core/datanode/src/main/i18n/zh/org/apache/iotdb/db/i18n/StorageEngineMessages.java |
Adds the Chinese missing-device error message. |
iotdb-core/datanode/src/main/i18n/en/org/apache/iotdb/db/i18n/StorageEngineMessages.java |
Adds the English error naming the device and TsFile. |
Review effort: Max
JackieTien97
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround. Status of my earlier comments: 9 of 12 are fixed, including both P1s. Still open by design: the batch read's IO bytes are not reported, the 0.6 ratio has no off switch, and the selective reader is an upstream follow-up.
Since this PR is a performance change, I benchmarked it against its base. A JUnit harness (not pushed) drives FileLoaderUtils.loadAlignedTimeSeriesMetadata over real TsFiles written with TsFileWriter (12 files, aligned devices, 4 chunks per series). It counts the bytes and read calls that reach the file and runs each variant in its own JVM (surefire defaults with -Xmx1024m, which gives a ~61 MB TimeSeriesMetadataCache). I measured f8b9c83 against ca8074d. The only batch-path change since then, 469394f, is time accounting, so the numbers apply to 6ab4b8e. Loaded metadata was identical to base in every scenario (checksums), so there is no correctness issue here.
Two runs per variant; ms per query (one device across 12 files), cold / warm:
| Scenario | base | PR | PR + the two changes below |
|---|---|---|---|
| Dense, 700 fields, select all, cache fits | 5.1–5.3 / 1.7–1.8 | 8.5 / 1.5–2.7 | 5.5–5.9 / 1.5–1.7 |
| Dense, 700 fields, select first 60% | 3.4–3.6 / 1.0–1.1 | 5.3–5.4 / 1.0 | 4.0–4.4 / 1.1 |
| Repeated select-all, working set larger than the metadata cache (steady state) | 4.5–9.9 ms, 1.7–4.1 MB per query | 10.1–16.1 ms, 11–15 MB per query | 4.4–5.4 ms, 1.0–2.0 MB per query |
| Sparse, ~20 of 700 fields per file (the motivating case) | 4.8–4.9 / 4.7 | 2.8–2.9 / 2.1 | 1.9–2.0 / 2.1 |
| 100-field table (below the gate) | 0.9 / 0.27 | unchanged | unchanged |
| Cold load, 96 devices x 300 fields, 8 / 32 threads (wall ms) | 225–232 / 237–247 | 208–220 / 193–210 | 199–213 / 186–189 |
The sparse case gains about 2x as intended, and the lock stripes do not hurt concurrency. There are two regressions, though. Each has an inline comment with the change that removed it locally:
- [P1] Under metadata-cache pressure, the "cache holds most of the request, so delegate to per-column
get()" fallback multiplies metadata reads 3–8x. - [P1] For dense wide devices, a cold call costs 45–70% more time than base. It moves the same bytes in fewer reads, so the extra cost is CPU: every selected measurement goes through two cache passes and a BloomFilter probe before the device read.
Two smaller points:
- With a 60% prefix projection, the device-wide read still loads a leaf group that no selected column needs, so it stays 10–30% slower even after both changes. That cost remains until the selective reader lands.
- The description says
getDeviceTimeseriesMetadatacosts about 2x IO. In these runs the batch read moved the same bytes as the per-column path (2019.9 KB vs 2021.0 KB per query, in 48 vs 72 reads). Also, the gate in the code is>=, while the description says "more than".
I'm happy to share the harness if it helps.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved issues alter per-series counters, allow repeated metadata reads and unnecessary sibling loading, and leave shared test state dependent on execution order.
4 open issues
2 resolved since last review
Review effort: Balanced
JackieTien97
left a comment
There was a problem hiding this comment.
The set-based reader is a nice redesign, and it resolves two of the three regressions from my previous review:
- Under cache pressure, steady-state metadata IO is now below base. Repeated select-all reads 1.0–1.3 MB per query against base's 1.7–2.0 MB, and takes 4.3–4.5 ms against 5.1–7.0 ms.
- A 60% prefix projection now reads the same bytes as base.
I reran the same harness on aa52c51 and on its merge base d5c1534. It uses real TsFiles (12 files, aligned and non-aligned devices, 4 chunks per series), counts bytes and reads at the file, and runs each benchmark in its own JVM (two runs per build, -Xmx1024m). Loaded metadata matched base in every scenario. The PR's tests plus ReadPointAlignedCrossCompactionTest (37 cases) pass locally.
ms per query (one device across 12 files), cold / warm:
| Scenario | base | PR |
|---|---|---|
| Dense, 700 fields, select all, cache fits | 5.7 / 1.9–2.2 | 6.6–7.5 / 1.9–2.0 |
| Dense, 700 fields, first 60% | 3.5–3.8 / 1.0–1.4 | 4.0–4.2 / 1.0–1.2 |
| Dense, 1 column | 1.0 | 0.5–0.6 |
| Re-query after evicting 1% / 30% | 5.5–5.9 / 6.9–7.0 | 3.0–3.3 / 5.2–6.0 |
| Repeated select-all, working set larger than the cache (steady state) | 5.1–7.0, 1.7–2.0 MB per query | 4.3–4.5, 1.0–1.3 MB per query |
| 10 random fields absent per file, warm | 2.1–2.5, 0 KB | 3.0–3.2, 1.9 MB / 46 reads per query |
| 10 fields added later (absent in every file), warm | 2.1–2.2, 0 KB | 2.4–2.5, 517 KB / 24 reads per query |
| Sparse, ~20 of 700 fields per file | 5.0–5.2 / 4.9–5.0, warm 0 KB | 2.1–2.3 / 2.2–2.3, warm 59 KB / 24 reads |
| 100-field table | 0.9–1.0 / 0.3 | 1.0 / 0.25 |
| Tree model: 300 non-aligned series, loaded one by one | 2.9–3.1 / 1.1 | 4.1–4.4 / 1.7–2.1 |
| Tree model: 5 of 300 non-aligned series | 0.53 / 0.01 | 0.29–0.40 / 0.03 |
| Cold load, 96 devices x 300 fields, 8 / 32 threads (wall ms) | 221–227 / 222–233 | 163–189 / 160–161 |
Still open (each has an inline comment):
- [P1] A selected column that is absent from a file is never cached as absent, so every warm query re-reads the leaf groups it maps into. Base does no metadata IO for such columns once warm. Adding columns to a wide table leaves every older file without them, so this will be common.
- [P1] The single-column
get()is 33–53% slower cold and 48–96% slower warm. This is the tree-model path (300 non-aligned series per device). Bytes are identical; the cost is per-call allocation, even on cache hits. - [P2] Dense cold loads are still 15–32% slower, and 7–18% for the 60% projection, in line with the numbers in the description.
- [P2] Offset-addressed (external) reads now consult the BloomFilter. The javadoc says this is skipped, and master did skip it.
Also, tsfile.version points at 2.5.1-261117-SNAPSHOT. That snapshot is built from apache/tsfile#1123, which is still open with no maintainer approval and was republished on 11-18. This PR can't merge until that change is reviewed, merged and released.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
9 open issues
This setstsfile.versionto a SNAPSHOT, which is generally not merge-safe/reproducible (builds… · New The Javadoc says bloom filtering is skipped whendeviceMetadataIndexNodeOffsetis non-null, but… · NewloadBloomFilterTimeis always recorded as 0 because it’s never updated around… · NewloadBloomFilterTimeis always recorded as 0 because it’s never updated around… · NewloadBloomFilterTimeis always recorded as 0 because it’s never updated around… · NewloadBloomFilterTimeis always recorded as 0 because it’s never updated around… · New This test createstarget/external-cache-disabled.tsfilebut never deletes it. That can pollute… · New This test createstarget/external-cache-disabled.tsfilebut never deletes it. That can pollute… · New This busywhile (true)loop can spin indefinitely ifhasNextFile()keeps returning… · New
4 resolved since last review
Review effort: Lite
A table scan that selects (nearly) all field columns of a device previously deserialized the device's TimeSeriesMetadata column by column, re-walking the device metadata index once per leaf node. Add a batch path that reads the whole device's metadata in a single pass via TsFileSequenceReader.readDeviceMetadata, populating the per-measurement cache and building the aligned metadata through the existing modification/loader logic. The batch path is taken when the selected field columns reach a configurable fraction (device_metadata_batch_scan_ratio, default 0.8) of the table's total field columns.
The batch path read a device's TimeseriesMetadata via TsFileSequenceReader.readDeviceMetadata, which deserializes only the header and statistics and leaves chunkMetadataList null. The downstream DiskAlignedChunkMetadataLoader dereferences that list in getCopiedChunkMetadataList, so a wide-table scan NPE'd at query time, and the null-chunk entries also polluted the shared per-measurement LRU cache (breaking later narrow queries on the same device). Switch to getDeviceTimeseriesMetadata, which deserializes chunk metadata too, and adapt the unit test to the list-based API, asserting that chunk metadata flows through to the result.
Remove the unreachable time-column-only branch in the batch loader (the batch gate already requires a non-empty selection), and validate device_metadata_batch_scan_ratio so a non-positive value falls back to the default instead of routing every table scan through the batch path. Document the valid range in the properties template.
Remove the ignoreNotExists gate so the batch path is also taken for files with a device-level time index, where previously only file-level-index files (or recovery) used it. When the batch read finds the device is missing from the file, replicate the per-column path: skip silently when ignoreNotExists is set, otherwise throw CorruptedTsFileException to surface the time-index/metadata inconsistency.
Skip the device read when the time column is known absent, so a bloom filter false positive on a field cannot force an absent-device read. Fall back to per-column lookups for a small remainder (one or two measurements), which reads a single leaf group instead of the whole device, and count the cold-call cache probe and lock wait in the disk lookup time. Strengthen the wide-scan tests: assert timestamps and every field value, compare a sparse batch scan against the per-column path, cover an all-null projection under ignoreAllNullRows, and add cache regressions for the filtered time column and the single-miss fallback.
TsFileSequenceReader is an external apache/tsfile dependency and has no batch API that accepts a measurement set, so the batch path still reads the whole device and filters afterwards. Note the constraint and the existing mitigations (bloom filter, per-column fallback) at the call site; a truly selective reader needs an upstream change.
Load the remaining misses through the per-column get() when the cache already holds most of the request: it touches only their leaf groups instead of re-reading the whole device. Replace the GC-sensitive WeakHashMap monitor with fixed lock stripes, and make the returned map complete for allSensors on every early return. Give aggregation table scans a totalFieldColumnCount so they can use the batch path, and make the fan-out gate span two leaf groups (>=), fixing its off-by-one and comment. Tests: production ignoreAllNullRows=false coverage including all-null projections, batch negative-caching parity, copy-on-return, a three-miss eviction case, the multi-leaf premise, and per-chunk offset/statistics comparison.
When a cold batch call delegates the remaining measurements to the per-column get(), the cache probes and lock wait were assigned to cache time while the cache hit count was zero, so neither branch reported them. Attribute the phase to disk time when there are no cache hits and allow a disk time with a zero count to be recorded.
Replace the device-wide batch read and its 0.6 projection gate with one getTimeseriesMetadata(Set) entry point: resolve cached columns first, then read each measurement leaf group that holds a selected column once, bloom-probing only the group probe and answering every requested column of the group from that read. get() becomes a single-column wrapper over it. This removes the static ratio, the parallel batch branch, the totalFieldColumnCount planner plumbing and the now-unused i18n exception constant. The cache and wide-device scan tests are rewritten for the new semantics.
An aligned device always has its time column, so when the requested set contains it and the cache or the bloom filter proves it absent, no selected series of that device can be read from this file. The per-column loader returned null at that point; the set-based loader kept probing the remaining measurements, and a bloom-filter false positive on any of them sent the device metadata index lookup into the reader, which throws when the device is not in the file (e.g. a seq file that only holds a subset of the devices read by cross compaction). Probe the time column first and stop the load when it resolves to null. Found by ReadPointAlignedCrossCompactionTest.test21.
The requested measurements were only added to the pending set by the cache pass. The cache-disabled and external-TsFile branch skipped that pass without seeding the set, so it stayed empty, no leaf group was read and the call returned nothing: read_tsfile queries returned zero rows on every external TsFile. Seed pending from the requested set in that branch and add a regression test that loads an external file through getTimeseriesMetadata with the device metadata offset.
Count disk lookups per requested measurement, cache hits included in the same total, as the per-column path did, so EXPLAIN ANALYZE and the monitoring counters stay comparable; leaf-read behavior stays covered by the cache tests rather than by the counter. When the metadata cache is disabled, filter the group read by the requested measurements instead of allSensors, so a non-aligned get() does not deserialize and drop its siblings' chunk metadata. Restore the shared test context to SERIES_SCAN after the external-file test, and document that the once-per-group guarantee needs a bloom filter.
Keep only the helpers with several call sites or real logic: inline the time-column and bloom predicates, rebuild the probe order where it is used, and return the cache-hit count from loadLeafGroups instead of wrapping it in a one-field class.
Replace the per-probe leaf-group loop with one call to TsFileSequenceReader.readTimeseriesMetadata(device, offset, measurements, returnedMeasurements, ...), which resolves the pending measurements against the leaf groups that hold them, reads each visited group once, and also returns the group siblings a single-column call needs for cache warming. The cache keeps the cache pass, the bloom-filter device gate, the negative placeholder and the per-series counters; the probe loop, probeFound, the group filter and the per-column bloom probes are gone. The aligned loader now always requests the time column. This needs the tsfile change that adds the set-based overload; the tsfile dependency must be bumped to a version containing it before this branch can be pushed.
The set-based readTimeseriesMetadata overload is not in the dated tsfile snapshot the build used before. Point the dependency at the 2.5.1 snapshot, which will contain it once apache/tsfile#1123 is merged and published. The IoTDB change that uses the new API must not be pushed until that snapshot exists in the configured repositories.
The build must not resolve the shared 2.5.1-SNAPSHOT, whose content can change under it. Depend on the dated 2.5.1-261117-SNAPSHOT that carries the selective metadata reader, and publish that snapshot before pushing the branch so CI no longer resolves a shared moving artifact.
Cache the TimeSeriesMetadataCacheKey hash in the constructor instead of allocating the Objects.hash argument array on every lookup and write, and pass the pending set straight to the tsfile reader when the aligned batch path requests exactly the set it caches.
Each cache-missed measurement is probed against the file's bloom filter: a measurement the filter rules out is answered with null without a read and without a negative cache entry, because probing the filter again is cheaper than one entry per absent column per file. The measurements the filter cannot rule out go to the reader in one call, and every one the reader does not return is remembered as absent, so a later query does not walk its leaf group again. Offset-addressed external TsFiles may not have a bloom filter that covers their series, so bloom filtering is skipped for them. The single-column special case of the device gate is gone because the per-column probe covers it. The tsfile dependency moves to the dated snapshot that carries the renamed set-based reader.
- Answer a single-series cache hit before building the batch request (singleton set, result map, pending set) or the device prefix. - Time the BloomFilter load, so loadBloomFilterTime is no longer always zero in EXPLAIN ANALYZE and the monitoring counters. - Re-check pending entries under the device lock: a thread that waited no longer re-reads what another thread has just cached. - Skip BloomFilter loading in the cache branch for offset-addressed files, as the method javadoc promises, not only in the disabled-cache branch. - Include the pending set in `returned`; a miss outside allSensors would otherwise be mistaken for an absent series and pollute the negative cache. Reuse allSensors without copying when nothing is cached, which is the single-column cold path. - Pass the query debug flag to the BloomFilter cache again. - Test hygiene: delete the external fixture, bound the hasFile retry, and assert one reader call per file in the wide aligned scan.
7e24c11 to
37ca9c7
Compare
- Drop the redundant time-column guard in the aligned loader: getAllSensors already adds the time column. - Build the device string once per call and reuse it for the lock and the BloomFilter prefix. - Recognize the aligned batch caller by set identity, so the equal contents of its two set arguments are known without comparing them. - Document the roles of pending, requested and allSensors in readAndMerge. - Remove every loaded measurement from pending and remember the remainder directly; the final fill pass is no longer needed. - Drop the cached hash field of the cache key.
- Fold buildAlignedTimeSeriesMetadata back into its only caller: the device-wide batch path that shared it is gone. - Rename FragmentInstanceContext.ignoreNotExistsDevice to isStorageEngineRecovering. It is set at query start from !StorageEngine.isReadyForNonReadWriteFunctions(), so it reports whether the data regions were still recovering, not whether a missing device can be ignored. The cache and reader parameters keep their policy names because they also cover the FILE_TIME_INDEX case.
There was a problem hiding this comment.
🟡 Changes recommended
Bloom-filter exclusions cause incorrect query statistics and redundant locked cache probes, while test cleanup leaves an external file reader open.
3 open issues
9 resolved since last review
This setstsfile.versionto a SNAPSHOT, which is generally not merge-safe/reproducible (builds… This busywhile (true)loop can spin indefinitely ifhasNextFile()keeps returning… This test createstarget/external-cache-disabled.tsfilebut never deletes it. That can pollute… This test createstarget/external-cache-disabled.tsfilebut never deletes it. That can pollute…loadBloomFilterTimeis always recorded as 0 because it’s never updated around…loadBloomFilterTimeis always recorded as 0 because it’s never updated around…loadBloomFilterTimeis always recorded as 0 because it’s never updated around…loadBloomFilterTimeis always recorded as 0 because it’s never updated around… The Javadoc says bloom filtering is skipped whendeviceMetadataIndexNodeOffsetis non-null, but…
Review effort: Max
- Check the aligned time column before the per-column cache pass: a file without data for the device then costs one lookup and at most one BloomFilter probe instead of one lookup per selected measurement. - Load the BloomFilter and reject excluded measurements before taking the device lock; the locked recheck now only sees the survivors, so a warm sparse scan does not recheck every excluded column under the lock. - Derive diskLookupCount in finally, so the device-gate early returns and the partial-cache paths are counted and the cache/disk time split uses the real miss count. - Hold an external reader reference in the external-TsFile test and release it before deleting the fixture; the internal close helper never saw the external registry entry.
There was a problem hiding this comment.
🟡 Changes recommended
When only the time-column entry is evicted, BloomFilter loading is misattributed between cache and disk phases, potentially producing negative disk-time statistics and metadata timers.
1 open issue
3 resolved since last review
Review effort: Balanced
The time-column check can load the BloomFilter before the cache phase ends. Subtracting that load from the disk interval, which does not contain it, could record negative disk time and metadata timers. Track the load before and after the cache phase separately and subtract each from its own interval. Add a regression case that evicts only the time-column entry and makes the BloomFilter load slow, so the accounting error would turn the recorded disk time negative.
apache/tsfile#1123 is merged on develop (squash 0d89263e); the dated snapshot 2.5.1-261120-SNAPSHOT is built from that commit and published to Apache snapshots.


Description
A device's
TimeseriesMetadatais stored in measurement leaf groups (up tomax_degree_of_index_nodeseries per group). Loading it column by column walks the device metadata index and deserializes the containing leaf group once per requested column, so a scan that selects almost all columns pays that traversal once per column, and every column absent from a file costs its own cache probe and BloomFilter check. In the motivating workload (one device with ~700 columns, ~20 written per day, 12 files) loading the metadata of one device across its files dominated the query.The TsFile side of this change is apache/tsfile#1123; this PR pins the dated snapshot built from that branch.
What changed
TsFile side (apache/tsfile#1123)
TsFileSequenceReader.readTimeseriesMetadataForMeasurements(device, offset, measurements, returnedMeasurements, mayExist, ignoreNotExistDevice, ioSizeRecorder)locates the device's measurement metadata index node (or reads the supplied offset for external TsFiles), routes each name to its floor child with a binary search over the node's children, and reads each visited internal node and each visited leaf group once. Chunk metadata is decoded only for the measurements the call returns.returnedMeasurementsmay be a superset ofmeasurements, which lets a single-column call warm its group siblings.mayExistis optional: a name is tested before it selects a group, so a group that only absent names point to is not read. It must have no false negatives; aBloomFilterqualifies.DataNode
TimeSeriesMetadataCache.getTimeseriesMetadatais now:count == 0, is answered with null and remembered with a negative cache entry, so a later query does not walk its leaf group again.Offset-addressed external TsFiles skip BloomFilter filtering, because their BloomFilter may not cover the series.
get()remains the single-column entry point used by tree-model scans. (Earlier revisions of this PR used a device-wide read gated by a 0.6 projection ratio; the selective reader replaces that design.)tsfile.versionmoves to2.5.1-261120-SNAPSHOT, cut from the mergeddevelop(apache/tsfile branchsnapshot/2.5.1-261120, squash commit0d89263e) and published to the Apache snapshots repository. It must be replaced by a released TsFile version before merge.Example
Table with 700 field columns; each of 12 TsFiles holds 20 of them (rotating);
SELECT *for one device:The per-column probe is the price of keeping negative entries bounded by the BloomFilter false-positive rate instead of one entry per absent column per file. A dense 700-column device adds the same probe per column but keeps the reads and bytes unchanged.
Benchmark
Measured on the latest revision with the same machine and TsFile snapshot for both variants, cold metadata cache unless noted; bytes and read counts come from
QueryStatistics. The base column was measured on master with the same harness. Each figure is the median of five timed rounds after warm-up; the dense cold case varies by up to about 20% between runs, so the table reports ranges.The sparse wide-table case that motivated the change is the main gain: cold 7.7–7.8 ms to 2.5–3.0 ms, warm 5.0–5.3 ms to 1.5–2.0 ms. The dense cold case varies by 15–20% between rounds on this machine (repeated runs of the previous revision show the same range, including a 30 ms outlier), so the table reports the observed range rather than a single sample; the bytes and read counts are identical across runs. The latest revision checks the aligned time column before the per-column cache pass and filters BloomFilter-excluded measurements before taking the device lock, so a warm sparse query no longer enters the locked recheck for columns that are absent from the file. Sparse cold still pays one BloomFilter probe per cache-missed column, which is what keeps negative entries bounded by the false-positive rate instead of one per absent column. The single-series
get()fast path answers a cache hit on the first LRU lookup, like master.Merge checklist
develop(squash commit0d89263e); the dated snapshot is cut from that commit.2.5.1-261120-SNAPSHOTinpom.xmlwith a released TsFile version before merge.Tests
TimeSeriesMetadataReadTestcovers batch vs per-measurement reads, absent names and devices, the offset-addressed variant, sibling warming, exact node/group read counts on an internal measurement index, aligned (time column) and table-model fixtures, and what an absent name costs with and withoutmayExist.TimeSeriesMetadataCacheTestadds "reader-confirmed absence is remembered, so the second query does not call the reader" and "a BloomFilter-excluded column is probed again instead of being remembered". It also covers cache hits, placeholders, BloomFilter exclusion and external TsFiles.AlignedSeriesScanBatchTest(dense, sparse and all-null projections end to end; the dense case asserts one reader call per file through a spied reader) andReadPointAlignedCrossCompactionTestcover the device-absent path.🤖 Generated with Claude Code