diff --git a/CHANGELOG.md b/CHANGELOG.md index d2988ee..b0597fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,28 @@ All notable changes to SheetStream. Versions follow [Semantic Versioning](https://semver.org/). +## 0.7.0 (2026-11-18) + +### Changed + +- **Text is stored in the cell or in the shared string table, whichever suits the column.** The new `PackageOptions.StringStorage` is `Auto` by default, with `Shared` (every text through the shared string table, which is what SheetStream did until now) and `Inline` (every text in its cell, the same as `UseSharedStrings = false`, which still works and wins) as the other choices. Until now every text went through the table: a hash lookup per cell, every distinct text held in memory until the save, and a table the size of the sheet for columns whose values never repeat. A sheet of 100,000 rows of ten text columns allocated 25 to 44 MB doing it and took about twice as long as it needed. With `Auto` SheetStream looks at the first cells of each column (128 if none repeats, otherwise 512) and decides: a column whose values repeat and are long or varied (the repeated texts average 16 characters or more, or the sample holds 200 or more different values) keeps using the table; every other column (ids, e-mail addresses, free text, and short values from a small set such as a status or a country) is written inline from then on, without being hashed or remembered. A column that looked unique is not written off at once, because a column can repeat among thousands of different values (customer or product names: 20,000 different ones in 100,000 rows) and show no repeat in its first cells. Such a column gets two more looks, at its 512th and its 8,192nd cell, of 512 cells each. Each look counts how many of the cells repeat a text the column has already put in the table and from that estimates how many different values the column has (the number of cell pairs compared divided by the number of pairs that were equal, the "birthday" estimate). The zip compressor only folds away a repeat that is within 32 KB of the last one, so a column whose different values together (their number times their average length) are more than that cannot be helped by the compressor, and sharing makes the file several times smaller (20,000 different values of 25 or 80 characters in 100,000 rows: from 6.4 MB and 17.1 MB inline to 1.9 MB and 2.6 MB shared, at the default zip level). If the estimated vocabulary is more than 24 KB, and the column repeats at all (at least 3 repeats in the look, and by the estimate at least one cell in a hundred so far), the column is shared from there on. A vocabulary that fits in the window stays inline however many cells there are, and so does one that the compressor reads easily: 1,920 customer names of 13 characters built from 48 first and 40 last names made a 5% larger file and a slower write when shared, at the default zip level. A shared column that stops repeating is written inline from then on. The same 100,000 by 10 sheets now allocate under 300 KB. **Files can differ in how the same text is stored** (a cell reads `t="inlineStr"` where it used to read `t="s"`), but Excel and every reader show the same text; the one difference we found is openpyxl 3.1.5 (and pandas through it), which shows the `_xHHHH_` escape of a control character as text in either layout and does not decode the `_x005F_` escape of a literal `_xHHHH_` in an inline cell (it does in a shared string), so such text is kept in the table even in an inline column. Third-party readers may show the escapes of text written inline (`StringStorage.Inline`, or a column `Auto` writes inline) as characters of the text; Excel and ExcelReader read both layouts alike. A sheet of fewer than 128 rows has its text stored exactly as with `StringStorage.Shared` (the sheet and `sharedStrings.xml` are the same bytes as `Shared` writes; with the default options the sheet XML still differs from 0.6.0 because cells no longer repeat their address). Two things to know. A column that repeats among more different values than the two looks can see (about 100,000 or more, so few repeats that they hide in a look), or that only starts repeating after its 8,700th cell, stays inline: the file is then as large as one from a writer that always writes inline, and larger than `Shared` would make it; set `StringStorage.Shared` for such data. And Excel opened a sheet made of short repeated texts faster when they were shared (about 15 to 30% in an informal test on a workstation), which is the reason to choose `Shared` when opening speed matters more than writing speed. The figures for this release come from the benchmark runs in `benchmarks/`, not from this note. +- **Text written inline is composed in one pass.** Inline text used to be scanned three times (to sanitise it, to find the characters that need an entity, and to encode it). It is now checked and encoded in one pass, eight characters at a time on .NET 8 and later, and a cell without a style or an address is composed in one piece. The file is the same, byte for byte (checked against a real `XmlWriter` for every text length around the block size and the size limit of the fast route, with every kind of special character at every position, and for `Auto` with columns that change from shared to inline). +- **Less work per text cell and per shared string.** On .NET 8 and later the 24 opening and 13 closing bytes of an inline text cell are written with a few 8-byte stores instead of a block copy and a byte loop, the cell's closing tags are written in the same pass as its text (one room check per cell), a plain unstyled cell is recognised with one flag, and the check for text that needs escaping reads sixteen characters per step. The shared string table is looked up and added to with one hash instead of two (.NET 8 and later), and `sharedStrings.xml` is composed as text in chunks and handed to the XML writer raw instead of through three XML writer calls per entry, which also makes a workbook written with `StringStorage.Shared` faster (6 ms on the 360,000 distinct texts of StringHeavy, 13 ms on the one million of the all-unique sheet). The files are the same byte for byte (the sheet and shared strings parts hash the same before and after for eight scenarios in all three storage modes, and the differential tests against a real `XmlWriter` pass unchanged); only the .NET 8 and later build changed, the `netstandard2.0` build keeps its loops. Measured on a quiet VM, the 100,000 by 10 text sheets with default options, median of 21 runs, next to SpreadCheetah in the same run: StringHeavy 82.4 to 76.0 ms at the Fastest zip level (SpreadCheetah 79.2 and 78.1 in those two runs) and 193.6 to 186.4 ms at Optimal (189.4 and 187.2); the all-unique sheet 62.7 to 56.0 ms and 172.6 to 168.0 ms (SpreadCheetah 54.7 to 56.0 and 165.4 to 167.3). The figures for this release come from the benchmark runs in `benchmarks/`, not from this note. +- **Smaller sheet files: cells no longer repeat their address.** A cell that sits in the column right after the previous cell of its row is written without its `r="B2"` attribute, which the Open XML specification makes optional (a cell without one belongs in the next column; the first cell of a row belongs in column A). A cell that follows a gap, whether from `Skip()` or an empty cell with no style, still carries its address, so every value stays exactly where you put it. Excel, ExcelReader, `Worksheets.Edit` and `OpenTemplate` all read these files as before, and so do EPPlus 4.5.3.3, ClosedXML 0.105.1, ExcelDataReader 3.9.0 and NPOI 2.7.6's XSSF reader (each checked on 2026-11-16 by reading a sheet with gaps and comparing every cell with what was written). Some tools do need the address on every cell, and we checked three of them: **MiniExcel 1.46.0** ignores the addresses of a sheet whose first cell has none, so a value that follows a gap is put in the wrong column (a sheet with no gaps reads correctly; this corrects an earlier note in this entry that listed MiniExcel as reading such files correctly), NPOI 2.7.6's XSSF event (SAX) reader gets no address for such a cell, and the Excel ADO / OleDb provider (ACE 12.0 and 16.0) fails with "System resource exceeded". Open XML SDK code that reads `Cell.CellReference` gets null for such a cell (the attribute is absent). Microsoft Spreadsheet Compare has been reported to need the address; we have not checked it. If you or your users rely on one of them, set `PackageOptions.WriteCellReferences = true`: every cell is then written with its address, as before this change, at the price of a larger file and a slower write. The option is new and off by default. +- **Writing numbers creates no garbage.** Every number, style index and shared-string index written to a cell used to be turned into a new string first. They are now formatted straight into a reused buffer, so writing a million cells of numbers, dates and repeated text no longer allocates per cell (about 25 megabytes less garbage for a 100,000 by 10 sheet), which also means fewer garbage collections and a lower peak under load. The text in the file is unchanged, byte for byte. On .NET Framework (the `netstandard2.0` build) doubles, decimals, dates and durations still go through a string, because that runtime cannot format them into a buffer; integers, style numbers and text are free of it there too. + +- **Faster cell writing.** A sheet's rows are now composed as UTF-8 text straight in the write buffer of the zip entry, instead of being handed to the XML writer, which re-encoded them and copied them again. Numbers, dates, durations, booleans, text (shared and inline), formulas, error cells and styled empty cells are formatted once, in place, and reach the compressor in large blocks; the `` element is written the same way. Rich text still renders its runs with the XML writer. Numbers that are short decimals (`1234.5`, `19.99`, `0.07`, which is nearly every price, quantity and date in a business sheet) are now turned into text by a small routine instead of the runtime's general shortest-round-trip formatter; it gives exactly the same text (checked against the runtime over hundreds of millions of values) and anything it cannot decide goes to the runtime as before. On .NET Framework the runtime formats every double, as before, because its formatter does not always find the shortest text and a file must not change with the runtime that wrote it (the build for `netstandard2.0` tells .NET Framework from .NET Core 3 and later and .NET 5 and later by the runtime's description; .NET Core 1 and 2 describe themselves as ".NET Core 4.6.x", are not told apart and are not supported). The file is identical in content, character for character (checked against a real `XmlWriter` for every kind of cell and every character, in new sheets, AutoFit sheets and sheets filled into a template, and against the previous build on the samples). Only the compressed bytes can differ slightly, because the data now reaches the compressor in different pieces. A package also keeps one write buffer for all its parts instead of one of 64 KB per part, so writing a sheet allocates about 150 KB instead of about 690 KB; `PackageOptions.BufferSize` sizes that one buffer (4 KB at least). The speed-up depends on the machine and the data; the figures for this release come from the benchmark runs in `benchmarks/`, not from this note. + +- **Benchmarks re-measured on a clean virtual machine, with raw results published.** The tables in the README and in `benchmarks/README.md` now come from one Azure `Standard_D8s_v6` machine (2026-11-18) instead of a developer workstation, with three launches per row for the five standard write scenarios and a SpreadCheetah 1.29.0 row at the same zip level next to every SheetStream row. The BenchmarkDotNet reports, the `--peak` output, the correctness-guard log and the hardware record of this run and of two earlier full runs are in `benchmarks/published-results`. The comparison with SpreadCheetah is mixed and stated as it measured: SheetStream is ahead on long, wide, very wide, million-row and styled sheets, level on mixed text, and about 2% behind on all-unique text at the `Optimal` zip level; SpreadCheetah allocates less per write. + +### Fixed + +- **A line break written as a carriage return (`"a\r\nb"`, `"a\rb"`) now reads back as written.** The text was written with a raw carriage return in the XML, and every XML parser turns that into a line feed (XML 1.0, section 2.11), so `"a\r\nb"` came back as `"a\nb"` in Excel, ExcelReader and every other reader. A carriage return is now written as the character reference ` `, in inline text, in the shared string table, in rich-text runs, in the cached text of a formula, and in the cells and shared strings that `OpenTemplate` and `Edit` copy from the workbook they open (including text made of nothing but white space, such as `"\r"` or `" \r\n "`, and the untouched inline, formula and rich-text cells of an edited sheet). The reference is the spelling every reader we tried gives back as a carriage return: Excel (opened through COM, `Range.Value2`), ExcelReader and openpyxl. Excel's own spelling, `_x000D_`, was not chosen: Excel and ExcelReader decode it, but openpyxl (and pandas through it) show it as the literal text `_x000D_`. Text without a carriage return is written byte for byte as before; a line feed alone, which is what Excel itself stores for Alt+Enter, is unchanged. Comments, document properties and other text that is not a cell still write a raw carriage return. +- **Text of more than 32,767 characters is refused when it is written.** Excel refuses to open a workbook that holds a cell with more than 32,767 characters ("Unable to get the Open property"); the whole workbook, not just the cell. SheetStream used to save such a workbook without a word, in every storage mode. **This changes behaviour for text Excel could not open anyway:** `WriteCell` (text, and any object that becomes text), `WriteInlineString`, `WriteRichText` (the runs are counted together), the cached text of `WriteFormula`, the text of `WriteHyperlink` and `WriteInternalLink`, `LoadFrom*`, and `Cells[...].Value` (also when editing a workbook; an object that is only turned into text when it is written, such as a `StringBuilder` or a type with its own `ToString`, is converted by the same rule at the assignment and checked there) now throw an `ArgumentException` that says what happened and how long the text is ("Excel cannot open a cell with more than 32,767 characters; this text has 40,000."), before anything of the cell is written, so the row is as it was and you can shorten the text and go on. A text of exactly 32,767 characters is written as before. The length counted is that of the text, the way Excel counts it: a control character or a literal `_x0041_` is one character or seven, not the longer spelling it has in the file. If you relied on writing long text for readers other than Excel, split it over several cells. +- **`Skip` refuses a count that would take the next cell past column XFD.** `Skip(int.MaxValue)` added to the column unchecked, so the column wrapped to a negative number; the next cell then wrote its opening bytes before the column letters failed, and a caller that caught the exception and carried on saved a sheet Excel could not open (with `StringStorage.Auto` it ended in an `IndexOutOfRangeException` instead). `Skip` now throws `ArgumentOutOfRangeException` for a count that goes past the column after XFD, and every way of writing a cell (text, numbers, formulas, hyperlinks) checks the column before it stages a byte, adds to the shared string table or registers a link, so a refused call leaves the row as it was and writing can go on. A hyperlink written at a column past XFD used to be remembered anyway and made `Save()` fail. +- **A second sheet is judged on its own text under `StringStorage.Auto`.** A column of unique text written on a second sheet (the same ids listed twice, say) was shared as soon as it was sampled, because the sample found the first sheet's entries in the table and counted them as repeats. About eleven times as many texts went into the shared string table as for one sheet (12,800 instead of 1,152 for 20,000 or 100,000 rows). A text that an earlier sheet put in the table now counts as a repeat on a sheet only from the second time that sheet uses it, so each sheet is judged as if it started with an empty table, while the cells still share the entries. The strings of a template (or of an edited workbook) are the exception: they are in the file anyway, so a column that uses them is judged as before, and counting them as new would have made the file 38% larger for 20,000 rows cycling through 300 phrases that the template lists (130 KB to 180 KB). The output for one sheet of a new package, and for one sheet filled into a template, is unchanged byte for byte (checked on 8 scenarios in the three storage modes and on 9 template fills); a column that really repeats is still shared on every sheet. +- **A formula written with `NaN` or an infinity as its result no longer stores a result.** `WriteFormula(formula, double.NaN)` (and the infinities) wrote `NaN`, `INF` or `-INF` as the stored result, which Excel cannot use as a number. The formula is now written without a stored result on every build, so Excel calculates it when the file is opened. + ## 0.6.0 (2026-11-14) A security and safety release. It closes, for every way of opening a workbook, the class of problem that 0.5.0 fixed only in part, and it changes when a workbook is written. Two changes can break existing code; they are listed first. diff --git a/README.md b/README.md index 6461be2..258fda3 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ SheetStream.NET is a zero-dependency, MIT-licensed .NET library that creates, fills, edits and reads Excel (`.xlsx`) files. It is not related to the Node.js package of the same name. -P4 Software built it when EPPlus moved to a commercial licence. Rows are written straight into the zip archive through an `XmlWriter`, so memory stays flat however many rows you export. It targets `netstandard2.0` and `net8.0`, which covers .NET Framework 4.7.2 and later, and .NET 8 and later. The namespace and assembly are `SheetStream`. +P4 Software built it when EPPlus moved to a commercial licence. Rows are written straight into the zip archive as they are produced, so memory stays flat however many rows you export. It targets `netstandard2.0` and `net8.0`, which covers .NET Framework 4.7.2 and later, and .NET 8 and later. The namespace and assembly are `SheetStream`. ## Install @@ -49,7 +49,7 @@ using (var package = ExcelPackage.Create("FinancialReport.xlsx")) { sheet.Row(i + 1) .WriteCell(i) - .WriteCell($"Enterprise SKU-{i}") // Shared string, de-duplicated + .WriteCell($"Enterprise SKU-{i}") // Text: shared or inline, chosen per column .WriteCell(1500.50 * i, currencyStyle); } @@ -136,33 +136,36 @@ SheetStream.NET writes while you go, which is why memory stays flat. The cost is ## Benchmarks -Writing 100,000 rows of 10 mixed, formatted columns. BenchmarkDotNet on .NET 8.0.31, Intel Core i7-8700, Windows 11, measured on one developer workstation (2026-10-13). Peak memory is the peak working set above an empty baseline process, from separate fresh-process runs. +Writing 100,000 rows of 10 mixed, formatted columns. BenchmarkDotNet 0.15.8 on .NET 10.0.15, Windows Server 2025, an Azure `Standard_D8s_v6` virtual machine (Intel Xeon Platinum 8573C, 8 vCPUs, 32 GB) used for nothing else, measured on 2026-11-18 with three fresh processes per row (30 measured writes). Peak memory is the peak working set above an empty baseline process, from separate fresh-process runs. SheetStream.NET and SpreadCheetah each appear at both zip levels, so every row has a row from the other library at the same level to be read against. | Library | Time | Allocated | Peak above baseline | File size | |---|--:|--:|--:|--:| -| SheetStream.NET (default compression) | 1,012.6 ms | 26.67 MB | 29.6 MB | 5,798 KB | -| SheetStream.NET (`CompressionLevel.Fastest`) | 534.1 ms | 26.67 MB | 31.0 MB | 7,006 KB | -| MiniExcel 1.46.0 | 596.4 ms | 343.36 MB | 37.3 MB | 7,421 KB | -| NPOI 2.7.6 SXSSF | 2,039.7 ms | 704.22 MB | 65.2 MB | 5,518 KB | -| EPPlus 4.5.3.3 | 2,286.7 ms | 390.19 MB | 183.8 MB | 5,601 KB | -| ClosedXML 0.105.1 | 3,103.2 ms | 1,104.16 MB | 231.1 MB | 6,030 KB | -| NPOI 2.7.6 XSSF | 4,658.4 ms | 1,186.16 MB | 545.1 MB | 5,304 KB | - -- At default settings MiniExcel is faster. SheetStream.NET defaults to the smallest files (`CompressionLevel.Optimal`); with `Fastest` the two are level on this sheet (534 ms against 596 ms is inside the error bars). -- Allocation is where SheetStream.NET is clearly ahead: about 13 times less than the next library, and the same at both compression levels. -- NPOI and EPPlus write slightly smaller files on this sheet. -- Reading the same 100,000-row file: SheetStream.NET 532.7 ms and 118.62 MB allocated, ExcelDataReader 915.9 ms and 211.03 MB, EPPlus 4.5.3.3 1,205.8 ms and 787.26 MB, MiniExcel 1,829.3 ms and 2,369.07 MB, ClosedXML 3,034.0 ms and 1,598.17 MB. These were measured before 0.5.0, whose reader limit checks make reading up to about 10% slower and allocate about 10% less. -- At 1,000,000 rows, SheetStream.NET with `Fastest` takes 4.8 s and allocates 275 MB; MiniExcel takes 5.3 s and 3.3 GB. The default setting takes 12.8 s for a file 17% smaller. +| SheetStream.NET (`CompressionLevel.Fastest`) | 80.0 ms | 164 KB | 16.7 MB | 5,304 KB | +| SpreadCheetah 1.29.0 (its default, Fastest) | 106.8 ms | 12 KB | 15.0 MB | 5,296 KB | +| SheetStream.NET (default compression, Optimal) | 248.3 ms | 164 KB | 14.4 MB | 2,917 KB | +| SpreadCheetah 1.29.0 (Optimal) | 274.0 ms | 12 KB | 12.8 MB | 2,916 KB | +| MiniExcel 1.46.0 | 335.5 ms | 309 MB | 114.6 MB | 10,420 KB | +| NPOI 2.7.6 SXSSF | 1,582.3 ms | 636 MB | 130.9 MB | 5,518 KB | +| EPPlus 4.5.3.3 | 2,172.3 ms | 390 MB | 287.0 MB | 5,601 KB | +| ClosedXML 0.105.1 | 2,441.8 ms | 1,013 MB | 351.5 MB | 5,955 KB | +| NPOI 2.7.6 XSSF | 4,354.0 ms | 1,149 MB | 619.9 MB | 5,304 KB | + +- **Against SpreadCheetah, at the same zip level:** SheetStream.NET is faster on this sheet (25% less time at `Fastest`, 9% at `Optimal`), and on wide sheets (1,000 × 1,000: 42% and 14% less), very wide sheets (1,000 × 10,000: 43% and 15%), a million rows (23% and 13%) and styled dates (14% and 3%). The two are level on a mix of repeated and unique text. SpreadCheetah is about 2% faster on its own benchmark shape, 100,000 rows of ten text columns in which every value is different, at the `Optimal` level. The same ratios came out of three complete runs. +- **Against MiniExcel:** SheetStream.NET at its default settings is faster in every scenario we measure except one: with every cell styled as a date, money or percentage (StyledDates), MiniExcel takes 301 ms against 341 ms at the default level (105 ms with `Fastest`). MiniExcel's files are 3.6 times larger on this sheet. +- **Allocation:** SpreadCheetah allocates the least (12 KB per write here), SheetStream.NET next (164 KB, the same for a million rows), then every other library by a wide margin. +- Reading the same 100,000-row file: SheetStream.NET 351 ms and 124 MB allocated, ExcelDataReader 693 ms and 249 MB, EPPlus 4.5.3.3 1,163 ms and 805 MB, MiniExcel 1,459 ms and 2,157 MB, ClosedXML 2,663 ms and 1,509 MB. +- At 1,000,000 rows, SheetStream.NET with `Fastest` takes 0.81 s, SpreadCheetah 1.05 s and MiniExcel 3.5 s; at the default level SheetStream.NET takes 2.5 s for a file 45% smaller. - The read benchmarks use a file SheetStream.NET wrote, and MiniExcel cannot bold the header, so its file carries slightly less formatting. -EPPlus and NPOI are pinned to their last plain open-source releases. Methodology, the wide-sheet scenarios and how to rerun everything: [benchmarks/README.md](benchmarks/README.md). Rerun it on your own hardware before relying on the numbers. - +These are the numbers of one machine; the ratios were the same in all three runs, the absolute times will differ on yours. EPPlus and NPOI are pinned to their last plain open-source releases. Every table, the raw output, the methodology and how to rerun everything, including the rows where SheetStream.NET is not first: [benchmarks/README.md](benchmarks/README.md). Rerun it on your own hardware before relying on the numbers. ## When to choose something else - You need to edit large workbooks in place. `ExcelPackage.Edit` holds the edited sheet in memory and is meant for small files. Use ClosedXML (MIT) or EPPlus. - You need formulas evaluated in .NET, for example to read a calculated total without Excel. Use ClosedXML or EPPlus, which have calculation engines. - You need sparklines, macros, encryption, or pivot tables with calculated fields, grouping or slicers. Not supported. See the [gap analysis](docs/gap-analysis-epplus-8.md) for the full list against EPPlus 8. - You only map simple objects to and from sheets and want the smallest API. MiniExcel does that well. +- Other tools read your files and need every cell's address. By default SheetStream.NET omits the + attribute on a cell that directly follows the previous one, which the Open XML specification allows and Excel reads; MiniExcel 1.46.0, NPOI 2.7.6's SAX reader and the Excel OleDb provider (ACE) misread such files. Set PackageOptions.WriteCellReferences = true, at the price of a larger file and a slower write. ## Migrating from EPPlus or ClosedXML diff --git a/benchmarks/.gitignore b/benchmarks/.gitignore index ddfda55..98f2720 100644 --- a/benchmarks/.gitignore +++ b/benchmarks/.gitignore @@ -1,2 +1,6 @@ # BenchmarkDotNet writes its raw reports here; the README carries the copied tables. BenchmarkDotNet.Artifacts/ + +# Older reports that run-on-fresh-vm.ps1 moves aside, and the time-stamped result folders it creates. +BenchmarkDotNet.Artifacts.before-*/ +results/ diff --git a/benchmarks/README.md b/benchmarks/README.md index de06a1b..39efa55 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -20,33 +20,40 @@ NPOI is measured twice on the write side: once with **XSSF**, the in-memory user All commands run from `benchmarks/SheetStream.Benchmarks`, and everything is Release. ```powershell -# The two original write scenarios (about 10 minutes on the machine below) -dotnet run -c Release -- --filter '*Long*' '*.WideBenchmarks.*' - -# One scenario at a time -dotnet run -c Release -- --filter '*Long*' # 100k rows x 10 columns -dotnet run -c Release -- --filter '*.WideBenchmarks.*' # 1,000 x 1,000 ('*Wide*' alone would also match VeryWide) -dotnet run -c Release -- --filter '*Million*' # 1,000,000 rows x 10 columns (needs 16 GB+ RAM, see below) -dotnet run -c Release -- --filter '*VeryWide*' # 1,000 rows x 10,000 columns -dotnet run -c Release -- --filter '*Read*' # read the 100k x 10 file with every reader (about 7 minutes) +# One scenario at a time. Each filter names exactly one benchmark class. +dotnet run -c Release -- --filter '*.LongBenchmarks.*' # 100k rows x 10 mixed columns +dotnet run -c Release -- --filter '*.WideBenchmarks.*' # 1,000 x 1,000 doubles +dotnet run -c Release -- --filter '*.SpreadCheetahOwnBenchmarks.*' # 100k x 10 distinct strings: SpreadCheetah's own scenario +dotnet run -c Release -- --filter '*.StringHeavyBenchmarks.*' # 100k x 10 strings, repeated and unique, with escaping +dotnet run -c Release -- --filter '*.StyledDatesBenchmarks.*' # 100k x 8 dates, date-times, money and percentages +dotnet run -c Release -- --filter '*.MillionBenchmarks.*' # 1,000,000 rows x 10 columns (needs 16 GB+ RAM, see below) +dotnet run -c Release -- --filter '*.VeryWideBenchmarks.*' # 1,000 rows x 10,000 columns +dotnet run -c Release -- --filter '*.ReadBenchmarks.*' # read the 100k x 10 file with every reader (about 7 minutes) + +# Several at once (each class takes a few minutes; the three-launch classes take about three times as long as one launch) +dotnet run -c Release -- --filter '*.LongBenchmarks.*' '*.WideBenchmarks.*' # Everything, including the 10M-cell scenarios (well over an hour) dotnet run -c Release -- --filter '*' # Peak working set, output size and full-size verification, each library in a fresh process. -# With no scenario it runs all five (Long, Wide, Million, VeryWide, Read); name one or more to run just those. +# With no scenario it runs all of them (see Scenarios.All); name one or more to run just those. dotnet run -c Release -- --peak # all scenarios dotnet run -c Release -- --peak Long Wide # about 3 minutes +dotnet run -c Release -- --peak SpreadCheetahOwn StringHeavy StyledDates dotnet run -c Release -- --peak Read # about 3.5 minutes dotnet run -c Release -- --peak Million VeryWide # tens of minutes; the in-memory libraries need 16 GB+ RAM -# Correctness guard only: shows which rows, cells and formats every writer actually wrote, -# and the checksum every reader arrived at +# Correctness guard only: shows which rows, cells and formats every writer actually wrote +# (values are compared cell by cell for the string and styled scenarios), the data profile of the +# string scenarios, and the checksum every reader arrived at dotnet run -c Release -- --verify ``` In Git Bash, quote the filter (`'*'`). Otherwise the shell expands `*` to file names. +**Filters match the full name of a benchmark, and the matching ignores letter case.** `'*Read*'` therefore also selects every `SpreadCheetah` benchmark in the suite (S-p-**read**-Cheetah), and `'*Wide*'` also selects VeryWide. Use the `'*..*'` form above, which selects exactly one class; `--list flat --filter ''` shows what a filter would run, without running anything. + BenchmarkDotNet's `-j Dry` switch adds a Dry job next to the one declared on each class rather than replacing it, so a "quick" `--filter '*Read*' -j Dry` still runs the full job as well. The **Million** scenario holds a million `LongRow` objects (about 80 MB) plus each library's working set. SheetStream and MiniExcel stay close to the baseline, but the in-memory models (EPPlus, ClosedXML, NPOI XSSF) build the whole workbook first and allocate ten times what they do on Long, so run Million on a machine with at least 16 GB of RAM. @@ -66,6 +73,22 @@ Every library gets the same input. The input is built once, before timing, from * **Wide**: 1,000 rows x 1,000 columns of doubles, with no header and no formatting. * **Million**: 1,000,000 rows x the same 10 columns as Long (10M cells). Same data generator, same `WriteLong` code; only the row count changes. * **VeryWide**: 1,000 rows x 10,000 columns of doubles (10M cells, the same count as Million). Same `WriteWide` code as Wide; only the column count changes. Million and VeryWide together show whether a library cares about the shape of a sheet or only about the number of cells. +* **SpreadCheetahOwn**: SpreadCheetah's own published benchmark shape, so every library is also measured on the scenario SpreadCheetah chose for itself. 100,000 rows x 10 columns, every cell a string, no header, no styles. Every value is distinct (`Value -`), so nothing can be de-duplicated and no value repeats in a row or a column. Per its README, SpreadCheetah's benchmark builds each string from its row and column position and writes to an in-memory stream; the text here is ours, the shape is theirs. +* **StringHeavy**: 100,000 rows x 10 string columns, no header, no styles, with a realistic mix. `--verify` prints this table (distinct values, longest value, cells that need XML escaping) for the data it generates: + + | Column | Content | Distinct values | + |---:|---|---:| + | 1 | order id (`SO-00000001`) | 100,000 | + | 2 | status | 5 | + | 3 | country | 32 | + | 4 | category | 12 | + | 5 | customer name (first + last, includes accented letters, an apostrophe and `Smith & Sons`) | 1,920 | + | 6 | e-mail | 100,000 | + | 7 | city | 50 | + | 8 | priority | 3 | + | 9 | SKU (hexadecimal, unique) | 100,000 | + | 10 | free-text note, at most 60 characters, 41,666 of them containing `&`, `<`, `>` or `"` | 100,000 | +* **StyledDates**: 100,000 rows x 8 columns plus a bold header row. Three dates (two `yyyy-mm-dd`, one `yyyy-mm-dd hh:mm:ss` with whole seconds), three money columns (`#,##0.00`) and two percentage columns (`0.00%`). Every data cell carries a style, so each row uses four distinct styles (date, date-time, money, percentage), and the header makes five. * **Read**: every reader reads the Long file (100,000 rows x 10 columns) that SheetStream wrote once, outside the timing, into a byte array. Each reader walks every data row and returns a checksum (the sum of the `Id` column plus the number of non-empty cells), so the work cannot be optimised away and the result can be checked against the generator. ## Methodology @@ -80,6 +103,9 @@ Every library gets the same input. The input is built once, before timing, from * Long passes the typed `LongRow[]` with `[ExcelColumn(Format = ...)]` on the formatted columns. * Wide passes one `Dictionary` per row, MiniExcel's documented shape for dynamic columns. This boxes every double. * Everything else is left at MiniExcel's defaults. + * **SpreadCheetah** (all scenarios): one cell array created once and reused for every row (the reuse pattern SpreadCheetah's own benchmark and the code its source generator produces both use; `AddRowAsync` writes the row out before it returns). The cell type is the cheapest one that fits: `DataCell` where no cell has a style (Wide, SpreadCheetahOwn, StringHeavy), `StyledCell` where every cell does (StyledDates), `Cell` where only some do (Long). Styles are added once. It is measured at its own default zip level (Fastest) and at Optimal, next to SheetStream at both levels. + * **SheetStream** (string scenarios): default options, which means `StringStorage.Auto`: SheetStream looks at the first 128 to 512 cells of every column and then either keeps sharing the column's text (the values repeat and are long or varied) or writes it inline (unique ids, e-mail addresses, free text, short values from a small set); a column that looked unique gets two more looks of 512 cells, at its 512th and 8,192nd cell, which share it if it repeats among thousands of values. Nothing is tuned for the benchmark. The Long, Million, SpreadCheetahOwn and StringHeavy scenarios also run two extra rows, **SheetStream (shared strings, as before)** and **(shared strings, Fastest zip)**, which set only `StringStorage = Shared`: every text through the shared string table, which is what SheetStream did before `StringStorage` existed. They let one run show before and after. + * **MiniExcel** (string scenarios): one reused `Dictionary` per sheet, overwritten for every row. MiniExcel's streaming writer consumes a row before asking for the next, so this is safe and is the cheapest way to feed it; the guard proves every row still arrives intact. StyledDates uses the typed `StyledRow[]` with `[ExcelColumn(Format = ...)]`, like Long. * **NPOI:** `CreateRow` / `CreateCell` / `SetCellValue`, with the three `ICellStyle`s created once and `Write(stream, leaveOpen: true)`. SXSSF uses its default 100-row window. It flushes older rows to a temporary file on disk, so its time includes that disk I/O. * **Each reader is written the way its docs recommend.** The input is the same byte array for all of them, handed over as a fresh read-only `MemoryStream` per invocation (created in `[IterationSetup]`, outside the timing), so nobody pays for disk I/O and a reader that disposes its stream does no harm. Every reader skips the header row and counts data rows only. * **SheetStream ReadRows:** `ExcelReader.Open(stream, leaveOpen: true)` and `ReadRows("Data")`, the untyped streaming path; numbers arrive as `double`, shared strings resolved, the date column as `DateTime`. @@ -91,8 +117,8 @@ Every library gets the same input. The input is built once, before timing, from * **EPPlus 4.5.3.3:** `new ExcelPackage(stream)`, then a `Cells[r, c].Value` loop over `Dimension`. Each `Cells[r, c]` allocates an `ExcelRange`; there is no cheaper documented path on the read side. * **NPOI XSSF:** `new XSSFWorkbook(stream)`, then `GetRow` / `GetCell` over `LastRowNum`. * **NPOI SAX:** `OPCPackage.Open(stream)`, `XSSFReader`, `ReadOnlySharedStringsTable` and an `ISheetContentsHandler`, driven through `XSSFEventBasedExcelExtractor.ProcessSheet`, which is the only parser entry point NPOI exposes for its port of POI's event model. The handler delivers every cell as a *formatted string*, so `Id` is parsed back from text; that cost is inherent to the API. The package is closed with `Revert()` because `OPCPackage.Open(Stream)` opens read-write and `Close()` would try to save. -* **BenchmarkDotNet job:** `[MemoryDiagnoser]` with `SimpleJob(launchCount: 1, warmupCount: 3, iterationCount: 10)` for Long, Wide and Read, and a lighter `SimpleJob(launchCount: 1, warmupCount: 1, iterationCount: 3)` for Million and VeryWide, where the in-memory libraries take 20 to 60 seconds per write. One invocation per iteration throughout. `[IterationSetup]` empties the buffer (or opens the input stream), outside the measurement. Each benchmark runs in its own process, as BenchmarkDotNet does by default, on the default runtime settings (workstation GC, tiered compilation). -* **Correctness guard:** every write benchmark's `GlobalSetup` writes a small case (50 rows for the Long-shaped scenarios, 40 x 40 for the Wide-shaped ones) with every library. It reads each output back with the Open XML SDK's SAX reader and throws if the row count or the number of cells in the last row is wrong. The read benchmark's `GlobalSetup` writes a 50-row file and throws if any reader's checksum differs from the generator's figure (`Checksum.Expected`, computed from the data definition, not from any reader). The `--peak` children repeat both checks on the full-size data after their measurement. +* **BenchmarkDotNet job:** `[MemoryDiagnoser]` with `SimpleJob(launchCount: 3, warmupCount: 3, iterationCount: 10)` for Long, Wide, SpreadCheetahOwn, StringHeavy and StyledDates (30 measured writes per row, in three fresh processes: a single process can settle into a state a few milliseconds slower than the next one on a 55 ms write, and three launches average that), `SimpleJob(launchCount: 1, warmupCount: 3, iterationCount: 10)` for Read, and a lighter `SimpleJob(launchCount: 1, warmupCount: 1, iterationCount: 3)` for Million and VeryWide, where the in-memory libraries take 20 to 60 seconds per write. One invocation per iteration throughout. `[IterationSetup]` empties the buffer (or opens the input stream), outside the measurement. Each benchmark runs in its own process, as BenchmarkDotNet does by default, on the default runtime settings (workstation GC, tiered compilation). +* **Correctness guard:** every write benchmark's `GlobalSetup` writes a small case (50 rows for the Long-shaped scenarios, 40 x 40 for the Wide-shaped ones, 60 rows for the three new scenarios) with every library. It reads each output back with the Open XML SDK's SAX reader and throws if the row count or the number of cells in the last row is wrong. For SpreadCheetahOwn, StringHeavy and StyledDates it also reads every cell back and throws unless each text equals the generator's text (shared and inline strings are resolved first), each number equals the generator's number to within 1e-7 (dates are compared as their serial numbers), and, for StyledDates, each data cell carries the number format of its column. The header's bold font is reported, not enforced, because MiniExcel cannot make a header bold. The value check was itself tested by corrupting one cell in one writer and confirming the guard names that cell. The read benchmark's `GlobalSetup` writes a 50-row file and throws if any reader's checksum differs from the generator's figure (`Checksum.Expected`, computed from the data definition, not from any reader). The `--peak` children repeat both checks on the full-size data after their measurement. * **Peak working set (`--peak`):** the executable starts itself again with `--peak-child `. Each child does the following: 1. Builds the same input (for Read: writes the 100k-row file with SheetStream into memory). 2. Writes (or reads) once, cold, including JIT. @@ -101,184 +127,355 @@ Every library gets the same input. The input is built once, before timing, from Each case runs in 3 fresh processes and the median is reported. A **Baseline** child builds the input and calls no library. It shows the floor every library sits on: the .NET runtime, the benchmark executable and the input data (for Read, the input file as well). The Baseline child never calls a writer or reader, so the libraries' assemblies are not loaded or JIT-compiled in it. Each library's "above baseline" figure therefore includes loading and compiling its own code. The output buffer's pages count only once they are written, so each writer's figure includes roughly its output size. -## Machine +## Reproduce these numbers -``` -BenchmarkDotNet v0.15.8, Windows 11 (10.0.26340.9616) -Intel Core i7-8700 CPU 3.20GHz (Max: 3.19GHz) (Coffee Lake), 1 CPU, 12 logical and 6 physical cores -.NET SDK 10.0.401 - [Host] : .NET 8.0.31 (8.0.31, 8.0.3126.42015), X64 RyuJIT x86-64-v3 - Job-IUYGTA : .NET 8.0.31 (8.0.31, 8.0.3126.42015), X64 RyuJIT x86-64-v3 +On a clean Windows machine (a new virtual machine is ideal) with only the .NET 10 SDK installed, copy or clone this repository, open the `benchmarks` folder and double-click `run-on-fresh-vm.cmd`, or run: + +```powershell +.\run-on-fresh-vm.ps1 -DryRun # print the plan and the hardware, change nothing +.\run-on-fresh-vm.ps1 # SpreadCheetahOwn, StringHeavy, StyledDates, Long and Wide, then --peak +.\run-on-fresh-vm.ps1 -Scenarios SpreadCheetahOwn # one scenario (comma-separated list, or All) ``` -This is a developer workstation, not an isolated benchmark box. The error columns show the noise this causes. SheetStream on Wide has the widest relative spread (±26% at 99.9%). +The script does the following, in order, and stops with a plain-language message if a step fails: -## Results +1. Prints the CPU, cores, memory, operating system, .NET SDK, code version and the exact package versions. +2. Builds the benchmark project in Release. +3. Runs `--verify`. **If any library fails the correctness check the script stops and takes no timings.** +4. Runs each chosen scenario through BenchmarkDotNet. +5. Runs `--peak` for the same scenarios. +6. Copies the BenchmarkDotNet reports, the logs and a `hardware.txt` into `benchmarks\results\-