CloudCutter E57 Benchmark 02: 244.54 GiB and 10.81 Billion Points

Benchmark E57-02 extends the first public CloudCutter baseline from a 26.09 GiB source to a substantially larger registered industrial dataset. The objective was not simply to prove that the application could finish. The test examines how reading, progressive visualization, memory use and multi-output export scale when the point count increases by more than eleven times.

The result is technically strong but deliberately mixed. CloudCutter completed both runs, kept its measured working set below 0.4 GiB and reproduced all six output inventories. Full reading scaled close to the point-count increase. At the same time, useful preview arrived too late, and the current safe exporter performed approximately one complete source point-stream read for every output file.

These are now the two principal optimization targets.

Benchmark summary

Dataset

The input was a non-public registered industrial-plant E57 export. Project-identifying information and the source file are not distributed, but its identity is fixed by exact metadata and SHA-256:

Filename: industrial_plant_registered-01.e57
Exact size: 262,571,366,400 bytes
Size: 244.539 GiB
Points: 10,809,494,448
E57 scans: 249
Structured / unstructured scans: 249 / 0
RGB scans: 249
Intensity scans: 170
SHA-256: A2126C6374EF8737A03BAFCC4436B74F73D89AECDDA33025FF0FB7412113297B

Compared with Benchmark E57-01, the source is 9.37 times larger by physical size and contains 11.26 times more points.

Test system

Both runs used the same normal desktop workstation:

GPU identification is included for system documentation. GPU utilization was not measured, and this report does not claim GPU acceleration for the E57 pipeline.

Method

The first run followed the established cold-start protocol:

1. Calculate and record the source SHA-256.
2. Restart Windows and do not open the source with another application.
3. Allow only CloudCutter's required metadata preflight before timed reading.
4. Select Cold start after Windows reboot.
5. Use Progressive preview + exact background refinement.
6. Create and save selections during the loading workflow.
7. Export six E57 files with XYZ and RGB.
8. Wait for automatic validation and create the report.

CloudCutter was then closed and reopened without restarting Windows for the repeated run. The same source, output definitions and export content were used.

Reading scales close to linearly

Benchmark E57-01 refined 960.08 million points in approximately 13 minutes 15 seconds, or 1.208 million points per second. With 11.26 times more points, strict linear scaling predicts approximately 2 hours 29 minutes for E57-02.

• Cold run: 2 h 48 min 44 s, approximately 13.1% slower than the linear estimate.
• Repeated run: 2 h 31 min 35 s, approximately 1.6% slower than the linear estimate.

The repeated result is therefore very close to point-count-linear scaling. No catastrophic growth in full-read time was observed.

The repeated run was about 10% faster across preview, full reading and export. Because the source point stream is much larger than available RAM and only one cold/repeated pair was recorded, this is described as a repeated-run effect—not proof that the complete source was served from Windows file cache.

The main user-facing problem: late preview

The initial preview stage completed after 51 minutes 7 seconds in the cold run and 46 minutes 29 seconds in the repeated run.

The current report label First preview does not identify the first moment at which points become visible. It records completion of the initial preview stage. During the test, visible and usable point data appeared much later in loading progress than with the 26.09 GiB source.

This distinction matters. Complete reading can scale acceptably while time-to-first-use still becomes unacceptable. Future reports will separate:

• first valid point decoded;
• first preview batch ready;
• first non-empty frame displayed;
• selection controls enabled;
• initial preview stage completed;
• full exact refinement completed.

The optimization target is to publish a small but useful spatial sample early and improve it continuously while reading continues.

Multi-output export result

One user command created six E57 outputs with XYZ and RGB:

The combined outputs contain 473,569,341 points—approximately 4.38% of the source point count.

All 6/6 files passed CloudCutter's libE57 metadata-open and Data3D point-count validation. The repeated run reproduced every per-file size and point count. Raw SHA-256 comparison and independent payload checks in CloudCompare remain separate verification steps and are not claimed by the automatic validator.

The export bottleneck is measurable at terabyte scale

The current safe exporter rereads the source point stream for every output. Derived from the rounded process-I/O averages and measured export duration, both runs performed approximately 1,054.4 GiB—about 1.03 TiB—of process-level logical reads during the six-file export.

This corresponds to approximately 175.7 GiB of point-stream reads per output pass. Five of the six source passes are repeated work.

For six outputs, replacing this design with one source pass can theoretically remove approximately 83.3% of source point-stream reads. The idealized single-pass lower bound is approximately 28–31 minutes before multi-writer routing and buffering overhead. This is not a promised future result, but it establishes the scale of the optimization opportunity.

At the current rate, 20 outputs from this dataset would require roughly 9–10 hours and approximately 3.4 TiB of logical source reads. The next correspondence exporter will therefore focus on one sequential source pass, bounded-memory routing and safe validation of every completed output.

Memory and CPU behavior

The measured working set remained small relative to the 244.54 GiB input:

• cold loading: 0.263 GiB;
• cold export: 0.297 GiB;
• repeated loading: 0.325 GiB;
• repeated export: 0.385 GiB.

This supports CloudCutter's bounded-memory design. The HTML report exposes working set, not total private or committed memory, so no broader RAM claim is made here.

Average total-capacity CPU utilization ranged from 1.64% to 5.66%, with a peak below 12%. This indicates optimization headroom, but does not prove that simple parallelization is the correct solution. Library serialization, synchronous I/O, batching and process synchronization must be profiled separately.

Benchmark findings that require correction

Two report-level inconsistencies were identified:

• The estimated XY footprint changed from 17,325 m² to 19,575 m² for the same source. The estimate is sample-dependent and should become deterministic after exact refinement or be labelled as a preview estimate.
• Active operator selection was 33 seconds in the cold run but 55 minutes 28 seconds in the repeated run despite identical output inventories. The repeated interval included waiting and is not a valid human-effort comparison.

These findings do not invalidate the read, export or output results. They identify where the benchmark instrumentation itself must become more precise.

What Benchmark E57-02 proves—and what it does not

Supported by the current evidence

• CloudCutter completed a 10.81-billion-point registered E57 workflow on a machine with 23.9 GiB RAM.
• Full reading scaled close to point-count-linearly in the repeated run.
• The measured working set remained below 0.4 GiB.
• One command created six RGB-designated E57 outputs.
• All six outputs passed metadata and point-count validation.
• The repeated run reproduced the same output sizes and point counts.
• The safe exporter still performs approximately one source point-stream pass per output.
• Time-to-useful-preview requires a separate optimization.

Not yet claimed

• byte-identical output files;
• complete automatic decoding of every RGB payload;
• warm-cache acceleration as an isolated cause;
• deterministic XY footprint estimation;
• a valid repeated-run operator-time comparison;
• single-pass export in the current version.

Next optimization stage

The benchmark changes the priority order for CloudCutter development:

1. instrument the real first-visible and first-usable preview events;
2. publish progressive preview batches independently of total source size;
3. make benchmark footprint and operator metrics deterministic and interpretable;
4. implement the single-pass correspondence exporter;
5. preserve output membership, RGB and E57 validity with bounded buffers;
6. repeat E57-01 and E57-02 with saved selection manifests;
7. publish the before/after results, including regressions.

The purpose of this series remains the same: publish evidence, including weaknesses, while using each benchmark to determine what should be improved next.

Reports and recordings

For supporting evidence, technical questions or collaboration: [email protected]

Publication history