Collaboration performance test report
Test environment, metrics, and results for real-time concurrent editing with the Univer collaboration engine.
We tested multiple editors in one document and simultaneous edits across many documents. After warming up the documents, we measured acknowledgment latency, throughput, the percentage of planned submissions that succeeded, and the collaboration process's CPU and memory usage.
Test environment
The collaboration service and PostgreSQL ran on separate servers. The machine sizes in the results refer to the collaboration node in the diagram.
PostgreSQL stores document records, changesets, snapshot records, and block references. OSS stores snapshot cell-data blocks.
All measurements are for a single collaboration node. When concurrent documents exceed its capacity, additional nodes can handle edits to different documents. They share a database and use pub/sub to synchronize rooms, presence, and edit broadcasts. See Multiple collaboration nodes.
| Component | Configuration |
|---|---|
| Univer Collaboration SDK version | 1.5.2 |
| OS and runtime | Ubuntu 26.04.1 LTS, Node.js 24.14.0 |
| Storage | PostgreSQL 18.6 and Alibaba Cloud OSS |
| Database adapter | PostgreSQL + S3 Adapter |
| Collaboration node | Alibaba Cloud ECS ecs.g9i.xlarge (4 vCPUs, 16 GiB) or ecs.g9i.2xlarge (8 vCPUs, 32 GiB), Intel Xeon 6982P-C, 40 GiB ESSD |
| Database server | Alibaba Cloud ECS ecs.c9i.large, 2 vCPUs, 4 GiB, Intel Xeon 6982P-C, 60 GiB ESSD |
| Load generator | Alibaba Cloud ECS ecs.u2a-c1m2.xlarge, 4 vCPUs, 8 GiB, AMD EPYC |
| Network | Alibaba Cloud Shenzhen region. Clients, collaboration node, and database communicate over the regional private network. OSS uses an internal endpoint. |
Collaboration service configuration
On both node sizes, we raised runtime.maxLoadedUnits from 1,200 to 4,000 so the node could keep up to 4,000 documents in memory. Other SDK runtime settings used their defaults, including automatic worker selection. The default snapshot policy triggers after 50 revisions since the last successful snapshot or when a single changeset reaches 5 MiB.
The benchmark service uses an initialized PostgreSQL pool and OSS client. The transport attaches to the Node.js HTTP server.
const dbAdapter = await PostgresS3DatabaseAdapter.create({ pool, s3: { client, bucket, prefix },});const service = new UniverCollabService({ dbAdapter, runtime: { maxLoadedUnits: 4000 },});const endpoint = new UniverCollabEndpoint(service);const transport = createNodeTransport();transport.register(endpoint);transport.attach(server);Test snapshots
Each document starts from a snapshot. Cell counts refer to cells containing data or formulas before warm-up.
| Snapshot | Populated cells | Contents |
|---|---|---|
blank | 0 | Empty worksheet |
10k-cells | About 10,000 | Numbers, text, and formulas |
100k-cells | About 100,000 | Numbers, text, and formulas |
Both populated sheets have 26 columns: 14 numeric, nine text, and three containing formulas that reference cells in the same row.
Method
Simulated clients
Each editor in the results is an independent session. Each session joins one document, submits a cell edit over HTTP, and receives the acknowledgment over WebSocket. The scripts call the collaboration API directly. Both the HTTP request and the wait for acknowledgment have a 10-second timeout.
Warm-up and sampling
Each run creates documents from the selected snapshot. Warm-up starts once all sessions have joined. The first edit loads the document on the server. A single document receives one warm-up edit. In multi-document runs, successive documents receive one to 49 edits, restarting at one for the 50th document. This staggers the starting revisions and snapshot writes.
Sampling starts after all documents have warmed up. We record first-submission latency separately during warm-up. Submission counts, latency percentiles, CPU, and memory in the tables come from the sampling period.
Edit workload
For the single-document test we increased the number of editors on blank. For each of the three snapshots we also increased the number of active documents. Each submission contains one sheet.mutation.set-range-values operation that writes a number to one cell.
| Condition | One document | Multiple documents |
|---|---|---|
| Editors per document | Varies by run (see results) | Two in every run |
| Edit operation | Staggered writes to each editor's own cell | Staggered writes to each editor's own cell |
| Frequency per editor | Once every 6 seconds | Once every 2 seconds |
| Sampling period per run | 300 seconds | 300 seconds |
Metrics
| Metric | Definition |
|---|---|
| Acknowledgment latency | Time from sending an edit until the same client receives the server acknowledgment. P50, P95, and P99 are the times within which 50%, 95%, and 99% of successful submissions were acknowledged. Max is the longest successful acknowledgment time. |
| Throughput | Successful submissions divided by sampling time, in submissions per second. |
| Completion rate | Successful submissions as a percentage of all planned submissions, including failed and unsent ones. |
| CPU mean / peak (%) | Mean and highest sampled CPU usage of the collaboration process. One fully utilized core counts as 100%, so totals can exceed 100%. |
| Peak memory (MiB) | Highest sampled resident set size (RSS) of the collaboration process. |
Pass criteria
A run passes when acknowledgment latency is P99 < 1,000 ms, the completion rate is > 90%, and the collaboration process stays up until every test document passes the consistency check. Latency and completion rate are evaluated before rounding.
A load level is evaluated across the runs reported here. A single run must pass, and if there are multiple runs, every run must pass. Interrupted samples and missing metrics are marked as missing with the reason.
Interpreting the results
Node.js scripts simulate editing sessions and send cell changes directly to the API. Acknowledgment latency measures the round trip from the script to the server. It excludes browser interaction and spreadsheet rendering.
Use these measurements for performance assessment and single-node capacity planning. Performance depends on deployment, network conditions, document size, and edit type and frequency.
The highest passing load is the largest tested level that met the criteria. Mean CPU usage approached node capacity at some passing levels. Treat these levels as capacity-planning references, not sustained production targets. Leave headroom for concurrency spikes and resource fluctuations, and validate sustained capacity with representative workloads in the target environment.
Results
Summary
The tables show the highest passing load per node size and document size. Load counts editors for one document and active documents for multi-document runs. Each row gives throughput, latency, CPU, and memory at that level. See detailed results for completion rates and higher loads.
Multiple editors in one document
Each node size passed one run with 500 editors.
| Snapshot | Node size | Highest passing editor count | Throughput (edits/s) | P95 (ms) | P99 (ms) | CPU mean / peak (%) | Peak memory (MiB) |
|---|---|---|---|---|---|---|---|
blank | 4 cores / 16 GiB | 500 | 83.3 | 13.3 | 17.4 | 27.8 / 54.8 | 777 |
blank | 8 cores / 32 GiB | 500 | 83.3 | 13.9 | 18.0 | 25.6 / 100.5 | 1,322 |
Concurrent editing across documents
| Snapshot | Node size | Highest passing document count | Throughput (edits/s) | P95 (ms) | P99 (ms) | CPU mean / peak (%) | Peak memory (MiB) |
|---|---|---|---|---|---|---|---|
blank | 4 cores / 16 GiB | 2,100 | 2,100.0 | 29.9 | 194.6 | 118.9 / 332.6 | 5,829 |
blank | 8 cores / 32 GiB | 2,900 | 2,900.0 | 17.8 | 56.2 | 145.3 / 363.9 | 7,404 |
10k-cells | 4 cores / 16 GiB | 1,800 | 1,800.0 | 106.2 | 311.9 | 241.3 / 379.3 | 6,304 |
10k-cells | 8 cores / 32 GiB | 2,400 | 2,400.0 | 26.9 | 482.0 | 286.3 / 749.3 | 8,071 |
100k-cells | 4 cores / 16 GiB | 380 | 380.0 | 91.1 | 200.9 | 389.6 / 401.8 | 12,440 |
100k-cells | 8 cores / 32 GiB | 750 | 750.0 | 33.5 | 72.3 | 649.7 / 782.1 | 29,276 |
Load estimates for 4 cores / 8 GiB and 8 cores / 16 GiB
The 4-core/8-GiB and 8-core/16-GiB figures are estimates based on measurements from 4-core/16-GiB and 8-core/32-GiB nodes respectively. We allowed 6.5 GiB and 13.7 GiB for the collaboration process. For each workload and snapshot we selected the highest passing level in the detailed results whose peak RSS fit the budget, using the highest RSS across runs.
Multiple editors in one document
| Snapshot | Target node size | Estimated editors | Peak memory (MiB) |
|---|---|---|---|
blank | 4 cores / 8 GiB | 500 | 777 |
blank | 8 cores / 16 GiB | 500 | 1,322 |
Concurrent editing across documents
| Snapshot | Target node size | Estimated active documents | Peak memory (MiB) |
|---|---|---|---|
blank | 4 cores / 8 GiB | 2,100 | 5,829 |
blank | 8 cores / 16 GiB | 2,900 | 7,404 |
10k-cells | 4 cores / 8 GiB | 1,800 | 6,304 |
10k-cells | 8 cores / 16 GiB | 2,400 | 8,071 |
100k-cells | 4 cores / 8 GiB | 350 | 5,413 |
100k-cells | 8 cores / 16 GiB | 600 | 8,943 |
Sizing node pools
In a multi-node deployment, the gateway can route documents by size to separate pools. These single-node estimates assume two editors per document, each editing once every two seconds.
- For small and medium documents, evaluate 4 cores and 8 GiB first. In the 4-core/16-GiB tests, the highest passing loads within the smaller memory budget were 2,100 blank sheets or 1,800 sheets with about 10,000 populated cells each.
- For large documents, evaluate 8 cores with 16 or 32 GiB. With about 100,000 populated cells per document, the 8-core/32-GiB node passed at 750 documents. The largest passing load within the 16-GiB configuration's memory budget was 600.
Detailed results
Single-run rows show measured values. When combining runs at the same level, we report arithmetic means for throughput, P50, P95, P99, Max, completion rate, and mean CPU. Peak CPU and memory are the highest observed across runs. Notes give the run count, the range of per-run maxima, and the P99 range where relevant. Each run is evaluated separately.
Multiple editors in one document
Snapshot · blank
4 cores / 16 GiB
| Editors | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 150 | 25.0 | 2.9 | 4.3 | 5.4 | 12.6 | 100.00% | 5.7 / 27.9 | 690 | — |
| 400 | 66.7 | 5.2 | 10.1 | 12.4 | 29.8 | 100.00% | 19.8 / 41.9 | 781 | — |
| 500 | 83.3 | 6.6 | 13.3 | 17.4 | 40.1 | 100.00% | 27.8 / 54.8 | 777 | — |
8 cores / 32 GiB
| Editors | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 150 | 25.0 | 3.4 | 4.4 | 5.5 | 10.7 | 100.00% | 5.1 / 66.5 | 1,123 | — |
| 400 | 66.7 | 6.2 | 10.8 | 13.4 | 28.2 | 100.00% | 18.2 / 75.9 | 1,196 | — |
| 500 | 83.3 | 7.5 | 13.9 | 18.0 | 36.9 | 100.00% | 25.6 / 100.5 | 1,322 | — |
Concurrent editing across documents
Snapshot · blank
4 cores / 16 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 500 | 500.0 | 1.1 | 1.6 | 3.7 | 25.0 | 100.00% | 33.8 / 89.3 | 2,057 | — |
| 1,000 | 1,000.0 | 1.1 | 2.4 | 10.7 | 57.0 | 100.00% | 61.4 / 193.1 | 3,133 | — |
| 1,500 | 1,500.0 | 1.3 | 4.8 | 40.1 | 361.6 | 100.00% | 86.3 / 279.0 | 4,134 | — |
| 2,100 | 2,100.0 | 1.7 | 29.9 | 194.6 | 1,156.2 | 100.00% | 118.9 / 332.6 | 5,829 | 3 runs, P99 range 141.9–254.0 ms, Max range 508.2–1,543.0 ms |
| 2,200 | 2,200.0 | 1.8 | 70.1 | 270.4 | 1,227.5 | 100.00% | 123.5 / 369.9 | 6,050 | Run 1 |
| 2,200 | 2,199.8 | 2.0 | 274.0 | 860.4 | 2,787.2 | 99.99% | 127.1 / 392.6 | 6,172 | Run 2 |
| 2,200 | 2,198.8 | 2.2 | 376.5 | 1,062.4 | 3,747.8 | 99.95% | 126.1 / 347.2 | 6,071 | Run 3, latency missed target |
| 2,300 | 2,298.1 | 3.2 | 878.6 | 1,675.8 | 4,313.9 | 99.92% | 135.8 / 370.8 | 6,336 | P99 missed target |
| 2,400 | 2,204.8 | 2,192.5 | 2,457.5 | 2,888.6 | 4,611.5 | 91.87% | 163.6 / 394.8 | 7,023 | P99 missed target |
8 cores / 32 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 500 | 500.0 | 1.0 | 1.3 | 2.2 | 26.1 | 100.00% | 30.6 / 78.1 | 2,823 | — |
| 1,000 | 1,000.0 | 1.1 | 1.7 | 3.9 | 58.9 | 100.00% | 53.8 / 163.4 | 3,961 | — |
| 1,500 | 1,500.0 | 1.2 | 2.6 | 9.6 | 51.8 | 100.00% | 76.3 / 165.1 | 4,694 | — |
| 2,200 | 2,200.0 | 1.4 | 6.5 | 15.0 | 189.2 | 100.00% | 112.2 / 428.1 | 6,030 | — |
| 2,600 | 2,600.0 | 1.8 | 13.0 | 38.8 | 334.1 | 100.00% | 132.2 / 477.5 | 6,891 | — |
| 2,700 | 2,700.0 | 1.8 | 11.7 | 35.0 | 270.5 | 100.00% | 135.1 / 661.7 | 7,026 | 3 runs, peak CPU 518.6%–661.7%, Max range 234.9–313.9 ms |
| 2,800 | 2,800.0 | 2.0 | 14.6 | 52.1 | 352.0 | 100.00% | 140.5 / 537.4 | 7,331 | 3 runs, peak CPU 464.9%–537.4%, Max range 273.3–430.0 ms |
| 2,900 | 2,900.0 | 2.2 | 17.8 | 56.2 | 395.2 | 100.00% | 145.3 / 363.9 | 7,404 | 1 run |
| 3,000 | 2,375.5 | 2,537.1 | 2,715.4 | 2,868.1 | 3,821.8 | 79.18% | 194.3 / 570.9 | 8,257 | 2 runs, P99 and completion rate missed targets, Max range 3,311.4–4,332.2 ms |
Snapshot · 10k-cells
4 cores / 16 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 500 | 500.0 | 1.1 | 3.2 | 7.7 | 206.5 | 100.00% | 71.1 / 165.8 | 2,514 | — |
| 1,000 | 1,000.0 | 1.3 | 5.1 | 16.4 | 115.6 | 100.00% | 133.6 / 330.5 | 3,879 | — |
| 1,600 | 1,600.0 | 2.0 | 42.7 | 256.7 | 1,165.7 | 100.00% | 215.2 / 371.1 | 5,677 | — |
| 1,800 | 1,800.0 | 2.7 | 106.2 | 311.9 | 1,197.3 | 100.00% | 241.3 / 379.3 | 6,304 | 3 runs, Max range 773.7–1,455.4 ms |
| 1,900 | 1,899.9 | 5.4 | 330.2 | 797.1 | 3,210.5 | 99.99% | 257.9 / 399.3 | 6,626 | Run 1 |
| 1,900 | 1,891.4 | 4.6 | 552.6 | 1,541.5 | 6,200.0 | 99.54% | 256.7 / 397.2 | 6,686 | Run 2, latency missed target |
| 2,000 | 1,868.4 | 2,084.5 | 2,401.9 | 2,964.0 | 7,658.3 | 93.42% | 284.8 / 397.6 | 7,104 | P99 missed target |
| 2,100 | 1,904.0 | 2,202.3 | 2,631.7 | 3,022.0 | 4,550.7 | 90.67% | 291.2 / 393.0 | 7,580 | P99 missed target |
8 cores / 32 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 500 | 500.0 | 1.0 | 2.0 | 4.1 | 27.3 | 100.00% | 69.3 / 139.9 | 3,408 | — |
| 1,000 | 1,000.0 | 1.1 | 2.8 | 6.8 | 30.3 | 100.00% | 127.2 / 238.8 | 4,601 | — |
| 2,000 | 2,000.0 | 1.5 | 8.2 | 21.2 | 184.4 | 100.00% | 249.9 / 634.8 | 7,222 | — |
| 2,200 | 2,200.0 | 1.7 | 11.6 | 33.8 | 244.0 | 100.00% | 268.6 / 713.0 | 7,716 | — |
| 2,300 | 2,300.0 | 1.9 | 14.4 | 197.2 | 1,043.4 | 100.00% | 284.4 / 696.9 | 8,071 | 3 runs, P99 range 39.6–368.8 ms, Max range 326.1–1,700.6 ms |
| 2,400 | 2,400.0 | 2.1 | 26.9 | 482.0 | 2,287.6 | 100.00% | 286.3 / 749.3 | 8,071 | 1 run |
| 2,500 | 2,499.8 | 3.3 | 600.4 | 1,228.1 | 3,346.4 | 99.99% | 302.5 / 744.7 | 8,308 | P99 missed target |
| 2,600 | 2,367.6 | 1,053.5 | 2,489.8 | 2,646.2 | 4,510.6 | 91.06% | 324.3 / 741.4 | 8,865 | P99 missed target |
| 2,700 | 2,222.7 | 2,467.7 | 2,583.5 | 2,687.6 | 3,627.4 | 82.32% | 334.4 / 776.5 | 9,238 | 2 runs, P99 and completion rate missed targets, Max range 3,547.4–3,707.3 ms |
Snapshot · 100k-cells
4 cores / 16 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 100 | 100.0 | 1.4 | 6.6 | 22.8 | 68.0 | 100.00% | 91.6 / 202.2 | 2,662 | — |
| 250 | 250.0 | 1.9 | 23.0 | 56.8 | 637.3 | 100.00% | 246.3 / 396.6 | 4,046 | — |
| 350 | 350.0 | 5.1 | 76.2 | 170.5 | 479.8 | 100.00% | 365.5 / 397.9 | 5,413 | — |
| 380 | 380.0 | 5.7 | 91.1 | 200.9 | 725.9 | 100.00% | 389.6 / 401.8 | 12,440 | 3 runs, Max range 615.2–856.9 ms |
| 390 | 390.0 | 5.8 | 106.0 | 275.3 | 1,339.3 | 100.00% | 391.1 / 399.6 | 13,639 | Out of memory after sampling |
| 400 | 394.7 | 6.2 | 108.1 | 210.3 | 675.8 | 98.67% | 377.4 / 398.3 | 14,226 | Out of memory, 1,385 failed submissions |
8 cores / 32 GiB
| Active documents | Throughput (edits/s) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Completion rate | CPU mean / peak (%) | Peak memory (MiB) | Notes |
|---|---|---|---|---|---|---|---|---|---|
| 100 | 100.0 | 1.3 | 2.8 | 21.7 | 36.7 | 100.00% | 91.8 / 200.1 | 3,392 | — |
| 250 | 250.0 | 1.2 | 5.8 | 23.0 | 284.2 | 100.00% | 242.0 / 471.1 | 5,013 | — |
| 500 | 500.0 | 1.5 | 16.2 | 33.9 | 121.6 | 100.00% | 482.6 / 716.4 | 7,721 | — |
| 600 | 600.0 | 2.0 | 26.7 | 65.6 | 302.1 | 100.00% | 575.9 / 772.1 | 8,943 | — |
| 700 | 700.0 | 2.4 | 29.9 | 64.5 | 288.1 | 100.00% | 641.4 / 782.5 | 25,692 | — |
| 750 | 750.0 | 2.6 | 33.5 | 72.3 | 280.7 | 100.00% | 649.7 / 782.1 | 29,276 | 3 runs, Max range 266.2–295.1 ms |
| 775 | 725.2 | 2.7 | 32.7 | 71.0 | 314.2 | 93.57% | 607.2 / 789.1 | 29,360 | Out of memory, 4,474 failed submissions |
We continue to optimize the collaboration service to reduce CPU and memory use and improve editing performance for large documents.
How is this guide?