[pbs-devel] [PATCH proxmox-backup] backup/verify: improve speed by sorting chunks by inode

[Fabian Grünbichler]

On April 13, 2021 4:35 pm, Dominik Csapak wrote:
> before reading the chunks from disk in the order of the index file,
> stat them first and sort them by inode number.
> 
> this can have a very positive impact on read speed on spinning disks,
> even with the additional stat'ing of the chunks.
> 
> memory footprint should be tolerable, for 1_000_000 chunks
> we need about ~16MiB of memory (Vec of 64bit position + 64bit inode)
> (assuming 4MiB Chunks, such an index would reference 4TiB of data)
> 
> two small benchmarks (single spinner, ext4) here showed an improvement from
> ~430 seconds to ~330 seconds for a 32GiB fixed index
> and from
> ~160 seconds to ~120 seconds for a 10GiB dynamic index
> 
> Signed-off-by: Dominik Csapak <d.csapak at proxmox.com>
> ---
> it would be great if other people could also benchmark this patch on
> different setups a little (in addition to me), to verify or disprove my results

zfs with single spinner + fast special device, with a (not counted ;)) 
warmup run and everything fitting into cache:

Benchmark #1: stock
  Time (mean ± σ):     21.407 s ±  0.819 s    [User: 20.1 ms, System: 15.2 ms]
  Range (min … max):   21.070 s … 23.078 s    6 runs

Benchmark #2: patched
  Time (mean ± σ):     47.119 s ±  0.018 s    [User: 29.5 ms, System: 15.1 ms]
  Range (min … max):   47.107 s … 47.154 s    6 runs

Summary
  'stock' ran
    2.20 ± 0.08 times faster than 'patched'

same setup, but ARC reduced so that verified data > ARC and we start 
bottle-necking on the spinner:

Benchmark #1: stock
  Time (mean ± σ):     367.821 s ±  0.801 s    [User: 195.9 ms, System: 80.0 ms]
  Range (min … max):   366.840 s … 368.802 s    4 runs

Benchmark #2: patched
  Time (mean ± σ):     406.391 s ±  1.304 s    [User: 188.3 ms, System: 100.8 ms]
  Range (min … max):   404.891 s … 407.919 s    4 runs

Summary
  'stock' ran
    1.10 ± 0.00 times faster than 'patched'

both benchmarks for verifying a datastore with ~12G of on-disk chunk 
data.