[pbs-devel] [PATCH proxmox-backup] backup/verify: improve speed by sorting chunks by inode

[Fabian Grünbichler]

On April 14, 2021 6:44 pm, Dietmar Maurer wrote:
>> zfs with single spinner + fast special device, with a (not counted ;)) 
>> warmup run and everything fitting into cache:
>> 
>> Benchmark #1: stock
>>   Time (mean ± σ):     21.407 s ±  0.819 s    [User: 20.1 ms, System: 15.2 ms]
>>   Range (min … max):   21.070 s … 23.078 s    6 runs
>> 
>> Benchmark #2: patched
>>   Time (mean ± σ):     47.119 s ±  0.018 s    [User: 29.5 ms, System: 15.1 ms]
>>   Range (min … max):   47.107 s … 47.154 s    6 runs
>> 
>> Summary
>>   'stock' ran
>>     2.20 ± 0.08 times faster than 'patched'
> 
> I assume you have about 3000 chunks? On XFS, I can do 3000 stat() calls per second (after cache cleared).
> So why the hell are stat calls that slow on ZFS with special device? Or what causes that delay (stat or sort)?

13432 chunks. the run you quoted is with everything in ARC, so this is 
not even hitting disk (neither special nor regular vdev). I haven't 
profiled what exactly is the cause of the slowdown.