[pbs-devel] [PATCH 0/9] Improve live-restore speed and replace AsyncIndexReader
Stefan Reiter
s.reiter at proxmox.com
Wed Jun 2 16:38:24 CEST 2021
This series is the third attempt[0] at substantially improving live-restore
performance. This time, a fully async- and concurrency safe LRU cache is
implemented, and a new CachedChunkReader is used to provide lock-free reading
from a remote chunk source. The big performance improvements come from removing
the global Mutex synchronising all read_image_at requests in the QEMU library
and the shared LRU cache.
The idea of using 'mmap' to alleviate memory pressure was dropped in favor of
using a shared cache between drives. This provides about the same performance
improvement in typical live-restore scenarios, but with a fixed memory
requirement, independant of drives. Adding 'mmap' caching is doable, but the
Rust code really doesn't look very nice with it, and I think it's not necessary
in this version...
Anyway, I figured out how to better benchmark the changes too, and can now
reproduce the performance gains very reliably. Below is a (gnu)plot of running
40 live-restores of a Windows 10 VM with a single 32GB disk, first with the
currently shipping libproxmox-backup-qemu0 (A) and then with the patched one
from this series (B). Testing was done with QEMU 6.0.
seconds
240 +-----------------------------------------------------------------------+
| + + + + + + A + |
230 |-+ A A A +-|
| A A A AA A A A A |
220 |-+ A AA A A +-|
| A A A A A A AA A A |
210 |-+ A A A A A A A-|
| A |
200 |-+ A +-|
| |
190 |-+ +-|
| |
180 |-A +-|
| |
170 |-+ +-|
| B B B |
160 |-B B B B B B B +-|
| B B B B B |
150 |-+ BB B BB B B B B B B B B-|
| B + B + + B + B B + B + B B + B |
140 +-----------------------------------------------------------------------+
0 5 10 15 20 25 30 35 40
iteration
1.0.3-1 A
patched B
[0] see: https://lists.proxmox.com/pipermail/pbs-devel/2021-April/002932.html
proxmox-backup: Stefan Reiter (7):
tools/BroadcastFuture: add testcase for better understanding
tools: add AsyncLruCache as a wrapper around sync LruCache
backup: add CachedChunkReader utilizing AsyncLruCache
backup: add AsyncRead/Seek to CachedChunkReader
replace AsyncIndexReader with SeekableCachedChunkReader
backup: remove AsyncIndexReader
tools/lru_cache: make minimum capacity 1
src/api2/admin/datastore.rs | 4 +-
src/backup.rs | 4 +-
src/backup/async_index_reader.rs | 215 -------------------------
src/backup/cached_chunk_reader.rs | 199 +++++++++++++++++++++++
src/bin/proxmox_backup_client/mount.rs | 4 +-
src/tools.rs | 1 +
src/tools/async_lru_cache.rs | 135 ++++++++++++++++
src/tools/broadcast_future.rs | 11 ++
src/tools/lru_cache.rs | 1 +
9 files changed, 353 insertions(+), 221 deletions(-)
delete mode 100644 src/backup/async_index_reader.rs
create mode 100644 src/backup/cached_chunk_reader.rs
create mode 100644 src/tools/async_lru_cache.rs
proxmox-backup-qemu: Stefan Reiter (2):
add shared_cache module
access: use CachedChunkReader
src/lib.rs | 7 ++++++-
src/restore.rs | 30 ++++++++----------------------
src/shared_cache.rs | 36 ++++++++++++++++++++++++++++++++++++
3 files changed, 50 insertions(+), 23 deletions(-)
create mode 100644 src/shared_cache.rs
--
2.30.2
More information about the pbs-devel
mailing list