[pbs-devel] applied-series: [PATCH v2 0/5] Fix PBS blockdriver for non-VM settings

[Thomas Lamprecht]

On 22.07.20 15:56, Stefan Reiter wrote:
> When using the PBS blockdriver with qemu-nbd (for example), it can happen that
> enough read requests are issued to saturate the tokio thread pool. Not an issue
> in general, but as me and Wolfgang painstakenly discovered a while back, it does
> break block_on, which is used in BufferedFixedReader. This means that reading
> larger amounts of data would hang the QEMU process [0].
> 
> Fix this by replacing the BufferedFixedReader with an AsyncIndexReader,
> implementing AsyncSeek for it in the process. This makes the entire API async,
> requiring no block_on anymore.
> 
> Incidentally, this also gave me my best benchmark results yet, coming in at
> above 1.6 Gb/s read speed via NBD on my local machine.
> 
> Additionally I discovered a seperate bug (fixed by patch 5), wherein read
> requests that we're not aligned to the chunk size would return bogus data. This
> too only seems to happen in non-VM connections (e.g. nbd, etc...).
> 
> v2:
> * Remove BufferedFixedReader entirely, use AsyncIndexReader instead
> * Implement AsyncSeek for AsyncIndexReader
> * Fix the second bug in Rust instead of QEMU C
> 
> 
> [0] ...and since the NBD kernel driver appears to be horribly broken, this often
> also crashes most of the system, but that's a different story. If you ever get
> in this situation, 'nbd-client -d /dev/nbdX' works (sometimes) to force
> disconnect the device ('qemu-nbd -d' intelligently issues a read before
> disconnecting, thus hanging before getting anything done...)
> 
> 
> backup: Stefan Reiter (3):
>   add and implement chunk_from_offset for IndexFile
>   implement AsyncSeek for AsyncIndexReader
>   remove BufferedFixedReader interface
> 
>  src/backup/async_index_reader.rs | 116 ++++++++++++++++++----
>  src/backup/dynamic_index.rs      |  18 ++++
>  src/backup/fixed_index.rs        | 165 +++----------------------------
>  src/backup/index.rs              |   4 +
>  4 files changed, 129 insertions(+), 174 deletions(-)
> 
> backup-qemu: Stefan Reiter (2):
>   use AsyncIndexReader for read_image_at
>   read_image_at: iterate until buffer is filled
> 
>  current-api.h  |  4 ++--
>  src/lib.rs     |  4 ++--
>  src/restore.rs | 35 +++++++++++++++++++++++------------
>  3 files changed, 27 insertions(+), 16 deletions(-)
> 

applied series, thanks!

using the noload mount option is really the key here, for ext4 at least ^^
# mount -o noload /dev/nbd0p1 /mnt/foo