[pbs-devel] [PATCH vma-to-pbs v3 2/6] add support for bulk import of a dump directory

[Fabian Grünbichler]

> Filip Schauer <f.schauer at proxmox.com> hat am 30.10.2024 14:59 CET geschrieben:
> 
>  
> On 29/10/2024 11:41, Fabian Grünbichler wrote:
> >> +        let re = Regex::new(
> >> +            r"vzdump-qemu-(\d+)-(\d{4}_\d{2}_\d{2}-\d{2}_\d{2}_\d{2}).vma(|.zst|.lzo|.gz)$",
> >> +        )?;
> >> +
> >> +        let mut vmas = Vec::new();
> >> +
> >> +        for entry in read_dir(dump_dir_path)? {
> >> +            let entry = entry?;
> >> +            let path = entry.path();
> >> +
> >> +            if !path.is_file() {
> >> +                continue;
> >> +            }
> >> +
> >> +            if let Some(file_name) = path.file_name().and_then(|n|  n.to_str()) {
> > if we find a file that is not valid UTF-8, should we warn about it?
> 
> 
> Any file name that contains invalid UTF-8 will not get matched by the
> regex anyway and v4 of this patch adds a debug message logging any file
> that does not match the regex.
> 
> 
> On 29/10/2024 11:41, Fabian Grünbichler wrote:
> >> +            let process = cmd.arg(vma_file_path).stdout(Stdio::piped()).spawn()?;
> >> +            let stdout = process.stdout.expect("Failed to capture stdout");
> >> +            Box::new(BufReader::new(stdout))
> > did you test whether with_capacity and a higher buffer size than 8K improves performance?

just realized that there is a pipe before it anyhow which likely has a 64k buffer..

> >
> >> +        }
> >> +        None => match &backup_args.vma_file_path {
> >> +            Some(vma_file_path) => matchFile::open(vma_file_path)  {
> >> +                Err(why) => return Err(anyhow!("Couldn't open file: {}", why)),
> >> +                Ok(file) => Box::new(BufReader::new(file)),
> > same here
> 
> 
> Tested buffer sizes from 1024 up to 1048576 and it made no measurable
> performance difference.