[pve-devel] Error between PVE and LVM

[Daniel Hunsaker]

It'll be a while. At best.

On Sat, Nov 29, 2014, 19:32 Cesar Peschiera <brain at click.com.py> wrote:

>  Hi Daniel
>
> Many thanks for your reply.
>
> Now that the problem was understood, as also the need, i would like to ask
> if anybody can add to the PVE GUI the option I most need? (and also any
> good administrator of storages).
>
> Best regards
> Cesar
>
> ----- Original Message -----
> *From:* Daniel Hunsaker <danhunsaker at gmail.com>
> *To:* Cesar Peschiera <brain at click.com.py> ; Alexandre DERUMIER
> <aderumier at odiso.com> ; pve-devel at pve.proxmox.com
>
> *Sent:* Saturday, November 29, 2014 3:38 PM
> *Subject:* Re: [pve-devel] Error between PVE and LVM
>
> OK, I think I see what you were saying there (having a bit of trouble with
> the grammar; my German is so rusty as to be useless, but learning it while
> I did gave me an appreciation for how hard it is to write/speak in a
> language I didn't grow up with, so I'm not complaining about it; I just
> mention it by way of apology for any remaining misunderstandings). You keep
> the VGs themselves small, to minimize administrative/maintenance task
> times. The situations in which you're looking to resize an LV are right
> after increasing the size of the VG the LV belongs to. This makes complete
> sense, and I feel a bit silly for not thinking of such a setup myself.
>
> On Sat, Nov 29, 2014, 11:20 Cesar Peschiera <brain at click.com.py> wrote:
>
>>  Hi Daniel
>>
>> Thanks for your clarification, it is good know it, and please let me to
>> do some comments:
>>
>> About of do to grow the max the VD of the VM: as a administration
>> strategy, if my hard disks don't have all  the space occupied with a
>> Physical Volume, will be very easy and online expand any thing of LVM which
>> is desired, as also can be util if you want to use such space free for
>> other things out of LVM.
>>
>> Moreover, as a administration strategy, if my DRBD resource has a space
>> very limited, the verification of DRBD storages will take less time to
>> complete (talking in hours of time), and when more later i want do grow the
>> DRBD resource for any  need (that always i can do it), entails to that
>> after will be need grow the Physical Volume and the Volume Group that is
>> within of the DRBD resource, then finally, i will gain time for that DRBD
>> completes the verification of his storages that are executed automatic and
>> periodically if is that it is limited the space of these resources.
>>
>> Unfortunately, the times of delay for that finish the tasks of
>> verifications of storages replicated or shared always was a topic of study
>> and strategy important for his administrators, due to that also the VMs are
>> using these same hard disks at the same time that a verification is in
>> progress, and obviously it can be more critical talking in speed terms when
>> a data base depend of these hard drives.
>>
>> And finally, I know that the all manufacturers of hard drives tell us
>> about of the percent of error in writes that have his hard drives, and is
>> for it (as also for other reasons), that any decent RAID controller have
>> the option of verify these units.
>>
>> So in conclusion, and talking in general mode, as are needed that the
>> systems to make these tasks:
>> 1) The backup of the VMs.
>> 2) The verification of the storages replicated.
>> 3) The verification of the HDDs.
>> 4) All these tasks must be do in online mode, but none at the same time
>> for avoid degradation of yield.
>>
>> It is for it that i need reduce the times of this verifications to the
>> max possible, and also so i need to have the data and storages as small as
>> possible, whether they are in a RAID Controller, as in LVM, DRBD, Gluster,
>> Ceph, hardware storage or any kind of storage that exists.
>>
>> Best regards
>>  Cesar
>>
>>
>>
>> ----- Original Message -----
>> *From:* Daniel Hunsaker <danhunsaker at gmail.com>
>>
>>  *To:* Cesar Peschiera <brain at click.com.py> ; Alexandre DERUMIER
>> <aderumier at odiso.com> ; pve-devel at pve.proxmox.com
>> *Sent:* Saturday, November 29, 2014 8:43 AM
>> *Subject:* Re: [pve-devel] Error between PVE and LVM
>>
>> It'll be some time before such options can be added to the underlying
>> Proxmox API.
>>
>> As to your comments about the resize not showing in the GUI even with qm
>> resize, that isn't quite true. Let's say you'd resize your LV normally
>> using `lvextend -L 100G`. I understand you'd prefer to use `100%` here, but
>> just bear with my example for the moment and pretend you'd use `100G`.
>> Changing that to `qm resize 100G` would not only resize the LV to 100 GB,
>> but also tell QEMU about the new size, which would in turn update the GUI.
>> That's because the qm command hooks into the same code the GUI uses to make
>> the changes it makes. So switching to `qm resize` will give you the VM/GUI
>> update at the cost of using % sizes (for now).
>>
>> I still don't understand what benefit you'd get from increasing a single
>> VM's LV to the full VG size. No other LV would then be able to grow within
>> that VG. The only setup I can imagine that would avoid that problem is one
>> where each LV has a dedicated VG, which is to say, each VG only has a
>> single LV. That setup seems pretty inefficient to me, not really taking
>> advantage of LVM's design principles. But then, maybe I'm just missing
>> something.
>>
>> On Sat, Nov 29, 2014, 00:37 Cesar Peschiera <brain at click.com.py> wrote:
>>
>>>  Hi Daniel
>>>
>>> As the moderns Windows systems have the options of grow or shrink the
>>> partitions of his HDDs online, add such features to the PVE GUI will be
>>> very pleasant.
>>>
>>> And if is possible also add the a option of "resize to the max allowed"
>>> (the space available in his Volume group) , will be fantastic.
>>>
>>> Best regards
>>>  Cesar
>>>
>>> ----- Original Message -----
>>> *From:* Daniel Hunsaker <danhunsaker at gmail.com>
>>> *To:* Alexandre DERUMIER <aderumier at odiso.com>
>>>
>>>  *Cc:* pve-devel at pve.proxmox.com ; Cesar Peschiera <brain at click.com.py>
>>> *Sent:* Friday, November 28, 2014 3:55 AM
>>> *Subject:* Re: [pve-devel] Error between PVE and LVM
>>>
>>> The only difference between lvresize and lvextend is that lvresize
>>> supports shrinking the volume as well as growing it. My comments aren't
>>> questions so much as observations on a patch series I'm planning to create
>>> and submit that will allow shrinking volumes as well as expanding them.
>>> Probably roll the max-free option in there as well if it hasn't been added
>>> already by then, though I wonder at the utility of such an operation, since
>>> you could only use it on one lv per vg.
>>>
>>> On Thu, Nov 27, 2014, 23:49 Alexandre DERUMIER <aderumier at odiso.com>
>>> wrote:
>>>
>>>> Sorry, but I'm a bit lost in all the discussion.
>>>>
>>>> are your questions (both daniel and cesar) about shrinking ?  or extend
>>>> ?
>>>>
>>>> (I don't have used lvm since a long time, don't known the difference
>>>> between lvresize and lvextend).
>>>>
>>>> @cesar, I don't understand why since the begin of this discus, you
>>>> resize manually the lvm disk.
>>>> (If the need is to do it command line, use qm resize, it'll extend the
>>>> lvm volume and tell to qemu the new size)
>>>>
>>>>
>>>>
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Daniel Hunsaker" <danhunsaker at gmail.com>
>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>, "Cesar Peschiera" <
>>>> brain at click.com.py>
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Envoyé: Vendredi 28 Novembre 2014 06:32:15
>>>> Objet: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>> Ah, good, it does support +size. In that case, simply swapping
>>>> `lvresize` into the code in place of `lvextend` (along with properly
>>>> handling -size to convert it to an absolute size for the command) would add
>>>> shrinking support to LVM storage. Just need to further explore the other
>>>> storage plugins' resize options to get shrinking for them as well. Makes
>>>> the patches a bit simpler.
>>>>
>>>> And yes, I always forget about `qm`. It's quite a bit simpler than
>>>> `pvesh`.
>>>>
>>>>
>>>> On Thu, Nov 27, 2014, 22:11 Alexandre DERUMIER < aderumier at odiso.com >
>>>> wrote:
>>>>
>>>>
>>>> >>But for this moment, i have two questions:
>>>> >>1) Do I have any simpler option to grow my LV(that is the HDD of the
>>>> VM) by
>>>> >>CLI?
>>>> >>2) If the answer is correct, what exactly should i execute?
>>>>
>>>> all the gui feature are available with cli, with "qm" command.
>>>>
>>>>
>>>>
>>>> #qm resize <vmid> <disk> <size>
>>>>
>>>> #man qm
>>>> qm resize <vmid> <disk> <size> [OPTIONS]
>>>>
>>>> Extend volume size.
>>>>
>>>> <vmid> integer (1 - N)
>>>>
>>>> The (unique) ID of the VM.
>>>>
>>>> <disk> (ide0 | ide1 | ide2 | ide3 | sata0 | sata1 | sata2 | sata3 |
>>>> sata4 | sata5 | scsi0 | scsi1 | scsi10 | scsi11 | scsi12 |
>>>> scsi13 | scsi2 | scsi3 | scsi4 | scsi5 | scsi6 | scsi7 | scsi8
>>>> | scsi9 | virtio0 | virtio1 | virtio10 | virtio11 | virtio12 |
>>>> virtio13 | virtio14 | virtio15 | virtio2 | virtio3 | virtio4 |
>>>> virtio5 | virtio6 | virtio7 | virtio8 | virtio9)
>>>>
>>>> The disk you want to resize.
>>>>
>>>> <size> \+?\d+(\.\d+)?[KMGT]?
>>>>
>>>> The new size. With the '+' sign the value is added to the
>>>> actual size of the volume and without it, the value is taken
>>>> as an absolute one. Shrinking disk size is not supported.
>>>>
>>>> -digest string
>>>>
>>>> Prevent changes if current configuration file has different
>>>> SHA1 digest. This can be used to prevent concurrent
>>>> modifications.
>>>>
>>>> -skiplock boolean
>>>>
>>>> Ignore locks - only root is allowed to use this option.
>>>>
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Cesar Peschiera" < brain at click.com.py >
>>>> À: "Daniel Hunsaker" < danhunsaker at gmail.com >, "Alexandre DERUMIER" <
>>>> aderumier at odiso.com >
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Envoyé: Jeudi 27 Novembre 2014 21:01:28
>>>> Objet: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>> Thanks Daniel, your words are encouraging for the future of PVE and for
>>>> me.
>>>>
>>>> But for this moment, i have two questions:
>>>> 1) Do I have any simpler option to grow my LV(that is the HDD of the
>>>> VM) by
>>>> CLI?
>>>> 2) If the answer is correct, what exactly should i execute?
>>>>
>>>> Best regards
>>>> Cesar
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: Daniel Hunsaker
>>>> To: Alexandre DERUMIER ; Cesar Peschiera
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Sent: Thursday, November 27, 2014 3:37 PM
>>>> Subject: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>>
>>>> If the GUI is resizing volumes, the API supports it, which means you
>>>> should
>>>> be able to use `pvesh` to do the operation in one command, instead of
>>>> using
>>>> the LVM commands and QEMU monitor directly. It does only support
>>>> specifying
>>>> the new size in bytes (which it seems to convert to MiB before actually
>>>> using), but it's still an option.
>>>>
>>>> As for the "max available" option, I'd personally find it more useful to
>>>> upgrade the API itself support the full range of `lvresize -L` values
>>>> (it
>>>> currently uses `lvextend`, which means volumes cannot be reduced in
>>>> size - a
>>>> fairly safe approach in case the filesystem inside the VM hasn't been
>>>> reduced in advance, but also a bit restrictive), or at least the largest
>>>> subset we could also support for other storage plugins. I'll see about
>>>> implementing that if nobody else gets to it first.
>>>>
>>>>
>>>> On Thu, Nov 27, 2014, 09:15 Alexandre DERUMIER < aderumier at odiso.com >
>>>> wrote:
>>>>
>>>> >>This process is correct when you use the GUI, but when you have space
>>>> >>limited in the hard disk, and you want to change some partitions by
>>>> CLI,
>>>> >>where finally will be working with the logical volumes, is when
>>>> starting
>>>> >>the
>>>> >>problem due that the VM not see the change applied.
>>>>
>>>> ah ok.
>>>>
>>>> This is normal, you need to tell to qemu what is the new size.
>>>> (This is what we are doing in the code : vm_mon_cmd($vmid,
>>>> "block_resize",
>>>> device => $deviceid, size => int($size)); )
>>>>
>>>> if you manually upgrade the disk size,
>>>> you need to use the monitor :
>>>>
>>>> #block_resize device size
>>>>
>>>> ex:
>>>>
>>>> #block_resize drive-virtio0 sizeinbytes
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Cesar Peschiera" < brain at click.com.py >
>>>> À: "Alexandre DERUMIER" < aderumier at odiso.com >
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Envoyé: Jeudi 27 Novembre 2014 17:00:37
>>>> Objet: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>> Hi Alexandre
>>>>
>>>> >This value correctly change after resize ?
>>>> If, before change, the logical volume had a smaller size.
>>>>
>>>> >We first extend the lvm disk, then we tell to qemu the new disk.
>>>> This process is correct when you use the GUI, but when you have space
>>>> limited in the hard disk, and you want to change some partitions by CLI,
>>>> where finally will be working with the logical volumes, is when
>>>> starting the
>>>> problem due that the VM not see the change applied.
>>>>
>>>> Please let me to do two suggestions:
>>>> - Maybe will be better than PVE GUI have a option that say: "resize to
>>>> max
>>>> available", or something.
>>>> I guess that this first option would be very good due to that the user
>>>> will
>>>> not need calculate the space available considering the space used in the
>>>> metadata of LVM.
>>>>
>>>> - Moreover, in previous versions of PVE, while that I could see the
>>>> reflected changes into the VM, in the PVE GUI, when i see the size of
>>>> his
>>>> hard disk, it shows his old size, then i had that remove the disk for
>>>> re add
>>>> it, only of this manner i could see his new size.
>>>>
>>>> >What kind of disk do you use in your guest ? virtio ? scsi ? ide ?
>>>> Virtio-block, moreover i have good references about virtio-scsi, do you
>>>> know
>>>> something about virtio-scsi for use it in windows systems?
>>>>
>>>> Many thanks again for your attention
>>>> Best regards
>>>> Cesar
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Alexandre DERUMIER" < aderumier at odiso.com >
>>>> To: "Cesar Peschiera" < brain at click.com.py >
>>>> Cc: < pve-devel at pve.proxmox.com >
>>>> Sent: Thursday, November 27, 2014 6:35 AM
>>>> Subject: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>>
>>>> So,
>>>>
>>>> >>shell# lvs
>>>> >>LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
>>>> >>vm-100-disk-1 drbdvg2 -wi------ 30.00g
>>>>
>>>> This value correctly change after resize ?
>>>>
>>>>
>>>>
>>>> the resize code is here:
>>>>
>>>> we first extend the lvm disk, then we tell to qemu the new disk.
>>>>
>>>> (What kind of disk do you use in your guest ? virtio ? scsi ? ide ?)
>>>>
>>>>
>>>>
>>>>
>>>> /usr/share/perl5/PVE/ QemuServer.pm
>>>>
>>>>
>>>> sub qemu_block_resize {
>>>> my ($vmid, $deviceid, $storecfg, $volid, $size) = @_;
>>>>
>>>> my $running = check_running($vmid);
>>>>
>>>> return if !PVE::Storage::volume_resize($ storecfg, $volid, $size,
>>>> $running);
>>>>
>>>> return if !$running;
>>>>
>>>> vm_mon_cmd($vmid, "block_resize", device => $deviceid, size =>
>>>> int($size));
>>>>
>>>> }
>>>>
>>>>
>>>> /usr/share/perl5/PVE/Storage/ LVMPlugin.pm
>>>>
>>>> sub volume_resize {
>>>> my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
>>>>
>>>> $size = ($size/1024/1024) . "M";
>>>>
>>>> my $path = $class->path($scfg, $volname);
>>>> my $cmd = ['/sbin/lvextend', '-L', $size, $path];
>>>> run_command($cmd, errmsg => "error resizing volume '$path'");
>>>>
>>>> return 1;
>>>> }
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Cesar Peschiera" < brain at click.com.py >
>>>> À: "Alexandre DERUMIER" < aderumier at odiso.com >
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Envoyé: Jeudi 27 Novembre 2014 09:11:29
>>>> Objet: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>> Hi Alexandre
>>>>
>>>> Thanks for your attention, here my answers and suggestions about of the
>>>> problem of your customer:
>>>>
>>>> >We have made no change since resize feature has been implemented.
>>>> >Can you describe a little bit more the problem on the guest side ?
>>>> >do you see disk size increase with parted/fdisk ?
>>>> I see the new size (vm-100-disk-1) of the logical volume by CLI, but it
>>>> isn't reflected into the VM.
>>>> In my case DRBD is in a upper layer to the LV, but the concept of LVM is
>>>> applicable for any Logical Volume in any kind of block device that
>>>> Linux can
>>>> recognise.
>>>> shell# lvs
>>>> LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
>>>> vm-100-disk-1 drbdvg2 -wi------ 30.00g
>>>> data pve -wi-ao--- 143.50g
>>>> root pve -wi-ao--- 20.00g
>>>> swap pve -wi-ao--- 20.00g
>>>>
>>>> >I have add a customer during a previous training session, which have
>>>> >problem with raw lvm disk in vms,
>>>> I don't have problems with the VMs (LVM or raw image file), only with
>>>> the
>>>> LVM resize by CLI.
>>>>
>>>> >because of proxmox host scanning all lvm disks on the host side. (Don't
>>>> >remember if it's have impact on resize).
>>>> It don't have impact on resize, and it is necessary for that LVM can
>>>> manage
>>>> the changes online included (VM and host), that it is my case.
>>>> Moreover, for my DRBD resources, i use this filter on the lvm.conf file
>>>> for
>>>> avoid scanning all lvm disks:
>>>>
>>>> filter = [ "r|/dev/sdb1|", "r|/dev/sdc1|", "r|/dev/sdd1|",
>>>> "r|/dev/sde1|",
>>>> "r|/dev/disk/|", "r|/dev/block/|", "a/.*/" ]
>>>> Where:
>>>> a=accept (include) , and
>>>> r=reject (exclude) the scans to speed startup.
>>>>
>>>> >We have need to add a filter in lvm.conf on the host, to exclude scan
>>>> of
>>>> >vms lvm disk.
>>>> Sure, i use the CLI
>>>>
>>>> An additional note of IBM:
>>>> In the best practices, LVM as block device is the mode more fast for
>>>> get the
>>>> better performance in reads and writes of disks.
>>>>
>>>> Official Web page of IBM in "Best practice: Use block devices for VM
>>>> storage":
>>>> http://www-01.ibm.com/support/ knowledgecenter/linuxonibm/
>>>> liaat/liaatbpblock.htm
>>>>
>>>> And finally, my question:
>>>> Can be corrected my problem?
>>>>
>>>> Best regards
>>>> Cesar
>>>>
>>>> ----- Original Message -----
>>>> From: "Alexandre DERUMIER" < aderumier at odiso.com >
>>>> To: "Cesar Peschiera" < brain at click.com.py >
>>>> Cc: < pve-devel at pve.proxmox.com >
>>>> Sent: Thursday, November 27, 2014 3:47 AM
>>>> Subject: Re: [pve-devel] Error between PVE and LVM
>>>>
>>>>
>>>> Hi,
>>>>
>>>> >>In previous versions of PVE, this task was possible do it with much
>>>> >>easily.
>>>>
>>>> We have made no change since resize feature has been implemented.
>>>> Can you describe a little bit more the problem on the guest side ?
>>>> do you see disk size increase with parted/fdisk ?
>>>>
>>>> Do you use raw lvm disk in your vms ? or partitions on top of lvm ?
>>>> I have the LVM partition, and the image of the virtual disk isn't on a
>>>> file
>>>> system (as raw, qcow2, or any other kind of file format), so my image
>>>> disk
>>>> is a LVM as block device.
>>>>
>>>> I have add a customer during a previous training session, which have
>>>> problem
>>>> with raw lvm disk in vms,
>>>>
>>>> because of proxmox host scanning all lvm disks on the host side. (Don't
>>>> remember if it's have impact on resize).
>>>>
>>>> We have need to add a filter in lvm.conf on the host, to exclude scan
>>>> of vms
>>>> lvm disk.
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Cesar Peschiera" < brain at click.com.py >
>>>> À: pve-devel at pve.proxmox.com
>>>> Envoyé: Jeudi 27 Novembre 2014 07:15:19
>>>> Objet: [pve-devel] Error between PVE and LVM
>>>>
>>>> Hi to the PVE team.
>>>>
>>>> I found a problem between PVE and LVM.
>>>>
>>>> Considering that if it is used LVM as block device for the virtual
>>>> disks of
>>>> the VMs, Linux give us a great comfort, but the problem is that if I in
>>>> "online mode" enlarge a Physical Volume and after enlarge a Logical
>>>> Volume
>>>> by CLI, in PVE, the VM can not see the new free hard disk space without
>>>> partition.
>>>>
>>>> In previous versions of PVE, this task was possible do it with much
>>>> easily.
>>>>
>>>> Moreover, i think that this feature is very util, due to that in the
>>>> actual
>>>> condition, it force me to power off the VM and start it again, so that
>>>> being
>>>> a server (talking about of the VM) that is in a production environment,
>>>> only
>>>> can I do it outside of working hours.
>>>>
>>>> So i would like to ask if the PVE team have interest in correcting this
>>>> problem.
>>>>
>>>> Best regards
>>>> Cesar
>>>>
>>>> ______________________________ _________________
>>>> pve-devel mailing list
>>>> pve-devel at pve.proxmox.com
>>>> http://pve.proxmox.com/cgi- bin/mailman/listinfo/pve-devel
>>>> ______________________________ _________________
>>>> pve-devel mailing list
>>>> pve-devel at pve.proxmox.com
>>>> http://pve.proxmox.com/cgi- bin/mailman/listinfo/pve-devel
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-devel/attachments/20141130/b60f79a3/attachment.htm>