[pve-devel] IFF_VNET_HDR ioctl on tap device

[Alexandre DERUMIER]

Hi,
I was reading a mail in kvm mailing about a debian-libvirt bug with network performance with virtio devices.

libvirt do an ioctl IFF_VNET_HDR on tap devices if qemu has support for IFF_VNET_HDR and kernel implement  TUNGETIFF ioctl().

Maybe can we implement this ?





libvirt code
------------

/**
 * virNetDevProbeVnetHdr:
 * @tapfd: a tun/tap file descriptor
 *
 * Check whether it is safe to enable the IFF_VNET_HDR flag on the
 * tap interface.
 *
 * Setting IFF_VNET_HDR enables QEMU's virtio_net driver to allow
 * guests to pass larger (GSO) packets, with partial checksums, to
 * the host. This greatly increases the achievable throughput.
 *
 * It is only useful to enable this when we're setting up a virtio
 * interface. And it is only *safe* to enable it when we know for
 * sure that a) qemu has support for IFF_VNET_HDR and b) the running
 * kernel implements the TUNGETIFF ioctl(), which qemu needs to query
 * the supplied tapfd.
 *
 * Returns 1 if VnetHdr is supported, 0 if not supported
 */
#ifdef IFF_VNET_HDR
static int 

virNetDevProbeVnetHdr(int tapfd)
{
# if defined(IFF_VNET_HDR) && defined(TUNGETFEATURES) && defined(TUNGETIFF)
    unsigned int features;
    struct ifreq dummy;

    if (ioctl(tapfd, TUNGETFEATURES, &features) != 0) {
        VIR_INFO("Not enabling IFF_VNET_HDR; "
                 "TUNGETFEATURES ioctl() not implemented");
        return 0;
    }

    if (!(features & IFF_VNET_HDR)) {
        VIR_INFO("Not enabling IFF_VNET_HDR; "
                 "TUNGETFEATURES ioctl() reports no IFF_VNET_HDR");
        return 0;
    }
   /* The kernel will always return -1 at this point.
     * If TUNGETIFF is not implemented then errno == EBADFD.
     */
    if (ioctl(tapfd, TUNGETIFF, &dummy) != -1 || errno != EBADFD) {
        VIR_INFO("Not enabling IFF_VNET_HDR; "
                 "TUNGETIFF ioctl() not implemented");
        return 0;
    }

    VIR_INFO("Enabling IFF_VNET_HDR");

    return 1;
# else
    (void) tapfd;
    VIR_INFO("Not enabling IFF_VNET_HDR; disabled at build time");
    return 0;
# endif
}
#endif






mail from mailing
-----------------



On 12.04.2012 11:42, Hans-Kristian Bakke wrote:
> Hi
> 
> For some reason I am not able to get good network performance using
> virtio/vhost-net on Debian KVM host (perhaps also valid for Ubuntu
> hosts then).

The issue has been identified, after Hans-Kristian gave me access
to his machine and I did alot of testing.  And as usual, the root
cause was very stupid... ;)

In last release of debian qemu-kvm package I changed the way how
debian package version string propagates to build procedure -- before
it was a patch grabbing values from debian/version, but in last
release I used --with-pkgversion configure flag.  This resulted in
the following change:

- QEMU emulator version 1.0 (qemu-kvm-1.0 Debian 1.0+dfsg-8), Copyright (c) 2003-2008 Fabrice Bellard
+ QEMU emulator version 1.0 (Debian qemu-kvm 1.0+dfsg-9), Copyright (c) 2003-2008 Fabrice Bellard

As it turns out, libvirt parses `qemu -version' output and looks for
" (qemu-kvm-" string in there, and if it is found, libvirt enables
some "extra" features.  Important for us was setting IFF_VNET_HDR
flag for a tap device, -- apparently this flag makes a HUGE difference
in networking speed, especially when using vhost_net.

Obviously this is a change unique to debian, and I never thought
about such an.. "interesting" effect it may give us.

This is a libvirt bug actually, since support of vnet_hdr can be
determined by other means, and since upstream qemu now has almost
everything from qemu-kvm, and it wants to be fast too.  But since
qemu[-kvm] has a long history of changing features, it is difficult
to blame libvirt that much...

Oh well.

Thanks!