<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Stanislav,<br>
<br>
I really think you should have a look where the problem is. This
is not Proxmox specific at all, so this list doesn't seem the
place to diagnose it.<br>
<br>
If DRBD guys think it's an upper layer problem, maybe they can
point you to the problematic layer, and then contact the mantainer
(probably someone in kernel development).<br>
<br>
I recall from ceph list that there were some barrier problems in
kernels < 2.6.33 . I don't know whether those are fixed in the
kernel from RHEL6 Proxmox uses...<br>
<br>
Cheers<br>
Eneko<br>
<br>
On 28/05/15 09:22, Stanislav German-Evtushenko wrote:<br>
</div>
<blockquote
cite="mid:CAPeRZrXOQ7Q=-i_YCCrtntUHyr8g8W7iv7odfH979=6eA5vxNA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>
<div>Hi Dietmar,<br>
<br>
</div>
I did it couple of times already and everytime I had
the same answer "upper layer problem". Well, as
we've done this long way up to this point I would
like to continue.<br>
</div>
<br>
I have just done the same test with mdadm and not
DRBD. And what I found that this problem was
reproducible on the software raid too, just as it was
claimed by Lars Ellenberg. It means that problem is
not only related to DRBD but to O_DIRECT mode
generally when we don't use host cache and a block
device reads data directly from userspace.<br>
<br>
</div>
The testcase is bellow.<br>
<br>
</div>
<div>1. Prepare<br>
<br>
</div>
<div>dd if=/dev/zero of=/tmp/mdadm1 bs=1M count=100<br>
dd if=/dev/zero of=/tmp/mdadm2 bs=1M count=100<br>
losetup /dev/loop1 /tmp/mdadm1<br>
losetup /dev/loop2 /tmp/mdadm2<br>
mdadm --create /dev/md0 --level=1 --raid-devices=2
/dev/loop{1,2}<br>
</div>
<br>
</div>
<div>2. Write data with O_DIRECT<br>
<br>
</div>
<div>./a.out /dev/md0<br>
<br>
</div>
3. Check consistency with vbindiff<br>
<br>
</div>
vbindiff /tmp/mdadm{1,2} #press enter multiple times to
skip metadata<br>
<br>
</div>
And here we find that data on "physical devices" is different
and md raid did not catch this.<br>
<br>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, May 28,
2015 at 7:40 AM, Dietmar Maurer <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:dietmar@proxmox.com"
target="_blank">dietmar@proxmox.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">>
What this means?<br>
<br>
I still think you should discuss
that on the DRBD list.<br>
<br>
</blockquote>
</div>
<br>
</div>
<div class="gmail_extra">Best regards,<br>
</div>
<div class="gmail_extra">Stanislav
German-Evtushenko<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
pve-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:pve-devel@pve.proxmox.com">pve-devel@pve.proxmox.com</a>
<a class="moz-txt-link-freetext" href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel">http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943575997
943493611
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
<a class="moz-txt-link-abbreviated" href="http://www.binovo.es">www.binovo.es</a></pre>
</body>
</html>