[pve-devel] MooseFS storage plugin

[Benjamin]

Hi Thomas, thanks for the reply! (Sorry, re-sending as I forgot to reply
all)

On Tue, Apr 19, 2022, 4:13 PM Thomas Lamprecht <t.lamprecht at proxmox.com>
wrote:

For Storage plugins the basic rules to be considered included are:
* Plugin code itself must be AGPLv3, if it's self written that'd be already
  covered by submitting it for inclusion alongside the CLA.


No worries, I've already licensed it under AGPLv3 and all code samples used
are also AGPLv3 and acknowledged in the readme... happy to sign the CLA.


* The underlying storage tech must be open source, as while some provider
allow
  an open API or FOSS client, the backend needs to be FOSS too. This is no
"legal
  issue" per se, but otherwise we cannot test it freely nor recommend to our
  users, and at that point its better to have it as external plugin.


Yep, MooseFS is open source. You could almost consider it "open core"
depending on how important the Pro features are, but it really is usable as
a good filesystem without them.


* Must be actively developed


* Ideally brings some features/characteristics that the existing storage
  plugins don't have.

MooseFS seems to be split into the core GPLv2 licensed part and a
proprietary
one, fwict. How big is the difference there, can someone use the free part
to
build a enterprise-grade service without needing to get proprietary code?
(spending money is naturally fine, don't get me wrong).


There are two key differences;

1. MooseFS Pro has highly available metadata servers, making the entire
system HA rather than the master being a single point of failure (albeit
one that is easy to backup, restore and failover as needed)
2. Pro has erasure coding tiers instead of just straight replication.

I've got a Pro licence for installations up to 150TiB, and will be actively
testing my plugin against both Pro and CE clusters once it's working.


Also, its feature set reminds a bit of ceph, which we already have support
for
not only accessing but also managing. Is there an advantage of MooseFS over
ceph RBD and cephFS technology or is it more preference?

With that info it could be easier to decide if we'd include a MooseFS plugin
directly, for now I cannot answer that. But even if we wouldn't include it,
the external plugin mechanism makes it relatively easy to make PVE support
it
nonetheless.


I would be intending to maintain it as an external plugin. I'd *love* to
get it made part of base Proxmox too, but it does duplicate some of the
existing plugins.

My main reason for liking MooseFS is its relative simplicity compared to
Ceph and Gluster. It's dead simple to setup, and extreme reliable with good
performance out of the box sans tuning.

It's also really, really hard to lose data with MooseFS from what I've seen.


Yes, the plugin ABI isn't that complex, and we often avoid more complex or
unordinary perl language features; albeit I may be biased with that :)


I've managed to get my plugin half working despite having never touched
Perl until this morning. I'll continue to poke at it and report back here
with more logs and debugging if that's okay.


I guess that got now already fixed by your commit that actually imports
File::Path[0], which provides mkpath() and auto-exports it.

https://github.com/Zorlin/pve-moosefs/commit/a1621cfba41c88414e1bdc8fd857fa341d62bbf2


Yep, that sorted it. I've still got some showstopper bugs, I may need to
follow a few Perl tutorials and get up to speed to understand things a bit
better.


cheers,
Thomas


Thanks for the warm welcome!
- Benjamin
PS:

> can someone use the free part to
build a enterprise-grade service without needing to get proprietary code?

In my opinion yes, up to a point. The master service being a SPOF is an
issue when you're wanting an enterprise grade service. Nevertheless in the
event of a failure you can have Corosync/Pacemaker auto promote a server
that was acting as a "meta logger" and have it take over. It requires a bit
of scripting but can be done reliably without too much work.

As a bonus, having a MooseFS plugin pretty much means you have a LizardFS
plugin too, they're fairly compatible/similar.