[pmg-devel] [PATCH v2 pmg-log-tracker] rewrite in rust

[Mira Limbeck]

On 11/8/19 11:20 AM, Thomas Lamprecht wrote:
> On 11/8/19 11:17 AM, Mira Limbeck wrote:
>> Memory usage was way too high (1.8G for the testcase, while the C log tracker used 150M).
> RSS? or what memory property did you measure?

Yes, RSS. The Rc<> had cycles and were only cleaned up at the end, so 
for ~1.8G syslogs it used almost as much.

With Weak<> they now get cleaned up correctly and the page faults are 
reduced by ~10x, which explains the reduced system time.

>
>> With the new changes it is now at ~190M and requires less time. Especially system time is now lower than the C version.
> great!
>
>> Will send a v3 soon.
>>
>> On 11/6/19 8:13 AM, Dietmar Maurer wrote:
>>> memory usage would also be interesting ...
>>>
>>> Also, why is system time higher in rust??
>>>
>>>> On 5 November 2019 19:04 Thomas Lamprecht <t.lamprecht at proxmox.com> wrote:
>>>>
>>>>    On 11/5/19 4:58 PM, Mira Limbeck wrote:
>>>>> Some simple benchmarks: (32 syslog files (syslog to syslog.31.gz))
>>>>>
>>>>> Rust: (median of 5 runs + 1 for cache)
>>>>> sudo pmg-log-tracker -s "2018-01-01 00:00:00" -vv > /dev/null  11.68s user 0.74s system 99% cpu 12.421 total
>>>>> sudo pmg-log-tracker -s "2018-01-01 00:00:00" -v > /dev/null  9.89s user 0.64s system 99% cpu 10.531 total
>>>>>
>>>>> C: (same as for the rust version)
>>>>> sudo pmg-log-tracker -s "2018-01-01 00:00:00" -vv > /dev/null  11.83s user 0.32s system 99% cpu 12.147 total
>>>>> sudo pmg-log-tracker -s "2018-01-01 00:00:00" -v > /dev/null  10.58s user 0.24s system 99% cpu 10.821 total
>>>>>
>>>> what about the benchmark environment? Where caches dropped in-between?
>>>>
>>>> Was a memory-backed tempfs for the backing "storage" tested also? To
>>>> have a comparison which is for sure not IO limited?