[pve-devel] [PATCH v3 many 00/66] fix #4156: introduce new notification system

[Lukas Wagner]

Hi again,

On 7/18/23 14:34, Dominik Csapak wrote:
> * i found one bug, but not quite sure yet where it comes from exactly,
>    putting in emojis into a field (e.g. a comment or author) it's accepted,
>    but editing a different entry fails with:
> 
> --->8---
> could not serialize configuration: writing 'notifications.cfg' failed: detected unexpected control character in section 'testgroup' key 'comment' (500)
> ---8<---
> 
> not sure where the utf-8 info gets lost. (or we could limit all fields to ascii?)
> such a notification target still works AFAICT (but if set as e.g. the author it's
> probably the wrong value)
> 
> (i used 😀 as a test)

So I investigated a bit and found a minimal reproducer. Turns out it's an encoding issue
in the FFI interface (perl->rust).

Let's assume that we have the following exported function in the pve-rs bindings:

   #[export]
   fn test_emoji(name: &str) {
       dbg!(&name);
   }



   use PVE::RS::Notify;
  
   my $str = "😊";
   PVE::RS::Notify::test_emoji($str);


   root at pve:~# perl test.pl
   [src/notify.rs:562] &name = "ð\u{9f}\u{98}\u{8a}"

To me it looks a bit like a UTF-16/UTF-8 mixup:

ð = 0x00F0 in UTF16
😊 = 0xF0 0x9F 0x98 0x8A in UTF-8

The issue can be fixed by doing a `$str = encode('utf-8', $str);` before calling
`test_emoji`.

However, I think this should be probably handled automagically by the perlmod bindings, if
at all possible?
@Wolfgang, what are your thoughts about this? Maximiliano said he is going to take a look
at the perlmod code, but if you have any idea about where to fix this issue, then
this would probably be helpful to him.

-- 
- Lukas