[pve-devel] [RFC common 2/2] fix #4501: next unused port: work around issue with too short expiretime

[Fiona Ebner]

Am 15.11.23 um 09:51 schrieb Fabian Grünbichler:
> On November 14, 2023 3:13 pm, Fiona Ebner wrote:
>> Am 14.11.23 um 15:02 schrieb Fiona Ebner:
>>> For QEMU migration via TCP, there's a bit of time between port
>>> reservation and usage, because currently, the port needs to be
>>> reserved before doing a fork, where the systemd scope needs to be set
>>> up and swtpm might need to be started before the QEMU binary can be
>>> invoked and actually use the port.
>>>
>>> To improve the situation, get the latest port recorded in the
>>> reservation file and start trying from the next port, wrapping around
>>> when hitting the end. Drastically reduces the chances to run into a
>>> conflict, because after a given port reservation, all other ports are
>>> tried first before returning to that port.
>>
>> Sorry, this is not true. It can be that in the meantime, a port for a
>> different range is reserved and that will remove the reservation for the
>> port in the migration range if expired. So we'd need to change the code
>> to remove only reservations from the current range to not lose track of
>> the latest previously used migration port.
> 
> the whole thing would also still be racy anyway across processes, right?
> not sure it's worth the additional effort compared to the other patches
> then.. if those are not enough (i.e., we still get real-world reports)
> then the "increase expiry further + explicit release" part could still
> be implemented as follow-up..
> 

No, it's not racy. The reserved ports are recorded in a file while
taking a lock, so each process will see what the others have last used.

My question is if the explicit release isn't much more effort than the
round-robin-style approach here, because it puts the burden on the
callers and you need a good way to actually check if the port is now
used successfully (without creating new races!) and a new helper for
removing the reservation. (That said, with round-robin we would need to
remember which range a port was for if we ever want to support
overlapping ranges).

As long as you have competition for early ports, you just need one
instance where the time between reservation and usage is longer than the
expiretime and you're very likely to hit the issue (except another
earlier port is free again). With round-robin, you need such an instance
and have all(!) other ports reserved/used in the meantime.