[PATCH 0/3] ethdev: fix SFF-8472 calibration overflow

[Roman Khromenok <roma55592@yandex.ru>]

This follows up on the review of the SFF-8472 Rx power calibration fix,
where the automated review pointed out the remaining issues in
sff_8472_calibration().

Calibration formula 1 (temperature, voltage, Tx bias, Tx power) scales
the 16-bit fields in place by a slope of up to 255.996 and adds a signed
16-bit offset. A result out of the field range is undefined behavior
for the float to integer conversion; a negative sum wraps around.
UBSan reports float-cast-overflow on randomly generated externally
calibrated SFP dumps.

Patch 1 computes formula 1 in double precision and saturates the result
to the field range, as already done for the Rx power. It is intended
for stable.
Patch 2 rounds the calibrated values to the nearest integer instead
of truncating them: RX_PWR(1) = 0.7 is stored as 0.69999999 and
a raw reading of 1000 gave 699 instead of 700.
Patch 3 adds unit tests for saturation at both ends and for rounding.

Based on dpdk-next-net for-main.

Depends-on: series-39455 ("ethdev: fix SFF-8472 external Rx power calibration")

Roman Khromenok (3):
  ethdev: fix SFF-8472 calibration overflow
  ethdev: round SFF-8472 calibrated values
  test: check SFF-8472 calibration saturation and rounding

 app/test/test_ethdev_module_eeprom.c | 96 ++++++++++++++++++++++++++++
 lib/ethdev/sff_8472.c                | 55 +++++++++++-----
 2 files changed, 134 insertions(+), 17 deletions(-)

-- 
2.47.3