Re: [PATCH 11/19] sched/numa: Restrict migrating in parallel to the same node.

Mel Gorman
2018-06-05T09:54:03Z

On Mon, Jun 04, 2018 at 03:30:20PM +0530, Srikar Dronamraju wrote:
> Since task migration under numa balancing can happen in parallel, more
> than one task might choose to move to the same node at the same time.
> This can cause load imbalances at the node level.
>
> The problem is more likely if there are more cores per node or more
> nodes in system.
>
> Use a per-node variable to indicate if task migration
> to the node under numa balance is currently active.
> This per-node variable will not track swapping of tasks.
>
> Testcase Time: Min Max Avg StdDev
> numa01.sh Real: 434.84 676.90 550.53 106.24
> numa01.sh Sys: 125.98 217.34 179.41 30.35
> numa01.sh User: 38318.48 53789.56 45864.17 6620.80
> numa02.sh Real: 60.06 61.27 60.59 0.45
> numa02.sh Sys: 14.25 17.86 16.09 1.28
> numa02.sh User: 5190.13 5225.67 5209.24 13.19
> numa03.sh Real: 748.21 960.25 823.15 73.51
> numa03.sh Sys: 96.68 122.10 110.42 11.29
> numa03.sh User: 58222.16 72595.27 63552.22 5048.87
> numa04.sh Real: 433.08 630.55 499.30 68.15
> numa04.sh Sys: 245.22 386.75 306.09 63.32
> numa04.sh User: 35014.68 46151.72 38530.26 3924.65
> numa05.sh Real: 394.77 410.07 401.41 5.99
> numa05.sh Sys: 212.40 301.82 256.23 35.41
> numa05.sh User: 33224.86 34201.40 33665.61 313.40
>
> Testcase Time: Min Max Avg StdDev %Change
> numa01.sh Real: 674.61 997.71 785.01 115.95 -29.86%
> numa01.sh Sys: 180.87 318.88 270.13 51.32 -33.58%
> numa01.sh User: 54001.30 71936.50 60495.48 6237.55 -24.18%
> numa02.sh Real: 60.62 62.30 61.46 0.62 -1.415%
> numa02.sh Sys: 15.01 33.63 24.38 6.81 -34.00%
> numa02.sh User: 5234.20 5325.60 5276.23 38.85 -1.269%
> numa03.sh Real: 827.62 946.85 914.48 44.58 -9.987%
> numa03.sh Sys: 135.55 172.40 158.46 12.75 -30.31%
> numa03.sh User: 64839.42 73195.44 70805.96 3061.20 -10.24%
> numa04.sh Real: 481.01 608.76 521.14 47.28 -4.190%
> numa04.sh Sys: 329.59 373.15 353.20 14.20 -13.33%
> numa04.sh User: 37649.09 40722.94 38806.32 1072.32 -0.711%
> numa05.sh Real: 399.21 415.38 409.88 5.54 -2.066%
> numa05.sh Sys: 319.46 418.57 363.31 37.62 -29.47%
> numa05.sh User: 33727.77 34732.68 34127.41 447.11 -1.353%
>
> The commit does cause some performance regression but is needed from
> a fairness/correctness perspective.
>

While it may cause some performance regressions, it may be due to either
a) some workloads benefit from overloading a node if the tasks idle
frequently or b) the regression may be due to delayed convergence. I'm
not 100% convinced this needs to be done from a correctness point of
view based on just this microbenchmark

--
Mel Gorman
SUSE Labs