From: kernel test robot <lkp@intel.com>
To: kaiyang2@cs.cmu.edu
Cc: oe-kbuild-all@lists.linux.dev
Subject: Re: [RFC PATCH 2/4] calculate memory.low for the local node and track its usage
Date: Sun, 22 Sep 2024 07:18:44 +0800 [thread overview]
Message-ID: <202409220804.TAoLKEBm-lkp@intel.com> (raw)
In-Reply-To: <20240920221202.1734227-3-kaiyang2@cs.cmu.edu>
Hi,
[This is a private test report for your RFC patch.]
kernel test robot noticed the following build warnings:
[auto build test WARNING on akpm-mm/mm-everything]
[also build test WARNING on linus/master next-20240920]
[cannot apply to tip/sched/core v6.11]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/kaiyang2-cs-cmu-edu/Add-get_cgroup_local_usage-for-estimating-the-top-tier-memory-usage/20240921-061404
base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/r/20240920221202.1734227-3-kaiyang2%40cs.cmu.edu
patch subject: [RFC PATCH 2/4] calculate memory.low for the local node and track its usage
config: x86_64-rhel-8.3 (https://download.01.org/0day-ci/archive/20240922/202409220804.TAoLKEBm-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240922/202409220804.TAoLKEBm-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202409220804.TAoLKEBm-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> mm/page_counter.c:268: warning: Function parameter or struct member 'nr_pages_local' not described in 'page_counter_set_low'
>> mm/page_counter.c:443: warning: Function parameter or struct member 'is_local' not described in 'page_counter_calculate_protection'
vim +268 mm/page_counter.c
bf8d5d52ffe89a Roman Gushchin 2018-06-07 258
230671533d6463 Roman Gushchin 2018-06-07 259 /**
230671533d6463 Roman Gushchin 2018-06-07 260 * page_counter_set_low - set the amount of protected memory
230671533d6463 Roman Gushchin 2018-06-07 261 * @counter: counter
230671533d6463 Roman Gushchin 2018-06-07 262 * @nr_pages: value to set
230671533d6463 Roman Gushchin 2018-06-07 263 *
230671533d6463 Roman Gushchin 2018-06-07 264 * The caller must serialize invocations on the same counter.
230671533d6463 Roman Gushchin 2018-06-07 265 */
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 266 void page_counter_set_low(struct page_counter *counter, unsigned long nr_pages,
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 267 unsigned long nr_pages_local)
230671533d6463 Roman Gushchin 2018-06-07 @268 {
230671533d6463 Roman Gushchin 2018-06-07 269 struct page_counter *c;
230671533d6463 Roman Gushchin 2018-06-07 270
f86b810c2610b0 Chris Down 2020-04-01 271 WRITE_ONCE(counter->low, nr_pages);
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 272 WRITE_ONCE(counter->locallow, nr_pages_local);
230671533d6463 Roman Gushchin 2018-06-07 273
230671533d6463 Roman Gushchin 2018-06-07 274 for (c = counter; c; c = c->parent)
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 275 propagate_protected_usage(c, atomic_long_read(&c->usage),
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 276 get_cgroup_local_usage(counter->memcg, false));
230671533d6463 Roman Gushchin 2018-06-07 277 }
230671533d6463 Roman Gushchin 2018-06-07 278
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 279 /**
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 280 * page_counter_memparse - memparse() for page counter limits
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 281 * @buf: string to parse
650c5e565492f9 Johannes Weiner 2015-02-11 282 * @max: string meaning maximum possible value
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 283 * @nr_pages: returns the result in number of pages
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 284 *
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 285 * Returns -EINVAL, or 0 and @nr_pages on success. @nr_pages will be
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 286 * limited to %PAGE_COUNTER_MAX.
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 287 */
650c5e565492f9 Johannes Weiner 2015-02-11 288 int page_counter_memparse(const char *buf, const char *max,
650c5e565492f9 Johannes Weiner 2015-02-11 289 unsigned long *nr_pages)
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 290 {
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 291 char *end;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 292 u64 bytes;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 293
650c5e565492f9 Johannes Weiner 2015-02-11 294 if (!strcmp(buf, max)) {
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 295 *nr_pages = PAGE_COUNTER_MAX;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 296 return 0;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 297 }
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 298
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 299 bytes = memparse(buf, &end);
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 300 if (*end != '\0')
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 301 return -EINVAL;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 302
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 303 *nr_pages = min(bytes / PAGE_SIZE, (u64)PAGE_COUNTER_MAX);
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 304
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 305 return 0;
3e32cb2e0a12b6 Johannes Weiner 2014-12-10 306 }
a8585ac6862198 Maarten Lankhorst 2024-07-03 307
a8585ac6862198 Maarten Lankhorst 2024-07-03 308
941ce635234162 Roman Gushchin 2024-07-26 309 #ifdef CONFIG_MEMCG
a8585ac6862198 Maarten Lankhorst 2024-07-03 310 /*
a8585ac6862198 Maarten Lankhorst 2024-07-03 311 * This function calculates an individual page counter's effective
a8585ac6862198 Maarten Lankhorst 2024-07-03 312 * protection which is derived from its own memory.min/low, its
a8585ac6862198 Maarten Lankhorst 2024-07-03 313 * parent's and siblings' settings, as well as the actual memory
a8585ac6862198 Maarten Lankhorst 2024-07-03 314 * distribution in the tree.
a8585ac6862198 Maarten Lankhorst 2024-07-03 315 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 316 * The following rules apply to the effective protection values:
a8585ac6862198 Maarten Lankhorst 2024-07-03 317 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 318 * 1. At the first level of reclaim, effective protection is equal to
a8585ac6862198 Maarten Lankhorst 2024-07-03 319 * the declared protection in memory.min and memory.low.
a8585ac6862198 Maarten Lankhorst 2024-07-03 320 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 321 * 2. To enable safe delegation of the protection configuration, at
a8585ac6862198 Maarten Lankhorst 2024-07-03 322 * subsequent levels the effective protection is capped to the
a8585ac6862198 Maarten Lankhorst 2024-07-03 323 * parent's effective protection.
a8585ac6862198 Maarten Lankhorst 2024-07-03 324 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 325 * 3. To make complex and dynamic subtrees easier to configure, the
a8585ac6862198 Maarten Lankhorst 2024-07-03 326 * user is allowed to overcommit the declared protection at a given
a8585ac6862198 Maarten Lankhorst 2024-07-03 327 * level. If that is the case, the parent's effective protection is
a8585ac6862198 Maarten Lankhorst 2024-07-03 328 * distributed to the children in proportion to how much protection
a8585ac6862198 Maarten Lankhorst 2024-07-03 329 * they have declared and how much of it they are utilizing.
a8585ac6862198 Maarten Lankhorst 2024-07-03 330 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 331 * This makes distribution proportional, but also work-conserving:
a8585ac6862198 Maarten Lankhorst 2024-07-03 332 * if one counter claims much more protection than it uses memory,
a8585ac6862198 Maarten Lankhorst 2024-07-03 333 * the unused remainder is available to its siblings.
a8585ac6862198 Maarten Lankhorst 2024-07-03 334 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 335 * 4. Conversely, when the declared protection is undercommitted at a
a8585ac6862198 Maarten Lankhorst 2024-07-03 336 * given level, the distribution of the larger parental protection
a8585ac6862198 Maarten Lankhorst 2024-07-03 337 * budget is NOT proportional. A counter's protection from a sibling
a8585ac6862198 Maarten Lankhorst 2024-07-03 338 * is capped to its own memory.min/low setting.
a8585ac6862198 Maarten Lankhorst 2024-07-03 339 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 340 * 5. However, to allow protecting recursive subtrees from each other
a8585ac6862198 Maarten Lankhorst 2024-07-03 341 * without having to declare each individual counter's fixed share
a8585ac6862198 Maarten Lankhorst 2024-07-03 342 * of the ancestor's claim to protection, any unutilized -
a8585ac6862198 Maarten Lankhorst 2024-07-03 343 * "floating" - protection from up the tree is distributed in
a8585ac6862198 Maarten Lankhorst 2024-07-03 344 * proportion to each counter's *usage*. This makes the protection
a8585ac6862198 Maarten Lankhorst 2024-07-03 345 * neutral wrt sibling cgroups and lets them compete freely over
a8585ac6862198 Maarten Lankhorst 2024-07-03 346 * the shared parental protection budget, but it protects the
a8585ac6862198 Maarten Lankhorst 2024-07-03 347 * subtree as a whole from neighboring subtrees.
a8585ac6862198 Maarten Lankhorst 2024-07-03 348 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 349 * Note that 4. and 5. are not in conflict: 4. is about protecting
a8585ac6862198 Maarten Lankhorst 2024-07-03 350 * against immediate siblings whereas 5. is about protecting against
a8585ac6862198 Maarten Lankhorst 2024-07-03 351 * neighboring subtrees.
a8585ac6862198 Maarten Lankhorst 2024-07-03 352 */
a8585ac6862198 Maarten Lankhorst 2024-07-03 353 static unsigned long effective_protection(unsigned long usage,
a8585ac6862198 Maarten Lankhorst 2024-07-03 354 unsigned long parent_usage,
a8585ac6862198 Maarten Lankhorst 2024-07-03 355 unsigned long setting,
a8585ac6862198 Maarten Lankhorst 2024-07-03 356 unsigned long parent_effective,
a8585ac6862198 Maarten Lankhorst 2024-07-03 357 unsigned long siblings_protected,
a8585ac6862198 Maarten Lankhorst 2024-07-03 358 bool recursive_protection)
a8585ac6862198 Maarten Lankhorst 2024-07-03 359 {
a8585ac6862198 Maarten Lankhorst 2024-07-03 360 unsigned long protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 361 unsigned long ep;
a8585ac6862198 Maarten Lankhorst 2024-07-03 362
a8585ac6862198 Maarten Lankhorst 2024-07-03 363 protected = min(usage, setting);
a8585ac6862198 Maarten Lankhorst 2024-07-03 364 /*
a8585ac6862198 Maarten Lankhorst 2024-07-03 365 * If all cgroups at this level combined claim and use more
a8585ac6862198 Maarten Lankhorst 2024-07-03 366 * protection than what the parent affords them, distribute
a8585ac6862198 Maarten Lankhorst 2024-07-03 367 * shares in proportion to utilization.
a8585ac6862198 Maarten Lankhorst 2024-07-03 368 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 369 * We are using actual utilization rather than the statically
a8585ac6862198 Maarten Lankhorst 2024-07-03 370 * claimed protection in order to be work-conserving: claimed
a8585ac6862198 Maarten Lankhorst 2024-07-03 371 * but unused protection is available to siblings that would
a8585ac6862198 Maarten Lankhorst 2024-07-03 372 * otherwise get a smaller chunk than what they claimed.
a8585ac6862198 Maarten Lankhorst 2024-07-03 373 */
a8585ac6862198 Maarten Lankhorst 2024-07-03 374 if (siblings_protected > parent_effective)
a8585ac6862198 Maarten Lankhorst 2024-07-03 375 return protected * parent_effective / siblings_protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 376
a8585ac6862198 Maarten Lankhorst 2024-07-03 377 /*
a8585ac6862198 Maarten Lankhorst 2024-07-03 378 * Ok, utilized protection of all children is within what the
a8585ac6862198 Maarten Lankhorst 2024-07-03 379 * parent affords them, so we know whatever this child claims
a8585ac6862198 Maarten Lankhorst 2024-07-03 380 * and utilizes is effectively protected.
a8585ac6862198 Maarten Lankhorst 2024-07-03 381 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 382 * If there is unprotected usage beyond this value, reclaim
a8585ac6862198 Maarten Lankhorst 2024-07-03 383 * will apply pressure in proportion to that amount.
a8585ac6862198 Maarten Lankhorst 2024-07-03 384 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 385 * If there is unutilized protection, the cgroup will be fully
a8585ac6862198 Maarten Lankhorst 2024-07-03 386 * shielded from reclaim, but we do return a smaller value for
a8585ac6862198 Maarten Lankhorst 2024-07-03 387 * protection than what the group could enjoy in theory. This
a8585ac6862198 Maarten Lankhorst 2024-07-03 388 * is okay. With the overcommit distribution above, effective
a8585ac6862198 Maarten Lankhorst 2024-07-03 389 * protection is always dependent on how memory is actually
a8585ac6862198 Maarten Lankhorst 2024-07-03 390 * consumed among the siblings anyway.
a8585ac6862198 Maarten Lankhorst 2024-07-03 391 */
a8585ac6862198 Maarten Lankhorst 2024-07-03 392 ep = protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 393
a8585ac6862198 Maarten Lankhorst 2024-07-03 394 /*
a8585ac6862198 Maarten Lankhorst 2024-07-03 395 * If the children aren't claiming (all of) the protection
a8585ac6862198 Maarten Lankhorst 2024-07-03 396 * afforded to them by the parent, distribute the remainder in
a8585ac6862198 Maarten Lankhorst 2024-07-03 397 * proportion to the (unprotected) memory of each cgroup. That
a8585ac6862198 Maarten Lankhorst 2024-07-03 398 * way, cgroups that aren't explicitly prioritized wrt each
a8585ac6862198 Maarten Lankhorst 2024-07-03 399 * other compete freely over the allowance, but they are
a8585ac6862198 Maarten Lankhorst 2024-07-03 400 * collectively protected from neighboring trees.
a8585ac6862198 Maarten Lankhorst 2024-07-03 401 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 402 * We're using unprotected memory for the weight so that if
a8585ac6862198 Maarten Lankhorst 2024-07-03 403 * some cgroups DO claim explicit protection, we don't protect
a8585ac6862198 Maarten Lankhorst 2024-07-03 404 * the same bytes twice.
a8585ac6862198 Maarten Lankhorst 2024-07-03 405 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 406 * Check both usage and parent_usage against the respective
a8585ac6862198 Maarten Lankhorst 2024-07-03 407 * protected values. One should imply the other, but they
a8585ac6862198 Maarten Lankhorst 2024-07-03 408 * aren't read atomically - make sure the division is sane.
a8585ac6862198 Maarten Lankhorst 2024-07-03 409 */
a8585ac6862198 Maarten Lankhorst 2024-07-03 410 if (!recursive_protection)
a8585ac6862198 Maarten Lankhorst 2024-07-03 411 return ep;
a8585ac6862198 Maarten Lankhorst 2024-07-03 412
a8585ac6862198 Maarten Lankhorst 2024-07-03 413 if (parent_effective > siblings_protected &&
a8585ac6862198 Maarten Lankhorst 2024-07-03 414 parent_usage > siblings_protected &&
a8585ac6862198 Maarten Lankhorst 2024-07-03 415 usage > protected) {
a8585ac6862198 Maarten Lankhorst 2024-07-03 416 unsigned long unclaimed;
a8585ac6862198 Maarten Lankhorst 2024-07-03 417
a8585ac6862198 Maarten Lankhorst 2024-07-03 418 unclaimed = parent_effective - siblings_protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 419 unclaimed *= usage - protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 420 unclaimed /= parent_usage - siblings_protected;
a8585ac6862198 Maarten Lankhorst 2024-07-03 421
a8585ac6862198 Maarten Lankhorst 2024-07-03 422 ep += unclaimed;
a8585ac6862198 Maarten Lankhorst 2024-07-03 423 }
a8585ac6862198 Maarten Lankhorst 2024-07-03 424
a8585ac6862198 Maarten Lankhorst 2024-07-03 425 return ep;
a8585ac6862198 Maarten Lankhorst 2024-07-03 426 }
a8585ac6862198 Maarten Lankhorst 2024-07-03 427
a8585ac6862198 Maarten Lankhorst 2024-07-03 428
a8585ac6862198 Maarten Lankhorst 2024-07-03 429 /**
a8585ac6862198 Maarten Lankhorst 2024-07-03 430 * page_counter_calculate_protection - check if memory consumption is in the normal range
a8585ac6862198 Maarten Lankhorst 2024-07-03 431 * @root: the top ancestor of the sub-tree being checked
a8585ac6862198 Maarten Lankhorst 2024-07-03 432 * @counter: the page_counter the counter to update
a8585ac6862198 Maarten Lankhorst 2024-07-03 433 * @recursive_protection: Whether to use memory_recursiveprot behavior.
a8585ac6862198 Maarten Lankhorst 2024-07-03 434 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 435 * Calculates elow/emin thresholds for given page_counter.
a8585ac6862198 Maarten Lankhorst 2024-07-03 436 *
a8585ac6862198 Maarten Lankhorst 2024-07-03 437 * WARNING: This function is not stateless! It can only be used as part
a8585ac6862198 Maarten Lankhorst 2024-07-03 438 * of a top-down tree iteration, not for isolated queries.
a8585ac6862198 Maarten Lankhorst 2024-07-03 439 */
a8585ac6862198 Maarten Lankhorst 2024-07-03 440 void page_counter_calculate_protection(struct page_counter *root,
a8585ac6862198 Maarten Lankhorst 2024-07-03 441 struct page_counter *counter,
6f4c005a5f8b8f Kaiyang Zhao 2024-09-20 442 bool recursive_protection, int is_local)
a8585ac6862198 Maarten Lankhorst 2024-07-03 @443 {
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
next prev parent reply other threads:[~2024-09-21 23:19 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-09-20 22:11 [RFC PATCH 0/4] memory tiering fairness by per-cgroup control of promotion and demotion kaiyang2
2024-09-20 22:11 ` [RFC PATCH 1/4] Add get_cgroup_local_usage for estimating the top-tier memory usage kaiyang2
2024-09-20 22:11 ` [RFC PATCH 2/4] calculate memory.low for the local node and track its usage kaiyang2
2024-09-21 23:18 ` kernel test robot [this message]
2024-09-22 8:39 ` kernel test robot
2024-10-15 22:05 ` Gregory Price
2024-09-20 22:11 ` [RFC PATCH 3/4] use memory.low local node protection for local node reclaim kaiyang2
2024-09-22 0:51 ` kernel test robot
2024-09-22 16:31 ` kernel test robot
2024-10-15 21:52 ` Gregory Price
2024-09-20 22:11 ` [RFC PATCH 4/4] reduce NUMA balancing scan size of cgroups over their local memory.low kaiyang2
2024-10-11 20:51 ` [RFC PATCH 0/4] memory tiering fairness by per-cgroup control of promotion and demotion Kaiyang Zhao
2024-11-08 19:01 ` kaiyang2
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=202409220804.TAoLKEBm-lkp@intel.com \
--to=lkp@intel.com \
--cc=kaiyang2@cs.cmu.edu \
--cc=oe-kbuild-all@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.