* Re: [PATCH v6 3/3] lib: zstd: Upgrade to latest upstream zstd version 1.4.6
[not found] <20201202203242.1187898-4-nickrterrell@gmail.com>
@ 2020-12-02 23:58 ` kernel test robot
2020-12-04 14:03 ` David Sterba
0 siblings, 1 reply; 2+ messages in thread
From: kernel test robot @ 2020-12-02 23:58 UTC (permalink / raw)
To: Nick Terrell, Herbert Xu
Cc: kbuild-all, linux-crypto, linux-btrfs, squashfs-devel,
linux-f2fs-devel, linux-kernel, Kernel Team, Nick Terrell,
Chris Mason, Petr Malat
[-- Attachment #1: Type: text/plain, Size: 12156 bytes --]
Hi Nick,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on cryptodev/master]
[also build test WARNING on kdave/for-next f2fs/dev-test kees/for-next/pstore linus/master v5.10-rc6]
[cannot apply to crypto/master squashfs/master next-20201201]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Nick-Terrell/Update-to-zstd-1-4-6/20201203-043418
base: https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
config: parisc-allyesconfig (attached as .config)
compiler: hppa-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/1ae5d159649a18815c67be65c370f7fd90e59e9f
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Nick-Terrell/Update-to-zstd-1-4-6/20201203-043418
git checkout 1ae5d159649a18815c67be65c370f7fd90e59e9f
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=parisc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast_extDict_generic':
>> lib/zstd/compress/zstd_double_fast.c:501:1: warning: the frame size of 3724 bytes is larger than 1280 bytes [-Wframe-larger-than=]
501 | }
| ^
lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast':
lib/zstd/compress/zstd_double_fast.c:336:1: warning: the frame size of 3792 bytes is larger than 1280 bytes [-Wframe-larger-than=]
336 | }
| ^
lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast_dictMatchState':
lib/zstd/compress/zstd_double_fast.c:356:1: warning: the frame size of 3808 bytes is larger than 1280 bytes [-Wframe-larger-than=]
356 | }
| ^
--
lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast_extDict_generic':
>> lib/zstd/compress/zstd_fast.c:476:1: warning: the frame size of 2736 bytes is larger than 1280 bytes [-Wframe-larger-than=]
476 | }
| ^
lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast':
lib/zstd/compress/zstd_fast.c:204:1: warning: the frame size of 1508 bytes is larger than 1280 bytes [-Wframe-larger-than=]
204 | }
| ^
lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast_dictMatchState':
lib/zstd/compress/zstd_fast.c:372:1: warning: the frame size of 1540 bytes is larger than 1280 bytes [-Wframe-larger-than=]
372 | }
| ^
vim +501 lib/zstd/compress/zstd_double_fast.c
357
358
359 static size_t ZSTD_compressBlock_doubleFast_extDict_generic(
360 ZSTD_matchState_t* ms, seqStore_t* seqStore, U32 rep[ZSTD_REP_NUM],
361 void const* src, size_t srcSize,
362 U32 const mls /* template */)
363 {
364 ZSTD_compressionParameters const* cParams = &ms->cParams;
365 U32* const hashLong = ms->hashTable;
366 U32 const hBitsL = cParams->hashLog;
367 U32* const hashSmall = ms->chainTable;
368 U32 const hBitsS = cParams->chainLog;
369 const BYTE* const istart = (const BYTE*)src;
370 const BYTE* ip = istart;
371 const BYTE* anchor = istart;
372 const BYTE* const iend = istart + srcSize;
373 const BYTE* const ilimit = iend - 8;
374 const BYTE* const base = ms->window.base;
375 const U32 endIndex = (U32)((size_t)(istart - base) + srcSize);
376 const U32 lowLimit = ZSTD_getLowestMatchIndex(ms, endIndex, cParams->windowLog);
377 const U32 dictStartIndex = lowLimit;
378 const U32 dictLimit = ms->window.dictLimit;
379 const U32 prefixStartIndex = (dictLimit > lowLimit) ? dictLimit : lowLimit;
380 const BYTE* const prefixStart = base + prefixStartIndex;
381 const BYTE* const dictBase = ms->window.dictBase;
382 const BYTE* const dictStart = dictBase + dictStartIndex;
383 const BYTE* const dictEnd = dictBase + prefixStartIndex;
384 U32 offset_1=rep[0], offset_2=rep[1];
385
386 DEBUGLOG(5, "ZSTD_compressBlock_doubleFast_extDict_generic (srcSize=%zu)", srcSize);
387
388 /* if extDict is invalidated due to maxDistance, switch to "regular" variant */
389 if (prefixStartIndex == dictStartIndex)
390 return ZSTD_compressBlock_doubleFast_generic(ms, seqStore, rep, src, srcSize, mls, ZSTD_noDict);
391
392 /* Search Loop */
393 while (ip < ilimit) { /* < instead of <=, because (ip+1) */
394 const size_t hSmall = ZSTD_hashPtr(ip, hBitsS, mls);
395 const U32 matchIndex = hashSmall[hSmall];
396 const BYTE* const matchBase = matchIndex < prefixStartIndex ? dictBase : base;
397 const BYTE* match = matchBase + matchIndex;
398
399 const size_t hLong = ZSTD_hashPtr(ip, hBitsL, 8);
400 const U32 matchLongIndex = hashLong[hLong];
401 const BYTE* const matchLongBase = matchLongIndex < prefixStartIndex ? dictBase : base;
402 const BYTE* matchLong = matchLongBase + matchLongIndex;
403
404 const U32 curr = (U32)(ip-base);
405 const U32 repIndex = curr + 1 - offset_1; /* offset_1 expected <= curr +1 */
406 const BYTE* const repBase = repIndex < prefixStartIndex ? dictBase : base;
407 const BYTE* const repMatch = repBase + repIndex;
408 size_t mLength;
409 hashSmall[hSmall] = hashLong[hLong] = curr; /* update hash table */
410
411 if ((((U32)((prefixStartIndex-1) - repIndex) >= 3) /* intentional underflow : ensure repIndex doesn't overlap dict + prefix */
412 & (repIndex > dictStartIndex))
413 && (MEM_read32(repMatch) == MEM_read32(ip+1)) ) {
414 const BYTE* repMatchEnd = repIndex < prefixStartIndex ? dictEnd : iend;
415 mLength = ZSTD_count_2segments(ip+1+4, repMatch+4, iend, repMatchEnd, prefixStart) + 4;
416 ip++;
417 ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, 0, mLength-MINMATCH);
418 } else {
419 if ((matchLongIndex > dictStartIndex) && (MEM_read64(matchLong) == MEM_read64(ip))) {
420 const BYTE* const matchEnd = matchLongIndex < prefixStartIndex ? dictEnd : iend;
421 const BYTE* const lowMatchPtr = matchLongIndex < prefixStartIndex ? dictStart : prefixStart;
422 U32 offset;
423 mLength = ZSTD_count_2segments(ip+8, matchLong+8, iend, matchEnd, prefixStart) + 8;
424 offset = curr - matchLongIndex;
425 while (((ip>anchor) & (matchLong>lowMatchPtr)) && (ip[-1] == matchLong[-1])) { ip--; matchLong--; mLength++; } /* catch up */
426 offset_2 = offset_1;
427 offset_1 = offset;
428 ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
429
430 } else if ((matchIndex > dictStartIndex) && (MEM_read32(match) == MEM_read32(ip))) {
431 size_t const h3 = ZSTD_hashPtr(ip+1, hBitsL, 8);
432 U32 const matchIndex3 = hashLong[h3];
433 const BYTE* const match3Base = matchIndex3 < prefixStartIndex ? dictBase : base;
434 const BYTE* match3 = match3Base + matchIndex3;
435 U32 offset;
436 hashLong[h3] = curr + 1;
437 if ( (matchIndex3 > dictStartIndex) && (MEM_read64(match3) == MEM_read64(ip+1)) ) {
438 const BYTE* const matchEnd = matchIndex3 < prefixStartIndex ? dictEnd : iend;
439 const BYTE* const lowMatchPtr = matchIndex3 < prefixStartIndex ? dictStart : prefixStart;
440 mLength = ZSTD_count_2segments(ip+9, match3+8, iend, matchEnd, prefixStart) + 8;
441 ip++;
442 offset = curr+1 - matchIndex3;
443 while (((ip>anchor) & (match3>lowMatchPtr)) && (ip[-1] == match3[-1])) { ip--; match3--; mLength++; } /* catch up */
444 } else {
445 const BYTE* const matchEnd = matchIndex < prefixStartIndex ? dictEnd : iend;
446 const BYTE* const lowMatchPtr = matchIndex < prefixStartIndex ? dictStart : prefixStart;
447 mLength = ZSTD_count_2segments(ip+4, match+4, iend, matchEnd, prefixStart) + 4;
448 offset = curr - matchIndex;
449 while (((ip>anchor) & (match>lowMatchPtr)) && (ip[-1] == match[-1])) { ip--; match--; mLength++; } /* catch up */
450 }
451 offset_2 = offset_1;
452 offset_1 = offset;
453 ZSTD_storeSeq(seqStore, (size_t)(ip-anchor), anchor, iend, offset + ZSTD_REP_MOVE, mLength-MINMATCH);
454
455 } else {
456 ip += ((ip-anchor) >> kSearchStrength) + 1;
457 continue;
458 } }
459
460 /* move to next sequence start */
461 ip += mLength;
462 anchor = ip;
463
464 if (ip <= ilimit) {
465 /* Complementary insertion */
466 /* done after iLimit test, as candidates could be > iend-8 */
467 { U32 const indexToInsert = curr+2;
468 hashLong[ZSTD_hashPtr(base+indexToInsert, hBitsL, 8)] = indexToInsert;
469 hashLong[ZSTD_hashPtr(ip-2, hBitsL, 8)] = (U32)(ip-2-base);
470 hashSmall[ZSTD_hashPtr(base+indexToInsert, hBitsS, mls)] = indexToInsert;
471 hashSmall[ZSTD_hashPtr(ip-1, hBitsS, mls)] = (U32)(ip-1-base);
472 }
473
474 /* check immediate repcode */
475 while (ip <= ilimit) {
476 U32 const current2 = (U32)(ip-base);
477 U32 const repIndex2 = current2 - offset_2;
478 const BYTE* repMatch2 = repIndex2 < prefixStartIndex ? dictBase + repIndex2 : base + repIndex2;
479 if ( (((U32)((prefixStartIndex-1) - repIndex2) >= 3) /* intentional overflow : ensure repIndex2 doesn't overlap dict + prefix */
480 & (repIndex2 > dictStartIndex))
481 && (MEM_read32(repMatch2) == MEM_read32(ip)) ) {
482 const BYTE* const repEnd2 = repIndex2 < prefixStartIndex ? dictEnd : iend;
483 size_t const repLength2 = ZSTD_count_2segments(ip+4, repMatch2+4, iend, repEnd2, prefixStart) + 4;
484 U32 const tmpOffset = offset_2; offset_2 = offset_1; offset_1 = tmpOffset; /* swap offset_2 <=> offset_1 */
485 ZSTD_storeSeq(seqStore, 0, anchor, iend, 0, repLength2-MINMATCH);
486 hashSmall[ZSTD_hashPtr(ip, hBitsS, mls)] = current2;
487 hashLong[ZSTD_hashPtr(ip, hBitsL, 8)] = current2;
488 ip += repLength2;
489 anchor = ip;
490 continue;
491 }
492 break;
493 } } }
494
495 /* save reps for next block */
496 rep[0] = offset_1;
497 rep[1] = offset_2;
498
499 /* Return the last literals size */
500 return (size_t)(iend - anchor);
> 501 }
502
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 67360 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH v6 3/3] lib: zstd: Upgrade to latest upstream zstd version 1.4.6
2020-12-02 23:58 ` [PATCH v6 3/3] lib: zstd: Upgrade to latest upstream zstd version 1.4.6 kernel test robot
@ 2020-12-04 14:03 ` David Sterba
0 siblings, 0 replies; 2+ messages in thread
From: David Sterba @ 2020-12-04 14:03 UTC (permalink / raw)
To: kernel test robot
Cc: Nick Terrell, Herbert Xu, kbuild-all, linux-crypto, linux-btrfs,
squashfs-devel, linux-f2fs-devel, linux-kernel, Kernel Team,
Chris Mason, Petr Malat
On Thu, Dec 03, 2020 at 07:58:16AM +0800, kernel test robot wrote:
> All warnings (new ones prefixed by >>):
>
> lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast_extDict_generic':
> >> lib/zstd/compress/zstd_double_fast.c:501:1: warning: the frame size of 3724 bytes is larger than 1280 bytes [-Wframe-larger-than=]
Frame size 3724?
> lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast':
> lib/zstd/compress/zstd_double_fast.c:336:1: warning: the frame size of 3792 bytes is larger than 1280 bytes [-Wframe-larger-than=]
3792
> lib/zstd/compress/zstd_double_fast.c: In function 'ZSTD_compressBlock_doubleFast_dictMatchState':
> lib/zstd/compress/zstd_double_fast.c:356:1: warning: the frame size of 3808 bytes is larger than 1280 bytes [-Wframe-larger-than=]
3808
> lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast_extDict_generic':
> >> lib/zstd/compress/zstd_fast.c:476:1: warning: the frame size of 2736 bytes is larger than 1280 bytes [-Wframe-larger-than=]
2736
> lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast':
> lib/zstd/compress/zstd_fast.c:204:1: warning: the frame size of 1508 bytes is larger than 1280 bytes [-Wframe-larger-than=]
1508
> lib/zstd/compress/zstd_fast.c: In function 'ZSTD_compressBlock_fast_dictMatchState':
> lib/zstd/compress/zstd_fast.c:372:1: warning: the frame size of 1540 bytes is larger than 1280 bytes [-Wframe-larger-than=]
1540
For userspace code it's nothing but in kernel it's a lot for a single
function. The largest number is almost one page, there were days where
this would be one half of the whole stack space. We can't waste precious
resources like that. Taking the userspace code as-is does not seem to
work.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-12-04 14:05 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20201202203242.1187898-4-nickrterrell@gmail.com>
2020-12-02 23:58 ` [PATCH v6 3/3] lib: zstd: Upgrade to latest upstream zstd version 1.4.6 kernel test robot
2020-12-04 14:03 ` David Sterba
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).