xfstests can't be installed by running make install

xfstests can't be installed by running make install

[Zorro Lang]

Hi,

xfstests can't be installed now, if I run "make install", it fails as [1].
I don't what has been changed, and I'm not really so familar with these
building things ... Is there anyone knows what's wrong with it? I tested
on Fedora 28 with GNU Make 4.2.1.

Thanks,
Zorro

[1]
# git pull
Already up to date.
# make
# make install
Building include
Building lib
Building ltp
Building src
Building log-writes
Building perf
Building aio-dio-regress
Building m4
Building common                                
Building tests
/usr/bin/gmake --no-print-directory Q=@ -C include install
gmake[1]: Nothing to be done for 'install'.
/usr/bin/gmake --no-print-directory Q=@ -C lib install
gmake[1]: Nothing to be done for 'install'.
/usr/bin/gmake --no-print-directory Q=@ -C ltp install
../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
/usr/bin/gmake --no-print-directory Q=@ -C src install                           
Building log-writes
Building perf
Building aio-dio-regress
/usr/bin/gmake -C log-writes install
../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
/usr/bin/gmake -C perf install
../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
/usr/bin/gmake -C aio-dio-regress install
../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_c!
 opy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
/usr/bin/gmake --no-print-directory Q=@ -C m4 install
gmake[1]: Nothing to be done for 'install'.
/usr/bin/gmake --no-print-directory Q=@ -C common install
../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
/usr/bin/gmake --no-print-directory Q=@ -C tests install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
/usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
......
......
(again and again and again ....)
......
......

Re: xfstests can't be installed by running make install

[Eryu Guan]

On Wed, Jul 11, 2018 at 10:33:14PM +0800, Zorro Lang wrote:
> Hi,
> 
> xfstests can't be installed now, if I run "make install", it fails as [1].
> I don't what has been changed, and I'm not really so familar with these
> building things ... Is there anyone knows what's wrong with it? I tested
> on Fedora 28 with GNU Make 4.2.1.
> 
> Thanks,
> Zorro
> 
> [1]
> # git pull
> Already up to date.
> # make
> # make install
> Building include
> Building lib
> Building ltp
> Building src
> Building log-writes
> Building perf
> Building aio-dio-regress
> Building m4
> Building common                                
> Building tests
> /usr/bin/gmake --no-print-directory Q=@ -C include install
> gmake[1]: Nothing to be done for 'install'.
> /usr/bin/gmake --no-print-directory Q=@ -C lib install
> gmake[1]: Nothing to be done for 'install'.
> /usr/bin/gmake --no-print-directory Q=@ -C ltp install
> ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
> /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
> ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
> /usr/bin/gmake --no-print-directory Q=@ -C src install                           
> Building log-writes
> Building perf
> Building aio-dio-regress
> /usr/bin/gmake -C log-writes install
> ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
> ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
> /usr/bin/gmake -C perf install
> ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
> ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
> /usr/bin/gmake -C aio-dio-regress install
> ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
> ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
> ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
> /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
> /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
> /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
> /usr/bin/gmake --no-print-directory Q=@ -C m4 install
> gmake[1]: Nothing to be done for 'install'.
> /usr/bin/gmake --no-print-directory Q=@ -C common install
> ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
> ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
> /usr/bin/gmake --no-print-directory Q=@ -C tests install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> ......
> ......
> (again and again and again ....)

This looks like a bug of make to me.

TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))

The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
"tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
So we get subdir list and go into each subdir and do install there.

But the same "wildcard" expression returns files too on fedora 28, e.g.
"tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
"tests/" is included in subdirs again, and we go to "tests/" to do
install there again and enter a infinite loop.

I can workaround the problem by appending a dot, i.e.

- TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))
+ TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/.)))

So we make sure only dirs are returned.

Thanks,
Eryu

> ......
> ......
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> On Wed, Jul 11, 2018 at 10:33:14PM +0800, Zorro Lang wrote:
> > Hi,
> > 
> > xfstests can't be installed now, if I run "make install", it fails as [1].
> > I don't what has been changed, and I'm not really so familar with these
> > building things ... Is there anyone knows what's wrong with it? I tested
> > on Fedora 28 with GNU Make 4.2.1.
> > 
> > Thanks,
> > Zorro
> > 
> > [1]
> > # git pull
> > Already up to date.
> > # make
> > # make install
> > Building include
> > Building lib
> > Building ltp
> > Building src
> > Building log-writes
> > Building perf
> > Building aio-dio-regress
> > Building m4
> > Building common                                
> > Building tests
> > /usr/bin/gmake --no-print-directory Q=@ -C include install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C lib install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C ltp install
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
> > ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
> > /usr/bin/gmake --no-print-directory Q=@ -C src install                           
> > Building log-writes
> > Building perf
> > Building aio-dio-regress
> > /usr/bin/gmake -C log-writes install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
> > ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
> > /usr/bin/gmake -C perf install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
> > ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
> > /usr/bin/gmake -C aio-dio-regress install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
> > ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test se!
 ek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
> > /usr/bin/gmake --no-print-directory Q=@ -C m4 install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C common install
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
> > ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
> > /usr/bin/gmake --no-print-directory Q=@ -C tests install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > ......
> > ......
> > (again and again and again ....)
> 
> This looks like a bug of make to me.
> 
> TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))
> 
> The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> So we get subdir list and go into each subdir and do install there.
> 
> But the same "wildcard" expression returns files too on fedora 28, e.g.
> "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
> "tests/" is included in subdirs again, and we go to "tests/" to do
> install there again and enter a infinite loop.
> 
> I can workaround the problem by appending a dot, i.e.
> 
> - TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))
> + TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/.)))
> 
> So we make sure only dirs are returned.

Thanks for looking into it. I'd glad to know you're fixing it :)
This change looks good to me. xfstests can work well on Fedora28 now, and
it doesn't bring in regression issue on old system likes RHEL-7.

Thanks,
Zorro

> 
> Thanks,
> Eryu
> 
> > ......
> > ......
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Dave Chinner]

On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> On Wed, Jul 11, 2018 at 10:33:14PM +0800, Zorro Lang wrote:
> > Hi,
> > 
> > xfstests can't be installed now, if I run "make install", it fails as [1].
> > I don't what has been changed, and I'm not really so familar with these
> > building things ... Is there anyone knows what's wrong with it? I tested
> > on Fedora 28 with GNU Make 4.2.1.
> > 
> > Thanks,
> > Zorro
> > 
> > [1]
> > # git pull
> > Already up to date.
> > # make
> > # make install
> > Building include
> > Building lib
> > Building ltp
> > Building src
> > Building log-writes
> > Building perf
> > Building aio-dio-regress
> > Building m4
> > Building common                                
> > Building tests
> > /usr/bin/gmake --no-print-directory Q=@ -C include install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C lib install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C ltp install
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
> > ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
> > /usr/bin/gmake --no-print-directory Q=@ -C src install                           
> > Building log-writes
> > Building perf
> > Building aio-dio-regress
> > /usr/bin/gmake -C log-writes install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
> > ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
> > /usr/bin/gmake -C perf install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
> > ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
> > /usr/bin/gmake -C aio-dio-regress install
> > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
> > ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test se!
 ek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
> > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
> > /usr/bin/gmake --no-print-directory Q=@ -C m4 install
> > gmake[1]: Nothing to be done for 'install'.
> > /usr/bin/gmake --no-print-directory Q=@ -C common install
> > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
> > ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
> > /usr/bin/gmake --no-print-directory Q=@ -C tests install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > ......
> > ......
> > (again and again and again ....)
> 
> This looks like a bug of make to me.
> 
> TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))

Is this a regression because of commit 103258a566b4 ("build: fix
install target using sudo")?

> The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> So we get subdir list and go into each subdir and do install there.
> 
> But the same "wildcard" expression returns files too on fedora 28, e.g.
> "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,

Should *never* return tests/Makefile, because that does not match
the [a-z]* regex - it's a lowercase first character match, not
uppercase. And the *only* things that should be in tests/ is the
Makefile and all the test subdirs, so it shouldn't be matching the
wrong thing. i.e. how are we getting tests/ as a result in the
TESTS_SUBDIRS set?

> "tests/" is included in subdirs again, and we go to "tests/" to do
> install there again and enter a infinite loop.

Again, is this a regression? If so, we need to understand why using
$(CURDIR) is returning different results from $(PWD)/$(TESTS_DIR)/

> I can workaround the problem by appending a dot, i.e.

At this point, this seems more like hacking a bandaid over whatever
problem is occurring rather than trying to solve/understand why this
is now happening.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfstests can't be installed by running make install

[Eryu Guan]

On Sun, Jul 15, 2018 at 03:43:20PM +1000, Dave Chinner wrote:
> On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> > On Wed, Jul 11, 2018 at 10:33:14PM +0800, Zorro Lang wrote:
> > > Hi,
> > > 
> > > xfstests can't be installed now, if I run "make install", it fails as [1].
> > > I don't what has been changed, and I'm not really so familar with these
> > > building things ... Is there anyone knows what's wrong with it? I tested
> > > on Fedora 28 with GNU Make 4.2.1.
> > > 
> > > Thanks,
> > > Zorro
> > > 
> > > [1]
> > > # git pull
> > > Already up to date.
> > > # make
> > > # make install
> > > Building include
> > > Building lib
> > > Building ltp
> > > Building src
> > > Building log-writes
> > > Building perf
> > > Building aio-dio-regress
> > > Building m4
> > > Building common                                
> > > Building tests
> > > /usr/bin/gmake --no-print-directory Q=@ -C include install
> > > gmake[1]: Nothing to be done for 'install'.
> > > /usr/bin/gmake --no-print-directory Q=@ -C lib install
> > > gmake[1]: Nothing to be done for 'install'.
> > > /usr/bin/gmake --no-print-directory Q=@ -C ltp install
> > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
> > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
> > > ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
> > > /usr/bin/gmake --no-print-directory Q=@ -C src install                           
> > > Building log-writes
> > > Building perf
> > > Building aio-dio-regress
> > > /usr/bin/gmake -C log-writes install
> > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
> > > ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
> > > /usr/bin/gmake -C perf install
> > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
> > > ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
> > > /usr/bin/gmake -C aio-dio-regress install
> > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
> > > ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
> > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
> > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
> > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
> > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
> > > /usr/bin/gmake --no-print-directory Q=@ -C m4 install
> > > gmake[1]: Nothing to be done for 'install'.
> > > /usr/bin/gmake --no-print-directory Q=@ -C common install
> > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
> > > ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
> > > /usr/bin/gmake --no-print-directory Q=@ -C tests install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > ......
> > > ......
> > > (again and again and again ....)
> > 
> > This looks like a bug of make to me.
> > 
> > TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))
> 
> Is this a regression because of commit 103258a566b4 ("build: fix
> install target using sudo")?

No, it fails even with that commit reverted.

> 
> > The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> > "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> > So we get subdir list and go into each subdir and do install there.
> > 
> > But the same "wildcard" expression returns files too on fedora 28, e.g.
> > "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
> 
> Should *never* return tests/Makefile, because that does not match
> the [a-z]* regex - it's a lowercase first character match, not
> uppercase. And the *only* things that should be in tests/ is the
> Makefile and all the test subdirs, so it shouldn't be matching the
> wrong thing. i.e. how are we getting tests/ as a result in the
> TESTS_SUBDIRS set?

That's why I think it's a bug of make. I did the following test on
Fedora 28 with make-4.2.1-6.fc28.x86_64.

[root@fedoravm tmp]# rpm -q make
make-4.2.1-6.fc28.x86_64
[root@fedoravm tmp]# pwd
/root/tmp
[root@fedoravm tmp]# ls -l
total 4
-rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
[root@fedoravm tmp]# cat Makefile 
STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
STRING2 = $(wildcard ./[a-z]*/)
default:
        @echo STRING1="$(STRING1)"
        @echo STRING2="$(STRING2)"
[root@fedoravm tmp]# make
STRING1=/root/tmp/testdir/ /root/tmp/Makefile
STRING2=./testdir/ ./Makefile
[root@fedoravm tmp]#

So 'Makefile' is matched and returned (and the usa ofCURDIR doesn't
matter), and that explains why we have "tests/" in TESTS_SUBDIRS.

The same test on RHEL7.4 works all as expected, 'Makefile' didn't match
the pattern.

> 
> > "tests/" is included in subdirs again, and we go to "tests/" to do
> > install there again and enter a infinite loop.
> 
> Again, is this a regression? If so, we need to understand why using
> $(CURDIR) is returning different results from $(PWD)/$(TESTS_DIR)/
> 
> > I can workaround the problem by appending a dot, i.e.
> 
> At this point, this seems more like hacking a bandaid over whatever
> problem is occurring rather than trying to solve/understand why this
> is now happening.

Agreed, I just expressed the fact that adding an ending dot could change
the behavior, I didn't think it's a proper fix until we know exactly
what causes it.

Thanks,
Eryu

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Sun, Jul 15, 2018 at 03:11:10PM +0800, Eryu Guan wrote:
> On Sun, Jul 15, 2018 at 03:43:20PM +1000, Dave Chinner wrote:
> > On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> > > On Wed, Jul 11, 2018 at 10:33:14PM +0800, Zorro Lang wrote:
> > > > Hi,
> > > > 
> > > > xfstests can't be installed now, if I run "make install", it fails as [1].
> > > > I don't what has been changed, and I'm not really so familar with these
> > > > building things ... Is there anyone knows what's wrong with it? I tested
> > > > on Fedora 28 with GNU Make 4.2.1.
> > > > 
> > > > Thanks,
> > > > Zorro
> > > > 
> > > > [1]
> > > > # git pull
> > > > Already up to date.
> > > > # make
> > > > # make install
> > > > Building include
> > > > Building lib
> > > > Building ltp
> > > > Building src
> > > > Building log-writes
> > > > Building perf
> > > > Building aio-dio-regress
> > > > Building m4
> > > > Building common                                
> > > > Building tests
> > > > /usr/bin/gmake --no-print-directory Q=@ -C include install
> > > > gmake[1]: Nothing to be done for 'install'.
> > > > /usr/bin/gmake --no-print-directory Q=@ -C lib install
> > > > gmake[1]: Nothing to be done for 'install'.
> > > > /usr/bin/gmake --no-print-directory Q=@ -C ltp install
> > > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp
> > > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp
> > > > ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp
> > > > /usr/bin/gmake --no-print-directory Q=@ -C src install                           
> > > > Building log-writes
> > > > Building perf
> > > > Building aio-dio-regress
> > > > /usr/bin/gmake -C log-writes install
> > > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes
> > > > ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes
> > > > /usr/bin/gmake -C perf install
> > > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf
> > > > ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf
> > > > /usr/bin/gmake -C aio-dio-regress install
> > > > ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress
> > > > ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress
> > > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src
> > > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_tes!
 t seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum /var/lib/xfstests/src
> > > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh /var/lib/xfstests/src
> > > > /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src
> > > > /usr/bin/gmake --no-print-directory Q=@ -C m4 install
> > > > gmake[1]: Nothing to be done for 'install'.
> > > > /usr/bin/gmake --no-print-directory Q=@ -C common install
> > > > ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common
> > > > ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common
> > > > /usr/bin/gmake --no-print-directory Q=@ -C tests install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install
> > > > ......
> > > > ......
> > > > (again and again and again ....)
> > > 
> > > This looks like a bug of make to me.
> > > 
> > > TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[a-z]*/)))
> > 
> > Is this a regression because of commit 103258a566b4 ("build: fix
> > install target using sudo")?
> 
> No, it fails even with that commit reverted.
> 
> > 
> > > The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> > > "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> > > So we get subdir list and go into each subdir and do install there.
> > > 
> > > But the same "wildcard" expression returns files too on fedora 28, e.g.
> > > "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
> > 
> > Should *never* return tests/Makefile, because that does not match
> > the [a-z]* regex - it's a lowercase first character match, not
> > uppercase. And the *only* things that should be in tests/ is the
> > Makefile and all the test subdirs, so it shouldn't be matching the
> > wrong thing. i.e. how are we getting tests/ as a result in the
> > TESTS_SUBDIRS set?
> 
> That's why I think it's a bug of make. I did the following test on
> Fedora 28 with make-4.2.1-6.fc28.x86_64.
> 
> [root@fedoravm tmp]# rpm -q make
> make-4.2.1-6.fc28.x86_64
> [root@fedoravm tmp]# pwd
> /root/tmp
> [root@fedoravm tmp]# ls -l
> total 4
> -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> [root@fedoravm tmp]# cat Makefile 
> STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> STRING2 = $(wildcard ./[a-z]*/)
> default:
>         @echo STRING1="$(STRING1)"
>         @echo STRING2="$(STRING2)"
> [root@fedoravm tmp]# make
> STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> STRING2=./testdir/ ./Makefile
> [root@fedoravm tmp]#
> 
> So 'Makefile' is matched and returned (and the usa ofCURDIR doesn't
> matter), and that explains why we have "tests/" in TESTS_SUBDIRS.
> 
> The same test on RHEL7.4 works all as expected, 'Makefile' didn't match
> the pattern.
> 
> > 
> > > "tests/" is included in subdirs again, and we go to "tests/" to do
> > > install there again and enter a infinite loop.
> > 
> > Again, is this a regression? If so, we need to understand why using
> > $(CURDIR) is returning different results from $(PWD)/$(TESTS_DIR)/
> > 
> > > I can workaround the problem by appending a dot, i.e.
> > 
> > At this point, this seems more like hacking a bandaid over whatever
> > problem is occurring rather than trying to solve/understand why this
> > is now happening.
> 
> Agreed, I just expressed the fact that adding an ending dot could change
> the behavior, I didn't think it's a proper fix until we know exactly
> what causes it.

Thanks all your reviewing. I think it's time to CC bug-make@gnu.org, to get
more help from make expert about what has been changed in latest make project.

Thanks,
Zorro

> 
> Thanks,
> Eryu
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Paul Smith]

On Mon, 2018-07-16 at 15:30 +0800, Zorro Lang wrote:
> > [root@fedoravm tmp]# ls -l
> > total 4
> > -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > [root@fedoravm tmp]# cat Makefile 
> > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > STRING2 = $(wildcard ./[a-z]*/)
> > default:
> >          @echo STRING1="$(STRING1)"
> >          @echo STRING2="$(STRING2)"
> > [root@fedoravm tmp]# make
> > STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> > STRING2=./testdir/ ./Makefile
> > [root@fedoravm tmp]#

GNU make uses the system libc version of the glob(3) and fnmatch(3)
functions to implement its wildcard function on any system which
provides GNU libc.

So, if there's a change which introduces a problem with wildcard it is
more likely to be related to the GNU libc implementation of glob() or
fnmatch().

Unless you've somehow compiled GNU make to use its internal version of
GNU glob()/fnmatch() instead of the system version.

I filed a bug about this with GNU libc a long time ago, and it was
apparently fixed in GNU libc 2.19 in 2014.  So, I'm not sure why you've
just started seeing it now.

https://sourceware.org/bugzilla/show_bug.cgi?id=10278

Re: xfstests can't be installed by running make install

[Eryu Guan]

On Mon, Jul 16, 2018 at 07:48:26AM -0400, Paul Smith wrote:
> On Mon, 2018-07-16 at 15:30 +0800, Zorro Lang wrote:
> > > [root@fedoravm tmp]# ls -l
> > > total 4
> > > -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> > > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > > [root@fedoravm tmp]# cat Makefile 
> > > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > > STRING2 = $(wildcard ./[a-z]*/)
> > > default:
> > >          @echo STRING1="$(STRING1)"
> > >          @echo STRING2="$(STRING2)"
> > > [root@fedoravm tmp]# make
> > > STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> > > STRING2=./testdir/ ./Makefile
> > > [root@fedoravm tmp]#
> 
> GNU make uses the system libc version of the glob(3) and fnmatch(3)
> functions to implement its wildcard function on any system which
> provides GNU libc.
> 
> So, if there's a change which introduces a problem with wildcard it is
> more likely to be related to the GNU libc implementation of glob() or
> fnmatch().
> 
> Unless you've somehow compiled GNU make to use its internal version of
> GNU glob()/fnmatch() instead of the system version.
> 
> I filed a bug about this with GNU libc a long time ago, and it was
> apparently fixed in GNU libc 2.19 in 2014.  So, I'm not sure why you've
> just started seeing it now.
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=10278

This problem here doesn't seem the same as the bug above, Fedora 28 has
glibc-2.27, which contains the fix for above bug, and the bug is about
trailing "/". But the problem here is we're asking for all lower case
filenames but wildcard returns upper case names too. e.g.

[root@fedoravm tmp]# pwd
/root/tmp
[root@fedoravm tmp]# ls -l
total 4
-rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
-rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
-rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
[root@fedoravm tmp]# cat Makefile
STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
STRING2 = $(wildcard ./[a-z]*/)
STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
STRING4 = $(wildcard $(CURDIR)/[a-z]*)
default:
        @echo STRING1="$(STRING1)"
        @echo STRING2="$(STRING2)"
        @echo STRING3="$(STRING3)"
        @echo STRING4="$(STRING4)"
[root@fedoravm tmp]# make
STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
STRING2=./aaa ./AAA ./testdir/ ./Makefile
STRING3=/root/tmp/testdir/.
STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
[root@fedoravm tmp]#

STRING4 is asking for all lower file names, but both "AAA" and
"Makefile" are returned.

Thanks,
Eryu

Re: xfstests can't be installed by running make install

[Florian Weimer]

* Eryu Guan:

> This problem here doesn't seem the same as the bug above, Fedora 28 has
> glibc-2.27, which contains the fix for above bug, and the bug is about
> trailing "/". But the problem here is we're asking for all lower case
> filenames but wildcard returns upper case names too. e.g.
>
> [root@fedoravm tmp]# pwd
> /root/tmp
> [root@fedoravm tmp]# ls -l
> total 4
> -rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
> -rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
> -rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
> drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> [root@fedoravm tmp]# cat Makefile
> STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> STRING2 = $(wildcard ./[a-z]*/)
> STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
> STRING4 = $(wildcard $(CURDIR)/[a-z]*)
> default:
>         @echo STRING1="$(STRING1)"
>         @echo STRING2="$(STRING2)"
>         @echo STRING3="$(STRING3)"
>         @echo STRING4="$(STRING4)"
> [root@fedoravm tmp]# make
> STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
> STRING2=./aaa ./AAA ./testdir/ ./Makefile
> STRING3=/root/tmp/testdir/.
> STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
> [root@fedoravm tmp]#
>
> STRING4 is asking for all lower file names, but both "AAA" and
> "Makefile" are returned.

This is related to this glibc bug:

  https://sourceware.org/bugzilla/show_bug.cgi?id=23393

The bug mentions the regular expression [0-9], but it also affects
patterns like [a-z].  I have not yet looked at fnmatch and glob in
detail, but based on the report here (and a quick test with “echo
[a-z]*”), they are affected by the same issue.

This is ultimately caused by a locale data update which was backported
into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
has this change for version 2.28 (not yet released).  It's currently
not considered a release blocker, if it's even considered a bug at
all.

I dimly recall earlier discussions regarding this matter quite some
time ago, perhaps in the POSIX context.  GNU grep appears to have a
workaround for [0-9], but not [a-z].  Cc:ing Jim Meyering in case he
has any insights.

Re: xfstests can't be installed by running make install

[Dave Chinner]

On Tue, Jul 17, 2018 at 10:15:47PM +0200, Florian Weimer wrote:
> * Eryu Guan:
> 
> > This problem here doesn't seem the same as the bug above, Fedora 28 has
> > glibc-2.27, which contains the fix for above bug, and the bug is about
> > trailing "/". But the problem here is we're asking for all lower case
> > filenames but wildcard returns upper case names too. e.g.
> >
> > [root@fedoravm tmp]# pwd
> > /root/tmp
> > [root@fedoravm tmp]# ls -l
> > total 4
> > -rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
> > -rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
> > -rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
> > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > [root@fedoravm tmp]# cat Makefile
> > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > STRING2 = $(wildcard ./[a-z]*/)
> > STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
> > STRING4 = $(wildcard $(CURDIR)/[a-z]*)
> > default:
> >         @echo STRING1="$(STRING1)"
> >         @echo STRING2="$(STRING2)"
> >         @echo STRING3="$(STRING3)"
> >         @echo STRING4="$(STRING4)"
> > [root@fedoravm tmp]# make
> > STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
> > STRING2=./aaa ./AAA ./testdir/ ./Makefile
> > STRING3=/root/tmp/testdir/.
> > STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
> > [root@fedoravm tmp]#
> >
> > STRING4 is asking for all lower file names, but both "AAA" and
> > "Makefile" are returned.
> 
> This is related to this glibc bug:
> 
>   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> 
> The bug mentions the regular expression [0-9], but it also affects
> patterns like [a-z].  I have not yet looked at fnmatch and glob in
> detail, but based on the report here (and a quick test with “echo
> [a-z]*”), they are affected by the same issue.
> 
> This is ultimately caused by a locale data update which was backported
> into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
> has this change for version 2.28 (not yet released).  It's currently
> not considered a release blocker, if it's even considered a bug at
> all.

So what you are saying (i.e. "not a bug") is that we should expect
widespread regex breakage in custom scripts when distros roll out
glibc 2.28 because it will break assumptions people have coded into
their scripts and custom code for the past 20+ years?

That sounds like fun. :/

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfstests can't be installed by running make install

[Florian Weimer]

* Dave Chinner:

(The regular expression or glob pattern [a-z] matches B.)

>> This is ultimately caused by a locale data update which was backported
>> into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
>> has this change for version 2.28 (not yet released).  It's currently
>> not considered a release blocker, if it's even considered a bug at
>> all.
>
> So what you are saying (i.e. "not a bug") is that we should expect
> widespread regex breakage in custom scripts when distros roll out
> glibc 2.28 because it will break assumptions people have coded into
> their scripts and custom code for the past 20+ years?

According to POSIX, [a-z] is undefined in locales other than C or
POSIX.  This isn't a recent change.  POSIX pretty much says that you
need to use [[:lower:]] instead.

I don't doubt that a lot of code doesn't follow these rules, and
saying that it is broken according to POSIX doesn't help users who run
into issues.  But on the glibc side, I don't see anyone else arguing
that this needs to be fixed in glibc.

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Tue, Jul 17, 2018 at 10:15:47PM +0200, Florian Weimer wrote:
> * Eryu Guan:
> 
> > This problem here doesn't seem the same as the bug above, Fedora 28 has
> > glibc-2.27, which contains the fix for above bug, and the bug is about
> > trailing "/". But the problem here is we're asking for all lower case
> > filenames but wildcard returns upper case names too. e.g.
> >
> > [root@fedoravm tmp]# pwd
> > /root/tmp
> > [root@fedoravm tmp]# ls -l
> > total 4
> > -rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
> > -rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
> > -rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
> > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > [root@fedoravm tmp]# cat Makefile
> > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > STRING2 = $(wildcard ./[a-z]*/)
> > STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
> > STRING4 = $(wildcard $(CURDIR)/[a-z]*)
> > default:
> >         @echo STRING1="$(STRING1)"
> >         @echo STRING2="$(STRING2)"
> >         @echo STRING3="$(STRING3)"
> >         @echo STRING4="$(STRING4)"
> > [root@fedoravm tmp]# make
> > STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
> > STRING2=./aaa ./AAA ./testdir/ ./Makefile
> > STRING3=/root/tmp/testdir/.
> > STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
> > [root@fedoravm tmp]#
> >
> > STRING4 is asking for all lower file names, but both "AAA" and
> > "Makefile" are returned.
> 
> This is related to this glibc bug:
> 
>   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> 
> The bug mentions the regular expression [0-9], but it also affects
> patterns like [a-z].  I have not yet looked at fnmatch and glob in
> detail, but based on the report here (and a quick test with “echo
> [a-z]*”), they are affected by the same issue.
> 
> This is ultimately caused by a locale data update which was backported
> into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
> has this change for version 2.28 (not yet released).  It's currently
> not considered a release blocker, if it's even considered a bug at
> all.
> 
> I dimly recall earlier discussions regarding this matter quite some
> time ago, perhaps in the POSIX context.  GNU grep appears to have a
> workaround for [0-9], but not [a-z].  Cc:ing Jim Meyering in case he
> has any insights.

JFYI:

I reported this bug to glibc bugzilla too, and I got some responses
as below:
https://sourceware.org/bugzilla/show_bug.cgi?id=23420#c1

Hmm... is Fedora glibc much different with upstream ... Need more
investigate.

Thanks,
Zorro

> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Wed, Jul 18, 2018 at 11:15:15AM +0800, Zorro Lang wrote:
> On Tue, Jul 17, 2018 at 10:15:47PM +0200, Florian Weimer wrote:
> > * Eryu Guan:
> > 
> > > This problem here doesn't seem the same as the bug above, Fedora 28 has
> > > glibc-2.27, which contains the fix for above bug, and the bug is about
> > > trailing "/". But the problem here is we're asking for all lower case
> > > filenames but wildcard returns upper case names too. e.g.
> > >
> > > [root@fedoravm tmp]# pwd
> > > /root/tmp
> > > [root@fedoravm tmp]# ls -l
> > > total 4
> > > -rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
> > > -rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
> > > -rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
> > > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > > [root@fedoravm tmp]# cat Makefile
> > > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > > STRING2 = $(wildcard ./[a-z]*/)
> > > STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
> > > STRING4 = $(wildcard $(CURDIR)/[a-z]*)
> > > default:
> > >         @echo STRING1="$(STRING1)"
> > >         @echo STRING2="$(STRING2)"
> > >         @echo STRING3="$(STRING3)"
> > >         @echo STRING4="$(STRING4)"
> > > [root@fedoravm tmp]# make
> > > STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
> > > STRING2=./aaa ./AAA ./testdir/ ./Makefile
> > > STRING3=/root/tmp/testdir/.
> > > STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
> > > [root@fedoravm tmp]#
> > >
> > > STRING4 is asking for all lower file names, but both "AAA" and
> > > "Makefile" are returned.
> > 
> > This is related to this glibc bug:
> > 
> >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393

Hi Florian,

Sorry I just saw your internal reply in Red Hat. If you'd like to talk
about this issue at here, I have to say this change is radical.

Before, we can do this:
# echo abcd > testfile
# echo ABCD >> testfile
# egrep  [a-z] testfile
abcd

But now it becomes this suddently:
# echo abcd > testfile
# echo ABCD >> testfile
# egrep  [a-z] testfile
abcd
ABCD

I'm afraid that we will get many complaints from customers ... And of
course it will break many cases and tools.

Thanks,
Zorro

> > 
> > The bug mentions the regular expression [0-9], but it also affects
> > patterns like [a-z].  I have not yet looked at fnmatch and glob in
> > detail, but based on the report here (and a quick test with “echo
> > [a-z]*”), they are affected by the same issue.
> > 
> > This is ultimately caused by a locale data update which was backported
> > into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
> > has this change for version 2.28 (not yet released).  It's currently
> > not considered a release blocker, if it's even considered a bug at
> > all.
> > 
> > I dimly recall earlier discussions regarding this matter quite some
> > time ago, perhaps in the POSIX context.  GNU grep appears to have a
> > workaround for [0-9], but not [a-z].  Cc:ing Jim Meyering in case he
> > has any insights.
> 
> JFYI:
> 
> I reported this bug to glibc bugzilla too, and I got some responses
> as below:
> https://sourceware.org/bugzilla/show_bug.cgi?id=23420#c1
> 
> Hmm... is Fedora glibc much different with upstream ... Need more
> investigate.
> 
> Thanks,
> Zorro
> 
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Wed, Jul 18, 2018 at 11:47:49AM +0800, Zorro Lang wrote:
> On Wed, Jul 18, 2018 at 11:15:15AM +0800, Zorro Lang wrote:
> > On Tue, Jul 17, 2018 at 10:15:47PM +0200, Florian Weimer wrote:
> > > * Eryu Guan:
> > > 
> > > > This problem here doesn't seem the same as the bug above, Fedora 28 has
> > > > glibc-2.27, which contains the fix for above bug, and the bug is about
> > > > trailing "/". But the problem here is we're asking for all lower case
> > > > filenames but wildcard returns upper case names too. e.g.
> > > >
> > > > [root@fedoravm tmp]# pwd
> > > > /root/tmp
> > > > [root@fedoravm tmp]# ls -l
> > > > total 4
> > > > -rw-r--r--. 1 root root   0 Jul 17 10:51 aaa
> > > > -rw-r--r--. 1 root root   0 Jul 17 10:51 AAA
> > > > -rw-r--r--. 1 root root 273 Jul 17 10:50 Makefile
> > > > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > > > [root@fedoravm tmp]# cat Makefile
> > > > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > > > STRING2 = $(wildcard ./[a-z]*/)
> > > > STRING3 = $(wildcard $(CURDIR)/[a-z]*/.)
> > > > STRING4 = $(wildcard $(CURDIR)/[a-z]*)
> > > > default:
> > > >         @echo STRING1="$(STRING1)"
> > > >         @echo STRING2="$(STRING2)"
> > > >         @echo STRING3="$(STRING3)"
> > > >         @echo STRING4="$(STRING4)"
> > > > [root@fedoravm tmp]# make
> > > > STRING1=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir/ /root/tmp/Makefile
> > > > STRING2=./aaa ./AAA ./testdir/ ./Makefile
> > > > STRING3=/root/tmp/testdir/.
> > > > STRING4=/root/tmp/aaa /root/tmp/AAA /root/tmp/testdir /root/tmp/Makefile
> > > > [root@fedoravm tmp]#
> > > >
> > > > STRING4 is asking for all lower file names, but both "AAA" and
> > > > "Makefile" are returned.
> > > 
> > > This is related to this glibc bug:
> > > 
> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> 
> Hi Florian,
> 
> Sorry I just saw your internal reply in Red Hat. If you'd like to talk
> about this issue at here, I have to say this change is radical.
> 
> Before, we can do this:
> # echo abcd > testfile
> # echo ABCD >> testfile
> # egrep  [a-z] testfile
> abcd
> 
> But now it becomes this suddently:
> # echo abcd > testfile
> # echo ABCD >> testfile
> # egrep  [a-z] testfile
> abcd
> ABCD

A stranger thing is:
egrep [A-Z] match ABCD and bcd, but not match 'a'...

I already can't understand the new rules ...

> 
> I'm afraid that we will get many complaints from customers ... And of
> course it will break many cases and tools.
> 
> Thanks,
> Zorro
> 
> > > 
> > > The bug mentions the regular expression [0-9], but it also affects
> > > patterns like [a-z].  I have not yet looked at fnmatch and glob in
> > > detail, but based on the report here (and a quick test with “echo
> > > [a-z]*”), they are affected by the same issue.
> > > 
> > > This is ultimately caused by a locale data update which was backported
> > > into Fedora 28 (glibc 2.27) and its derivatives.  Upstream glibc only
> > > has this change for version 2.28 (not yet released).  It's currently
> > > not considered a release blocker, if it's even considered a bug at
> > > all.
> > > 
> > > I dimly recall earlier discussions regarding this matter quite some
> > > time ago, perhaps in the POSIX context.  GNU grep appears to have a
> > > workaround for [0-9], but not [a-z].  Cc:ing Jim Meyering in case he
> > > has any insights.
> > 
> > JFYI:
> > 
> > I reported this bug to glibc bugzilla too, and I got some responses
> > as below:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=23420#c1
> > 
> > Hmm... is Fedora glibc much different with upstream ... Need more
> > investigate.
> > 
> > Thanks,
> > Zorro
> > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Florian Weimer]

* Zorro Lang:

>> > > This is related to this glibc bug:
>> > > 
>> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
>> 

> A stranger thing is:
> egrep [A-Z] match ABCD and bcd, but not match 'a'...

That's the same issue as [0-9] not matching ９.

> I already can't understand the new rules ...

The range operator matches characters according to their collation
weight, and sincce the weight of 'a' is less than the weight of 'A',
'a' is not included in the [A-Z] range.

This could be fixed by including all characters with the same primary
weight as the endpoints (so that [ā-ẑ] and [a-z] would end up being
the same).  It makes the behavior more logical, but it doesn't fix
existing scripts.

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Wed, Jul 18, 2018 at 08:04:05AM +0200, Florian Weimer wrote:
> * Zorro Lang:
> 
> >> > > This is related to this glibc bug:
> >> > > 
> >> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> >> 
> 
> > A stranger thing is:
> > egrep [A-Z] match ABCD and bcd, but not match 'a'...
> 
> That's the same issue as [0-9] not matching ９.
> 
> > I already can't understand the new rules ...
> 
> The range operator matches characters according to their collation
> weight, and sincce the weight of 'a' is less than the weight of 'A',
> 'a' is not included in the [A-Z] range.

How to define/calculate the *weight* in your context? Why you say the
weight of 'a' is less than the weight of 'A'

> 
> This could be fixed by including all characters with the same primary
> weight as the endpoints (so that [ā-ẑ] and [a-z] would end up being
> the same).  It makes the behavior more logical, but it doesn't fix
> existing scripts.

We find that the $LANG will affect how glibc deal with the wildcard.
We all test on LANG=en_US.UTF=8, but if I set export LANG=C, then
[a-z] and [A-Z] are all as expected, and xfstests make install works.

Thanks,
Zorro

> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Florian Weimer]

* Zorro Lang:

> On Wed, Jul 18, 2018 at 08:04:05AM +0200, Florian Weimer wrote:
>> * Zorro Lang:
>> 
>> >> > > This is related to this glibc bug:
>> >> > > 
>> >> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
>> >> 
>> 
>> > A stranger thing is:
>> > egrep [A-Z] match ABCD and bcd, but not match 'a'...
>> 
>> That's the same issue as [0-9] not matching ９.
>> 
>> > I already can't understand the new rules ...
>> 
>> The range operator matches characters according to their collation
>> weight, and sincce the weight of 'a' is less than the weight of 'A',
>> 'a' is not included in the [A-Z] range.
>
> How to define/calculate the *weight* in your context? Why you say the
> weight of 'a' is less than the weight of 'A'

This is a concept from POSIX collation, based on a locale definition:

I hope this link is reasonably stable:

<http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html#tag_07_03_02>

Basically, collation is in alternative way of sorting strings,
different from codepoint order, and it is specifically designed to
take cultural conventions into account.  Traditionally, most regular
expression range expression such as [a-z] follow collation order,
although this is not required by POSIX for non-C/non-POSIX locales.

>> This could be fixed by including all characters with the same primary
>> weight as the endpoints (so that [ā-ẑ] and [a-z] would end up being
>> the same).  It makes the behavior more logical, but it doesn't fix
>> existing scripts.
>
> We find that the $LANG will affect how glibc deal with the wildcard.
> We all test on LANG=en_US.UTF=8, but if I set export LANG=C, then
> [a-z] and [A-Z] are all as expected, and xfstests make install works.

Right, this is expected: POSIX requires the behavior you need for the
"C" locale.

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Wed, Jul 18, 2018 at 10:47:09AM +0200, Florian Weimer wrote:
> * Zorro Lang:
> 
> > On Wed, Jul 18, 2018 at 08:04:05AM +0200, Florian Weimer wrote:
> >> * Zorro Lang:
> >> 
> >> >> > > This is related to this glibc bug:
> >> >> > > 
> >> >> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> >> >> 
> >> 
> >> > A stranger thing is:
> >> > egrep [A-Z] match ABCD and bcd, but not match 'a'...
> >> 
> >> That's the same issue as [0-9] not matching ９.
> >> 
> >> > I already can't understand the new rules ...
> >> 
> >> The range operator matches characters according to their collation
> >> weight, and sincce the weight of 'a' is less than the weight of 'A',
> >> 'a' is not included in the [A-Z] range.
> >
> > How to define/calculate the *weight* in your context? Why you say the
> > weight of 'a' is less than the weight of 'A'
> 
> This is a concept from POSIX collation, based on a locale definition:
> 
> I hope this link is reasonably stable:
> 
> <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html#tag_07_03_02>
> 
> Basically, collation is in alternative way of sorting strings,
> different from codepoint order, and it is specifically designed to
> take cultural conventions into account.  Traditionally, most regular
> expression range expression such as [a-z] follow collation order,
> although this is not required by POSIX for non-C/non-POSIX locales.
> 
> >> This could be fixed by including all characters with the same primary
> >> weight as the endpoints (so that [ā-ẑ] and [a-z] would end up being
> >> the same).  It makes the behavior more logical, but it doesn't fix
> >> existing scripts.
> >
> > We find that the $LANG will affect how glibc deal with the wildcard.
> > We all test on LANG=en_US.UTF=8, but if I set export LANG=C, then
> > [a-z] and [A-Z] are all as expected, and xfstests make install works.
> 
> Right, this is expected: POSIX requires the behavior you need for the
> "C" locale.

I was trying to change all these things to [:digit:], [:lower:], [:upper:],
[:alpha:] and [:alnum:]. But there're many, and the worse thing is there're
many things like [1-9], [3-8], [1-9a-f], [0-9a-f-] etc...

So I have to stop, and think about if there's a better way? How about we
fix the Makefile issue by change [a-z] to [[:lower:]], then export LANG=C
in xfstests/check file, and recommand export it in local.file?

Please tell me, if you have better idea.

Thanks so much,
Zorro

> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Wed, Jul 18, 2018 at 06:12:05PM +0800, Zorro Lang wrote:
> On Wed, Jul 18, 2018 at 10:47:09AM +0200, Florian Weimer wrote:
> > * Zorro Lang:
> > 
> > > On Wed, Jul 18, 2018 at 08:04:05AM +0200, Florian Weimer wrote:
> > >> * Zorro Lang:
> > >> 
> > >> >> > > This is related to this glibc bug:
> > >> >> > > 
> > >> >> > >   https://sourceware.org/bugzilla/show_bug.cgi?id=23393
> > >> >> 
> > >> 
> > >> > A stranger thing is:
> > >> > egrep [A-Z] match ABCD and bcd, but not match 'a'...
> > >> 
> > >> That's the same issue as [0-9] not matching ９.
> > >> 
> > >> > I already can't understand the new rules ...
> > >> 
> > >> The range operator matches characters according to their collation
> > >> weight, and sincce the weight of 'a' is less than the weight of 'A',
> > >> 'a' is not included in the [A-Z] range.
> > >
> > > How to define/calculate the *weight* in your context? Why you say the
> > > weight of 'a' is less than the weight of 'A'
> > 
> > This is a concept from POSIX collation, based on a locale definition:
> > 
> > I hope this link is reasonably stable:
> > 
> > <http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html#tag_07_03_02>
> > 
> > Basically, collation is in alternative way of sorting strings,
> > different from codepoint order, and it is specifically designed to
> > take cultural conventions into account.  Traditionally, most regular
> > expression range expression such as [a-z] follow collation order,
> > although this is not required by POSIX for non-C/non-POSIX locales.
> > 
> > >> This could be fixed by including all characters with the same primary
> > >> weight as the endpoints (so that [ā-ẑ] and [a-z] would end up being
> > >> the same).  It makes the behavior more logical, but it doesn't fix
> > >> existing scripts.
> > >
> > > We find that the $LANG will affect how glibc deal with the wildcard.
> > > We all test on LANG=en_US.UTF=8, but if I set export LANG=C, then
> > > [a-z] and [A-Z] are all as expected, and xfstests make install works.
> > 
> > Right, this is expected: POSIX requires the behavior you need for the
> > "C" locale.
> 
> I was trying to change all these things to [:digit:], [:lower:], [:upper:],
> [:alpha:] and [:alnum:]. But there're many, and the worse thing is there're
> many things like [1-9], [3-8], [1-9a-f], [0-9a-f-] etc...
> 
> So I have to stop, and think about if there's a better way? How about we
> fix the Makefile issue by change [a-z] to [[:lower:]], then export LANG=C
> in xfstests/check file, and recommand export it in local.file?
> 
> Please tell me, if you have better idea.

Another way maybe we can define SED_PROG="LC_ALL=C sed",
GREP_PROG="LC_ALL=C grep" etc ... But Makefile still need to be fixed
singly.


> 
> Thanks so much,
> Zorro
> 
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[Florian Weimer]

* Zorro Lang:

> I was trying to change all these things to [:digit:], [:lower:], [:upper:],
> [:alpha:] and [:alnum:]. But there're many, and the worse thing is there're
> many things like [1-9], [3-8], [1-9a-f], [0-9a-f-] etc...

I finally found a summary of the old discussion I tried to recall:

<https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html>

Ten years on, and we still haven't solved this.

I think it's time to implement the Rational Range Interpretation
across all GNU tools.

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Thu, Jul 19, 2018 at 06:21:05PM +0200, Florian Weimer wrote:
> * Zorro Lang:
> 
> > I was trying to change all these things to [:digit:], [:lower:], [:upper:],
> > [:alpha:] and [:alnum:]. But there're many, and the worse thing is there're
> > many things like [1-9], [3-8], [1-9a-f], [0-9a-f-] etc...
> 
> I finally found a summary of the old discussion I tried to recall:
> 
> <https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html>
> 
> Ten years on, and we still haven't solved this.
> 
> I think it's time to implement the Rational Range Interpretation
> across all GNU tools.

Thanks Florian, I saw this patch from your team:
https://www.sourceware.org/ml/libc-alpha/2018-07/msg00620.html

I've sent a patch to fstests@, try to avoid this problem:
https://marc.info/?l=fstests&m=153199466428997&w=2

We already exported LANG=C and LC_ALL=C, I'm trying to change the
"[a-z]" to "[:lower:]" in Makefile for bypassing this confused
issue. (But yes, I need to wait reviewing :)

> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: xfstests can't be installed by running make install

[spagoveanu]

On Mon, Jul 16, 2018 at 07:48:26AM -0400, Paul Smith wrote:
> On Mon, 2018-07-16 at 15:30 +0800, Zorro Lang wrote:
> > > [root@fedoravm tmp]# ls -l
> > > total 4
> > > -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> > > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > > [root@fedoravm tmp]# cat Makefile 
> > > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > > STRING2 = $(wildcard ./[a-z]*/)
> > > default:
> > >          @echo STRING1="$(STRING1)"
> > >          @echo STRING2="$(STRING2)"
> > > [root@fedoravm tmp]# make
> > > STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> > > STRING2=./testdir/ ./Makefile

This is the same bug I've reported (with a working fix) in

http://lists.gnu.org/archive/html/bug-make/2018-06/msg00009.html

The '[a-z] matching uppercase chars because of LC_COLLATE' aggravating
"feature" is a red herring here; './[a-z]*/' shouldn't match 'makefile'
either; it should only match directories.

> > > [root@fedoravm tmp]#
> 
> GNU make uses the system libc version of the glob(3) and fnmatch(3)
> functions to implement its wildcard function on any system which
> provides GNU libc.
> 
> So, if there's a change which introduces a problem with wildcard it is
> more likely to be related to the GNU libc implementation of glob() or
> fnmatch().

There is a bug in glibc's glob() implementation. Here is a simple test
case that's doing the exactly same things gnu make is doing to trigger it
(use the GLOB_ALTDIRFUNC extension mechanism and set dirent.d_type
to DT_UNKNOWN). The bug can also be triggered without GLOB_ALTDIRFUNC
by using a filesystem without support for the d_type field (eg. minix fs).

$ cc -Wall -O2 glob_onlydir.c -o glob_onlydir
$ mkdir -p test/dir
$ touch test/file
$ ./glob_onlydir 'test/*'
test/dir
test/file
$ cat glob_onlydir.c
#include <stdio.h>
#include <glob.h>
#include <dirent.h>
#include <sys/stat.h>
static void *my_readdir(void *v){
	struct dirent *d = readdir(v);
	if(d) d->d_type = DT_UNKNOWN;
	return d;
}
int main(int ac, char **av){
	int i, j;
	glob_t g = {0};
	g.gl_opendir = (void*(*)(const char*))opendir;
	g.gl_closedir = (void(*)(void*))closedir;
	g.gl_stat = (int(*)(const char*, void*))stat;
	g.gl_lstat = (int(*)(const char*, void*))lstat;
	g.gl_readdir = my_readdir;
	for(i = 1; i < ac; i++)
		if(glob(av[i], GLOB_ONLYDIR|GLOB_ALTDIRFUNC, 0, &g) == 0)
			for(j = 0; j < g.gl_pathc; j++)
				printf("%s\n", g.gl_pathv[j]);
	return 0;
}

Re: xfstests can't be installed by running make install

[Dave Chinner]

On Sun, Jul 15, 2018 at 03:11:10PM +0800, Eryu Guan wrote:
> On Sun, Jul 15, 2018 at 03:43:20PM +1000, Dave Chinner wrote:
> > On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> > > The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> > > "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> > > So we get subdir list and go into each subdir and do install there.
> > > 
> > > But the same "wildcard" expression returns files too on fedora 28, e.g.
> > > "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
> > 
> > Should *never* return tests/Makefile, because that does not match
> > the [a-z]* regex - it's a lowercase first character match, not
> > uppercase. And the *only* things that should be in tests/ is the
> > Makefile and all the test subdirs, so it shouldn't be matching the
> > wrong thing. i.e. how are we getting tests/ as a result in the
> > TESTS_SUBDIRS set?
> 
> That's why I think it's a bug of make. I did the following test on
> Fedora 28 with make-4.2.1-6.fc28.x86_64.
> 
> [root@fedoravm tmp]# rpm -q make
> make-4.2.1-6.fc28.x86_64
> [root@fedoravm tmp]# pwd
> /root/tmp
> [root@fedoravm tmp]# ls -l
> total 4
> -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> [root@fedoravm tmp]# cat Makefile 
> STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> STRING2 = $(wildcard ./[a-z]*/)
> default:
>         @echo STRING1="$(STRING1)"
>         @echo STRING2="$(STRING2)"
> [root@fedoravm tmp]# make
> STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> STRING2=./testdir/ ./Makefile
> [root@fedoravm tmp]#

So make 4.2.1 on fedora 28 has broken regex matching?

I just ran this on a debian based test machine:

$ make --version
GNU Make 4.1
....
$ make
STRING1=/home/dave/tmp_make/testdir/
STRING2=./testdir/
$

That works, but it's old. I just upgraded it to the lastest unstable
package (4.2.1-1.1), which also upgraded glibc to 2.27-5. Looks like
the make version matches fedora 28, but:

$ make --version
GNU Make 4.2.1
....
$ make
STRING1=/home/dave/tmp_make/testdir/
STRING2=./testdir/
$

the regex behaves correctly. So this looks like it might be
something isolated to the fedora 28 distro package build or glibc
version?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: xfstests can't be installed by running make install

[Zorro Lang]

On Tue, Jul 17, 2018 at 03:24:08PM +1000, Dave Chinner wrote:
> On Sun, Jul 15, 2018 at 03:11:10PM +0800, Eryu Guan wrote:
> > On Sun, Jul 15, 2018 at 03:43:20PM +1000, Dave Chinner wrote:
> > > On Thu, Jul 12, 2018 at 12:39:21AM +0800, Eryu Guan wrote:
> > > > The "wildcard" part is supposed to find all subdirs in tests dir, e.g.
> > > > "tests/ext4 tests/generic tests/xfs ...", files shouldn't be included.
> > > > So we get subdir list and go into each subdir and do install there.
> > > > 
> > > > But the same "wildcard" expression returns files too on fedora 28, e.g.
> > > > "tests/Makefile tests/ext4 tests/generic tests/xfs ...", as a result,
> > > 
> > > Should *never* return tests/Makefile, because that does not match
> > > the [a-z]* regex - it's a lowercase first character match, not
> > > uppercase. And the *only* things that should be in tests/ is the
> > > Makefile and all the test subdirs, so it shouldn't be matching the
> > > wrong thing. i.e. how are we getting tests/ as a result in the
> > > TESTS_SUBDIRS set?
> > 
> > That's why I think it's a bug of make. I did the following test on
> > Fedora 28 with make-4.2.1-6.fc28.x86_64.
> > 
> > [root@fedoravm tmp]# rpm -q make
> > make-4.2.1-6.fc28.x86_64
> > [root@fedoravm tmp]# pwd
> > /root/tmp
> > [root@fedoravm tmp]# ls -l
> > total 4
> > -rw-r--r--. 1 root root 206 Jul 15 14:58 Makefile
> > drwxr-xr-x. 1 root root   0 Jul 15 14:59 testdir
> > [root@fedoravm tmp]# cat Makefile 
> > STRING1 = $(wildcard $(CURDIR)/[a-z]*/)
> > STRING2 = $(wildcard ./[a-z]*/)
> > default:
> >         @echo STRING1="$(STRING1)"
> >         @echo STRING2="$(STRING2)"
> > [root@fedoravm tmp]# make
> > STRING1=/root/tmp/testdir/ /root/tmp/Makefile
> > STRING2=./testdir/ ./Makefile
> > [root@fedoravm tmp]#
> 
> So make 4.2.1 on fedora 28 has broken regex matching?
> 
> I just ran this on a debian based test machine:
> 
> $ make --version
> GNU Make 4.1
> ....
> $ make
> STRING1=/home/dave/tmp_make/testdir/
> STRING2=./testdir/
> $
> 
> That works, but it's old. I just upgraded it to the lastest unstable
> package (4.2.1-1.1), which also upgraded glibc to 2.27-5. Looks like
> the make version matches fedora 28, but:
> 
> $ make --version
> GNU Make 4.2.1
> ....
> $ make
> STRING1=/home/dave/tmp_make/testdir/
> STRING2=./testdir/
> $
> 
> the regex behaves correctly. So this looks like it might be
> something isolated to the fedora 28 distro package build or glibc
> version?

>From my testing:
1) make-4.2.1-4 built by glibc-2.26 works fine with glibc-2.26-27.
2) make-4.2.1-7 built by glibc-2.27 reproduce this bug with glibc-2.27-30.
3) make-4.2.1-4 built by glibc-2.26 reproduce this bug with glibc-2.27-30.
   Downgrade make forcibly by:
   # rpm -e --nodeps make
   # yum install make-4.2.1-4.xxx.rpm

Thanks,
Zorro

> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html