From: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
To: Xiong Zhou <xzhou-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
jack-AlSwsSmVLrQ@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: multi-threads libvmmalloc fork test hang
Date: Thu, 27 Oct 2016 15:37:20 +0200 [thread overview]
Message-ID: <20161027133720.GF19743@quack2.suse.cz> (raw)
In-Reply-To: <20161027112230.wsumgs62fqdxt3sc-obHvtwIU90hQcClZ3XN9yxcY2uh10dtjAL8bYrjMMd8@public.gmane.org>
Thanks for the report. I'll try to reproduce it...
Honza
On Thu 27-10-16 19:22:30, Xiong Zhou wrote:
> # description
>
> nvml test suite vmmalloc_fork test hang.
>
> $ ps -eo stat,comm | grep vmma
> S+ vmmalloc_fork
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
>
> dmesg:
>
> [ 250.499097] INFO: task vmmalloc_fork:9805 blocked for more than 120 seconds.
> [ 250.530667] Not tainted 4.9.09fe68ca+ #27
> [ 250.550901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 250.585752] vmmalloc_fork D[ 250.598362] ffffffff8171813c 0 9805 9765 0x00000080
> [ 250.623445] ffff88075dc68f80[ 250.636052] 0000000000000000 ffff88076058db00 ffff88017c5b0000 ffff880763b19340[ 250.668510] ffffc9000fe1bbb0 ffffffff8171813c ffffc9000fe1bc20 ffffc9000fe1bbe0[ 250.704220] ffffffff82248898 ffff88076058db00 ffffffff82248898Call Trace:
> [ 250.738382] [<ffffffff8171813c>] ? __schedule+0x21c/0x6a0
> [ 250.763404] [<ffffffff817185f6>] schedule+0x36/0x80
> [ 250.786177] [<ffffffff81284471>] get_unlocked_mapping_entry+0xc1/0x120
> [ 250.815869] [<ffffffff81283810>] ? iomap_dax_rw+0x110/0x110
> [ 250.841350] [<ffffffff81284c0a>] grab_mapping_entry+0x4a/0x220
> [ 250.868442] [<ffffffff812851e9>] iomap_dax_fault+0xa9/0x3b0
> [ 250.894437] [<ffffffffa02b15fe>] xfs_filemap_fault+0xce/0xf0 [xfs]
> [ 250.922805] [<ffffffff811d3159>] __do_fault+0x79/0x100
> [ 250.947035] [<ffffffff811d7a2b>] do_fault+0x49b/0x690
> [ 250.970964] [<ffffffffa02b146c>] ? xfs_filemap_pmd_fault+0x9c/0x160 [xfs]
> [ 251.001812] [<ffffffff811d94ba>] handle_mm_fault+0x61a/0xa50
> [ 251.027736] [<ffffffff8106c3da>] __do_page_fault+0x22a/0x4a0
> [ 251.053700] [<ffffffff8106c680>] do_page_fault+0x30/0x80
> [ 251.077962] [<ffffffff81003b55>] ? do_syscall_64+0x175/0x180
> [ 251.103835] [<ffffffff8171e208>] page_fault+0x28/0x30
>
>
> # kernel versions:
>
> v4.6 pass in seconds
> v4.7 hang
> v4.9-rc1 hang
> Linus tree to commit 9fe68ca hang
>
> bisect points to
> first bad commit: [ac401cc782429cc8560ce4840b1405d603740917] dax: New fault locking
>
> v4.7 with these 3 commits reverted pass:
> 4d9a2c8 - Jan Kara, 6 months ago : dax: Remove i_mmap_lock protection
> bc2466e - Jan Kara, 6 months ago : dax: Use radix tree entry lock to protect cow faults
> ac401cc - Jan Kara, 6 months ago : dax: New fault locking
>
> # nvml version:
> https://github.com/pmem/nvml.git
> to commit:
> feab4d6f65102139ce460890c898fcad09ce20ae
>
> # How reproducible:
> always
>
> # Test steps:
>
> <git clone and pmem0 setup>
>
> $cd nvml
> $make install -j64
>
> $cat > src/test/testconfig.sh <<EOF
> PMEM_FS_DIR=/daxmnt
> NON_PMEM_FS_DIR=/tmp
> EOF
>
> $mkfs.xfs /dev/pmem0
> $mkdir -p /daxmnt/
> $mount -o dax /dev/pmem0 /daxmnt/
>
> $make -C src/test/vmmalloc_fork/ TEST_TIME=60m clean
> $make -C src/test/vmmalloc_fork/ TEST_TIME=60m check
> $umount /daxmnt
--
Jan Kara <jack-IBi9RG/b67k@public.gmane.org>
SUSE Labs, CR
WARNING: multiple messages have this Message-ID (diff)
From: Jan Kara <jack@suse.cz>
To: Xiong Zhou <xzhou@redhat.com>
Cc: linux-nvdimm@ml01.01.org, jack@suse.cz,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: multi-threads libvmmalloc fork test hang
Date: Thu, 27 Oct 2016 15:37:20 +0200 [thread overview]
Message-ID: <20161027133720.GF19743@quack2.suse.cz> (raw)
In-Reply-To: <20161027112230.wsumgs62fqdxt3sc@xzhoul.usersys.redhat.com>
Thanks for the report. I'll try to reproduce it...
Honza
On Thu 27-10-16 19:22:30, Xiong Zhou wrote:
> # description
>
> nvml test suite vmmalloc_fork test hang.
>
> $ ps -eo stat,comm | grep vmma
> S+ vmmalloc_fork
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
> Sl+ vmmalloc_fork
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
> Z+ vmmalloc_fork <defunct>
>
> dmesg:
>
> [ 250.499097] INFO: task vmmalloc_fork:9805 blocked for more than 120 seconds.
> [ 250.530667] Not tainted 4.9.09fe68ca+ #27
> [ 250.550901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [ 250.585752] vmmalloc_fork D[ 250.598362] ffffffff8171813c 0 9805 9765 0x00000080
> [ 250.623445] ffff88075dc68f80[ 250.636052] 0000000000000000 ffff88076058db00 ffff88017c5b0000 ffff880763b19340[ 250.668510] ffffc9000fe1bbb0 ffffffff8171813c ffffc9000fe1bc20 ffffc9000fe1bbe0[ 250.704220] ffffffff82248898 ffff88076058db00 ffffffff82248898Call Trace:
> [ 250.738382] [<ffffffff8171813c>] ? __schedule+0x21c/0x6a0
> [ 250.763404] [<ffffffff817185f6>] schedule+0x36/0x80
> [ 250.786177] [<ffffffff81284471>] get_unlocked_mapping_entry+0xc1/0x120
> [ 250.815869] [<ffffffff81283810>] ? iomap_dax_rw+0x110/0x110
> [ 250.841350] [<ffffffff81284c0a>] grab_mapping_entry+0x4a/0x220
> [ 250.868442] [<ffffffff812851e9>] iomap_dax_fault+0xa9/0x3b0
> [ 250.894437] [<ffffffffa02b15fe>] xfs_filemap_fault+0xce/0xf0 [xfs]
> [ 250.922805] [<ffffffff811d3159>] __do_fault+0x79/0x100
> [ 250.947035] [<ffffffff811d7a2b>] do_fault+0x49b/0x690
> [ 250.970964] [<ffffffffa02b146c>] ? xfs_filemap_pmd_fault+0x9c/0x160 [xfs]
> [ 251.001812] [<ffffffff811d94ba>] handle_mm_fault+0x61a/0xa50
> [ 251.027736] [<ffffffff8106c3da>] __do_page_fault+0x22a/0x4a0
> [ 251.053700] [<ffffffff8106c680>] do_page_fault+0x30/0x80
> [ 251.077962] [<ffffffff81003b55>] ? do_syscall_64+0x175/0x180
> [ 251.103835] [<ffffffff8171e208>] page_fault+0x28/0x30
>
>
> # kernel versions:
>
> v4.6 pass in seconds
> v4.7 hang
> v4.9-rc1 hang
> Linus tree to commit 9fe68ca hang
>
> bisect points to
> first bad commit: [ac401cc782429cc8560ce4840b1405d603740917] dax: New fault locking
>
> v4.7 with these 3 commits reverted pass:
> 4d9a2c8 - Jan Kara, 6 months ago : dax: Remove i_mmap_lock protection
> bc2466e - Jan Kara, 6 months ago : dax: Use radix tree entry lock to protect cow faults
> ac401cc - Jan Kara, 6 months ago : dax: New fault locking
>
> # nvml version:
> https://github.com/pmem/nvml.git
> to commit:
> feab4d6f65102139ce460890c898fcad09ce20ae
>
> # How reproducible:
> always
>
> # Test steps:
>
> <git clone and pmem0 setup>
>
> $cd nvml
> $make install -j64
>
> $cat > src/test/testconfig.sh <<EOF
> PMEM_FS_DIR=/daxmnt
> NON_PMEM_FS_DIR=/tmp
> EOF
>
> $mkfs.xfs /dev/pmem0
> $mkdir -p /daxmnt/
> $mount -o dax /dev/pmem0 /daxmnt/
>
> $make -C src/test/vmmalloc_fork/ TEST_TIME=60m clean
> $make -C src/test/vmmalloc_fork/ TEST_TIME=60m check
> $umount /daxmnt
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
next prev parent reply other threads:[~2016-10-27 13:37 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-27 11:22 multi-threads libvmmalloc fork test hang Xiong Zhou
2016-10-27 11:22 ` Xiong Zhou
[not found] ` <20161027112230.wsumgs62fqdxt3sc-obHvtwIU90hQcClZ3XN9yxcY2uh10dtjAL8bYrjMMd8@public.gmane.org>
2016-10-27 13:37 ` Jan Kara [this message]
2016-10-27 13:37 ` Jan Kara
[not found] ` <20161027133720.GF19743-4I4JzKEfoa/jFM9bn6wA6Q@public.gmane.org>
2017-01-03 16:58 ` Ross Zwisler
2017-01-03 16:58 ` Ross Zwisler
2017-01-03 21:36 ` [PATCH] dax: fix deadlock with DAX 4k holes Ross Zwisler
2017-01-03 21:36 ` Ross Zwisler
2017-01-03 21:36 ` Ross Zwisler
2017-01-04 7:18 ` Jan Kara
2017-01-04 7:18 ` Jan Kara
2017-01-04 7:18 ` Jan Kara
2017-01-04 14:26 ` Xiong Zhou
2017-01-04 14:26 ` Xiong Zhou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161027133720.GF19743@quack2.suse.cz \
--to=jack-alswssmvlrq@public.gmane.org \
--cc=linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org \
--cc=xzhou-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.