Re: memleaks, acpi + ext4 + tty

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Fri, Aug 28, 2009 at 3:09 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Fri, Aug 28, 2009 at 2:50 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>> On Fri, Aug 28, 2009 at 9:52 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>> On Fri, Aug 28, 2009 at 9:32 AM, Catalin Marinas<catalin.marinas@arm.com> wrote:
>>>> "Luis R. Rodriguez" <mcgrof@gmail.com> wrote:
>>>>> I have an assorted collection of kmemleak reports for acpi, ext4 and
>>>>> tty, not sure how to read these yet to fix so figure I'd at least post
>>>>> them. To reproduce I can just dd=/dev/zero to some big file and played
>>>>> some video.
>>>>
>>>> If you do a few echo scan > /sys/kernel/debug/kmemleak, do they
>>>> disappear (i.e. transient false positives)?
>>>
>>> Sure, I will once on rc8.
>>>
>>>> Which kernel version is this?
>>>
>>> v2.6.31-rc7-33172-gf4a9f9a
>>>
>>> This is from wireless-testing, which has wireless patches on top of
>>> rc7. John just rebased to rc8 so will give that a shot at work.
>>>
>>>>> unreferenced object 0xffff88003e0015c0 (size 64):
>>>>>   comm "swapper", pid 1, jiffies 4294892352
>>>>>   backtrace:
>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>     [<ffffffff81118a03>] kmem_cache_alloc_node+0x193/0x200
>>>>>     [<ffffffff8152509e>] process_zones+0x70/0x1cd
>>>>>     [<ffffffff81525230>] pageset_cpuup_callback+0x35/0x92
>>>>>     [<ffffffff8152c9b7>] notifier_call_chain+0x47/0x90
>>>>>     [<ffffffff81078549>] __raw_notifier_call_chain+0x9/0x10
>>>>>     [<ffffffff81523f25>] _cpu_up+0x75/0x130
>>>>>     [<ffffffff8152403a>] cpu_up+0x5a/0x6a
>>>>>     [<ffffffff8181969e>] kernel_init+0xcc/0x1ba
>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>
>>>> Can't really tell. Maybe a false positive caused by kmemleak not
>>>> scanning the pgdata node_zones. Can you post your .config file?
>>>
>>> Sure, attached.
>>>
>>>>> unreferenced object 0xffff88003cb5f700 (size 64):
>>>>>   comm "swapper", pid 1, jiffies 4294892459
>>>>>   backtrace:
>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>     [<ffffffff812bb549>] kzalloc+0xf/0x11
>>>>>     [<ffffffff812bbb53>] acpi_add_single_object+0x58e/0xd3c
>>>>>     [<ffffffff812bc51c>] acpi_bus_scan+0x125/0x1af
>>>>>     [<ffffffff81842361>] acpi_scan_init+0xc8/0xe9
>>>>>     [<ffffffff8184211c>] acpi_init+0x21f/0x265
>>>>>     [<ffffffff8100a05b>] do_one_initcall+0x4b/0x1b0
>>>>>     [<ffffffff81819736>] kernel_init+0x164/0x1ba
>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>
>>>> I get ACPI reports as well and they may be real leaks. However, I
>>>> didn't have time to analyse the code (pretty complicated reference
>>>> counting).
>>>
>>> Heh OK thanks for reviewing them though.
>>>
>>>>> unreferenced object 0xffff880039571800 (size 1024):
>>>>>   comm "exe", pid 1168, jiffies 4294893410
>>>>>   backtrace:
>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>     [<ffffffff811e1d71>] ext4_mb_init+0x1a1/0x590
>>>>>     [<ffffffff811d2da3>] ext4_fill_super+0x1df3/0x26c0
>>>>>     [<ffffffff8112774f>] get_sb_bdev+0x16f/0x1b0
>>>>>     [<ffffffff811c8fd3>] ext4_get_sb+0x13/0x20
>>>>>     [<ffffffff81127216>] vfs_kern_mount+0x76/0x180
>>>>>     [<ffffffff8112738d>] do_kern_mount+0x4d/0x130
>>>>>     [<ffffffff8113fc57>] do_mount+0x307/0x8b0
>>>>>     [<ffffffff8114028f>] sys_mount+0x8f/0xe0
>>>>>     [<ffffffff81011f02>] system_call_fastpath+0x16/0x1b
>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>
>>>> The ext4 reports are real leaks and patch was posted here -
>>>> http://lkml.org/lkml/2009/7/15/62. However, it hasn't been merged into
>>>> mainline yet (I cc'ed Aneesh).
>>>>
>>>> The patch is merged in my "kmemleak-fixes" branch on
>>>> git://linux-arm.org/linux-2.6.git.
>>>
>>> Will try to suck them out and try them.
>>
>> OK -- tested rc8 + a pull of your tree into mine. The bootup was
>> really slow and something was just not going right. After a while
>> memleak complained it had 8 kmemleak logs but I was not able to get my
>> system usable enough to cat the file.
>>
>> In cases like these I wish I would hookup my ctrl-alt-del to kexec() a
>> safe kernel.
>>
>> After a long period of time it seems X wished it would start, it tried
>> and then flashed back to the tty. This kept repeating in a loop.
>>
>> I am not sure if the culprit was rc8 or the kmemleak branch merge --
>> I'll find out after I boot into rc8 in a few.
>
> rc8 busted my bootup, the issues are present with just
> wireless-testing. I highly doubt the issues are wireless-testing
> related so I will not bisect there. Since I am unable to get anything
> useful from the kernel to determine what may have gone sour, any
> suggestions on a path to bisect, or should I just do the whole tree?

I tried 2.6.31-rc8 from hpa's linux-2.6-allstable.git tree instead of
Linus [1] as I already had that tree, git describe says:

v2.6.31-rc8-15-gadda766

Testing this would be the same as testing Linus' blessed rc8 --
correct me I'm wrong. Contrary to what I expected this tree with the
same config works well!

I have compiled a fresh checkout of wireless-testing origin/master to
double check the issue and it is indeed only present on
wireless-testing. A diff stat between John's merge of 2.6.31-rc8 and
current master branch on wireless-testing [2] doesn't reveal much
other than wireless specific stuff, as expected, so it seems this may
after all be introduced in a recent patches in wireless-testing. I
still find this a bit odd given I see no others reporting major
issues. My boot doesn't go very far, it stalls for a while after input
devices are being detected, then it spits out a kmemleak warning about
13 kmemleaks. Here's a picture [3]. I didn't bother waiting as I did
last time for X to try to come up, something is really wrong. I'll
bisect wireless-testing in the morning, starting with a good marker at
merge-2009-08-28 as that is when John pulled 2.6.31-rc8 (and I confirm
a diff stat between that and v2.6.31-rc8 yields nothing as it should)
and current master as the bad marker. I have 9 steps to go, will leave
first step compiling overnight.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-allstable.git
[2] git diff --stat merge-2009-08-28..HEAD
[3] http://bombadil.infradead.org/~mcgrof/images/2009/08/lag-wl-2009-08-31.jpg
[4] git diff --stat merge-2009-08-28..v2.6.31-rc8

  Luis

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 1:23 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Fri, Aug 28, 2009 at 3:09 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>> On Fri, Aug 28, 2009 at 2:50 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>> On Fri, Aug 28, 2009 at 9:52 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>> On Fri, Aug 28, 2009 at 9:32 AM, Catalin Marinas<catalin.marinas@arm.com> wrote:
>>>>> "Luis R. Rodriguez" <mcgrof@gmail.com> wrote:
>>>>>> I have an assorted collection of kmemleak reports for acpi, ext4 and
>>>>>> tty, not sure how to read these yet to fix so figure I'd at least post
>>>>>> them. To reproduce I can just dd=/dev/zero to some big file and played
>>>>>> some video.
>>>>>
>>>>> If you do a few echo scan > /sys/kernel/debug/kmemleak, do they
>>>>> disappear (i.e. transient false positives)?
>>>>
>>>> Sure, I will once on rc8.
>>>>
>>>>> Which kernel version is this?
>>>>
>>>> v2.6.31-rc7-33172-gf4a9f9a
>>>>
>>>> This is from wireless-testing, which has wireless patches on top of
>>>> rc7. John just rebased to rc8 so will give that a shot at work.
>>>>
>>>>>> unreferenced object 0xffff88003e0015c0 (size 64):
>>>>>>   comm "swapper", pid 1, jiffies 4294892352
>>>>>>   backtrace:
>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>     [<ffffffff81118a03>] kmem_cache_alloc_node+0x193/0x200
>>>>>>     [<ffffffff8152509e>] process_zones+0x70/0x1cd
>>>>>>     [<ffffffff81525230>] pageset_cpuup_callback+0x35/0x92
>>>>>>     [<ffffffff8152c9b7>] notifier_call_chain+0x47/0x90
>>>>>>     [<ffffffff81078549>] __raw_notifier_call_chain+0x9/0x10
>>>>>>     [<ffffffff81523f25>] _cpu_up+0x75/0x130
>>>>>>     [<ffffffff8152403a>] cpu_up+0x5a/0x6a
>>>>>>     [<ffffffff8181969e>] kernel_init+0xcc/0x1ba
>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>
>>>>> Can't really tell. Maybe a false positive caused by kmemleak not
>>>>> scanning the pgdata node_zones. Can you post your .config file?
>>>>
>>>> Sure, attached.
>>>>
>>>>>> unreferenced object 0xffff88003cb5f700 (size 64):
>>>>>>   comm "swapper", pid 1, jiffies 4294892459
>>>>>>   backtrace:
>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>     [<ffffffff812bb549>] kzalloc+0xf/0x11
>>>>>>     [<ffffffff812bbb53>] acpi_add_single_object+0x58e/0xd3c
>>>>>>     [<ffffffff812bc51c>] acpi_bus_scan+0x125/0x1af
>>>>>>     [<ffffffff81842361>] acpi_scan_init+0xc8/0xe9
>>>>>>     [<ffffffff8184211c>] acpi_init+0x21f/0x265
>>>>>>     [<ffffffff8100a05b>] do_one_initcall+0x4b/0x1b0
>>>>>>     [<ffffffff81819736>] kernel_init+0x164/0x1ba
>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>
>>>>> I get ACPI reports as well and they may be real leaks. However, I
>>>>> didn't have time to analyse the code (pretty complicated reference
>>>>> counting).
>>>>
>>>> Heh OK thanks for reviewing them though.
>>>>
>>>>>> unreferenced object 0xffff880039571800 (size 1024):
>>>>>>   comm "exe", pid 1168, jiffies 4294893410
>>>>>>   backtrace:
>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>     [<ffffffff811e1d71>] ext4_mb_init+0x1a1/0x590
>>>>>>     [<ffffffff811d2da3>] ext4_fill_super+0x1df3/0x26c0
>>>>>>     [<ffffffff8112774f>] get_sb_bdev+0x16f/0x1b0
>>>>>>     [<ffffffff811c8fd3>] ext4_get_sb+0x13/0x20
>>>>>>     [<ffffffff81127216>] vfs_kern_mount+0x76/0x180
>>>>>>     [<ffffffff8112738d>] do_kern_mount+0x4d/0x130
>>>>>>     [<ffffffff8113fc57>] do_mount+0x307/0x8b0
>>>>>>     [<ffffffff8114028f>] sys_mount+0x8f/0xe0
>>>>>>     [<ffffffff81011f02>] system_call_fastpath+0x16/0x1b
>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>
>>>>> The ext4 reports are real leaks and patch was posted here -
>>>>> http://lkml.org/lkml/2009/7/15/62. However, it hasn't been merged into
>>>>> mainline yet (I cc'ed Aneesh).
>>>>>
>>>>> The patch is merged in my "kmemleak-fixes" branch on
>>>>> git://linux-arm.org/linux-2.6.git.
>>>>
>>>> Will try to suck them out and try them.
>>>
>>> OK -- tested rc8 + a pull of your tree into mine. The bootup was
>>> really slow and something was just not going right. After a while
>>> memleak complained it had 8 kmemleak logs but I was not able to get my
>>> system usable enough to cat the file.
>>>
>>> In cases like these I wish I would hookup my ctrl-alt-del to kexec() a
>>> safe kernel.
>>>
>>> After a long period of time it seems X wished it would start, it tried
>>> and then flashed back to the tty. This kept repeating in a loop.
>>>
>>> I am not sure if the culprit was rc8 or the kmemleak branch merge --
>>> I'll find out after I boot into rc8 in a few.
>>
>> rc8 busted my bootup, the issues are present with just
>> wireless-testing. I highly doubt the issues are wireless-testing
>> related so I will not bisect there. Since I am unable to get anything
>> useful from the kernel to determine what may have gone sour, any
>> suggestions on a path to bisect, or should I just do the whole tree?
>
> I tried 2.6.31-rc8 from hpa's linux-2.6-allstable.git tree instead of
> Linus [1] as I already had that tree, git describe says:
>
> v2.6.31-rc8-15-gadda766
>
> Testing this would be the same as testing Linus' blessed rc8 --
> correct me I'm wrong. Contrary to what I expected this tree with the
> same config works well!
>
> I have compiled a fresh checkout of wireless-testing origin/master to
> double check the issue and it is indeed only present on
> wireless-testing. A diff stat between John's merge of 2.6.31-rc8 and
> current master branch on wireless-testing [2] doesn't reveal much
> other than wireless specific stuff, as expected, so it seems this may
> after all be introduced in a recent patches in wireless-testing. I
> still find this a bit odd given I see no others reporting major
> issues. My boot doesn't go very far, it stalls for a while after input
> devices are being detected, then it spits out a kmemleak warning about
> 13 kmemleaks. Here's a picture [3]. I didn't bother waiting as I did
> last time for X to try to come up, something is really wrong. I'll
> bisect wireless-testing in the morning, starting with a good marker at
> merge-2009-08-28 as that is when John pulled 2.6.31-rc8 (and I confirm
> a diff stat between that and v2.6.31-rc8 yields nothing as it should)
> and current master as the bad marker. I have 9 steps to go, will leave
> first step compiling overnight.
>
> [1] git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-allstable.git
> [2] git diff --stat merge-2009-08-28..HEAD
> [3] http://bombadil.infradead.org/~mcgrof/images/2009/08/lag-wl-2009-08-31.jpg
> [4] git diff --stat merge-2009-08-28..v2.6.31-rc8

Hah, well this makes no sense:

mcgrof@tux ~/wireless-testing (git::(no branch))$ git bisect bad
a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b is first bad commit
commit a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b
Author: John W. Linville <linville@tuxdriver.com>
Date:   Wed Feb 27 16:04:18 2008 -0500

    Add localversion-wireless to identify builds from this tree.

    Signed-off-by: John W. Linville <linville@tuxdriver.com>

:000000 100644 0000000000000000000000000000000000000000
6a05d60db3b21d9c0a0b93b831c6ea453dc98785 A	localversion-wireless

I'll try a fresh branch on merge-2009-08-28 ..

  Luis

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 10:50 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Mon, Aug 31, 2009 at 1:23 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>> On Fri, Aug 28, 2009 at 3:09 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>> On Fri, Aug 28, 2009 at 2:50 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>> On Fri, Aug 28, 2009 at 9:52 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>>> On Fri, Aug 28, 2009 at 9:32 AM, Catalin Marinas<catalin.marinas@arm.com> wrote:
>>>>>> "Luis R. Rodriguez" <mcgrof@gmail.com> wrote:
>>>>>>> I have an assorted collection of kmemleak reports for acpi, ext4 and
>>>>>>> tty, not sure how to read these yet to fix so figure I'd at least post
>>>>>>> them. To reproduce I can just dd=/dev/zero to some big file and played
>>>>>>> some video.
>>>>>>
>>>>>> If you do a few echo scan > /sys/kernel/debug/kmemleak, do they
>>>>>> disappear (i.e. transient false positives)?
>>>>>
>>>>> Sure, I will once on rc8.
>>>>>
>>>>>> Which kernel version is this?
>>>>>
>>>>> v2.6.31-rc7-33172-gf4a9f9a
>>>>>
>>>>> This is from wireless-testing, which has wireless patches on top of
>>>>> rc7. John just rebased to rc8 so will give that a shot at work.
>>>>>
>>>>>>> unreferenced object 0xffff88003e0015c0 (size 64):
>>>>>>>   comm "swapper", pid 1, jiffies 4294892352
>>>>>>>   backtrace:
>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>     [<ffffffff81118a03>] kmem_cache_alloc_node+0x193/0x200
>>>>>>>     [<ffffffff8152509e>] process_zones+0x70/0x1cd
>>>>>>>     [<ffffffff81525230>] pageset_cpuup_callback+0x35/0x92
>>>>>>>     [<ffffffff8152c9b7>] notifier_call_chain+0x47/0x90
>>>>>>>     [<ffffffff81078549>] __raw_notifier_call_chain+0x9/0x10
>>>>>>>     [<ffffffff81523f25>] _cpu_up+0x75/0x130
>>>>>>>     [<ffffffff8152403a>] cpu_up+0x5a/0x6a
>>>>>>>     [<ffffffff8181969e>] kernel_init+0xcc/0x1ba
>>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>
>>>>>> Can't really tell. Maybe a false positive caused by kmemleak not
>>>>>> scanning the pgdata node_zones. Can you post your .config file?
>>>>>
>>>>> Sure, attached.
>>>>>
>>>>>>> unreferenced object 0xffff88003cb5f700 (size 64):
>>>>>>>   comm "swapper", pid 1, jiffies 4294892459
>>>>>>>   backtrace:
>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>>     [<ffffffff812bb549>] kzalloc+0xf/0x11
>>>>>>>     [<ffffffff812bbb53>] acpi_add_single_object+0x58e/0xd3c
>>>>>>>     [<ffffffff812bc51c>] acpi_bus_scan+0x125/0x1af
>>>>>>>     [<ffffffff81842361>] acpi_scan_init+0xc8/0xe9
>>>>>>>     [<ffffffff8184211c>] acpi_init+0x21f/0x265
>>>>>>>     [<ffffffff8100a05b>] do_one_initcall+0x4b/0x1b0
>>>>>>>     [<ffffffff81819736>] kernel_init+0x164/0x1ba
>>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>
>>>>>> I get ACPI reports as well and they may be real leaks. However, I
>>>>>> didn't have time to analyse the code (pretty complicated reference
>>>>>> counting).
>>>>>
>>>>> Heh OK thanks for reviewing them though.
>>>>>
>>>>>>> unreferenced object 0xffff880039571800 (size 1024):
>>>>>>>   comm "exe", pid 1168, jiffies 4294893410
>>>>>>>   backtrace:
>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>>     [<ffffffff811e1d71>] ext4_mb_init+0x1a1/0x590
>>>>>>>     [<ffffffff811d2da3>] ext4_fill_super+0x1df3/0x26c0
>>>>>>>     [<ffffffff8112774f>] get_sb_bdev+0x16f/0x1b0
>>>>>>>     [<ffffffff811c8fd3>] ext4_get_sb+0x13/0x20
>>>>>>>     [<ffffffff81127216>] vfs_kern_mount+0x76/0x180
>>>>>>>     [<ffffffff8112738d>] do_kern_mount+0x4d/0x130
>>>>>>>     [<ffffffff8113fc57>] do_mount+0x307/0x8b0
>>>>>>>     [<ffffffff8114028f>] sys_mount+0x8f/0xe0
>>>>>>>     [<ffffffff81011f02>] system_call_fastpath+0x16/0x1b
>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>
>>>>>> The ext4 reports are real leaks and patch was posted here -
>>>>>> http://lkml.org/lkml/2009/7/15/62. However, it hasn't been merged into
>>>>>> mainline yet (I cc'ed Aneesh).
>>>>>>
>>>>>> The patch is merged in my "kmemleak-fixes" branch on
>>>>>> git://linux-arm.org/linux-2.6.git.
>>>>>
>>>>> Will try to suck them out and try them.
>>>>
>>>> OK -- tested rc8 + a pull of your tree into mine. The bootup was
>>>> really slow and something was just not going right. After a while
>>>> memleak complained it had 8 kmemleak logs but I was not able to get my
>>>> system usable enough to cat the file.
>>>>
>>>> In cases like these I wish I would hookup my ctrl-alt-del to kexec() a
>>>> safe kernel.
>>>>
>>>> After a long period of time it seems X wished it would start, it tried
>>>> and then flashed back to the tty. This kept repeating in a loop.
>>>>
>>>> I am not sure if the culprit was rc8 or the kmemleak branch merge --
>>>> I'll find out after I boot into rc8 in a few.
>>>
>>> rc8 busted my bootup, the issues are present with just
>>> wireless-testing. I highly doubt the issues are wireless-testing
>>> related so I will not bisect there. Since I am unable to get anything
>>> useful from the kernel to determine what may have gone sour, any
>>> suggestions on a path to bisect, or should I just do the whole tree?
>>
>> I tried 2.6.31-rc8 from hpa's linux-2.6-allstable.git tree instead of
>> Linus [1] as I already had that tree, git describe says:
>>
>> v2.6.31-rc8-15-gadda766
>>
>> Testing this would be the same as testing Linus' blessed rc8 --
>> correct me I'm wrong. Contrary to what I expected this tree with the
>> same config works well!
>>
>> I have compiled a fresh checkout of wireless-testing origin/master to
>> double check the issue and it is indeed only present on
>> wireless-testing. A diff stat between John's merge of 2.6.31-rc8 and
>> current master branch on wireless-testing [2] doesn't reveal much
>> other than wireless specific stuff, as expected, so it seems this may
>> after all be introduced in a recent patches in wireless-testing. I
>> still find this a bit odd given I see no others reporting major
>> issues. My boot doesn't go very far, it stalls for a while after input
>> devices are being detected, then it spits out a kmemleak warning about
>> 13 kmemleaks. Here's a picture [3]. I didn't bother waiting as I did
>> last time for X to try to come up, something is really wrong. I'll
>> bisect wireless-testing in the morning, starting with a good marker at
>> merge-2009-08-28 as that is when John pulled 2.6.31-rc8 (and I confirm
>> a diff stat between that and v2.6.31-rc8 yields nothing as it should)
>> and current master as the bad marker. I have 9 steps to go, will leave
>> first step compiling overnight.
>>
>> [1] git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-allstable.git
>> [2] git diff --stat merge-2009-08-28..HEAD
>> [3] http://bombadil.infradead.org/~mcgrof/images/2009/08/lag-wl-2009-08-31.jpg
>> [4] git diff --stat merge-2009-08-28..v2.6.31-rc8
>
> Hah, well this makes no sense:
>
> mcgrof@tux ~/wireless-testing (git::(no branch))$ git bisect bad
> a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b is first bad commit
> commit a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b
> Author: John W. Linville <linville@tuxdriver.com>
> Date:   Wed Feb 27 16:04:18 2008 -0500
>
>    Add localversion-wireless to identify builds from this tree.
>
>    Signed-off-by: John W. Linville <linville@tuxdriver.com>
>
> :000000 100644 0000000000000000000000000000000000000000
> 6a05d60db3b21d9c0a0b93b831c6ea453dc98785 A      localversion-wireless
>
> I'll try a fresh branch on merge-2009-08-28 ..

OK I tried this, I even 'rm -rf * ; git checkout -f' and ..
merge-2009-08-28 tag yields the same issues, long lag upon bootup with
some kmemleaks I cannot even get to check. So somehow something is
different between merge-2009-08-28 and Linus' rc8. This is just
bizarre so to be even safer I'm just going to do a fresh git clone on
wireless-testing.

  Luis

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 11:37 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Mon, Aug 31, 2009 at 10:50 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>> On Mon, Aug 31, 2009 at 1:23 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>> On Fri, Aug 28, 2009 at 3:09 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>> On Fri, Aug 28, 2009 at 2:50 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>>> On Fri, Aug 28, 2009 at 9:52 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>>>>> On Fri, Aug 28, 2009 at 9:32 AM, Catalin Marinas<catalin.marinas@arm.com> wrote:
>>>>>>> "Luis R. Rodriguez" <mcgrof@gmail.com> wrote:
>>>>>>>> I have an assorted collection of kmemleak reports for acpi, ext4 and
>>>>>>>> tty, not sure how to read these yet to fix so figure I'd at least post
>>>>>>>> them. To reproduce I can just dd=/dev/zero to some big file and played
>>>>>>>> some video.
>>>>>>>
>>>>>>> If you do a few echo scan > /sys/kernel/debug/kmemleak, do they
>>>>>>> disappear (i.e. transient false positives)?
>>>>>>
>>>>>> Sure, I will once on rc8.
>>>>>>
>>>>>>> Which kernel version is this?
>>>>>>
>>>>>> v2.6.31-rc7-33172-gf4a9f9a
>>>>>>
>>>>>> This is from wireless-testing, which has wireless patches on top of
>>>>>> rc7. John just rebased to rc8 so will give that a shot at work.
>>>>>>
>>>>>>>> unreferenced object 0xffff88003e0015c0 (size 64):
>>>>>>>>   comm "swapper", pid 1, jiffies 4294892352
>>>>>>>>   backtrace:
>>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>>     [<ffffffff81118a03>] kmem_cache_alloc_node+0x193/0x200
>>>>>>>>     [<ffffffff8152509e>] process_zones+0x70/0x1cd
>>>>>>>>     [<ffffffff81525230>] pageset_cpuup_callback+0x35/0x92
>>>>>>>>     [<ffffffff8152c9b7>] notifier_call_chain+0x47/0x90
>>>>>>>>     [<ffffffff81078549>] __raw_notifier_call_chain+0x9/0x10
>>>>>>>>     [<ffffffff81523f25>] _cpu_up+0x75/0x130
>>>>>>>>     [<ffffffff8152403a>] cpu_up+0x5a/0x6a
>>>>>>>>     [<ffffffff8181969e>] kernel_init+0xcc/0x1ba
>>>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>>
>>>>>>> Can't really tell. Maybe a false positive caused by kmemleak not
>>>>>>> scanning the pgdata node_zones. Can you post your .config file?
>>>>>>
>>>>>> Sure, attached.
>>>>>>
>>>>>>>> unreferenced object 0xffff88003cb5f700 (size 64):
>>>>>>>>   comm "swapper", pid 1, jiffies 4294892459
>>>>>>>>   backtrace:
>>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>>>     [<ffffffff812bb549>] kzalloc+0xf/0x11
>>>>>>>>     [<ffffffff812bbb53>] acpi_add_single_object+0x58e/0xd3c
>>>>>>>>     [<ffffffff812bc51c>] acpi_bus_scan+0x125/0x1af
>>>>>>>>     [<ffffffff81842361>] acpi_scan_init+0xc8/0xe9
>>>>>>>>     [<ffffffff8184211c>] acpi_init+0x21f/0x265
>>>>>>>>     [<ffffffff8100a05b>] do_one_initcall+0x4b/0x1b0
>>>>>>>>     [<ffffffff81819736>] kernel_init+0x164/0x1ba
>>>>>>>>     [<ffffffff810130ca>] child_rip+0xa/0x20
>>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>>
>>>>>>> I get ACPI reports as well and they may be real leaks. However, I
>>>>>>> didn't have time to analyse the code (pretty complicated reference
>>>>>>> counting).
>>>>>>
>>>>>> Heh OK thanks for reviewing them though.
>>>>>>
>>>>>>>> unreferenced object 0xffff880039571800 (size 1024):
>>>>>>>>   comm "exe", pid 1168, jiffies 4294893410
>>>>>>>>   backtrace:
>>>>>>>>     [<ffffffff81121fad>] create_object+0x13d/0x2d0
>>>>>>>>     [<ffffffff81122265>] kmemleak_alloc+0x25/0x60
>>>>>>>>     [<ffffffff81119f3b>] __kmalloc+0x16b/0x250
>>>>>>>>     [<ffffffff811e1d71>] ext4_mb_init+0x1a1/0x590
>>>>>>>>     [<ffffffff811d2da3>] ext4_fill_super+0x1df3/0x26c0
>>>>>>>>     [<ffffffff8112774f>] get_sb_bdev+0x16f/0x1b0
>>>>>>>>     [<ffffffff811c8fd3>] ext4_get_sb+0x13/0x20
>>>>>>>>     [<ffffffff81127216>] vfs_kern_mount+0x76/0x180
>>>>>>>>     [<ffffffff8112738d>] do_kern_mount+0x4d/0x130
>>>>>>>>     [<ffffffff8113fc57>] do_mount+0x307/0x8b0
>>>>>>>>     [<ffffffff8114028f>] sys_mount+0x8f/0xe0
>>>>>>>>     [<ffffffff81011f02>] system_call_fastpath+0x16/0x1b
>>>>>>>>     [<ffffffffffffffff>] 0xffffffffffffffff
>>>>>>>
>>>>>>> The ext4 reports are real leaks and patch was posted here -
>>>>>>> http://lkml.org/lkml/2009/7/15/62. However, it hasn't been merged into
>>>>>>> mainline yet (I cc'ed Aneesh).
>>>>>>>
>>>>>>> The patch is merged in my "kmemleak-fixes" branch on
>>>>>>> git://linux-arm.org/linux-2.6.git.
>>>>>>
>>>>>> Will try to suck them out and try them.
>>>>>
>>>>> OK -- tested rc8 + a pull of your tree into mine. The bootup was
>>>>> really slow and something was just not going right. After a while
>>>>> memleak complained it had 8 kmemleak logs but I was not able to get my
>>>>> system usable enough to cat the file.
>>>>>
>>>>> In cases like these I wish I would hookup my ctrl-alt-del to kexec() a
>>>>> safe kernel.
>>>>>
>>>>> After a long period of time it seems X wished it would start, it tried
>>>>> and then flashed back to the tty. This kept repeating in a loop.
>>>>>
>>>>> I am not sure if the culprit was rc8 or the kmemleak branch merge --
>>>>> I'll find out after I boot into rc8 in a few.
>>>>
>>>> rc8 busted my bootup, the issues are present with just
>>>> wireless-testing. I highly doubt the issues are wireless-testing
>>>> related so I will not bisect there. Since I am unable to get anything
>>>> useful from the kernel to determine what may have gone sour, any
>>>> suggestions on a path to bisect, or should I just do the whole tree?
>>>
>>> I tried 2.6.31-rc8 from hpa's linux-2.6-allstable.git tree instead of
>>> Linus [1] as I already had that tree, git describe says:
>>>
>>> v2.6.31-rc8-15-gadda766
>>>
>>> Testing this would be the same as testing Linus' blessed rc8 --
>>> correct me I'm wrong. Contrary to what I expected this tree with the
>>> same config works well!
>>>
>>> I have compiled a fresh checkout of wireless-testing origin/master to
>>> double check the issue and it is indeed only present on
>>> wireless-testing. A diff stat between John's merge of 2.6.31-rc8 and
>>> current master branch on wireless-testing [2] doesn't reveal much
>>> other than wireless specific stuff, as expected, so it seems this may
>>> after all be introduced in a recent patches in wireless-testing. I
>>> still find this a bit odd given I see no others reporting major
>>> issues. My boot doesn't go very far, it stalls for a while after input
>>> devices are being detected, then it spits out a kmemleak warning about
>>> 13 kmemleaks. Here's a picture [3]. I didn't bother waiting as I did
>>> last time for X to try to come up, something is really wrong. I'll
>>> bisect wireless-testing in the morning, starting with a good marker at
>>> merge-2009-08-28 as that is when John pulled 2.6.31-rc8 (and I confirm
>>> a diff stat between that and v2.6.31-rc8 yields nothing as it should)
>>> and current master as the bad marker. I have 9 steps to go, will leave
>>> first step compiling overnight.
>>>
>>> [1] git://git.kernel.org/pub/scm/linux/kernel/git/hpa/linux-2.6-allstable.git
>>> [2] git diff --stat merge-2009-08-28..HEAD
>>> [3] http://bombadil.infradead.org/~mcgrof/images/2009/08/lag-wl-2009-08-31.jpg
>>> [4] git diff --stat merge-2009-08-28..v2.6.31-rc8
>>
>> Hah, well this makes no sense:
>>
>> mcgrof@tux ~/wireless-testing (git::(no branch))$ git bisect bad
>> a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b is first bad commit
>> commit a4e774ca75e5f2d8347b4d9746a2e0a9a4fc521b
>> Author: John W. Linville <linville@tuxdriver.com>
>> Date:   Wed Feb 27 16:04:18 2008 -0500
>>
>>    Add localversion-wireless to identify builds from this tree.
>>
>>    Signed-off-by: John W. Linville <linville@tuxdriver.com>
>>
>> :000000 100644 0000000000000000000000000000000000000000
>> 6a05d60db3b21d9c0a0b93b831c6ea453dc98785 A      localversion-wireless
>>
>> I'll try a fresh branch on merge-2009-08-28 ..
>
> OK I tried this, I even 'rm -rf * ; git checkout -f' and ..
> merge-2009-08-28 tag yields the same issues, long lag upon bootup with
> some kmemleaks I cannot even get to check. So somehow something is
> different between merge-2009-08-28 and Linus' rc8. This is just
> bizarre so to be even safer I'm just going to do a fresh git clone on
> wireless-testing.

Hey John so I tested wireless-testing on the merge-2009-08-28 tag on a
fresh git pull and verified this is indeed busted for me. Although I
had tried hpa's linux-2.6-allstable on HEAD just to be sure I am now
building Linus' tree from a fresh git clone on the v2.6.31-rc8 tag
just to be double check this was indeed not a 2.6.31-rc8 issue but
instead *something* on wireless-testing. What that something is is
unclear to me still, I guess after all these tests I'll run a manual
diff as git doesn't seem to be picking anything up.

  Luis

Re: memleaks, acpi + ext4 + tty

[John W. Linville]

On Mon, Aug 31, 2009 at 12:30:15PM -0700, Luis R. Rodriguez wrote:
> On Mon, Aug 31, 2009 at 11:37 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:

> > OK I tried this, I even 'rm -rf * ; git checkout -f' and ..
> > merge-2009-08-28 tag yields the same issues, long lag upon bootup with
> > some kmemleaks I cannot even get to check. So somehow something is
> > different between merge-2009-08-28 and Linus' rc8. This is just
> > bizarre so to be even safer I'm just going to do a fresh git clone on
> > wireless-testing.
> 
> Hey John so I tested wireless-testing on the merge-2009-08-28 tag on a
> fresh git pull and verified this is indeed busted for me. Although I
> had tried hpa's linux-2.6-allstable on HEAD just to be sure I am now
> building Linus' tree from a fresh git clone on the v2.6.31-rc8 tag
> just to be double check this was indeed not a 2.6.31-rc8 issue but
> instead *something* on wireless-testing. What that something is is
> unclear to me still, I guess after all these tests I'll run a manual
> diff as git doesn't seem to be picking anything up.

As you determined, there is no difference between merge-2009-08-28
and v2.6.31-rc8.  If one works and the other doesn't, then the main
thing not working will be git...

What is linux-2.6-allstable?  My guess is that the culprit is between
v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 12:33 PM, John W.
Linville<linville@tuxdriver.com> wrote:
> On Mon, Aug 31, 2009 at 12:30:15PM -0700, Luis R. Rodriguez wrote:
>> On Mon, Aug 31, 2009 at 11:37 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>
>> > OK I tried this, I even 'rm -rf * ; git checkout -f' and ..
>> > merge-2009-08-28 tag yields the same issues, long lag upon bootup with
>> > some kmemleaks I cannot even get to check. So somehow something is
>> > different between merge-2009-08-28 and Linus' rc8. This is just
>> > bizarre so to be even safer I'm just going to do a fresh git clone on
>> > wireless-testing.
>>
>> Hey John so I tested wireless-testing on the merge-2009-08-28 tag on a
>> fresh git pull and verified this is indeed busted for me. Although I
>> had tried hpa's linux-2.6-allstable on HEAD just to be sure I am now
>> building Linus' tree from a fresh git clone on the v2.6.31-rc8 tag
>> just to be double check this was indeed not a 2.6.31-rc8 issue but
>> instead *something* on wireless-testing. What that something is is
>> unclear to me still, I guess after all these tests I'll run a manual
>> diff as git doesn't seem to be picking anything up.
>
> As you determined, there is no difference between merge-2009-08-28
> and v2.6.31-rc8.  If one works and the other doesn't, then the main
> thing not working will be git...

Hm that would suck big time. At any rate, even if it was git
wireless-testing seems to have an issue right now. I'll confirm
shortly with Linus' own tree.

> What is linux-2.6-allstable?

hpa's collection of 2.6 stable kernels, including extraversion stuff.

> My guess is that the culprit is between
> v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.

But the issue is *not* in linux-2.6-allstable, it is only present on
wireless-testing, and I went as far back as merge-2009-08-28. I was
using whatever tag was available prior to you moving to rc8, and that
worked.

  Luis

Re: memleaks, acpi + ext4 + tty

[John W. Linville]

On Mon, Aug 31, 2009 at 12:43:01PM -0700, Luis R. Rodriguez wrote:
> On Mon, Aug 31, 2009 at 12:33 PM, John W.
> Linville<linville@tuxdriver.com> wrote:

> > My guess is that the culprit is between
> > v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
> 
> But the issue is *not* in linux-2.6-allstable, it is only present on
> wireless-testing, and I went as far back as merge-2009-08-28. I was
> using whatever tag was available prior to you moving to rc8, and that
> worked.

OK, then I can only guess that something went wrong in your bisection
(e.g. dirty build, something not marked properly, etc).  That, or
different .config files between your linux-2.6-allstable build and
the merge-2009-08-28 build of wireless-testing?

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 12:47 PM, John W.
Linville<linville@tuxdriver.com> wrote:
> On Mon, Aug 31, 2009 at 12:43:01PM -0700, Luis R. Rodriguez wrote:
>> On Mon, Aug 31, 2009 at 12:33 PM, John W.
>> Linville<linville@tuxdriver.com> wrote:
>
>> > My guess is that the culprit is between
>> > v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
>>
>> But the issue is *not* in linux-2.6-allstable, it is only present on
>> wireless-testing, and I went as far back as merge-2009-08-28. I was
>> using whatever tag was available prior to you moving to rc8, and that
>> worked.
>
> OK, then I can only guess that something went wrong in your bisection
> (e.g. dirty build, something not marked properly, etc).

Well so remember I did a clean git clone of wireless-testing and the
issue was still there, but I determined just now it seems rc8 from
Linus does indeed also have the issue. For some reason the issue was
not present on hpa's rc8 on linux-2.6-allstable. Will double check on
that again.

>  That, or
> different .config files between your linux-2.6-allstable build and
> the merge-2009-08-28 build of wireless-testing?

I ensured to diff configs them between builds :T

  Luis

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 1:02 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Mon, Aug 31, 2009 at 12:47 PM, John W.
> Linville<linville@tuxdriver.com> wrote:
>> On Mon, Aug 31, 2009 at 12:43:01PM -0700, Luis R. Rodriguez wrote:
>>> On Mon, Aug 31, 2009 at 12:33 PM, John W.
>>> Linville<linville@tuxdriver.com> wrote:
>>
>>> > My guess is that the culprit is between
>>> > v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
>>>
>>> But the issue is *not* in linux-2.6-allstable, it is only present on
>>> wireless-testing, and I went as far back as merge-2009-08-28. I was
>>> using whatever tag was available prior to you moving to rc8, and that
>>> worked.
>>
>> OK, then I can only guess that something went wrong in your bisection
>> (e.g. dirty build, something not marked properly, etc).
>
> Well so remember I did a clean git clone of wireless-testing and the
> issue was still there, but I determined just now it seems rc8 from
> Linus does indeed also have the issue. For some reason the issue was
> not present on hpa's rc8 on linux-2.6-allstable. Will double check on
> that again.

OK so hpa's tree just had all of Linus' stuff pulled, not just rc8. It
turns out Eric Paris' inotify patches are required to fix 2.6.31-rc8
for me.. I tested just one patch and it fixed my bootup lag:

http://git.infradead.org/users/eparis/notify.git/commit/b962e7312ae87006aed6f68ceee94bdf8db08338

FWIW for that patch:

Tested-by: Luis R. Rodriguez <lrodriguez@atheros.com>

So -- you if you're like me and had issues with bootup lag on
wireless-testing, you can probably fix your wireless-testing by
pulling his patches:

git pull git://git.infradead.org/users/eparis/notify.git for-linus

I saw Linus had some other fixes but I'll wait for rc9 for that as my
box seems reasonably stable right now.

  Luis

Re: memleaks, acpi + ext4 + tty

[Eric Paris]

On Mon, 2009-08-31 at 16:54 -0700, Luis R. Rodriguez wrote:
> On Mon, Aug 31, 2009 at 1:02 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> > On Mon, Aug 31, 2009 at 12:47 PM, John W.
> > Linville<linville@tuxdriver.com> wrote:
> >> On Mon, Aug 31, 2009 at 12:43:01PM -0700, Luis R. Rodriguez wrote:
> >>> On Mon, Aug 31, 2009 at 12:33 PM, John W.
> >>> Linville<linville@tuxdriver.com> wrote:
> >>
> >>> > My guess is that the culprit is between
> >>> > v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
> >>>
> >>> But the issue is *not* in linux-2.6-allstable, it is only present on
> >>> wireless-testing, and I went as far back as merge-2009-08-28. I was
> >>> using whatever tag was available prior to you moving to rc8, and that
> >>> worked.
> >>
> >> OK, then I can only guess that something went wrong in your bisection
> >> (e.g. dirty build, something not marked properly, etc).
> >
> > Well so remember I did a clean git clone of wireless-testing and the
> > issue was still there, but I determined just now it seems rc8 from
> > Linus does indeed also have the issue. For some reason the issue was
> > not present on hpa's rc8 on linux-2.6-allstable. Will double check on
> > that again.
> 
> OK so hpa's tree just had all of Linus' stuff pulled, not just rc8. It
> turns out Eric Paris' inotify patches are required to fix 2.6.31-rc8
> for me.. I tested just one patch and it fixed my bootup lag:
> 
> http://git.infradead.org/users/eparis/notify.git/commit/b962e7312ae87006aed6f68ceee94bdf8db08338
> 
> FWIW for that patch:
> 
> Tested-by: Luis R. Rodriguez <lrodriguez@atheros.com>
> 
> So -- you if you're like me and had issues with bootup lag on
> wireless-testing, you can probably fix your wireless-testing by
> pulling his patches:
> 
> git pull git://git.infradead.org/users/eparis/notify.git for-linus
> 
> I saw Linus had some other fixes but I'll wait for rc9 for that as my
> box seems reasonably stable right now.

Yes, -rc8 broke pretty badly for a number of people.  Linus did pull all
of the fixes that I know of.  I wouldn't suggest pulling just that one
commit.  All 3 of the post -rc8 patches in my tree fix -rc8
regressions   :(

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 5:26 PM, Eric Paris<eparis@redhat.com> wrote:
> On Mon, 2009-08-31 at 16:54 -0700, Luis R. Rodriguez wrote:
>> On Mon, Aug 31, 2009 at 1:02 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>> > On Mon, Aug 31, 2009 at 12:47 PM, John W.
>> > Linville<linville@tuxdriver.com> wrote:
>> >> On Mon, Aug 31, 2009 at 12:43:01PM -0700, Luis R. Rodriguez wrote:
>> >>> On Mon, Aug 31, 2009 at 12:33 PM, John W.
>> >>> Linville<linville@tuxdriver.com> wrote:
>> >>
>> >>> > My guess is that the culprit is between
>> >>> > v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
>> >>>
>> >>> But the issue is *not* in linux-2.6-allstable, it is only present on
>> >>> wireless-testing, and I went as far back as merge-2009-08-28. I was
>> >>> using whatever tag was available prior to you moving to rc8, and that
>> >>> worked.
>> >>
>> >> OK, then I can only guess that something went wrong in your bisection
>> >> (e.g. dirty build, something not marked properly, etc).
>> >
>> > Well so remember I did a clean git clone of wireless-testing and the
>> > issue was still there, but I determined just now it seems rc8 from
>> > Linus does indeed also have the issue. For some reason the issue was
>> > not present on hpa's rc8 on linux-2.6-allstable. Will double check on
>> > that again.
>>
>> OK so hpa's tree just had all of Linus' stuff pulled, not just rc8. It
>> turns out Eric Paris' inotify patches are required to fix 2.6.31-rc8
>> for me.. I tested just one patch and it fixed my bootup lag:
>>
>> http://git.infradead.org/users/eparis/notify.git/commit/b962e7312ae87006aed6f68ceee94bdf8db08338
>>
>> FWIW for that patch:
>>
>> Tested-by: Luis R. Rodriguez <lrodriguez@atheros.com>
>>
>> So -- you if you're like me and had issues with bootup lag on
>> wireless-testing, you can probably fix your wireless-testing by
>> pulling his patches:
>>
>> git pull git://git.infradead.org/users/eparis/notify.git for-linus
>>
>> I saw Linus had some other fixes but I'll wait for rc9 for that as my
>> box seems reasonably stable right now.
>
> Yes, -rc8 broke pretty badly for a number of people.  Linus did pull all
> of the fixes that I know of.  I wouldn't suggest pulling just that one
> commit.  All 3 of the post -rc8 patches in my tree fix -rc8
> regressions   :(

Thanks, the note -- so I guess best is to just pull from Linus ontop
of wireless-testing.

 Luis

Re: memleaks, acpi + ext4 + tty

[Zhu Yi]

On Tue, 2009-09-01 at 08:31 +0800, Luis R. Rodriguez wrote:
> >>
> http://git.infradead.org/users/eparis/notify.git/commit/b962e7312ae87006aed6f68ceee94bdf8db08338
> >>
> >> FWIW for that patch:
> >>
> >> Tested-by: Luis R. Rodriguez <lrodriguez@atheros.com>
> >>
> >> So -- you if you're like me and had issues with bootup lag on
> >> wireless-testing, you can probably fix your wireless-testing by
> >> pulling his patches:
> >>
> >> git pull git://git.infradead.org/users/eparis/notify.git for-linus
> >>
> >> I saw Linus had some other fixes but I'll wait for rc9 for that as
> my
> >> box seems reasonably stable right now.
> >
> > Yes, -rc8 broke pretty badly for a number of people.  Linus did pull
> all
> > of the fixes that I know of.  I wouldn't suggest pulling just that
> one
> > commit.  All 3 of the post -rc8 patches in my tree fix -rc8
> > regressions   :(
> 
> Thanks, the note -- so I guess best is to just pull from Linus ontop
> of wireless-testing.

I confirm I'm also suffered with the problem on today's wireless-testing
tip (udevadm --settle hangs on boot) and the above patch does fix the
issue for me. Thanks both for identify and fix the problem.

Thanks,
-yi

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 11:33 PM, Zhu Yi<yi.zhu@intel.com> wrote:
> On Tue, 2009-09-01 at 08:31 +0800, Luis R. Rodriguez wrote:
>> >>
>> http://git.infradead.org/users/eparis/notify.git/commit/b962e7312ae87006aed6f68ceee94bdf8db08338
>> >>
>> >> FWIW for that patch:
>> >>
>> >> Tested-by: Luis R. Rodriguez <lrodriguez@atheros.com>
>> >>
>> >> So -- you if you're like me and had issues with bootup lag on
>> >> wireless-testing, you can probably fix your wireless-testing by
>> >> pulling his patches:
>> >>
>> >> git pull git://git.infradead.org/users/eparis/notify.git for-linus
>> >>
>> >> I saw Linus had some other fixes but I'll wait for rc9 for that as
>> my
>> >> box seems reasonably stable right now.
>> >
>> > Yes, -rc8 broke pretty badly for a number of people.  Linus did pull
>> all
>> > of the fixes that I know of.  I wouldn't suggest pulling just that
>> one
>> > commit.  All 3 of the post -rc8 patches in my tree fix -rc8
>> > regressions   :(
>>
>> Thanks, the note -- so I guess best is to just pull from Linus ontop
>> of wireless-testing.
>
> I confirm I'm also suffered with the problem on today's wireless-testing
> tip (udevadm --settle hangs on boot) and the above patch does fix the
> issue for me. Thanks both for identify and fix the problem.

Just a heads up -- the relevant fixes are now merged on
wireless-testing, thanks John.

  Luis

Re: memleaks, acpi + ext4 + tty

[Luis R. Rodriguez]

On Mon, Aug 31, 2009 at 12:43 PM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
> On Mon, Aug 31, 2009 at 12:33 PM, John W.
> Linville<linville@tuxdriver.com> wrote:
>> On Mon, Aug 31, 2009 at 12:30:15PM -0700, Luis R. Rodriguez wrote:
>>> On Mon, Aug 31, 2009 at 11:37 AM, Luis R. Rodriguez<mcgrof@gmail.com> wrote:
>>
>>> > OK I tried this, I even 'rm -rf * ; git checkout -f' and ..
>>> > merge-2009-08-28 tag yields the same issues, long lag upon bootup with
>>> > some kmemleaks I cannot even get to check. So somehow something is
>>> > different between merge-2009-08-28 and Linus' rc8. This is just
>>> > bizarre so to be even safer I'm just going to do a fresh git clone on
>>> > wireless-testing.
>>>
>>> Hey John so I tested wireless-testing on the merge-2009-08-28 tag on a
>>> fresh git pull and verified this is indeed busted for me. Although I
>>> had tried hpa's linux-2.6-allstable on HEAD just to be sure I am now
>>> building Linus' tree from a fresh git clone on the v2.6.31-rc8 tag
>>> just to be double check this was indeed not a 2.6.31-rc8 issue but
>>> instead *something* on wireless-testing. What that something is is
>>> unclear to me still, I guess after all these tests I'll run a manual
>>> diff as git doesn't seem to be picking anything up.
>>
>> As you determined, there is no difference between merge-2009-08-28
>> and v2.6.31-rc8.  If one works and the other doesn't, then the main
>> thing not working will be git...
>
> Hm that would suck big time. At any rate, even if it was git
> wireless-testing seems to have an issue right now. I'll confirm
> shortly with Linus' own tree.
>
>> What is linux-2.6-allstable?
>
> hpa's collection of 2.6 stable kernels, including extraversion stuff.
>
>> My guess is that the culprit is between
>> v2.6.31-rc8 and whatever is at the head of linux-2.6-allstable.
>
> But the issue is *not* in linux-2.6-allstable, it is only present on
> wireless-testing, and I went as far back as merge-2009-08-28. I was
> using whatever tag was available prior to you moving to rc8, and that
> worked.

Good news is Linus' rc8 tag *has the same issue* so the reason why
wireless-testing has it is rc8 *does indeed have it*.

So Peter, if there is something extra or missing from
linux-2.6-allstable head it certainly fixed my issues, will have to
try a new respin of it to ensure it wasn't a fluke. Odd thing too
though is even my distribution kernel (karmic rc8 generic) works fine.

At least now I can safely bisect between rc7 and rc8, or pull the
kmemleaks branch and see if that fixed.

  Luis

Re: memleaks, acpi + ext4 + tty

[H. Peter Anvin]

On 08/31/2009 12:58 PM, Luis R. Rodriguez wrote:
> 
> Good news is Linus' rc8 tag *has the same issue* so the reason why
> wireless-testing has it is rc8 *does indeed have it*.
> 
> So Peter, if there is something extra or missing from
> linux-2.6-allstable head it certainly fixed my issues, will have to
> try a new respin of it to ensure it wasn't a fluke. Odd thing too
> though is even my distribution kernel (karmic rc8 generic) works fine.
> 

linux-2.6-allstable head is == linus.

	-hpa