[RFC] CMA page migration failure due to buffers on bh_lru

[RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hello,

I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.

But I have many failures to allocate memory from CMA area.
I found the same situation to Laura Abbott's patch descrbing, https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.

If Joonsoo's patch is applied and/or CMA feature is applied more and more,
buffer-heads problem is going to be serious definitely.

Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hello,

I am trying to apply CMA feature for my platform.
My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
Now my platform can use CMA area effectively.

But I have many failures to allocate memory from CMA area.
I found the same situation to Laura Abbott's patch descrbing, https://lkml.org/lkml/2012/8/31/313,
that releases buffer-heads attached at CPU's LRU list.

If Joonsoo's patch is applied and/or CMA feature is applied more and more,
buffer-heads problem is going to be serious definitely.

Please look into the Laura's patch again.
I think it must be applied with Joonsoo's patch.

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Michal Nazarewicz]

On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
> Hello,
>
> I am trying to apply CMA feature for my platform.
> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
> Now my platform can use CMA area effectively.
>
> But I have many failures to allocate memory from CMA area.
> I found the same situation to Laura Abbott's patch descrbing,
> https://lkml.org/lkml/2012/8/31/313,
> that releases buffer-heads attached at CPU's LRU list.
>
> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
> buffer-heads problem is going to be serious definitely.
>
> Please look into the Laura's patch again.
> I think it must be applied with Joonsoo's patch.

Just to make sure I understood you correctly, you're saying Laura's
patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Michal Nazarewicz]

On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
> Hello,
>
> I am trying to apply CMA feature for my platform.
> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
> Now my platform can use CMA area effectively.
>
> But I have many failures to allocate memory from CMA area.
> I found the same situation to Laura Abbott's patch descrbing,
> https://lkml.org/lkml/2012/8/31/313,
> that releases buffer-heads attached at CPU's LRU list.
>
> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
> buffer-heads problem is going to be serious definitely.
>
> Please look into the Laura's patch again.
> I think it must be applied with Joonsoo's patch.

Just to make sure I understood you correctly, you're saying Laura's
patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +--<mpn@google.com>--<xmpp:mina86@jabber.org>--ooO--(_)--Ooo--

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

2014-06-27 i??i ? 12:57, Michal Nazarewicz i?' e,?:
> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>> Hello,
>>
>> I am trying to apply CMA feature for my platform.
>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>> Now my platform can use CMA area effectively.
>>
>> But I have many failures to allocate memory from CMA area.
>> I found the same situation to Laura Abbott's patch descrbing,
>> https://lkml.org/lkml/2012/8/31/313,
>> that releases buffer-heads attached at CPU's LRU list.
>>
>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>> buffer-heads problem is going to be serious definitely.
>>
>> Please look into the Laura's patch again.
>> I think it must be applied with Joonsoo's patch.
>
> Just to make sure I understood you correctly, you're saying Laura's
> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>

Yes, it is.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

2014-06-27 오전 12:57, Michal Nazarewicz 쓴 글:
> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>> Hello,
>>
>> I am trying to apply CMA feature for my platform.
>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>> Now my platform can use CMA area effectively.
>>
>> But I have many failures to allocate memory from CMA area.
>> I found the same situation to Laura Abbott's patch descrbing,
>> https://lkml.org/lkml/2012/8/31/313,
>> that releases buffer-heads attached at CPU's LRU list.
>>
>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>> buffer-heads problem is going to be serious definitely.
>>
>> Please look into the Laura's patch again.
>> I think it must be applied with Joonsoo's patch.
>
> Just to make sure I understood you correctly, you're saying Laura's
> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>

Yes, it is.

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Laura Abbott]

(cc-ing Hugh since he had comments on the patch before)

On 6/26/2014 4:23 PM, Gioh Kim wrote:
> 
> 
> 2014-06-27 i??i ? 12:57, Michal Nazarewicz i?' e,?:
>> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>>> Hello,
>>>
>>> I am trying to apply CMA feature for my platform.
>>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>>> Now my platform can use CMA area effectively.
>>>
>>> But I have many failures to allocate memory from CMA area.
>>> I found the same situation to Laura Abbott's patch descrbing,
>>> https://lkml.org/lkml/2012/8/31/313,
>>> that releases buffer-heads attached at CPU's LRU list.
>>>
>>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>>> buffer-heads problem is going to be serious definitely.
>>>
>>> Please look into the Laura's patch again.
>>> I think it must be applied with Joonsoo's patch.
>>
>> Just to make sure I understood you correctly, you're saying Laura's
>> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>>
> 
> Yes, it is.

I submitted this before and it was suggested that this was more
related to filesystems

http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2

I never saw more discussion and pushed this into the 'CMA hacks' pile. 
So far we've been keeping the patch out of tree and it's useful to know
that others have found the same problem. I'm willing to resubmit the
patch for further discussion.

Thanks,
Laura

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Laura Abbott]

(cc-ing Hugh since he had comments on the patch before)

On 6/26/2014 4:23 PM, Gioh Kim wrote:
> 
> 
> 2014-06-27 오전 12:57, Michal Nazarewicz 쓴 글:
>> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>>> Hello,
>>>
>>> I am trying to apply CMA feature for my platform.
>>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>>> Now my platform can use CMA area effectively.
>>>
>>> But I have many failures to allocate memory from CMA area.
>>> I found the same situation to Laura Abbott's patch descrbing,
>>> https://lkml.org/lkml/2012/8/31/313,
>>> that releases buffer-heads attached at CPU's LRU list.
>>>
>>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>>> buffer-heads problem is going to be serious definitely.
>>>
>>> Please look into the Laura's patch again.
>>> I think it must be applied with Joonsoo's patch.
>>
>> Just to make sure I understood you correctly, you're saying Laura's
>> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>>
> 
> Yes, it is.

I submitted this before and it was suggested that this was more
related to filesystems

http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2

I never saw more discussion and pushed this into the 'CMA hacks' pile. 
So far we've been keeping the patch out of tree and it's useful to know
that others have found the same problem. I'm willing to resubmit the
patch for further discussion.

Thanks,
Laura

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hi,Laura.

I have a question.

Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread.
But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.

Is my concern correct?


2014-06-30 i??i ? 4:49, Laura Abbott i?' e,?:
> (cc-ing Hugh since he had comments on the patch before)
>
> On 6/26/2014 4:23 PM, Gioh Kim wrote:
>>
>>
>> 2014-06-27 i??i ? 12:57, Michal Nazarewicz i?' e,?:
>>> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>>>> Hello,
>>>>
>>>> I am trying to apply CMA feature for my platform.
>>>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>>>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>>>> Now my platform can use CMA area effectively.
>>>>
>>>> But I have many failures to allocate memory from CMA area.
>>>> I found the same situation to Laura Abbott's patch descrbing,
>>>> https://lkml.org/lkml/2012/8/31/313,
>>>> that releases buffer-heads attached at CPU's LRU list.
>>>>
>>>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>>>> buffer-heads problem is going to be serious definitely.
>>>>
>>>> Please look into the Laura's patch again.
>>>> I think it must be applied with Joonsoo's patch.
>>>
>>> Just to make sure I understood you correctly, you're saying Laura's
>>> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>>>
>>
>> Yes, it is.
>
> I submitted this before and it was suggested that this was more
> related to filesystems
>
> http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2
>
> I never saw more discussion and pushed this into the 'CMA hacks' pile.
> So far we've been keeping the patch out of tree and it's useful to know
> that others have found the same problem. I'm willing to resubmit the
> patch for further discussion.
>
> Thanks,
> Laura
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hi,Laura.

I have a question.

Does the __evict_bh_lru() not need bh_lru_lock()?
The get_cpu_var() has already preenpt_disable() and can prevent other thread.
But get_cpu_var cannot prevent IRQ context such like page-fault.
I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.

Is my concern correct?


2014-06-30 오전 4:49, Laura Abbott 쓴 글:
> (cc-ing Hugh since he had comments on the patch before)
>
> On 6/26/2014 4:23 PM, Gioh Kim wrote:
>>
>>
>> 2014-06-27 오전 12:57, Michal Nazarewicz 쓴 글:
>>> On Tue, Jun 24 2014, Gioh Kim <gioh.kim@lge.com> wrote:
>>>> Hello,
>>>>
>>>> I am trying to apply CMA feature for my platform.
>>>> My kernel version, 3.10.x, is not allocating memory from CMA area so that I applied
>>>> a Joonsoo Kim's patch (https://lkml.org/lkml/2014/5/28/64).
>>>> Now my platform can use CMA area effectively.
>>>>
>>>> But I have many failures to allocate memory from CMA area.
>>>> I found the same situation to Laura Abbott's patch descrbing,
>>>> https://lkml.org/lkml/2012/8/31/313,
>>>> that releases buffer-heads attached at CPU's LRU list.
>>>>
>>>> If Joonsoo's patch is applied and/or CMA feature is applied more and more,
>>>> buffer-heads problem is going to be serious definitely.
>>>>
>>>> Please look into the Laura's patch again.
>>>> I think it must be applied with Joonsoo's patch.
>>>
>>> Just to make sure I understood you correctly, you're saying Laura's
>>> patch at <https://lkml.org/lkml/2012/8/31/313> fixes your issue?
>>>
>>
>> Yes, it is.
>
> I submitted this before and it was suggested that this was more
> related to filesystems
>
> http://marc.info/?l=linaro-mm-sig&m=137645770708817&w=2
>
> I never saw more discussion and pushed this into the 'CMA hacks' pile.
> So far we've been keeping the patch out of tree and it's useful to know
> that others have found the same problem. I'm willing to resubmit the
> patch for further discussion.
>
> Thanks,
> Laura
>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Laura Abbott]

On 6/30/2014 6:07 PM, Gioh Kim wrote:
> Hi,Laura.
> 
> I have a question.
> 
> Does the __evict_bh_lru() not need bh_lru_lock()?
> The get_cpu_var() has already preenpt_disable() and can prevent other thread.
> But get_cpu_var cannot prevent IRQ context such like page-fault.
> I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
> 
> Is my concern correct?
> 
> 

__evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
either. It's possible I missed something though.

Thanks,
Laura

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Laura Abbott]

On 6/30/2014 6:07 PM, Gioh Kim wrote:
> Hi,Laura.
> 
> I have a question.
> 
> Does the __evict_bh_lru() not need bh_lru_lock()?
> The get_cpu_var() has already preenpt_disable() and can prevent other thread.
> But get_cpu_var cannot prevent IRQ context such like page-fault.
> I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
> 
> Is my concern correct?
> 
> 

__evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
either. It's possible I missed something though.

Thanks,
Laura

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Andrew Morton]

On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <lauraa@codeaurora.org> wrote:

> On 6/30/2014 6:07 PM, Gioh Kim wrote:
> > Hi,Laura.
> > 
> > I have a question.
> > 
> > Does the __evict_bh_lru() not need bh_lru_lock()?
> > The get_cpu_var() has already preenpt_disable() and can prevent other thread.
> > But get_cpu_var cannot prevent IRQ context such like page-fault.
> > I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
> > 
> > Is my concern correct?
> > 
> > 
> 
> __evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
> I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
> either. It's possible I missed something though.

I fear that running on_each_cpu() within try_to_free_buffers() is going
to be horridly expensive in some cases.

Maybe CMA can use invalidate_bh_lrus() to shoot down everything before
trying the allocation attempt.  That should increase the success rate
greatly and doesn't burden page reclaim.  The bh LRU isn't terribly
important from a performance point of view, so emptying it occasionally
won't hurt.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Andrew Morton]

On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <lauraa@codeaurora.org> wrote:

> On 6/30/2014 6:07 PM, Gioh Kim wrote:
> > Hi,Laura.
> > 
> > I have a question.
> > 
> > Does the __evict_bh_lru() not need bh_lru_lock()?
> > The get_cpu_var() has already preenpt_disable() and can prevent other thread.
> > But get_cpu_var cannot prevent IRQ context such like page-fault.
> > I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
> > 
> > Is my concern correct?
> > 
> > 
> 
> __evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
> I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
> either. It's possible I missed something though.

I fear that running on_each_cpu() within try_to_free_buffers() is going
to be horridly expensive in some cases.

Maybe CMA can use invalidate_bh_lrus() to shoot down everything before
trying the allocation attempt.  That should increase the success rate
greatly and doesn't burden page reclaim.  The bh LRU isn't terribly
important from a performance point of view, so emptying it occasionally
won't hurt.

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hi, Laura,

I has replaced the evict_bh_lrus(bh) with invalidate_bh_lrus() and it is working fine.
How about submit new patch with invalidate_bh_lrus()?
I would appreciate it.



2014-07-02 i??i?? 2:46, Andrew Morton i?' e,?:
> On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <lauraa@codeaurora.org> wrote:
>
>> On 6/30/2014 6:07 PM, Gioh Kim wrote:
>>> Hi,Laura.
>>>
>>> I have a question.
>>>
>>> Does the __evict_bh_lru() not need bh_lru_lock()?
>>> The get_cpu_var() has already preenpt_disable() and can prevent other thread.
>>> But get_cpu_var cannot prevent IRQ context such like page-fault.
>>> I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
>>>
>>> Is my concern correct?
>>>
>>>
>>
>> __evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
>> I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
>> either. It's possible I missed something though.
>
> I fear that running on_each_cpu() within try_to_free_buffers() is going
> to be horridly expensive in some cases.
>
> Maybe CMA can use invalidate_bh_lrus() to shoot down everything before
> trying the allocation attempt.  That should increase the success rate
> greatly and doesn't burden page reclaim.  The bh LRU isn't terribly
> important from a performance point of view, so emptying it occasionally
> won't hurt.
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] CMA page migration failure due to buffers on bh_lru

[Gioh Kim]

Hi, Laura,

I has replaced the evict_bh_lrus(bh) with invalidate_bh_lrus() and it is working fine.
How about submit new patch with invalidate_bh_lrus()?
I would appreciate it.



2014-07-02 오후 2:46, Andrew Morton 쓴 글:
> On Mon, 30 Jun 2014 19:02:45 -0700 Laura Abbott <lauraa@codeaurora.org> wrote:
>
>> On 6/30/2014 6:07 PM, Gioh Kim wrote:
>>> Hi,Laura.
>>>
>>> I have a question.
>>>
>>> Does the __evict_bh_lru() not need bh_lru_lock()?
>>> The get_cpu_var() has already preenpt_disable() and can prevent other thread.
>>> But get_cpu_var cannot prevent IRQ context such like page-fault.
>>> I think if a page-fault occured and a file is read in IRQ context it can change cpu-lru.
>>>
>>> Is my concern correct?
>>>
>>>
>>
>> __evict_bh_lru is called via on_each_cpu_cond which I believe will disable irqs.
>> I based the code on the existing invalidate_bh_lru which did not take the bh_lru_lock
>> either. It's possible I missed something though.
>
> I fear that running on_each_cpu() within try_to_free_buffers() is going
> to be horridly expensive in some cases.
>
> Maybe CMA can use invalidate_bh_lrus() to shoot down everything before
> trying the allocation attempt.  That should increase the success rate
> greatly and doesn't burden page reclaim.  The bh LRU isn't terribly
> important from a performance point of view, so emptying it occasionally
> won't hurt.
>
>