* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 15:43 ` Tomas Winkler
@ 2008-06-12 16:35 ` Tomas Winkler
2008-06-12 17:05 ` Johannes Berg
2008-06-12 17:03 ` Johannes Berg
` (2 subsequent siblings)
3 siblings, 1 reply; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 16:35 UTC (permalink / raw)
To: Johannes Berg
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 6:43 PM, Tomas Winkler <tomasw@gmail.com> wrote:
> On Thu, Jun 12, 2008 at 4:54 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
>> On Thu, 2008-06-12 at 09:38 -0400, Rik van Riel wrote:
>>> On Thu, 12 Jun 2008 12:07:34 +0200
>>> "Zdenek Kabelac" <zdenek.kabelac@gmail.com> wrote:
>>>
>>> > It looks like while there was a huge amount of buffers and caches -
>>> > system was unable to allocate few pages for kmalloc in iwl3945 driver
>>> > after resume.
>>>
>>> It looks like this is because it wants to allocate 2**5 contiguous
>>> pages, which is 128kB of contiguous kernel memory.
>>
>> 64-bit system I assume?
>> The allocation should be 256 * 20 * sizeof(struct sk_buff *).
>
>> Try the patch below. It should improve code generation too.
>>
>> I discussed this with Tomas previously and he says the hw is capable of
>> doing 20 fragments per frame (wonder why, Broadcom can do an unlimited
>> number...) but he complained about the networking stack not being able
>> to.
>
> This is scatter gather buffers that can be kicked in one DMA transaction.
>
> Well, the hardware needs to support IP checksumming for that, hence,
>> afaik, only two fragments can ever be used (one for hw header, one for
>> frame)
> This I still don't understand why but everybody is already tired to
> explaining me why.. :) Just need to find time to dig into it.
>
>> This cuts the allocation to 10%, or (under) a page in all cases.
>
> Probably. it would be safe to use vmalloc for allocating txb anyway.
> I'll give it a try.
So vmalloc didn't break anything on the 32bit machine I'm just about
to install 64 one so it will take hour or two.. I will issue some
official patch after that.
> There was already discussion on LKML about memory allocation problems
> on X86_64, which might explain this regression. This didn't happen
> before.
This is the thread title if you are interested.
'x86/kernel/pci_dma.c: gfp |= __GFP_NORETRY'
Tomas
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 16:35 ` Tomas Winkler
@ 2008-06-12 17:05 ` Johannes Berg
2008-06-12 17:39 ` Tomas Winkler
0 siblings, 1 reply; 23+ messages in thread
From: Johannes Berg @ 2008-06-12 17:05 UTC (permalink / raw)
To: Tomas Winkler
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
[-- Attachment #1: Type: text/plain, Size: 760 bytes --]
> > Probably. it would be safe to use vmalloc for allocating txb anyway.
> > I'll give it a try.
>
> So vmalloc didn't break anything on the 32bit machine I'm just about
> to install 64 one so it will take hour or two.. I will issue some
> official patch after that.
Well, I disagree, and I'll push my patch as soon as somebody confirms
that it doesn't break anything.
> > There was already discussion on LKML about memory allocation problems
> > on X86_64, which might explain this regression. This didn't happen
> > before.
>
> This is the thread title if you are interested.
> 'x86/kernel/pci_dma.c: gfp |= __GFP_NORETRY'
Like I said, it doesn't matter, there's no need to _waste_
18*256*sizeof(void *) bytes memory.
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:05 ` Johannes Berg
@ 2008-06-12 17:39 ` Tomas Winkler
2008-06-12 17:46 ` Johannes Berg
0 siblings, 1 reply; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 17:39 UTC (permalink / raw)
To: Johannes Berg
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 8:05 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> > Probably. it would be safe to use vmalloc for allocating txb anyway.
>> > I'll give it a try.
>>
>> So vmalloc didn't break anything on the 32bit machine I'm just about
>> to install 64 one so it will take hour or two.. I will issue some
>> official patch after that.
>
> Well, I disagree, and I'll push my patch as soon as somebody confirms
> that it doesn't break anything.
Remember you are not a maintainer of this driver and second we are
open to all suggestions you don't have to use this kind of
statements...
>
>> > There was already discussion on LKML about memory allocation problems
>> > on X86_64, which might explain this regression. This didn't happen
>> > before.
>>
>> This is the thread title if you are interested.
>> 'x86/kernel/pci_dma.c: gfp |= __GFP_NORETRY'
>
> Like I said, it doesn't matter, there's no need to _waste_
> 18*256*sizeof(void *) bytes memory.
It does matter this is not pci allocation we are saving in your patch.
> johannes
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:39 ` Tomas Winkler
@ 2008-06-12 17:46 ` Johannes Berg
2008-06-12 18:03 ` Tomas Winkler
2008-06-12 18:41 ` John W. Linville
0 siblings, 2 replies; 23+ messages in thread
From: Johannes Berg @ 2008-06-12 17:46 UTC (permalink / raw)
To: Tomas Winkler
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]
> > Well, I disagree, and I'll push my patch as soon as somebody confirms
> > that it doesn't break anything.
>
> Remember you are not a maintainer of this driver and second we are
> open to all suggestions you don't have to use this kind of
> statements...
Yeah, you're right, I can't really do that. But I can submit the patch
to akpm, and I'm sure he'll take it after you provide your counter
argument about hope never dying again ;)
Frankly, I don't see why you're so opposed to this patch even if it
doesn't solve anything it probably leads to better code generation and
using a lot less memory.
Also, I know you cannot actually need those descriptors since mac80211
will never ever pass such frames, and _that_ is an area I do have at
least some influence over, so I'll surely notice when that changes.
> >> > There was already discussion on LKML about memory allocation problems
> >> > on X86_64, which might explain this regression. This didn't happen
> >> > before.
> >>
> >> This is the thread title if you are interested.
> >> 'x86/kernel/pci_dma.c: gfp |= __GFP_NORETRY'
> >
> > Like I said, it doesn't matter, there's no need to _waste_
> > 18*256*sizeof(void *) bytes memory.
>
> It does matter this is not pci allocation we are saving in your patch.
Well, thing is, my patch saves 18 KiB memory on 32-bit and 36 on 64-bit,
so I think we should merge it regardless. Yes, the pci allocation is
icky, and yes, it would be good to just do it once instead of over and
over again, but even if you change it to do _all_ those allocations just
once we should not be wasting those 18/36 KiB memory for nothing.
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:46 ` Johannes Berg
@ 2008-06-12 18:03 ` Tomas Winkler
2008-06-12 18:15 ` Johannes Berg
2008-06-12 18:41 ` John W. Linville
1 sibling, 1 reply; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 18:03 UTC (permalink / raw)
To: Johannes Berg
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 8:46 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> > Well, I disagree, and I'll push my patch as soon as somebody confirms
>> > that it doesn't break anything.
>>
>> Remember you are not a maintainer of this driver and second we are
>> open to all suggestions you don't have to use this kind of
>> statements...
>
> Yeah, you're right, I can't really do that. But I can submit the patch
> to akpm, and I'm sure he'll take it after you provide your counter
> argument about hope never dying again ;)
Here you go again
> Frankly, I don't see why you're so opposed to this patch even if it
> doesn't solve anything it probably leads to better code generation and
> using a lot less memory.
I'm not against it. You;v decided that I'm fighting you because I gave
another solution.
Frankly we probably don't need this allocation at all. maybe one skb
is just enough
even with my never dying hope all fragments are in skb fragment list.
This still probably won't save pci memory allocation problem
Tomas
> Also, I know you cannot actually need those descriptors since mac80211
> will never ever pass such frames, and _that_ is an area I do have at
> least some influence over, so I'll surely notice when that changes.
>
>> >> > There was already discussion on LKML about memory allocation problems
>> >> > on X86_64, which might explain this regression. This didn't happen
>> >> > before.
>> >>
>> >> This is the thread title if you are interested.
>> >> 'x86/kernel/pci_dma.c: gfp |= __GFP_NORETRY'
>> >
>> > Like I said, it doesn't matter, there's no need to _waste_
>> > 18*256*sizeof(void *) bytes memory.
>>
>> It does matter this is not pci allocation we are saving in your patch.
>
> Well, thing is, my patch saves 18 KiB memory on 32-bit and 36 on 64-bit,
> so I think we should merge it regardless. Yes, the pci allocation is
> icky, and yes, it would be good to just do it once instead of over and
> over again, but even if you change it to do _all_ those allocations just
> once we should not be wasting those 18/36 KiB memory for nothing.
>
> johannes
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 18:03 ` Tomas Winkler
@ 2008-06-12 18:15 ` Johannes Berg
2008-06-12 20:11 ` Zdenek Kabelac
0 siblings, 1 reply; 23+ messages in thread
From: Johannes Berg @ 2008-06-12 18:15 UTC (permalink / raw)
To: Tomas Winkler
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
[-- Attachment #1: Type: text/plain, Size: 778 bytes --]
> I'm not against it. You;v decided that I'm fighting you because I gave
> another solution.
Ok, no, I'm not saying you shouldn't rewrite all the code to get rid of
it, but I think you can use a patch like mine interim as such a rewrite
is unlikely to go into 2.6.26, is it?
> Frankly we probably don't need this allocation at all. maybe one skb
> is just enough
That would be nice, indeed.
> even with my never dying hope all fragments are in skb fragment list.
:)
> This still probably won't save pci memory allocation problem
Yeah, true, that one needs to be done, but it could probably be done
only once when hw is probed rather than every time it is brought up.
Most likely not something you'll get to fix in 2.6.26 either though.
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 18:15 ` Johannes Berg
@ 2008-06-12 20:11 ` Zdenek Kabelac
2008-06-12 22:17 ` Tomas Winkler
2008-06-13 0:43 ` Andrew Morton
0 siblings, 2 replies; 23+ messages in thread
From: Zdenek Kabelac @ 2008-06-12 20:11 UTC (permalink / raw)
To: Johannes Berg
Cc: Tomas Winkler, Rik van Riel, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
2008/6/12 Johannes Berg <johannes@sipsolutions.net>:
>
>> I'm not against it. You;v decided that I'm fighting you because I gave
>> another solution.
>
> Ok, no, I'm not saying you shouldn't rewrite all the code to get rid of
> it, but I think you can use a patch like mine interim as such a rewrite
> is unlikely to go into 2.6.26, is it?
>
>> Frankly we probably don't need this allocation at all. maybe one skb
>> is just enough
>
> That would be nice, indeed.
>
>> even with my never dying hope all fragments are in skb fragment list.
>
> :)
>
>> This still probably won't save pci memory allocation problem
>
> Yeah, true, that one needs to be done, but it could probably be done
> only once when hw is probed rather than every time it is brought up.
> Most likely not something you'll get to fix in 2.6.26 either though.
Well - it's great that there will be saved few kB in allocation of
never used pointers in iwl driver - but does this really solve the
problem that kernel gets relatively quickly out of memory for
allocations of this size - I guess iwl isn't the only driver
requesting 32 sequential pages.
Is it possible to track how this memory gets fragment/lost - who owns
the block and why they are not back in the pool?
btw with 8hour uptime at this moment I can see this:
DMA: 26*4kB 37*8kB 72*16kB 65*32kB 3*64kB 0*128kB 0*256kB 0*512kB
0*1024kB 0*2048kB 1*4096kB = 7920kB
DMA32: 203*4kB 79*8kB 26*16kB 11*32kB 6*64kB 9*128kB 3*256kB 2*512kB
2*1024kB 0*2048kB 0*4096kB = 7588kB
so at this moment I can see quiet a lot of free DMA memory - but in my
trace at the thread beginig after several suspend/resumes this memory
was gone....
Zdenek
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 20:11 ` Zdenek Kabelac
@ 2008-06-12 22:17 ` Tomas Winkler
2008-06-13 0:43 ` Andrew Morton
1 sibling, 0 replies; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 22:17 UTC (permalink / raw)
To: Zdenek Kabelac
Cc: Johannes Berg, Rik van Riel, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 11:11 PM, Zdenek Kabelac
<zdenek.kabelac@gmail.com> wrote:
> 2008/6/12 Johannes Berg <johannes@sipsolutions.net>:
>>
>>> I'm not against it. You;v decided that I'm fighting you because I gave
>>> another solution.
>>
>> Ok, no, I'm not saying you shouldn't rewrite all the code to get rid of
>> it, but I think you can use a patch like mine interim as such a rewrite
>> is unlikely to go into 2.6.26, is it?
>>
>>> Frankly we probably don't need this allocation at all. maybe one skb
>>> is just enough
>>
>> That would be nice, indeed.
>>
>>> even with my never dying hope all fragments are in skb fragment list.
>>
>> :)
>>
>>> This still probably won't save pci memory allocation problem
>>
>> Yeah, true, that one needs to be done, but it could probably be done
>> only once when hw is probed rather than every time it is brought up.
>> Most likely not something you'll get to fix in 2.6.26 either though.
>
>
> Well - it's great that there will be saved few kB in allocation of
> never used pointers in iwl driver - but does this really solve the
> problem that kernel gets relatively quickly out of memory for
> allocations of this size - I guess iwl isn't the only driver
> requesting 32 sequential pages.
>
> Is it possible to track how this memory gets fragment/lost - who owns
> the block and why they are not back in the pool?
>
> btw with 8hour uptime at this moment I can see this:
>
> DMA: 26*4kB 37*8kB 72*16kB 65*32kB 3*64kB 0*128kB 0*256kB 0*512kB
> 0*1024kB 0*2048kB 1*4096kB = 7920kB
> DMA32: 203*4kB 79*8kB 26*16kB 11*32kB 6*64kB 9*128kB 3*256kB 2*512kB
> 2*1024kB 0*2048kB 0*4096kB = 7588kB
>
> so at this moment I can see quiet a lot of free DMA memory - but in my
> trace at the thread beginig after several suspend/resumes this memory
> was gone....
Currently the driver frees the memory on down and allocates it back on
up. This is done even in initial
reset for sake of flow simplicity.
I'm not sure yet why the memory is actually accumulating, whether the
bug is in the driver or memory system is not clear to me yet. I
haven't seen this on older kernels or driver.
As I wrote I'm also polishing a patch that doesn't do this free-alloc
loop hope this will remedy somehow this problem. It has a drawback as
it will hold on memory even if devices is down.
Tomas
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 20:11 ` Zdenek Kabelac
2008-06-12 22:17 ` Tomas Winkler
@ 2008-06-13 0:43 ` Andrew Morton
1 sibling, 0 replies; 23+ messages in thread
From: Andrew Morton @ 2008-06-13 0:43 UTC (permalink / raw)
To: Zdenek Kabelac
Cc: johannes, tomasw, riel, linux-kernel, yi.zhu, reinette.chatre,
linux-wireless
On Thu, 12 Jun 2008 22:11:32 +0200
"Zdenek Kabelac" <zdenek.kabelac@gmail.com> wrote:
> Well - it's great that there will be saved few kB in allocation of
> never used pointers in iwl driver - but does this really solve the
> problem that kernel gets relatively quickly out of memory for
> allocations of this size - I guess iwl isn't the only driver
> requesting 32 sequential pages.
I hope it is the only one. Doing a 128kb GFP_ATOMIC allocation is
hopelessly unreliable. Even a 128k GFP_KERNEL allocation will fail all
over the place.
Please convert the driver to allocate no more than 4k at a time.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:46 ` Johannes Berg
2008-06-12 18:03 ` Tomas Winkler
@ 2008-06-12 18:41 ` John W. Linville
1 sibling, 0 replies; 23+ messages in thread
From: John W. Linville @ 2008-06-12 18:41 UTC (permalink / raw)
To: Johannes Berg
Cc: Tomas Winkler, Rik van Riel, Zdenek Kabelac,
Linux Kernel Mailing List, yi.zhu, reinette.chatre,
linux-wireless
On Thu, Jun 12, 2008 at 07:46:52PM +0200, Johannes Berg wrote:
>
> > > Well, I disagree, and I'll push my patch as soon as somebody confirms
> > > that it doesn't break anything.
> >
> > Remember you are not a maintainer of this driver and second we are
> > open to all suggestions you don't have to use this kind of
> > statements...
>
> Yeah, you're right, I can't really do that. But I can submit the patch
> to akpm, and I'm sure he'll take it after you provide your counter
> argument about hope never dying again ;)
Is there some reason you think I would need to be cut out of the loop??
akpm isn't the only one who likes solutions...
John
--
John W. Linville
linville@tuxdriver.com
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 15:43 ` Tomas Winkler
2008-06-12 16:35 ` Tomas Winkler
@ 2008-06-12 17:03 ` Johannes Berg
2008-06-12 17:35 ` Tomas Winkler
2008-06-12 17:10 ` Rik van Riel
2008-06-12 21:30 ` Jiri Slaby
3 siblings, 1 reply; 23+ messages in thread
From: Johannes Berg @ 2008-06-12 17:03 UTC (permalink / raw)
To: Tomas Winkler
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]
> > Try the patch below. It should improve code generation too.
> >
> > I discussed this with Tomas previously and he says the hw is capable of
> > doing 20 fragments per frame (wonder why, Broadcom can do an unlimited
> > number...) but he complained about the networking stack not being able
> > to.
>
> This is scatter gather buffers that can be kicked in one DMA transaction.
>
> Well, the hardware needs to support IP checksumming for that, hence,
> > afaik, only two fragments can ever be used (one for hw header, one for
> > frame)
> This I still don't understand why but everybody is already tired to
> explaining me why.. :) Just need to find time to dig into it.
And you can safely decrease the allocation to 10% as I do in my patch
because once you understand you'll see that you cannot possibly use
more. Hence, you can ack this patch ;)
> > This cuts the allocation to 10%, or (under) a page in all cases.
>
> Probably. it would be safe to use vmalloc for allocating txb anyway.
> I'll give it a try.
Yeah, but why bother if we can just allocate 10% of the size, waste a
lot less memory etc. mac80211 isn't going to pass in a scatter/gather
frame anyway.
> There was already discussion on LKML about memory allocation problems
> on X86_64, which might explain this regression. This didn't happen
> before.
Doesn't really matter, iwlwifi is _wasting_ this allocation, it cannot
possibly use all those buffers anyway.
The more interesting thing is the pci_alloc_consistent allocation right
below that is also _huge_, but that's because of the stupid hardware
design, or can the hardware cope with having the descriptors non-linear
in memory?
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:03 ` Johannes Berg
@ 2008-06-12 17:35 ` Tomas Winkler
2008-06-12 17:39 ` Johannes Berg
0 siblings, 1 reply; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 17:35 UTC (permalink / raw)
To: Johannes Berg
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 8:03 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> > Try the patch below. It should improve code generation too.
>> >
>> > I discussed this with Tomas previously and he says the hw is capable of
>> > doing 20 fragments per frame (wonder why, Broadcom can do an unlimited
>> > number...) but he complained about the networking stack not being able
>> > to.
>>
>> This is scatter gather buffers that can be kicked in one DMA transaction.
>>
>> Well, the hardware needs to support IP checksumming for that, hence,
>> > afaik, only two fragments can ever be used (one for hw header, one for
>> > frame)
>> This I still don't understand why but everybody is already tired to
>> explaining me why.. :) Just need to find time to dig into it.
>
> And you can safely decrease the allocation to 10% as I do in my patch
> because once you understand you'll see that you cannot possibly use
> more. Hence, you can ack this patch ;)
>
>> > This cuts the allocation to 10%, or (under) a page in all cases.
>>
>> Probably. it would be safe to use vmalloc for allocating txb anyway.
>> I'll give it a try.
>
> Yeah, but why bother if we can just allocate 10% of the size, waste a
> lot less memory etc. mac80211 isn't going to pass in a scatter/gather
> frame anyway.
Hope never dies. I actually have seen this speed up the throughput so
I will dig into it anyway.
>> There was already discussion on LKML about memory allocation problems
>> on X86_64, which might explain this regression. This didn't happen
>> before.
>
> Doesn't really matter, iwlwifi is _wasting_ this allocation, it cannot
> possibly use all those buffers anyway.
This matter actually for consistent allocation.
> The more interesting thing is the pci_alloc_consistent allocation right
> below that is also _huge_, but that's because of the stupid hardware
> design, or can the hardware cope with having the descriptors non-linear
> in memory?
We talk after your next HW design. How will configure 265 * 16
descriptors separately.
Tomas
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:35 ` Tomas Winkler
@ 2008-06-12 17:39 ` Johannes Berg
2008-06-12 17:50 ` Tomas Winkler
0 siblings, 1 reply; 23+ messages in thread
From: Johannes Berg @ 2008-06-12 17:39 UTC (permalink / raw)
To: Tomas Winkler
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]
> > Yeah, but why bother if we can just allocate 10% of the size, waste a
> > lot less memory etc. mac80211 isn't going to pass in a scatter/gather
> > frame anyway.
>
> Hope never dies. I actually have seen this speed up the throughput so
> I will dig into it anyway.
Well, you can always add it back later if you make the networking stack
and mac80211 support it. It's a two-line patch after all.
> > The more interesting thing is the pci_alloc_consistent allocation right
> > below that is also _huge_, but that's because of the stupid hardware
> > design, or can the hardware cope with having the descriptors non-linear
> > in memory?
>
> We talk after your next HW design. How will configure 265 * 16
> descriptors separately.
Well, considering that other hardware does manage to do things
differently (say Broadcom because I know their DMA engine), I don't know
why your hw designers went wild with this. All you need is an
"end-of-frame" flag. But that's not really interesting to discuss,
unless this is actually controlled by the microcode and you can change
it.
johannes
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 17:39 ` Johannes Berg
@ 2008-06-12 17:50 ` Tomas Winkler
0 siblings, 0 replies; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 17:50 UTC (permalink / raw)
To: Johannes Berg
Cc: Rik van Riel, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, Jun 12, 2008 at 8:39 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> > Yeah, but why bother if we can just allocate 10% of the size, waste a
>> > lot less memory etc. mac80211 isn't going to pass in a scatter/gather
>> > frame anyway.
>>
>> Hope never dies. I actually have seen this speed up the throughput so
>> I will dig into it anyway.
>
> Well, you can always add it back later if you make the networking stack
> and mac80211 support it. It's a two-line patch after all.
I will handle this.
>> > The more interesting thing is the pci_alloc_consistent allocation right
>> > below that is also _huge_, but that's because of the stupid hardware
>> > design, or can the hardware cope with having the descriptors non-linear
>> > in memory?
>>
>> We talk after your next HW design. How will configure 265 * 16
>> descriptors separately.
>
> Well, considering that other hardware does manage to do things
> differently (say Broadcom because I know their DMA engine), I don't know
> why your hw designers went wild with this. All you need is an
> "end-of-frame" flag. But that's not really interesting to discuss,
> unless this is actually controlled by the microcode and you can change
> it.
>
This is have to do something with HW packet scheduling
> johannes
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 15:43 ` Tomas Winkler
2008-06-12 16:35 ` Tomas Winkler
2008-06-12 17:03 ` Johannes Berg
@ 2008-06-12 17:10 ` Rik van Riel
2008-06-12 21:30 ` Jiri Slaby
3 siblings, 0 replies; 23+ messages in thread
From: Rik van Riel @ 2008-06-12 17:10 UTC (permalink / raw)
To: Tomas Winkler
Cc: Johannes Berg, Zdenek Kabelac, Linux Kernel Mailing List, yi.zhu,
reinette.chatre, linux-wireless
On Thu, 12 Jun 2008 18:43:37 +0300
"Tomas Winkler" <tomasw@gmail.com> wrote:
> This is scatter gather buffers that can be kicked in one DMA transaction.
> This I still don't understand why but everybody is already tired to
> explaining me why.. :) Just need to find time to dig into it.
The only thing that makes no sense to me is why your
driver "needs" to allocate 10x as much memory in that
buffer than it will ever use.
What is the problem with the simpler solution, which
just reduces the size of the buffer to an amount of
memory that might actually get used?
--
All Rights Reversed
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 15:43 ` Tomas Winkler
` (2 preceding siblings ...)
2008-06-12 17:10 ` Rik van Riel
@ 2008-06-12 21:30 ` Jiri Slaby
2008-06-12 22:26 ` Tomas Winkler
3 siblings, 1 reply; 23+ messages in thread
From: Jiri Slaby @ 2008-06-12 21:30 UTC (permalink / raw)
To: Tomas Winkler
Cc: Johannes Berg, Rik van Riel, Zdenek Kabelac,
Linux Kernel Mailing List, yi.zhu, reinette.chatre,
linux-wireless
On 06/12/2008 05:43 PM, Tomas Winkler wrote:
> On Thu, Jun 12, 2008 at 4:54 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
>> This cuts the allocation to 10%, or (under) a page in all cases.
>
> Probably. it would be safe to use vmalloc for allocating txb anyway.
> I'll give it a try.
Why it wouldn't be "safe". I suggested it to you already, since allocating 64k
by kmalloc for descriptors accessed only in kernel is crud. Moreover you're
mixing the buffer with its descriptors here? Or what you're considering to vmalloc?
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: Problem: Out of memory after 2days with 2GB RAM
2008-06-12 21:30 ` Jiri Slaby
@ 2008-06-12 22:26 ` Tomas Winkler
0 siblings, 0 replies; 23+ messages in thread
From: Tomas Winkler @ 2008-06-12 22:26 UTC (permalink / raw)
To: Jiri Slaby
Cc: Johannes Berg, Rik van Riel, Zdenek Kabelac,
Linux Kernel Mailing List, yi.zhu, reinette.chatre,
linux-wireless
On Fri, Jun 13, 2008 at 12:30 AM, Jiri Slaby <jirislaby@gmail.com> wrote:
> On 06/12/2008 05:43 PM, Tomas Winkler wrote:
>>
>> On Thu, Jun 12, 2008 at 4:54 PM, Johannes Berg
>> <johannes@sipsolutions.net> wrote:
>>>
>>> This cuts the allocation to 10%, or (under) a page in all cases.
>>
>> Probably. it would be safe to use vmalloc for allocating txb anyway.
>> I'll give it a try.
>
> Why it wouldn't be "safe". I suggested it to you already, since allocating
> 64k by kmalloc for descriptors accessed only in kernel is crud. Moreover
> you're mixing the buffer with its descriptors here? Or what you're
> considering to vmalloc?
>
Not that. I just wasn't sure when I dropped the line I'm not doing it
under some spinlock or something like that.
Tomas
^ permalink raw reply [flat|nested] 23+ messages in thread