Re: XFS stack overhead

Eric Sandeen
2009-10-21T19:22:23Z

Ingo Molnar wrote:
> * Eric Sandeen <sandeen@redhat.com> wrote:
>
>> Arjan van de Ven wrote:
>>> On Wed, 21 Oct 2009 10:50:02 -0500
>>> Eric Sandeen <sandeen@redhat.com> wrote:
>>>
>>>> Ingo Molnar wrote:
>>>>> (Cc:-ed Arjan too.)
>>>>>
>>>>> * Dave Jones <davej@redhat.com> wrote:
>>>>>
>>>>>> 113c5413cf9051cc50b88befdc42e3402bb92115 introduced a change that
>>>>>> made CC_STACKPROTECTOR_ALL not-selectable if someone enables
>>>>>> CC_STACKPROTECTOR.
>>>>>>
>>>>>> We've noticed in Fedora that this has introduced noticable
>>>>>> overhead on some functions, including those which don't even have
>>>>>> any on-stack variables.
>>>>>>
>>>>>> According to the gcc manpage, -fstack-protector will protect
>>>>>> functions with as little as 8 bytes of stack usage. So we're
>>>>>> introducing a huge amount of overhead, to close a small amount of
>>>>>> vulnerability (the >0 && <8 case).
>>>>>>
>>>>>> The overhead as it stands right now means this whole option is
>>>>>> unusable for a distro kernel without reverting the above commit.
>>>>> Exactly what workload showed overhead, and how much?
>>>>>
>>>>> Ingo
>>>> I had xfs blowing up pretty nicely; granted, xfs is not svelte but it
>>>> was never this bad before.
>>>>
>>> do you have any indication that SP actually increases the stack
>>> footprint by that much? it's only a few bytes....
>>>
>>>
>> Here's a sample of some of the largest xfs stack users,
>> and the effect stack-protector had on them. This was just
>> done with objdump -d xfs.ko | scripts/checkstack.pl; I don't
>> know if there's extra runtime stack overhead w/ stackprotector?
>>
>> -Eric
>>
>> function nostack stackprot delta delta %
>> xfs_bmapi 376 408 32 9%
>> xfs_bulkstat 328 344 16 5%
>> _xfs_trans_commit 296 312 16 5%
>> xfs_iomap_write_delay 264 280 16 6%
>> xfs_file_ioctl 248 312 64 26%
>> xfs_symlink 248 264 16 6%
>> xfs_bunmapi 232 280 48 21%
>> xlog_do_recovery_pass 232 248 16 7%
>> xfs_trans_unreserve_and_mod_sb 224 240 16 7%
>> xfs_bmap_del_extent 216 248 32 15%
>> xfs_cluster_write 216 232 16 7%
>> xfs_file_compat_ioctl 216 296 80 37%
>> xfs_attr_set_int 200 216 16 8%
>> xfs_bmap_add_extent_delay_real 200 248 48 24%
>
> Note that those are very large stack frames to begin with.
>
> 3496 bytes - that's a _lot_ - can anyone even run XFS with 4K stacks on?

The above isn't a callchain; those are just the biggest users.

Yes, xfs works w/4k stacks but sometimes not over complex storage.

> With stackprotector it's 3928 - a 12% increase - which certainly does
> not help - but the basic problem is the large stack footprint to begin
> with.

I can find plenty of examples of > 300 bytes stack users in the core
kernel write path too, I'm just using xfs as an example...

> Also, the posting apparently mixes 'stack overhead' with 'runtime
> overhead'.

right, that's why I asked, I'm not sure if stackprotector has runtime
overhead as well.

-Eric

> Ingo