good explanation of __read_mostly, __init, __exit macros,

good explanation of __read_mostly, __init, __exit macros,

[Aft nix]

Hi,

The macro expansion of __read_mostly :

#define __read_mostly __attribute__((__section__(".data..read_mostly"))

This one is from cache.h

__init:
#define __init		__section(.init.text) __cold notrace

from init.h

__exit:

#define __exit          __section(.exit.text) __exitused __cold notrace

After searching through net i have not found any good explanation of
what is happening there.

Additonal question : I have heard about various "linker magic"
employed in kernel development. Any information
regarding this will be wonderful.

cheers

-- 
-aft

good explanation of __read_mostly, __init, __exit macros,

[Sannu K]

On Mon, Jul 16, 2012 at 4:52 PM, Aft nix <aftnix@gmail.com> wrote:

> Hi,
>
> The macro expansion of __read_mostly :
>
> #define __read_mostly __attribute__((__section__(".data..read_mostly"))
>
> This one is from cache.h
>
> __init:
> #define __init          __section(.init.text) __cold notrace
>
> from init.h
>
> __exit:
>
> #define __exit          __section(.exit.text) __exitused __cold notrace
>
> After searching through net i have not found any good explanation of
> what is happening there.
>
> Additonal question : I have heard about various "linker magic"
> employed in kernel development. Any information
> regarding this will be wonderful.
>
> cheers
>
> --
> -aft
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>


Using __init means the function code will be removed - saving RAM space
i.e., one cannot call the function declared with __init once the
initialization is done. I just know the meaning but I am not sure of how it
works. This may provide some info on your question even though it did not
answer your question.

Hope this Helps,
Sannu K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120716/fd6f6def/attachment.html 

good explanation of __read_mostly, __init, __exit macros,

[Filipe Rinaldi]

On 16 July 2012 12:22, Aft nix <aftnix@gmail.com> wrote:
> Hi,
>
> The macro expansion of __read_mostly :
>
> #define __read_mostly __attribute__((__section__(".data..read_mostly"))
>
> This one is from cache.h
>
> __init:
> #define __init          __section(.init.text) __cold notrace
>
> from init.h
>
> __exit:
>
> #define __exit          __section(.exit.text) __exitused __cold notrace
>
> After searching through net i have not found any good explanation of
> what is happening there.
>
> Additonal question : I have heard about various "linker magic"
> employed in kernel development. Any information
> regarding this will be wonderful.
>
> cheers
>
> --
> -aft
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies


Hi Aft,

These macros are used to assign a function or initialised variable
content to a specific section in memory. If you search for the linker
script of any architecture (*.lds.S files), you will see that these
variables and functions are placed in specific sections with names
like "__init_begin" and "__init_end". After the initialisation, Linux
can re-use for example the "init" memory.

-Filipe

good explanation of __read_mostly, __init, __exit macros,

[Robert P. J. Day]

On Mon, 16 Jul 2012, Filipe Rinaldi wrote:

> On 16 July 2012 12:22, Aft nix <aftnix@gmail.com> wrote:
> > Hi,
> >
> > The macro expansion of __read_mostly :
> >
> > #define __read_mostly __attribute__((__section__(".data..read_mostly"))
> >
> > This one is from cache.h
> >
> > __init:
> > #define __init          __section(.init.text) __cold notrace
> >
> > from init.h
> >
> > __exit:
> >
> > #define __exit          __section(.exit.text) __exitused __cold notrace
> >
> > After searching through net i have not found any good explanation of
> > what is happening there.
> >
> > Additonal question : I have heard about various "linker magic"
> > employed in kernel development. Any information
> > regarding this will be wonderful.
> >
> > cheers
> >
> > --
> > -aft
> >
> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies at kernelnewbies.org
> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
> Hi Aft,
>
> These macros are used to assign a function or initialised variable
> content to a specific section in memory. If you search for the linker
> script of any architecture (*.lds.S files), you will see that these
> variables and functions are placed in specific sections with names
> like "__init_begin" and "__init_end". After the initialisation, Linux
> can re-use for example the "init" memory.

  that's what's happening when you see that boot-time message:

"Freeing unused memory ..."

or whatever it is, something to that effect.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

good explanation of __read_mostly, __init, __exit macros,

[Gaurav Jain]

One instance where this 'linker magic' is employed is in the treatment of
the variable jiffies. jiffies has been traditionally an 'unsigned long'.
Therefore, it occupies 32 bits on 32 bit arch. and 64 bits on 64-bit arch.
However, having jiffies just 32-bit long can result in overflows pretty
quickly (few days) if the HZ value is large. So, nowadays (2.6 kernel), we
have one more variable - jiffies_64, which is 64 bit long. To maintain code
compatibility, this 'jiffies_64' is overlayed on the original 'jiffies'
variable, so that both variables begin at the same address. Older code can
continue to operate as is, while new code can make use of the 'jiffies_64'
variable, as and when required.

Please refer Chapter:11 (Timers and Time Management), Topic: Internal
Representation of Jiffies
Linux Kernel Development, 3rd Edition - Robert Love to read about this.

Best Regards
Gaurav

On Mon, Jul 16, 2012 at 6:10 PM, Robert P. J. Day <rpjday@crashcourse.ca>wrote:

> On Mon, 16 Jul 2012, Filipe Rinaldi wrote:
>
> > On 16 July 2012 12:22, Aft nix <aftnix@gmail.com> wrote:
> > > Hi,
> > >
> > > The macro expansion of __read_mostly :
> > >
> > > #define __read_mostly __attribute__((__section__(".data..read_mostly"))
> > >
> > > This one is from cache.h
> > >
> > > __init:
> > > #define __init          __section(.init.text) __cold notrace
> > >
> > > from init.h
> > >
> > > __exit:
> > >
> > > #define __exit          __section(.exit.text) __exitused __cold notrace
> > >
> > > After searching through net i have not found any good explanation of
> > > what is happening there.
> > >
> > > Additonal question : I have heard about various "linker magic"
> > > employed in kernel development. Any information
> > > regarding this will be wonderful.
> > >
> > > cheers
> > >
> > > --
> > > -aft
> > >
> > > _______________________________________________
> > > Kernelnewbies mailing list
> > > Kernelnewbies at kernelnewbies.org
> > > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >
> >
> > Hi Aft,
> >
> > These macros are used to assign a function or initialised variable
> > content to a specific section in memory. If you search for the linker
> > script of any architecture (*.lds.S files), you will see that these
> > variables and functions are placed in specific sections with names
> > like "__init_begin" and "__init_end". After the initialisation, Linux
> > can re-use for example the "init" memory.
>
>   that's what's happening when you see that boot-time message:
>
> "Freeing unused memory ..."
>
> or whatever it is, something to that effect.
>
> rday
>
> --
>
> ========================================================================
> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>                         http://crashcourse.ca
>
> Twitter:                                       http://twitter.com/rpjday
> LinkedIn:                               http://ca.linkedin.com/in/rpjday
> ========================================================================
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Gaurav Jain
Associate Software Engineer
VxVM Escalations Team, SAMG
Symantec Software India Pvt. Ltd.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120716/e3a05bf5/attachment-0001.html 

good explanation of __read_mostly, __init, __exit macros,

[Gaurav Jain]

Forgot to mention that this 'overlaying' is where 'linker magic' comes into
picture. I myself don't know the details though.

On Mon, Jul 16, 2012 at 7:56 PM, Gaurav Jain <gjainroorkee@gmail.com> wrote:

> One instance where this 'linker magic' is employed is in the treatment of
> the variable jiffies. jiffies has been traditionally an 'unsigned long'.
> Therefore, it occupies 32 bits on 32 bit arch. and 64 bits on 64-bit arch.
> However, having jiffies just 32-bit long can result in overflows pretty
> quickly (few days) if the HZ value is large. So, nowadays (2.6 kernel), we
> have one more variable - jiffies_64, which is 64 bit long. To maintain code
> compatibility, this 'jiffies_64' is overlayed on the original 'jiffies'
> variable, so that both variables begin at the same address. Older code can
> continue to operate as is, while new code can make use of the 'jiffies_64'
> variable, as and when required.
>
> Please refer Chapter:11 (Timers and Time Management), Topic: Internal
> Representation of Jiffies
> Linux Kernel Development, 3rd Edition - Robert Love to read about this.
>
> Best Regards
> Gaurav
>
>
> On Mon, Jul 16, 2012 at 6:10 PM, Robert P. J. Day <rpjday@crashcourse.ca>wrote:
>
>> On Mon, 16 Jul 2012, Filipe Rinaldi wrote:
>>
>> > On 16 July 2012 12:22, Aft nix <aftnix@gmail.com> wrote:
>> > > Hi,
>> > >
>> > > The macro expansion of __read_mostly :
>> > >
>> > > #define __read_mostly
>> __attribute__((__section__(".data..read_mostly"))
>> > >
>> > > This one is from cache.h
>> > >
>> > > __init:
>> > > #define __init          __section(.init.text) __cold notrace
>> > >
>> > > from init.h
>> > >
>> > > __exit:
>> > >
>> > > #define __exit          __section(.exit.text) __exitused __cold
>> notrace
>> > >
>> > > After searching through net i have not found any good explanation of
>> > > what is happening there.
>> > >
>> > > Additonal question : I have heard about various "linker magic"
>> > > employed in kernel development. Any information
>> > > regarding this will be wonderful.
>> > >
>> > > cheers
>> > >
>> > > --
>> > > -aft
>> > >
>> > > _______________________________________________
>> > > Kernelnewbies mailing list
>> > > Kernelnewbies at kernelnewbies.org
>> > > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>> >
>> >
>> > Hi Aft,
>> >
>> > These macros are used to assign a function or initialised variable
>> > content to a specific section in memory. If you search for the linker
>> > script of any architecture (*.lds.S files), you will see that these
>> > variables and functions are placed in specific sections with names
>> > like "__init_begin" and "__init_end". After the initialisation, Linux
>> > can re-use for example the "init" memory.
>>
>>   that's what's happening when you see that boot-time message:
>>
>> "Freeing unused memory ..."
>>
>> or whatever it is, something to that effect.
>>
>> rday
>>
>> --
>>
>> ========================================================================
>> Robert P. J. Day                                 Ottawa, Ontario, CANADA
>>                         http://crashcourse.ca
>>
>> Twitter:                                       http://twitter.com/rpjday
>> LinkedIn:                               http://ca.linkedin.com/in/rpjday
>> ========================================================================
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>
>
> --
> Gaurav Jain
> Associate Software Engineer
> VxVM Escalations Team, SAMG
> Symantec Software India Pvt. Ltd.
>
>
>
>


-- 
Gaurav Jain
Associate Software Engineer
VxVM Escalations Team, SAMG
Symantec Software India Pvt. Ltd.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120716/9b2d0e25/attachment.html 

good explanation of __read_mostly, __init, __exit macros,

[Mulyadi Santosa]

Hi..

On Mon, Jul 16, 2012 at 6:22 PM, Aft nix <aftnix@gmail.com> wrote:
> Hi,
>
> The macro expansion of __read_mostly :
>
> #define __read_mostly __attribute__((__section__(".data..read_mostly"))
>
> This one is from cache.h
>
> __init:
> #define __init          __section(.init.text) __cold notrace
>
> from init.h
>
> __exit:
>
> #define __exit          __section(.exit.text) __exitused __cold notrace
>
> After searching through net i have not found any good explanation of
> what is happening there.

like others had said, it means, the variables marked as read mostly
are grouped into special section in kernel memory layout, named
read_mostly. It's a subsection inside "data", which is I believe
belong to initialized variable.

By grouping this into single special region, it is hoped that you
reduce (if possible none happen) cache invalidation. Such thing happen
when bits in a cache line are updated. In such occasion, the whole
cache line must be updated by re-fetching the related memory address.

Thus, if all these bits are read only, once fetched, it will stays in
cache line as long as possible.

PS: there is no guarantee that this grouping will effectively
works...that's why you need to align it to the size of cache line.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

good explanation of __read_mostly, __init, __exit macros,

[Aft nix]

On Tue, Jul 17, 2012 at 12:39 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com> wrote:
> Hi..
>
> On Mon, Jul 16, 2012 at 6:22 PM, Aft nix <aftnix@gmail.com> wrote:
>> Hi,
>>
>> The macro expansion of __read_mostly :
>>
>> #define __read_mostly __attribute__((__section__(".data..read_mostly"))
>>
>> This one is from cache.h
>>
>> __init:
>> #define __init          __section(.init.text) __cold notrace
>>
>> from init.h
>>
>> __exit:
>>
>> #define __exit          __section(.exit.text) __exitused __cold notrace
>>
>> After searching through net i have not found any good explanation of
>> what is happening there.
>
> like others had said, it means, the variables marked as read mostly
> are grouped into special section in kernel memory layout, named
> read_mostly. It's a subsection inside "data", which is I believe
> belong to initialized variable.
>
> By grouping this into single special region, it is hoped that you
> reduce (if possible none happen) cache invalidation. Such thing happen
> when bits in a cache line are updated. In such occasion, the whole
> cache line must be updated by re-fetching the related memory address.
>
> Thus, if all these bits are read only, once fetched, it will stays in
> cache line as long as possible.
>
> PS: there is no guarantee that this grouping will effectively
> works...that's why you need to align it to the size of cache line.
>

Hi Santosa,

This special section, "data..read_mostly" , is it maintained by
kernel? then Linker has to know this information. How that is done?
I mean lets say a.c is compiled into a.o. a.o already has the offsets
for its data. Linker resolves the final address. So how the linker is
invoked
in a way so that this data flagged with __read__mostly will end up in
the special section?

Thing is, i'm getting the idea what these macros mean, but not
understanding the mechanisms behind them.

cheers.

> --
> regards,
>
> Mulyadi Santosa
> Freelance Linux trainer and consultant
>
> blog: the-hydra.blogspot.com
> training: mulyaditraining.blogspot.com



-- 
-aft

good explanation of __read_mostly, __init, __exit macros,

[Mulyadi Santosa]

Hi :)

On Tue, Jul 17, 2012 at 1:49 AM, Aft nix <aftnix@gmail.com> wrote:
> This special section, "data..read_mostly" , is it maintained by
> kernel?

by maintained, you mean allocated? then yes....

> then Linker has to know this information. How that is done?

it is provided by the linker script.... the fastest I can find is in
directory include/asm-generic/vmlinux.lds.h. This file is not final
linker script, it's kinda a "template" that will be further modified
by Kbuild (I guess)

> I mean lets say a.c is compiled into a.o. a.o already has the offsets
> for its data. Linker resolves the final address. So how the linker is
> invoked
> in a way so that this data flagged with __read__mostly will end up in
> the special section?

it's actually the same like putting your initialized variable in .data
section, your code in .text and so on. I just get the big picture that
it is done by creating section in the object file. By section, I think
it's simply done by creating a mark to denote the section.

please kindly study  ELF documentation for further info

> Thing is, i'm getting the idea what these macros mean, but not
> understanding the mechanisms behind them.

the whole chain is complex actually, but try to get the simple view
first. it's a grouping, read mostly data into a section/group in a
file (in this case, kernel image), kernel loaded.....kernel arrange
itself..put appropriate section into related addresses....done

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

good explanation of __read_mostly, __init, __exit macros,

[Arif Hossain]

On Tue, 2012-07-17 at 02:03 +0700, Mulyadi Santosa wrote:
> Hi :)
> 
> On Tue, Jul 17, 2012 at 1:49 AM, Aft nix <aftnix@gmail.com> wrote:
> > This special section, "data..read_mostly" , is it maintained by
> > kernel?
> 
> by maintained, you mean allocated? then yes....
> 
> > then Linker has to know this information. How that is done?
> 
> it is provided by the linker script.... the fastest I can find is in
> directory include/asm-generic/vmlinux.lds.h. This file is not final
> linker script, it's kinda a "template" that will be further modified
> by Kbuild (I guess)
> 
> > I mean lets say a.c is compiled into a.o. a.o already has the offsets
> > for its data. Linker resolves the final address. So how the linker is
> > invoked
> > in a way so that this data flagged with __read__mostly will end up in
> > the special section?
> 
> it's actually the same like putting your initialized variable in .data
> section, your code in .text and so on. I just get the big picture that
> it is done by creating section in the object file. By section, I think
> it's simply done by creating a mark to denote the section.
> 
> please kindly study  ELF documentation for further info
> 
> > Thing is, i'm getting the idea what these macros mean, but not
> > understanding the mechanisms behind them.
> 
> the whole chain is complex actually, but try to get the simple view
> first. it's a grouping, read mostly data into a section/group in a
> file (in this case, kernel image), kernel loaded.....kernel arrange
> itself..put appropriate section into related addresses....done
> 

Thanks for the detailed explanation. I've got the idea behind
__read_mostly. I've found a interesting document "Dead code elimination
" which i guess relates to "__init" and "__exit". 

Here is the link :
elinux.org/images/2/2d/ELC2010-gc-sections_Denys_Vlasenko.pdf

I'm going to study this post if any question arises.

cheers.

good explanation of __read_mostly, __init, __exit macros,

[Mulyadi Santosa]

On Tue, Jul 17, 2012 at 6:07 PM, Arif Hossain <aftnix@gmail.com> wrote:
> Thanks for the detailed explanation. I've got the idea behind
> __read_mostly. I've found a interesting document "Dead code elimination
> " which i guess relates to "__init" and "__exit".
>
> Here is the link :
> elinux.org/images/2/2d/ELC2010-gc-sections_Denys_Vlasenko.pdf
>
> I'm going to study this post if any question arises.

you welcome :) and sorry if I am not that smart .... :)

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com