[PATCH v4 00/13] Compiler Attributes

[PATCH v4 00/13] Compiler Attributes

[Miguel Ojeda]

The Compiler Attributes series is an effort to disentangle
the include/linux/compiler*.h headers and bring them up to date.

The main idea behind the series is to use feature checking macros
(i.e. __has_attribute) instead of compiler version checks (e.g. GCC_VERSION),
which are compiler-agnostic (so they can be shared, reducing the size
of compiler-specific headers) and version-agnostic.

Other related improvements have been performed in the headers as well,
which on top of the use of __has_attribute it has amounted to a significant
simplification of these headers (e.g. GCC_VERSION is now only guarding 4
non-attribute macros).

This series should also help the efforts to support compiling the kernel
with clang and icc. A fair amount of documentation and comments have also
been added, clarified or removed; and the headers are now more readable,
which should help kernel developers in general.

The series was triggered due to the move to gcc >= 4.6. In turn, this series
has also triggered Sparse to gain the ability to recognize __has_attribute
on its own.

You can also fetch it from:

  https://github.com/ojeda/linux/tree/compiler-attributes-v4

Enjoy!

Cheers,
Miguel

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Cc: Eli Friedman <efriedma@codeaurora.org>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Joe Perches <joe@perches.com>
Cc: Dominique Martinet <asmadeus@codewreck.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-sparse@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>

v3 -> v4

  This time it is an important fix I found while doing randconfigs, plus
  the documentation sentence that Nick suggested and the MAINTAINERS file.

  Compile-tested for a while on (x86_64, gcc-7.3) on a few randconfs.

  New:

  * Add MAINTAINERS entry for the new file (compiler_attributes.h)
    so that we try to maintain that one as clean as possible.
    (if someone else wants to join, feel free!)

  Modified:

  * Fix inline macro

    __always_inline cannot be used in the inline macro,
    because people marking a function as __always_inline would
    expand to inline which in turn would expand back to __always_inline
    (under some configs), which would not be expanded again.
    Added a comment about this in the inline macro.

    Also added another comment about __always_inline in compiler_attributes.h
    to note that users expect to not have to write inline (as well as
    the attribute). We cannot simply remove "inline" there (which would
    solve the problem described above) because using the attribute is
    not enough for gcc. A function marked as always_inline seems to require
    to be marked as inline itself so that the attribute is applied:

      "warning: always_inline function might not be inlinable [-Wattributes]"

    From the gcc docs:

      "For functions declared inline, this attribute inlines the function
       independent of any restrictions that otherwise apply to inlining."

    See https://godbolt.org/z/LpzUPj for an example.

  From reviews:

  * Add sentence about feature-detection in Docs (Nick)

Miguel Ojeda (13):
  Compiler Attributes: remove unused attributes
  Compiler Attributes: always use the extra-underscores syntax
  Compiler Attributes: remove unneeded tests
  Compiler Attributes: homogenize __must_be_array
  Compiler Attributes: naked was fixed in gcc 4.6
  Compiler Attributes: naked can be shared
  Compiler Attributes: remove unneeded sparse (__CHECKER__) tests
  Compiler Attributes: add missing SPDX ID in compiler_types.h
  Compiler Attributes: use feature checks instead of version checks
  Compiler Attributes: KENTRY used twice the "used" attribute
  Compiler Attributes: remove uses of __attribute__ from compiler.h
  Compiler Attributes: add Doc/process/programming-language.rst
  Compiler Attributes: Add MAINTAINERS entry

 Documentation/process/index.rst               |   1 +
 .../process/programming-language.rst          |  45 ++++
 MAINTAINERS                                   |   5 +
 include/linux/compiler-clang.h                |   5 -
 include/linux/compiler-gcc.h                  |  84 +-----
 include/linux/compiler-intel.h                |   9 -
 include/linux/compiler.h                      |  19 +-
 include/linux/compiler_attributes.h           | 244 ++++++++++++++++++
 include/linux/compiler_types.h                | 105 ++------
 9 files changed, 329 insertions(+), 188 deletions(-)
 create mode 100644 Documentation/process/programming-language.rst
 create mode 100644 include/linux/compiler_attributes.h

-- 
2.17.1

[PATCH v4 12/13] Compiler Attributes: add Doc/process/programming-language.rst

[Miguel Ojeda]

Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Cc: Eli Friedman <efriedma@codeaurora.org>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Joe Perches <joe@perches.com>
Cc: Dominique Martinet <asmadeus@codewreck.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-sparse@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
---
 Documentation/process/index.rst               |  1 +
 .../process/programming-language.rst          | 45 +++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 Documentation/process/programming-language.rst

diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst
index 37bd0628b6ee..c56f24a22d2a 100644
--- a/Documentation/process/index.rst
+++ b/Documentation/process/index.rst
@@ -23,6 +23,7 @@ Below are the essential guides that every developer should read.
    code-of-conflict
    development-process
    submitting-patches
+   programming-language
    coding-style
    maintainer-pgp-guide
    email-clients
diff --git a/Documentation/process/programming-language.rst b/Documentation/process/programming-language.rst
new file mode 100644
index 000000000000..e5f5f065dc24
--- /dev/null
+++ b/Documentation/process/programming-language.rst
@@ -0,0 +1,45 @@
+.. _programming_language:
+
+Programming Language
+====================
+
+The kernel is written in the C programming language [c-language]_.
+More precisely, the kernel is typically compiled with ``gcc`` [gcc]_
+under ``-std=gnu89`` [gcc-c-dialect-options]_: the GNU dialect of ISO C90
+(including some C99 features).
+
+This dialect contains many extensions to the language [gnu-extensions]_,
+and many of them are used within the kernel as a matter of course.
+
+There is some support for compiling the kernel with ``clang`` [clang]_
+and ``icc`` [icc]_ for several of the architectures, although at the time
+of writing it is not completed, requiring third-party patches.
+
+Attributes
+----------
+
+One of the common extensions used throughout the kernel are attributes
+[gcc-attribute-syntax]_. Attributes allow to introduce
+implementation-defined semantics to language entities (like variables,
+functions or types) without having to make significant syntactic changes
+to the language (e.g. adding a new keyword) [n2049]_.
+
+In some cases, attributes are optional (i.e. a compiler not supporting them
+should still produce proper code, even if it is slower or does not perform
+as many compile-time checks/diagnostics).
+
+The kernel defines pseudo-keywords (e.g. ``__pure``) instead of using
+directly the GNU attribute syntax (e.g. ``__attribute__((__pure__))``)
+in order to feature detect which ones can be used and/or to shorten the code.
+
+Please refer to ``include/linux/compiler_attributes.h`` for more information.
+
+.. [c-language] http://www.open-std.org/jtc1/sc22/wg14/www/standards
+.. [gcc] https://gcc.gnu.org
+.. [clang] https://clang.llvm.org
+.. [icc] https://software.intel.com/en-us/c-compilers
+.. [gcc-c-dialect-options] https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html
+.. [gnu-extensions] https://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html
+.. [gcc-attribute-syntax] https://gcc.gnu.org/onlinedocs/gcc/Attribute-Syntax.html
+.. [n2049] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2049.pdf
+
-- 
2.17.1

Re: [PATCH v4 00/13] Compiler Attributes

[Luc Van Oostenryck]

On Sat, Sep 08, 2018 at 11:24:46PM +0200, Miguel Ojeda wrote:
> The Compiler Attributes series is an effort to disentangle
> the include/linux/compiler*.h headers and bring them up to date.
> 
> The main idea behind the series is to use feature checking macros
> (i.e. __has_attribute) instead of compiler version checks (e.g. GCC_VERSION),
> which are compiler-agnostic (so they can be shared, reducing the size
> of compiler-specific headers) and version-agnostic.
> 
> Other related improvements have been performed in the headers as well,
> which on top of the use of __has_attribute it has amounted to a significant
> simplification of these headers (e.g. GCC_VERSION is now only guarding 4
> non-attribute macros).
> 
> This series should also help the efforts to support compiling the kernel
> with clang and icc. A fair amount of documentation and comments have also
> been added, clarified or removed; and the headers are now more readable,
> which should help kernel developers in general.
> 
> The series was triggered due to the move to gcc >= 4.6. In turn, this series
> has also triggered Sparse to gain the ability to recognize __has_attribute
> on its own.
> 
> You can also fetch it from:
> 
>   https://github.com/ojeda/linux/tree/compiler-attributes-v4
> 
> Enjoy!

It's a nice and welcomed cleanup!
Everything look good to me, so feel free to add my
    Reviewed-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>


Cheers,
-- Luc

Re: [PATCH v4 00/13] Compiler Attributes

[Miguel Ojeda]

Hi Luc,

On Sun, Sep 9, 2018 at 10:02 AM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sat, Sep 08, 2018 at 11:24:46PM +0200, Miguel Ojeda wrote:
>> The Compiler Attributes series is an effort to disentangle
>> the include/linux/compiler*.h headers and bring them up to date.
>>
>> The main idea behind the series is to use feature checking macros
>> (i.e. __has_attribute) instead of compiler version checks (e.g. GCC_VERSION),
>> which are compiler-agnostic (so they can be shared, reducing the size
>> of compiler-specific headers) and version-agnostic.
>>
>> Other related improvements have been performed in the headers as well,
>> which on top of the use of __has_attribute it has amounted to a significant
>> simplification of these headers (e.g. GCC_VERSION is now only guarding 4
>> non-attribute macros).
>>
>> This series should also help the efforts to support compiling the kernel
>> with clang and icc. A fair amount of documentation and comments have also
>> been added, clarified or removed; and the headers are now more readable,
>> which should help kernel developers in general.
>>
>> The series was triggered due to the move to gcc >= 4.6. In turn, this series
>> has also triggered Sparse to gain the ability to recognize __has_attribute
>> on its own.
>>
>> You can also fetch it from:
>>
>>   https://github.com/ojeda/linux/tree/compiler-attributes-v4
>>
>> Enjoy!
>
> It's a nice and welcomed cleanup!
> Everything look good to me, so feel free to add my
>     Reviewed-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>

Thanks a lot for the review! :-)

Cheers,
Miguel

Re: [PATCH v4 00/13] Compiler Attributes

[Miguel Ojeda]

Hi all,

On Sat, Sep 8, 2018 at 11:24 PM, Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> You can also fetch it from:
>
>   https://github.com/ojeda/linux/tree/compiler-attributes-v4

In case it helps, I have also rebased it onto next-20180907 at:

  https://github.com/ojeda/linux/tree/compiler-attributes-for-next

The only conflict was the removal of __aligned_largest in f3569fd613f6
("crypto: shash - Remove VLA usage in unaligned hashing").

(Luc's Reviewed-by is not included in that version)

Cheers,
Miguel

Re: [PATCH v4 12/13] Compiler Attributes: add Doc/process/programming-language.rst

[Jonathan Corbet]

On Sat,  8 Sep 2018 23:24:58 +0200
Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:

>  Documentation/process/index.rst               |  1 +
>  .../process/programming-language.rst          | 45 +++++++++++++++++++
>  2 files changed, 46 insertions(+)
>  create mode 100644 Documentation/process/programming-language.rst
> 
So I have some overall thoughts on the documentation; my apologies for
not getting to this until you got to v4...

1) I think the document is mistitled.  It's not really about the language
   that the kernel used, it's about compiler attributes.  So I would make
   both the name of the document and it introduction reflect that.

2) This is an ideal opportunity to document what all of those attributes
   actually mean.  I would guess that is the information many developers
   will come here looking for, and they'll go away frustrated.  The ideal
   thing to do, IMO, would be do say what each attribute means (rather
   than just which compilers support it) in a DOC section in the new
   compiler_attributes.h header, then use RST directives to pull all that
   information into this document.

Thanks,

jon

Re: [PATCH v4 12/13] Compiler Attributes: add Doc/process/programming-language.rst

[Miguel Ojeda]

Hi Jonathan,

On Sun, Sep 9, 2018 at 8:19 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> On Sat,  8 Sep 2018 23:24:58 +0200
> Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:
>
>>  Documentation/process/index.rst               |  1 +
>>  .../process/programming-language.rst          | 45 +++++++++++++++++++
>>  2 files changed, 46 insertions(+)
>>  create mode 100644 Documentation/process/programming-language.rst
>>
> So I have some overall thoughts on the documentation; my apologies for
> not getting to this until you got to v4...

No problem at all! :-)

>
> 1) I think the document is mistitled.  It's not really about the language
>    that the kernel used, it's about compiler attributes.  So I would make
>    both the name of the document and it introduction reflect that.

The idea here is to create a document explaining the programming
language that the kernel uses (which we hadn't), taking the chance to
describe the attributes (which in a way are extensions to the C
language); in the future expanding it with other (non-attribute)
extensions that we use. Of course, that is not done, but I thought
starting it was a nice idea (meant for people coming from a C
background that do not know the "dialect" used in Linux).

Also other kind of very commonly used macros and functions used
throughout the kernel could be summarized there as a summary for
newcomers (e.g. stuff like `printk`/`pr_*`, `likely`/`unlikely`,
`panic`, `EXPORT_SYMBOL`, `BUG_ON`/`WARN_ON`...), without writing full
documentation there (but pointing to where they are described, either
in the Docs or a source code if not).

What do you think?

>
> 2) This is an ideal opportunity to document what all of those attributes
>    actually mean.  I would guess that is the information many developers

I thought about that, but the issue is that compilers already describe
them: in the case of GCC, all of them are documented; some are in
clang; icc is not well documented --- see the previous threads about
this), so we would be duplicating that information (and it will become
outdated as compilers are released and improve the documentation).
This is the reason why the series removes most of the copy-pasted
documentation from gcc and instead supplies a link for each attribute
compiler_attributes.h to gcc & clang online docs.

>    will come here looking for, and they'll go away frustrated.  The ideal
>    thing to do, IMO, would be do say what each attribute means (rather
>    than just which compilers support it) in a DOC section in the new
>    compiler_attributes.h header, then use RST directives to pull all that
>    information into this document.

Initially my idea was to provide a table with the name and the links
to each compiler docs; so that it could be easily used. However, when
thinking about it, it seems that most people consulting such file
would come from the actual source code, so they would have to move
from there to the Doc/ file; so I did it the other way around (IIRC
Nick also mentioned his preference for keeping them in the source
code).

Now, the idea of keeping them in the header but pulling them to the
RST can be a good idea but I was unsure how to do it best. I took a
look at how other RST files did it (the pull), but I thought (maybe
wrongly) that I wouldn't be able to create a nice table (i.e. instead
it would be a long list of attributes, which most people will not use
-- and in my idea of a document describing all the extensions, it
would be taking most of the space), so I discarded it. I am not sure
about the future goals for the Docs/, so excuse me if I have the
completely wrong idea, but in a way, in the current state, I see the
kernel docs as articles/chapters/book on high-level concepts about the
kernel, rather than a technical reference (i.e. the source code with a
better formatting).

Thanks a lot for reviewing!

Cheers,
Miguel

Re: [PATCH v4 00/13] Compiler Attributes

[Nick Desaulniers]

On Sat, Sep 8, 2018 at 2:25 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> The Compiler Attributes series is an effort to disentangle
> the include/linux/compiler*.h headers and bring them up to date.
>
> The main idea behind the series is to use feature checking macros
> (i.e. __has_attribute) instead of compiler version checks (e.g. GCC_VERSION),
> which are compiler-agnostic (so they can be shared, reducing the size
> of compiler-specific headers) and version-agnostic.
>
> Other related improvements have been performed in the headers as well,
> which on top of the use of __has_attribute it has amounted to a significant
> simplification of these headers (e.g. GCC_VERSION is now only guarding 4
> non-attribute macros).
>
> This series should also help the efforts to support compiling the kernel
> with clang and icc. A fair amount of documentation and comments have also
> been added, clarified or removed; and the headers are now more readable,
> which should help kernel developers in general.
>
> The series was triggered due to the move to gcc >= 4.6. In turn, this series
> has also triggered Sparse to gain the ability to recognize __has_attribute
> on its own.
>
> You can also fetch it from:
>
>   https://github.com/ojeda/linux/tree/compiler-attributes-v4
>
> Enjoy!
>
> Cheers,
> Miguel
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> Cc: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
> Cc: Eli Friedman <efriedma@codeaurora.org>
> Cc: Christopher Li <sparse@chrisli.org>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Geert Uytterhoeven <geert@linux-m68k.org>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: Joe Perches <joe@perches.com>
> Cc: Dominique Martinet <asmadeus@codewreck.org>
> Cc: Nick Desaulniers <ndesaulniers@google.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: linux-sparse@vger.kernel.org
> Cc: linux-doc@vger.kernel.org
> Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
>
> v3 -> v4
>
>   This time it is an important fix I found while doing randconfigs, plus
>   the documentation sentence that Nick suggested and the MAINTAINERS file.
>
>   Compile-tested for a while on (x86_64, gcc-7.3) on a few randconfs.
>
>   New:
>
>   * Add MAINTAINERS entry for the new file (compiler_attributes.h)
>     so that we try to maintain that one as clean as possible.
>     (if someone else wants to join, feel free!)
>
>   Modified:
>
>   * Fix inline macro
>
>     __always_inline cannot be used in the inline macro,
>     because people marking a function as __always_inline would
>     expand to inline which in turn would expand back to __always_inline
>     (under some configs), which would not be expanded again.
>     Added a comment about this in the inline macro.
>
>     Also added another comment about __always_inline in compiler_attributes.h
>     to note that users expect to not have to write inline (as well as
>     the attribute). We cannot simply remove "inline" there (which would
>     solve the problem described above) because using the attribute is
>     not enough for gcc. A function marked as always_inline seems to require
>     to be marked as inline itself so that the attribute is applied:
>
>       "warning: always_inline function might not be inlinable [-Wattributes]"
>
>     From the gcc docs:
>
>       "For functions declared inline, this attribute inlines the function
>        independent of any restrictions that otherwise apply to inlining."
>
>     See https://godbolt.org/z/LpzUPj for an example.
>
>   From reviews:
>
>   * Add sentence about feature-detection in Docs (Nick)
>
> Miguel Ojeda (13):
>   Compiler Attributes: remove unused attributes
>   Compiler Attributes: always use the extra-underscores syntax
>   Compiler Attributes: remove unneeded tests
>   Compiler Attributes: homogenize __must_be_array
>   Compiler Attributes: naked was fixed in gcc 4.6
>   Compiler Attributes: naked can be shared
>   Compiler Attributes: remove unneeded sparse (__CHECKER__) tests
>   Compiler Attributes: add missing SPDX ID in compiler_types.h
>   Compiler Attributes: use feature checks instead of version checks
>   Compiler Attributes: KENTRY used twice the "used" attribute
>   Compiler Attributes: remove uses of __attribute__ from compiler.h
>   Compiler Attributes: add Doc/process/programming-language.rst
>   Compiler Attributes: Add MAINTAINERS entry
>
>  Documentation/process/index.rst               |   1 +
>  .../process/programming-language.rst          |  45 ++++
>  MAINTAINERS                                   |   5 +
>  include/linux/compiler-clang.h                |   5 -
>  include/linux/compiler-gcc.h                  |  84 +-----
>  include/linux/compiler-intel.h                |   9 -
>  include/linux/compiler.h                      |  19 +-
>  include/linux/compiler_attributes.h           | 244 ++++++++++++++++++
>  include/linux/compiler_types.h                | 105 ++------
>  9 files changed, 329 insertions(+), 188 deletions(-)
>  create mode 100644 Documentation/process/programming-language.rst
>  create mode 100644 include/linux/compiler_attributes.h
>
> --
> 2.17.1
>

For the ones I didn't already review:
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

-- 
Thanks,
~Nick Desaulniers