[PATCH v3 0/8] Improve performance of VM translation on x86_64

[PATCH v3 0/8] Improve performance of VM translation on x86_64

[Alexander Duyck]

This patch series is meant to address several issues I encountered with VM
translations on x86_64.  In my testing I found that swiotlb was incurring up
to a 5% processing overhead due to calls to __phys_addr.  To address that I
have updated swiotlb to use physical addresses instead of virtual addresses
to reduce the need to call __phys_addr.  However those patches didn't address
the other callers.  With these patches applied I am able to achieve an
additional 1% to 2% performance gain on top of the changes to swiotlb.

The first 2 patches are the performance optimizations that result in the 1% to
2% increase in overall performance.  The remaining patches are various
cleanups for a number of spots where __pa or virt_to_phys was being called
and was not needed or __pa_symbol could have been used.

It doesn't seem like the v2 patch set was accepted so I am submitting an
updated v3 set that is rebased off of linux-next with a few additional
improvements to the existing patches.  Specifically the first patch now also
updates __virt_addr_valid so that it is almost identical in layout to
__phys_addr.  Also I found one additional spot in init_64.c that could use
__pa_symbol instead of virt_to_page calls so I updated the first __pa_symbol
patch for the x86 init calls.

With this patch set applied I am noticing a 1-2% improvement in performance in
my routing tests.  Without my earlier swiotlb changes applied it was getting as
high as 6-7% because that code originally relied heavily on virt_to_phys.

The overall effect on size varies depending on what kernel options are
enabled.  I have notices that almost all of the network device drivers have
dropped in size by around 100 bytes.  I suspect this is due to the fact that
the virt_to_page call in dma_map_single is now less expensive.  However the
default build for x86_64 increases the vmlinux size by 3.5K with this change
applied.

---

Alexander Duyck (8):
      x86/lguest: Use __pa_symbol instead of __pa on C visible symbols
      x86/acpi: Use __pa_symbol instead of __pa on C visible symbols
      x86/xen: Use __pa_symbol instead of __pa on C visible symbols
      x86/ftrace: Use __pa_symbol instead of __pa on C visible symbols
      x86: Use __pa_symbol instead of __pa on C visible symbols
      x86: Drop 4 unnecessary calls to __pa_symbol
      x86: Make it so that __pa_symbol can only process kernel symbols on x86_64
      x86: Improve __phys_addr performance by making use of carry flags and inlining


 arch/x86/include/asm/page.h          |    3 +-
 arch/x86/include/asm/page_32.h       |    1 +
 arch/x86/include/asm/page_64_types.h |   20 +++++++++++-
 arch/x86/kernel/acpi/sleep.c         |    2 +
 arch/x86/kernel/cpu/intel.c          |    2 +
 arch/x86/kernel/ftrace.c             |    4 +-
 arch/x86/kernel/head32.c             |    4 +-
 arch/x86/kernel/head64.c             |    4 +-
 arch/x86/kernel/setup.c              |   16 +++++-----
 arch/x86/kernel/x8664_ksyms_64.c     |    3 ++
 arch/x86/lguest/boot.c               |    3 +-
 arch/x86/mm/init_64.c                |   18 +++++------
 arch/x86/mm/pageattr.c               |    8 ++---
 arch/x86/mm/physaddr.c               |   55 +++++++++++++++++++++++++---------
 arch/x86/platform/efi/efi.c          |    4 +-
 arch/x86/realmode/init.c             |    8 ++---
 arch/x86/xen/mmu.c                   |   21 +++++++------
 17 files changed, 111 insertions(+), 65 deletions(-)

-- 

[PATCH v3 1/8] x86: Improve __phys_addr performance by making use of carry flags and inlining

[Alexander Duyck]

This patch is meant to improve overall system performance when making use of
the __phys_addr call.  To do this I have implemented several changes.

First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
similar to how this is currently handled in 32 bit.  However in order to do
this it is required to export phys_base so that it is available if __phys_addr
is used in kernel modules.

The second change was to streamline the code by making use of the carry flag
on an add operation instead of performing a compare on a 64 bit value.  The
advantage to this is that it allows us to significantly reduce the overall
size of the call.  On my Xeon E5 system the entire __phys_addr inline call
consumes a little less than 32 bytes and 5 instructions.  I also applied
similar logic to the debug version of the function.  My testing shows that the
debug version of the function with this patch applied is slightly faster than
the non-debug version without the patch.

When building the kernel with the first two changes applied I saw build
warnings about __START_KERNEL_map and PAGE_OFFSET constants not fitting in
their type.  In order to resolve the build warning I changed their type from
UL to ULL.

Finally I also applied the same logic changes to __virt_addr_valid since it
used the same general code flow as __phys_addr and could achieve similar gains
though these changes.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

v3:  Added changes to __virt_addr_valid to keep it in sync with __phys_addr

 arch/x86/include/asm/page_64_types.h |   17 +++++++++++++-
 arch/x86/kernel/x8664_ksyms_64.c     |    3 +++
 arch/x86/mm/physaddr.c               |   40 +++++++++++++++++++++-------------
 3 files changed, 43 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
index 320f7bb..1ca93d3 100644
--- a/arch/x86/include/asm/page_64_types.h
+++ b/arch/x86/include/asm/page_64_types.h
@@ -30,14 +30,14 @@
  * hypervisor to fit.  Choosing 16 slots here is arbitrary, but it's
  * what Xen requires.
  */
-#define __PAGE_OFFSET           _AC(0xffff880000000000, UL)
+#define __PAGE_OFFSET           _AC(0xffff880000000000, ULL)
 
 #define __PHYSICAL_START	((CONFIG_PHYSICAL_START +	 	\
 				  (CONFIG_PHYSICAL_ALIGN - 1)) &	\
 				 ~(CONFIG_PHYSICAL_ALIGN - 1))
 
 #define __START_KERNEL		(__START_KERNEL_map + __PHYSICAL_START)
-#define __START_KERNEL_map	_AC(0xffffffff80000000, UL)
+#define __START_KERNEL_map	_AC(0xffffffff80000000, ULL)
 
 /* See Documentation/x86/x86_64/mm.txt for a description of the memory map. */
 #define __PHYSICAL_MASK_SHIFT	46
@@ -58,7 +58,20 @@ void copy_page(void *to, void *from);
 extern unsigned long max_pfn;
 extern unsigned long phys_base;
 
+static inline unsigned long __phys_addr_nodebug(unsigned long x)
+{
+	unsigned long y = x - __START_KERNEL_map;
+
+	/* use the carry flag to determine if x was < __START_KERNEL_map */
+	x = y + ((x > y) ? phys_base : (__START_KERNEL_map - PAGE_OFFSET));
+
+	return x;
+}
+#ifdef CONFIG_DEBUG_VIRTUAL
 extern unsigned long __phys_addr(unsigned long);
+#else
+#define __phys_addr(x)		__phys_addr_nodebug(x)
+#endif
 #define __phys_reloc_hide(x)	(x)
 
 #define vmemmap ((struct page *)VMEMMAP_START)
diff --git a/arch/x86/kernel/x8664_ksyms_64.c b/arch/x86/kernel/x8664_ksyms_64.c
index 1330dd1..b014d94 100644
--- a/arch/x86/kernel/x8664_ksyms_64.c
+++ b/arch/x86/kernel/x8664_ksyms_64.c
@@ -59,6 +59,9 @@ EXPORT_SYMBOL(memcpy);
 EXPORT_SYMBOL(__memcpy);
 EXPORT_SYMBOL(memmove);
 
+#ifndef CONFIG_DEBUG_VIRTUAL
+EXPORT_SYMBOL(phys_base);
+#endif
 EXPORT_SYMBOL(empty_zero_page);
 #ifndef CONFIG_PARAVIRT
 EXPORT_SYMBOL(native_load_gs_index);
diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c
index d2e2735..fd40d75 100644
--- a/arch/x86/mm/physaddr.c
+++ b/arch/x86/mm/physaddr.c
@@ -8,33 +8,43 @@
 
 #ifdef CONFIG_X86_64
 
+#ifdef CONFIG_DEBUG_VIRTUAL
 unsigned long __phys_addr(unsigned long x)
 {
-	if (x >= __START_KERNEL_map) {
-		x -= __START_KERNEL_map;
-		VIRTUAL_BUG_ON(x >= KERNEL_IMAGE_SIZE);
-		x += phys_base;
+	unsigned long y = x - __START_KERNEL_map;
+
+	/* use the carry flag to determine if x was < __START_KERNEL_map */
+	if (unlikely(x > y)) {
+		x = y + phys_base;
+
+		VIRTUAL_BUG_ON(y >= KERNEL_IMAGE_SIZE);
 	} else {
-		VIRTUAL_BUG_ON(x < PAGE_OFFSET);
-		x -= PAGE_OFFSET;
-		VIRTUAL_BUG_ON(!phys_addr_valid(x));
+		x = y + (__START_KERNEL_map - PAGE_OFFSET);
+
+		/* carry flag will be set if starting x was >= PAGE_OFFSET */
+		VIRTUAL_BUG_ON((x > y) || !phys_addr_valid(x));
 	}
+
 	return x;
 }
 EXPORT_SYMBOL(__phys_addr);
+#endif
 
 bool __virt_addr_valid(unsigned long x)
 {
-	if (x >= __START_KERNEL_map) {
-		x -= __START_KERNEL_map;
-		if (x >= KERNEL_IMAGE_SIZE)
+	unsigned long y = x - __START_KERNEL_map;
+
+	/* use the carry flag to determine if x was < __START_KERNEL_map */
+	if (unlikely(x > y)) {
+		x = y + phys_base;
+
+		if (y >= KERNEL_IMAGE_SIZE)
 			return false;
-		x += phys_base;
 	} else {
-		if (x < PAGE_OFFSET)
-			return false;
-		x -= PAGE_OFFSET;
-		if (!phys_addr_valid(x))
+		x = y + (__START_KERNEL_map - PAGE_OFFSET);
+
+		/* carry flag will be set if starting x was >= PAGE_OFFSET */
+		if ((x > y) || !phys_addr_valid(x))
 			return false;
 	}
 

Re: [PATCH v3 1/8] x86: Improve __phys_addr performance by making use of carry flags and inlining

[Kirill A. Shutemov]

On Mon, Nov 05, 2012 at 11:04:06AM -0800, Alexander Duyck wrote:
> This patch is meant to improve overall system performance when making use of
> the __phys_addr call.  To do this I have implemented several changes.
> 
> First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
> similar to how this is currently handled in 32 bit.  However in order to do
> this it is required to export phys_base so that it is available if __phys_addr
> is used in kernel modules.
> 
> The second change was to streamline the code by making use of the carry flag
> on an add operation instead of performing a compare on a 64 bit value.  The
> advantage to this is that it allows us to significantly reduce the overall
> size of the call.  On my Xeon E5 system the entire __phys_addr inline call
> consumes a little less than 32 bytes and 5 instructions.  I also applied
> similar logic to the debug version of the function.  My testing shows that the
> debug version of the function with this patch applied is slightly faster than
> the non-debug version without the patch.
> 
> When building the kernel with the first two changes applied I saw build
> warnings about __START_KERNEL_map and PAGE_OFFSET constants not fitting in
> their type.  In order to resolve the build warning I changed their type from
> UL to ULL.

What kind of warning messages did you see?
It's strange: sizeof(unsinged long) == sizeof(unsinged long long) on
x86_64

> 
> Finally I also applied the same logic changes to __virt_addr_valid since it
> used the same general code flow as __phys_addr and could achieve similar gains
> though these changes.
> 
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
> ---
> 
> v3:  Added changes to __virt_addr_valid to keep it in sync with __phys_addr
> 
>  arch/x86/include/asm/page_64_types.h |   17 +++++++++++++-
>  arch/x86/kernel/x8664_ksyms_64.c     |    3 +++
>  arch/x86/mm/physaddr.c               |   40 +++++++++++++++++++++-------------
>  3 files changed, 43 insertions(+), 17 deletions(-)
> 
> diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
> index 320f7bb..1ca93d3 100644
> --- a/arch/x86/include/asm/page_64_types.h
> +++ b/arch/x86/include/asm/page_64_types.h
> @@ -30,14 +30,14 @@
>   * hypervisor to fit.  Choosing 16 slots here is arbitrary, but it's
>   * what Xen requires.
>   */
> -#define __PAGE_OFFSET           _AC(0xffff880000000000, UL)
> +#define __PAGE_OFFSET           _AC(0xffff880000000000, ULL)
>  
>  #define __PHYSICAL_START	((CONFIG_PHYSICAL_START +	 	\
>  				  (CONFIG_PHYSICAL_ALIGN - 1)) &	\
>  				 ~(CONFIG_PHYSICAL_ALIGN - 1))
>  
>  #define __START_KERNEL		(__START_KERNEL_map + __PHYSICAL_START)
> -#define __START_KERNEL_map	_AC(0xffffffff80000000, UL)
> +#define __START_KERNEL_map	_AC(0xffffffff80000000, ULL)
>  
>  /* See Documentation/x86/x86_64/mm.txt for a description of the memory map. */
>  #define __PHYSICAL_MASK_SHIFT	46
> @@ -58,7 +58,20 @@ void copy_page(void *to, void *from);
>  extern unsigned long max_pfn;
>  extern unsigned long phys_base;
>  
> +static inline unsigned long __phys_addr_nodebug(unsigned long x)
> +{
> +	unsigned long y = x - __START_KERNEL_map;

With change above you assign ULL const to unsigned long variable. hm?

> +
> +	/* use the carry flag to determine if x was < __START_KERNEL_map */
> +	x = y + ((x > y) ? phys_base : (__START_KERNEL_map - PAGE_OFFSET));
> +
> +	return x;
> +}
> +#ifdef CONFIG_DEBUG_VIRTUAL
>  extern unsigned long __phys_addr(unsigned long);
> +#else
> +#define __phys_addr(x)		__phys_addr_nodebug(x)
> +#endif
>  #define __phys_reloc_hide(x)	(x)
>  
>  #define vmemmap ((struct page *)VMEMMAP_START)
> diff --git a/arch/x86/kernel/x8664_ksyms_64.c b/arch/x86/kernel/x8664_ksyms_64.c
> index 1330dd1..b014d94 100644
> --- a/arch/x86/kernel/x8664_ksyms_64.c
> +++ b/arch/x86/kernel/x8664_ksyms_64.c
> @@ -59,6 +59,9 @@ EXPORT_SYMBOL(memcpy);
>  EXPORT_SYMBOL(__memcpy);
>  EXPORT_SYMBOL(memmove);
>  
> +#ifndef CONFIG_DEBUG_VIRTUAL
> +EXPORT_SYMBOL(phys_base);
> +#endif
>  EXPORT_SYMBOL(empty_zero_page);
>  #ifndef CONFIG_PARAVIRT
>  EXPORT_SYMBOL(native_load_gs_index);
> diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c
> index d2e2735..fd40d75 100644
> --- a/arch/x86/mm/physaddr.c
> +++ b/arch/x86/mm/physaddr.c
> @@ -8,33 +8,43 @@
>  
>  #ifdef CONFIG_X86_64
>  
> +#ifdef CONFIG_DEBUG_VIRTUAL
>  unsigned long __phys_addr(unsigned long x)
>  {
> -	if (x >= __START_KERNEL_map) {
> -		x -= __START_KERNEL_map;
> -		VIRTUAL_BUG_ON(x >= KERNEL_IMAGE_SIZE);
> -		x += phys_base;
> +	unsigned long y = x - __START_KERNEL_map;
> +
> +	/* use the carry flag to determine if x was < __START_KERNEL_map */
> +	if (unlikely(x > y)) {
> +		x = y + phys_base;
> +
> +		VIRTUAL_BUG_ON(y >= KERNEL_IMAGE_SIZE);
>  	} else {
> -		VIRTUAL_BUG_ON(x < PAGE_OFFSET);
> -		x -= PAGE_OFFSET;
> -		VIRTUAL_BUG_ON(!phys_addr_valid(x));
> +		x = y + (__START_KERNEL_map - PAGE_OFFSET);
> +
> +		/* carry flag will be set if starting x was >= PAGE_OFFSET */
> +		VIRTUAL_BUG_ON((x > y) || !phys_addr_valid(x));
>  	}
> +
>  	return x;
>  }
>  EXPORT_SYMBOL(__phys_addr);
> +#endif
>  
>  bool __virt_addr_valid(unsigned long x)
>  {
> -	if (x >= __START_KERNEL_map) {
> -		x -= __START_KERNEL_map;
> -		if (x >= KERNEL_IMAGE_SIZE)
> +	unsigned long y = x - __START_KERNEL_map;
> +
> +	/* use the carry flag to determine if x was < __START_KERNEL_map */
> +	if (unlikely(x > y)) {
> +		x = y + phys_base;
> +
> +		if (y >= KERNEL_IMAGE_SIZE)
>  			return false;
> -		x += phys_base;
>  	} else {
> -		if (x < PAGE_OFFSET)
> -			return false;
> -		x -= PAGE_OFFSET;
> -		if (!phys_addr_valid(x))
> +		x = y + (__START_KERNEL_map - PAGE_OFFSET);
> +
> +		/* carry flag will be set if starting x was >= PAGE_OFFSET */
> +		if ((x > y) || !phys_addr_valid(x))
>  			return false;
>  	}
>  
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
 Kirill A. Shutemov

Re: [PATCH v3 1/8] x86: Improve __phys_addr performance by making use of carry flags and inlining

[Alexander Duyck]

On 11/05/2012 12:24 PM, Kirill A. Shutemov wrote:
> On Mon, Nov 05, 2012 at 11:04:06AM -0800, Alexander Duyck wrote:
>> This patch is meant to improve overall system performance when making use of
>> the __phys_addr call.  To do this I have implemented several changes.
>>
>> First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
>> similar to how this is currently handled in 32 bit.  However in order to do
>> this it is required to export phys_base so that it is available if __phys_addr
>> is used in kernel modules.
>>
>> The second change was to streamline the code by making use of the carry flag
>> on an add operation instead of performing a compare on a 64 bit value.  The
>> advantage to this is that it allows us to significantly reduce the overall
>> size of the call.  On my Xeon E5 system the entire __phys_addr inline call
>> consumes a little less than 32 bytes and 5 instructions.  I also applied
>> similar logic to the debug version of the function.  My testing shows that the
>> debug version of the function with this patch applied is slightly faster than
>> the non-debug version without the patch.
>>
>> When building the kernel with the first two changes applied I saw build
>> warnings about __START_KERNEL_map and PAGE_OFFSET constants not fitting in
>> their type.  In order to resolve the build warning I changed their type from
>> UL to ULL.
> What kind of warning messages did you see?
> It's strange: sizeof(unsinged long) == sizeof(unsinged long long) on
> x86_64

One of the warnings is included below:

In file included from /usr/src/kernels/linux-next/arch/x86/include/asm/page_types.h:37,
                 from /usr/src/kernels/linux-next/arch/x86/include/asm/pgtable_types.h:5,
                 from /usr/src/kernels/linux-next/arch/x86/include/asm/boot.h:11,
                 from arch/x86/realmode/rm/../../boot/boot.h:26,
                 from arch/x86/realmode/rm/../../boot/regs.c:19,
                 from arch/x86/realmode/rm/regs.c:1:
/usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h: In function '__phys_addr_nodebug':
/usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:63: warning: integer constant is too large for 'unsigned long' type
/usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type
/usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type

The warnings all seemed to originate from several different spots
throughout the x86 tree.  All of the warning messages include
arch/x86/boot/boot.h:26 and then from there up the included from list is
always the same.

Thanks,

Alex

Re: [PATCH v3 1/8] x86: Improve __phys_addr performance by making use of carry flags and inlining

[Kirill A. Shutemov]

On Mon, Nov 05, 2012 at 01:56:28PM -0800, Alexander Duyck wrote:
> On 11/05/2012 12:24 PM, Kirill A. Shutemov wrote:
> > On Mon, Nov 05, 2012 at 11:04:06AM -0800, Alexander Duyck wrote:
> >> This patch is meant to improve overall system performance when making use of
> >> the __phys_addr call.  To do this I have implemented several changes.
> >>
> >> First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
> >> similar to how this is currently handled in 32 bit.  However in order to do
> >> this it is required to export phys_base so that it is available if __phys_addr
> >> is used in kernel modules.
> >>
> >> The second change was to streamline the code by making use of the carry flag
> >> on an add operation instead of performing a compare on a 64 bit value.  The
> >> advantage to this is that it allows us to significantly reduce the overall
> >> size of the call.  On my Xeon E5 system the entire __phys_addr inline call
> >> consumes a little less than 32 bytes and 5 instructions.  I also applied
> >> similar logic to the debug version of the function.  My testing shows that the
> >> debug version of the function with this patch applied is slightly faster than
> >> the non-debug version without the patch.
> >>
> >> When building the kernel with the first two changes applied I saw build
> >> warnings about __START_KERNEL_map and PAGE_OFFSET constants not fitting in
> >> their type.  In order to resolve the build warning I changed their type from
> >> UL to ULL.
> > What kind of warning messages did you see?
> > It's strange: sizeof(unsinged long) == sizeof(unsinged long long) on
> > x86_64
> 
> One of the warnings is included below:
> 
> In file included from /usr/src/kernels/linux-next/arch/x86/include/asm/page_types.h:37,
>                  from /usr/src/kernels/linux-next/arch/x86/include/asm/pgtable_types.h:5,
>                  from /usr/src/kernels/linux-next/arch/x86/include/asm/boot.h:11,
>                  from arch/x86/realmode/rm/../../boot/boot.h:26,
>                  from arch/x86/realmode/rm/../../boot/regs.c:19,
>                  from arch/x86/realmode/rm/regs.c:1:
> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h: In function '__phys_addr_nodebug':
> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:63: warning: integer constant is too large for 'unsigned long' type
> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type
> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type
> 
> The warnings all seemed to originate from several different spots
> throughout the x86 tree.  All of the warning messages include
> arch/x86/boot/boot.h:26 and then from there up the included from list is
> always the same.

Realmode code compiles with -m32. I guess it's just wrong that it tries to
include <asm/page_64_types.h>.

-- 
 Kirill A. Shutemov

Re: [PATCH v3 1/8] x86: Improve __phys_addr performance by making use of carry flags and inlining

[Alexander Duyck]

On 11/05/2012 02:08 PM, Kirill A. Shutemov wrote:
> On Mon, Nov 05, 2012 at 01:56:28PM -0800, Alexander Duyck wrote:
>> On 11/05/2012 12:24 PM, Kirill A. Shutemov wrote:
>>> On Mon, Nov 05, 2012 at 11:04:06AM -0800, Alexander Duyck wrote:
>>>> This patch is meant to improve overall system performance when making use of
>>>> the __phys_addr call.  To do this I have implemented several changes.
>>>>
>>>> First if CONFIG_DEBUG_VIRTUAL is not defined __phys_addr is made an inline,
>>>> similar to how this is currently handled in 32 bit.  However in order to do
>>>> this it is required to export phys_base so that it is available if __phys_addr
>>>> is used in kernel modules.
>>>>
>>>> The second change was to streamline the code by making use of the carry flag
>>>> on an add operation instead of performing a compare on a 64 bit value.  The
>>>> advantage to this is that it allows us to significantly reduce the overall
>>>> size of the call.  On my Xeon E5 system the entire __phys_addr inline call
>>>> consumes a little less than 32 bytes and 5 instructions.  I also applied
>>>> similar logic to the debug version of the function.  My testing shows that the
>>>> debug version of the function with this patch applied is slightly faster than
>>>> the non-debug version without the patch.
>>>>
>>>> When building the kernel with the first two changes applied I saw build
>>>> warnings about __START_KERNEL_map and PAGE_OFFSET constants not fitting in
>>>> their type.  In order to resolve the build warning I changed their type from
>>>> UL to ULL.
>>> What kind of warning messages did you see?
>>> It's strange: sizeof(unsinged long) == sizeof(unsinged long long) on
>>> x86_64
>> One of the warnings is included below:
>>
>> In file included from /usr/src/kernels/linux-next/arch/x86/include/asm/page_types.h:37,
>>                  from /usr/src/kernels/linux-next/arch/x86/include/asm/pgtable_types.h:5,
>>                  from /usr/src/kernels/linux-next/arch/x86/include/asm/boot.h:11,
>>                  from arch/x86/realmode/rm/../../boot/boot.h:26,
>>                  from arch/x86/realmode/rm/../../boot/regs.c:19,
>>                  from arch/x86/realmode/rm/regs.c:1:
>> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h: In function '__phys_addr_nodebug':
>> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:63: warning: integer constant is too large for 'unsigned long' type
>> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type
>> /usr/src/kernels/linux-next/arch/x86/include/asm/page_64_types.h:66: warning: integer constant is too large for 'unsigned long' type
>>
>> The warnings all seemed to originate from several different spots
>> throughout the x86 tree.  All of the warning messages include
>> arch/x86/boot/boot.h:26 and then from there up the included from list is
>> always the same.
> Realmode code compiles with -m32. I guess it's just wrong that it tries to
> include <asm/page_64_types.h>.
>

I have been reviewing things and I think the problem is due to the fact
that we have content in page_64_types.h that doesn't really belong
there.  I will add a patch that moves the virtual to physical address
translation header contents over to page_64.h.  That will keep it
consistent with where it is in the 32 bit build (page_32.h) and avoids
the build conflicts.

Expect a v4 of the patches with this fixed in the next few hours.

Thanks,

Alex

[PATCH v3 2/8] x86: Make it so that __pa_symbol can only process kernel symbols on x86_64

[Alexander Duyck]

I submitted an earlier patch that make __phys_addr an inline.  This obviously
results in an increase in the code size.  One step I can take to reduce that
is to make it so that the __pa_symbol call does a direct translation for
kernel addresses instead of covering all of virtual memory.

On my system this reduced the size for __pa_symbol from 5 instructions
totalling 30 bytes to 3 instructions totalling 16 bytes.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/include/asm/page.h          |    3 ++-
 arch/x86/include/asm/page_32.h       |    1 +
 arch/x86/include/asm/page_64_types.h |    3 +++
 arch/x86/mm/physaddr.c               |   15 +++++++++++++++
 4 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h
index 8ca8283..3698a6a 100644
--- a/arch/x86/include/asm/page.h
+++ b/arch/x86/include/asm/page.h
@@ -44,7 +44,8 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr,
  * case properly. Once all supported versions of gcc understand it, we can
  * remove this Voodoo magic stuff. (i.e. once gcc3.x is deprecated)
  */
-#define __pa_symbol(x)	__pa(__phys_reloc_hide((unsigned long)(x)))
+#define __pa_symbol(x) \
+	__phys_addr_symbol(__phys_reloc_hide((unsigned long)(x)))
 
 #define __va(x)			((void *)((unsigned long)(x)+PAGE_OFFSET))
 
diff --git a/arch/x86/include/asm/page_32.h b/arch/x86/include/asm/page_32.h
index da4e762..4d550d0 100644
--- a/arch/x86/include/asm/page_32.h
+++ b/arch/x86/include/asm/page_32.h
@@ -15,6 +15,7 @@ extern unsigned long __phys_addr(unsigned long);
 #else
 #define __phys_addr(x)		__phys_addr_nodebug(x)
 #endif
+#define __phys_addr_symbol(x)	__phys_addr(x)
 #define __phys_reloc_hide(x)	RELOC_HIDE((x), 0)
 
 #ifdef CONFIG_FLATMEM
diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
index 1ca93d3..a130589 100644
--- a/arch/x86/include/asm/page_64_types.h
+++ b/arch/x86/include/asm/page_64_types.h
@@ -69,8 +69,11 @@ static inline unsigned long __phys_addr_nodebug(unsigned long x)
 }
 #ifdef CONFIG_DEBUG_VIRTUAL
 extern unsigned long __phys_addr(unsigned long);
+extern unsigned long __phys_addr_symbol(unsigned long);
 #else
 #define __phys_addr(x)		__phys_addr_nodebug(x)
+#define __phys_addr_symbol(x) \
+	((unsigned long)(x) - __START_KERNEL_map + phys_base)
 #endif
 #define __phys_reloc_hide(x)	(x)
 
diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c
index fd40d75..8420708 100644
--- a/arch/x86/mm/physaddr.c
+++ b/arch/x86/mm/physaddr.c
@@ -28,6 +28,21 @@ unsigned long __phys_addr(unsigned long x)
 	return x;
 }
 EXPORT_SYMBOL(__phys_addr);
+
+unsigned long __phys_addr_symbol(unsigned long x)
+{
+	unsigned long y = x - __START_KERNEL_map;
+
+	/* use the carry flag to determine if x was < __START_KERNEL_map */
+	VIRTUAL_BUG_ON(x < y);
+
+	x = y + phys_base;
+
+	VIRTUAL_BUG_ON(y >= KERNEL_IMAGE_SIZE);
+
+	return x;
+}
+EXPORT_SYMBOL(__phys_addr_symbol);
 #endif
 
 bool __virt_addr_valid(unsigned long x)

[PATCH v3 3/8] x86: Drop 4 unnecessary calls to __pa_symbol

[Alexander Duyck]

While debugging the __pa_symbol inline patch I found that there were a couple
spots where __pa_symbol was used as follows:
__pa_symbol(x) - __pa_symbol(y)

The compiler had reduced them to:
x - y

Since we also support a debug case where __pa_symbol is a function call it
would probably be useful to just change the two cases I found so that they are
always just treated as "x - y".  As such I am casting the values to
phys_addr_t and then doing simple subtraction so that the correct type and
value is returned.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/kernel/head32.c |    4 ++--
 arch/x86/kernel/head64.c |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/head32.c b/arch/x86/kernel/head32.c
index c18f59d..f15db0c 100644
--- a/arch/x86/kernel/head32.c
+++ b/arch/x86/kernel/head32.c
@@ -30,8 +30,8 @@ static void __init i386_default_early_setup(void)
 
 void __init i386_start_kernel(void)
 {
-	memblock_reserve(__pa_symbol(&_text),
-			 __pa_symbol(&__bss_stop) - __pa_symbol(&_text));
+	memblock_reserve(__pa_symbol(_text),
+			 (phys_addr_t)__bss_stop - (phys_addr_t)_text);
 
 #ifdef CONFIG_BLK_DEV_INITRD
 	/* Reserve INITRD */
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 037df57..42f5df1 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -97,8 +97,8 @@ void __init x86_64_start_reservations(char *real_mode_data)
 {
 	copy_bootdata(__va(real_mode_data));
 
-	memblock_reserve(__pa_symbol(&_text),
-			 __pa_symbol(&__bss_stop) - __pa_symbol(&_text));
+	memblock_reserve(__pa_symbol(_text),
+			 (phys_addr_t)__bss_stop - (phys_addr_t)_text);
 
 #ifdef CONFIG_BLK_DEV_INITRD
 	/* Reserve INITRD */

[PATCH v3 4/8] x86: Use __pa_symbol instead of __pa on C visible symbols

[Alexander Duyck]

When I made an attempt at separating __pa_symbol and __pa I found that there
were a number of cases where __pa was used on an obvious symbol.

I also caught one non-obvious case as _brk_start and _brk_end are based on the
address of __brk_base which is a C visible symbol.

In mark_rodata_ro I was able to reduce the overhead of kernel symbol to
virtual memory translation by using a combination of __va(__pa_symbol())
instead of page_address(virt_to_page()).

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

v3:  Added changes to init_64.c function mark_rodata_ro to avoid unnecessary
     conversion to and from a page when all that is wanted is a virtual
     address.

 arch/x86/kernel/cpu/intel.c |    2 +-
 arch/x86/kernel/setup.c     |   16 ++++++++--------
 arch/x86/mm/init_64.c       |   18 ++++++++----------
 arch/x86/mm/pageattr.c      |    8 ++++----
 arch/x86/platform/efi/efi.c |    4 ++--
 arch/x86/realmode/init.c    |    8 ++++----
 6 files changed, 27 insertions(+), 29 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 198e019..2249e7e 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -168,7 +168,7 @@ int __cpuinit ppro_with_ram_bug(void)
 #ifdef CONFIG_X86_F00F_BUG
 static void __cpuinit trap_init_f00f_bug(void)
 {
-	__set_fixmap(FIX_F00F_IDT, __pa(&idt_table), PAGE_KERNEL_RO);
+	__set_fixmap(FIX_F00F_IDT, __pa_symbol(idt_table), PAGE_KERNEL_RO);
 
 	/*
 	 * Update the IDT descriptor and reload the IDT so that
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index e800bc6..4343570 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -300,8 +300,8 @@ static void __init cleanup_highmap(void)
 static void __init reserve_brk(void)
 {
 	if (_brk_end > _brk_start)
-		memblock_reserve(__pa(_brk_start),
-				 __pa(_brk_end) - __pa(_brk_start));
+		memblock_reserve(__pa_symbol(_brk_start),
+				 _brk_end - _brk_start);
 
 	/* Mark brk area as locked down and no longer taking any
 	   new allocations */
@@ -761,12 +761,12 @@ void __init setup_arch(char **cmdline_p)
 	init_mm.end_data = (unsigned long) _edata;
 	init_mm.brk = _brk_end;
 
-	code_resource.start = virt_to_phys(_text);
-	code_resource.end = virt_to_phys(_etext)-1;
-	data_resource.start = virt_to_phys(_etext);
-	data_resource.end = virt_to_phys(_edata)-1;
-	bss_resource.start = virt_to_phys(&__bss_start);
-	bss_resource.end = virt_to_phys(&__bss_stop)-1;
+	code_resource.start = __pa_symbol(_text);
+	code_resource.end = __pa_symbol(_etext)-1;
+	data_resource.start = __pa_symbol(_etext);
+	data_resource.end = __pa_symbol(_edata)-1;
+	bss_resource.start = __pa_symbol(__bss_start);
+	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
 #ifdef CONFIG_CMDLINE_BOOL
 #ifdef CONFIG_CMDLINE_OVERRIDE
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index 3baff25..0374a10 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -770,12 +770,10 @@ void set_kernel_text_ro(void)
 void mark_rodata_ro(void)
 {
 	unsigned long start = PFN_ALIGN(_text);
-	unsigned long rodata_start =
-		((unsigned long)__start_rodata + PAGE_SIZE - 1) & PAGE_MASK;
+	unsigned long rodata_start = PFN_ALIGN(__start_rodata);
 	unsigned long end = (unsigned long) &__end_rodata_hpage_align;
-	unsigned long text_end = PAGE_ALIGN((unsigned long) &__stop___ex_table);
-	unsigned long rodata_end = PAGE_ALIGN((unsigned long) &__end_rodata);
-	unsigned long data_start = (unsigned long) &_sdata;
+	unsigned long text_end = PFN_ALIGN(&__stop___ex_table);
+	unsigned long rodata_end = PFN_ALIGN(&__end_rodata);
 
 	printk(KERN_INFO "Write protecting the kernel read-only data: %luk\n",
 	       (end - start) >> 10);
@@ -800,12 +798,12 @@ void mark_rodata_ro(void)
 #endif
 
 	free_init_pages("unused kernel memory",
-			(unsigned long) page_address(virt_to_page(text_end)),
-			(unsigned long)
-				 page_address(virt_to_page(rodata_start)));
+			(unsigned long) __va(__pa_symbol(text_end)),
+			(unsigned long) __va(__pa_symbol(rodata_start)));
+
 	free_init_pages("unused kernel memory",
-			(unsigned long) page_address(virt_to_page(rodata_end)),
-			(unsigned long) page_address(virt_to_page(data_start)));
+			(unsigned long) __va(__pa_symbol(rodata_end)),
+			(unsigned long) __va(__pa_symbol(_sdata)));
 }
 
 #endif
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index a718e0d..40f92f3 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -94,12 +94,12 @@ static inline void split_page_count(int level) { }
 
 static inline unsigned long highmap_start_pfn(void)
 {
-	return __pa(_text) >> PAGE_SHIFT;
+	return __pa_symbol(_text) >> PAGE_SHIFT;
 }
 
 static inline unsigned long highmap_end_pfn(void)
 {
-	return __pa(roundup(_brk_end, PMD_SIZE)) >> PAGE_SHIFT;
+	return __pa_symbol(roundup(_brk_end, PMD_SIZE)) >> PAGE_SHIFT;
 }
 
 #endif
@@ -276,8 +276,8 @@ static inline pgprot_t static_protections(pgprot_t prot, unsigned long address,
 	 * The .rodata section needs to be read-only. Using the pfn
 	 * catches all aliases.
 	 */
-	if (within(pfn, __pa((unsigned long)__start_rodata) >> PAGE_SHIFT,
-		   __pa((unsigned long)__end_rodata) >> PAGE_SHIFT))
+	if (within(pfn, __pa_symbol(__start_rodata) >> PAGE_SHIFT,
+		   __pa_symbol(__end_rodata) >> PAGE_SHIFT))
 		pgprot_val(forbidden) |= _PAGE_RW;
 
 #if defined(CONFIG_X86_64) && defined(CONFIG_DEBUG_RODATA)
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index ad44391..1b60026 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -410,8 +410,8 @@ void __init efi_reserve_boot_services(void)
 		 * - Not within any part of the kernel
 		 * - Not the bios reserved area
 		*/
-		if ((start+size >= virt_to_phys(_text)
-				&& start <= virt_to_phys(_end)) ||
+		if ((start+size >= __pa_symbol(_text)
+				&& start <= __pa_symbol(_end)) ||
 			!e820_all_mapped(start, start+size, E820_RAM) ||
 			memblock_is_region_reserved(start, size)) {
 			/* Could not reserve, skip it */
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index cbca565..8045026 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -62,9 +62,9 @@ void __init setup_real_mode(void)
 		__va(real_mode_header->trampoline_header);
 
 #ifdef CONFIG_X86_32
-	trampoline_header->start = __pa(startup_32_smp);
+	trampoline_header->start = __pa_symbol(startup_32_smp);
 	trampoline_header->gdt_limit = __BOOT_DS + 7;
-	trampoline_header->gdt_base = __pa(boot_gdt);
+	trampoline_header->gdt_base = __pa_symbol(boot_gdt);
 #else
 	/*
 	 * Some AMD processors will #GP(0) if EFER.LMA is set in WRMSR
@@ -78,8 +78,8 @@ void __init setup_real_mode(void)
 	*trampoline_cr4_features = read_cr4();
 
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
-	trampoline_pgd[0] = __pa(level3_ident_pgt) + _KERNPG_TABLE;
-	trampoline_pgd[511] = __pa(level3_kernel_pgt) + _KERNPG_TABLE;
+	trampoline_pgd[0] = __pa_symbol(level3_ident_pgt) + _KERNPG_TABLE;
+	trampoline_pgd[511] = __pa_symbol(level3_kernel_pgt) + _KERNPG_TABLE;
 #endif
 }
 

[PATCH v3 5/8] x86/ftrace: Use __pa_symbol instead of __pa on C visible symbols

[Alexander Duyck]

Instead of using __pa which is meant to be a general function for converting
virtual addresses to physical addresses we can use __pa_symbol which is the
preferred way of decoding kernel text virtual addresses to physical addresses.

In this case we are not directly converting C visible symbols however if we
know that the instruction pointer is somewhere between _text and _etext we
know that we are going to be translating an address form the kernel text
space.

Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/kernel/ftrace.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 1d41402..42a392a 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -89,7 +89,7 @@ do_ftrace_mod_code(unsigned long ip, const void *new_code)
 	 * kernel identity mapping to modify code.
 	 */
 	if (within(ip, (unsigned long)_text, (unsigned long)_etext))
-		ip = (unsigned long)__va(__pa(ip));
+		ip = (unsigned long)__va(__pa_symbol(ip));
 
 	return probe_kernel_write((void *)ip, new_code, MCOUNT_INSN_SIZE);
 }
@@ -279,7 +279,7 @@ static int ftrace_write(unsigned long ip, const char *val, int size)
 	 * kernel identity mapping to modify code.
 	 */
 	if (within(ip, (unsigned long)_text, (unsigned long)_etext))
-		ip = (unsigned long)__va(__pa(ip));
+		ip = (unsigned long)__va(__pa_symbol(ip));
 
 	return probe_kernel_write((void *)ip, val, size);
 }

[PATCH v3 6/8] x86/xen: Use __pa_symbol instead of __pa on C visible symbols

[Alexander Duyck]

This change updates a few of the functions to use __pa_symbol when
translating C visible symbols instead of __pa.  By using __pa_symbol we are
able to drop a few extra lines of code as don't have to test to see if the
virtual pointer is a part of the kernel text or just standard virtual memory.

Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/xen/mmu.c |   21 +++++++++++----------
 1 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 4a05b39..a63e5f9 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -1486,7 +1486,8 @@ static int xen_pgd_alloc(struct mm_struct *mm)
 
 		if (user_pgd != NULL) {
 			user_pgd[pgd_index(VSYSCALL_START)] =
-				__pgd(__pa(level3_user_vsyscall) | _PAGE_TABLE);
+				__pgd(__pa_symbol(level3_user_vsyscall) |
+				      _PAGE_TABLE);
 			ret = 0;
 		}
 
@@ -1958,10 +1959,10 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
 	 * pgd.
 	 */
 	if (xen_feature(XENFEAT_writable_page_tables)) {
-		native_write_cr3(__pa(init_level4_pgt));
+		native_write_cr3(__pa_symbol(init_level4_pgt));
 	} else {
 		xen_mc_batch();
-		__xen_write_cr3(true, __pa(init_level4_pgt));
+		__xen_write_cr3(true, __pa_symbol(init_level4_pgt));
 		xen_mc_issue(PARAVIRT_LAZY_CPU);
 	}
 	/* We can't that easily rip out L3 and L2, as the Xen pagetables are
@@ -1984,10 +1985,10 @@ static RESERVE_BRK_ARRAY(pmd_t, swapper_kernel_pmd, PTRS_PER_PMD);
 
 static void __init xen_write_cr3_init(unsigned long cr3)
 {
-	unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir));
+	unsigned long pfn = PFN_DOWN(__pa_symbol(swapper_pg_dir));
 
-	BUG_ON(read_cr3() != __pa(initial_page_table));
-	BUG_ON(cr3 != __pa(swapper_pg_dir));
+	BUG_ON(read_cr3() != __pa_symbol(initial_page_table));
+	BUG_ON(cr3 != __pa_symbol(swapper_pg_dir));
 
 	/*
 	 * We are switching to swapper_pg_dir for the first time (from
@@ -2011,7 +2012,7 @@ static void __init xen_write_cr3_init(unsigned long cr3)
 	pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE, pfn);
 
 	pin_pagetable_pfn(MMUEXT_UNPIN_TABLE,
-			  PFN_DOWN(__pa(initial_page_table)));
+			  PFN_DOWN(__pa_symbol(initial_page_table)));
 	set_page_prot(initial_page_table, PAGE_KERNEL);
 	set_page_prot(initial_kernel_pmd, PAGE_KERNEL);
 
@@ -2036,7 +2037,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
 
 	copy_page(initial_page_table, pgd);
 	initial_page_table[KERNEL_PGD_BOUNDARY] =
-		__pgd(__pa(initial_kernel_pmd) | _PAGE_PRESENT);
+		__pgd(__pa_symbol(initial_kernel_pmd) | _PAGE_PRESENT);
 
 	set_page_prot(initial_kernel_pmd, PAGE_KERNEL_RO);
 	set_page_prot(initial_page_table, PAGE_KERNEL_RO);
@@ -2045,8 +2046,8 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
 	pin_pagetable_pfn(MMUEXT_UNPIN_TABLE, PFN_DOWN(__pa(pgd)));
 
 	pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE,
-			  PFN_DOWN(__pa(initial_page_table)));
-	xen_write_cr3(__pa(initial_page_table));
+			  PFN_DOWN(__pa_symbol(initial_page_table)));
+	xen_write_cr3(__pa_symbol(initial_page_table));
 
 	memblock_reserve(__pa(xen_start_info->pt_base),
 			 xen_start_info->nr_pt_frames * PAGE_SIZE);

Re: [PATCH v3 6/8] x86/xen: Use __pa_symbol instead of __pa on C visible symbols

[Konrad Rzeszutek Wilk]

On Mon, Nov 05, 2012 at 11:06:00AM -0800, Alexander Duyck wrote:
> This change updates a few of the functions to use __pa_symbol when
> translating C visible symbols instead of __pa.  By using __pa_symbol we are
> able to drop a few extra lines of code as don't have to test to see if the
> virtual pointer is a part of the kernel text or just standard virtual memory.
> 
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Acked-by.

> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
> ---
> 
>  arch/x86/xen/mmu.c |   21 +++++++++++----------
>  1 files changed, 11 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
> index 4a05b39..a63e5f9 100644
> --- a/arch/x86/xen/mmu.c
> +++ b/arch/x86/xen/mmu.c
> @@ -1486,7 +1486,8 @@ static int xen_pgd_alloc(struct mm_struct *mm)
>  
>  		if (user_pgd != NULL) {
>  			user_pgd[pgd_index(VSYSCALL_START)] =
> -				__pgd(__pa(level3_user_vsyscall) | _PAGE_TABLE);
> +				__pgd(__pa_symbol(level3_user_vsyscall) |
> +				      _PAGE_TABLE);
>  			ret = 0;
>  		}
>  
> @@ -1958,10 +1959,10 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
>  	 * pgd.
>  	 */
>  	if (xen_feature(XENFEAT_writable_page_tables)) {
> -		native_write_cr3(__pa(init_level4_pgt));
> +		native_write_cr3(__pa_symbol(init_level4_pgt));
>  	} else {
>  		xen_mc_batch();
> -		__xen_write_cr3(true, __pa(init_level4_pgt));
> +		__xen_write_cr3(true, __pa_symbol(init_level4_pgt));
>  		xen_mc_issue(PARAVIRT_LAZY_CPU);
>  	}
>  	/* We can't that easily rip out L3 and L2, as the Xen pagetables are
> @@ -1984,10 +1985,10 @@ static RESERVE_BRK_ARRAY(pmd_t, swapper_kernel_pmd, PTRS_PER_PMD);
>  
>  static void __init xen_write_cr3_init(unsigned long cr3)
>  {
> -	unsigned long pfn = PFN_DOWN(__pa(swapper_pg_dir));
> +	unsigned long pfn = PFN_DOWN(__pa_symbol(swapper_pg_dir));
>  
> -	BUG_ON(read_cr3() != __pa(initial_page_table));
> -	BUG_ON(cr3 != __pa(swapper_pg_dir));
> +	BUG_ON(read_cr3() != __pa_symbol(initial_page_table));
> +	BUG_ON(cr3 != __pa_symbol(swapper_pg_dir));
>  
>  	/*
>  	 * We are switching to swapper_pg_dir for the first time (from
> @@ -2011,7 +2012,7 @@ static void __init xen_write_cr3_init(unsigned long cr3)
>  	pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE, pfn);
>  
>  	pin_pagetable_pfn(MMUEXT_UNPIN_TABLE,
> -			  PFN_DOWN(__pa(initial_page_table)));
> +			  PFN_DOWN(__pa_symbol(initial_page_table)));
>  	set_page_prot(initial_page_table, PAGE_KERNEL);
>  	set_page_prot(initial_kernel_pmd, PAGE_KERNEL);
>  
> @@ -2036,7 +2037,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
>  
>  	copy_page(initial_page_table, pgd);
>  	initial_page_table[KERNEL_PGD_BOUNDARY] =
> -		__pgd(__pa(initial_kernel_pmd) | _PAGE_PRESENT);
> +		__pgd(__pa_symbol(initial_kernel_pmd) | _PAGE_PRESENT);
>  
>  	set_page_prot(initial_kernel_pmd, PAGE_KERNEL_RO);
>  	set_page_prot(initial_page_table, PAGE_KERNEL_RO);
> @@ -2045,8 +2046,8 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
>  	pin_pagetable_pfn(MMUEXT_UNPIN_TABLE, PFN_DOWN(__pa(pgd)));
>  
>  	pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE,
> -			  PFN_DOWN(__pa(initial_page_table)));
> -	xen_write_cr3(__pa(initial_page_table));
> +			  PFN_DOWN(__pa_symbol(initial_page_table)));
> +	xen_write_cr3(__pa_symbol(initial_page_table));
>  
>  	memblock_reserve(__pa(xen_start_info->pt_base),
>  			 xen_start_info->nr_pt_frames * PAGE_SIZE);

[PATCH v3 7/8] x86/acpi: Use __pa_symbol instead of __pa on C visible symbols

[Alexander Duyck]

This change just updates one spot where __pa was being used when __pa_symbol
should have been used.  By using __pa_symbol we are able to drop a few extra
lines of code as we don't have to test to see if the virtual pointer is a
part of the kernel text or just standard virtual memory.

Cc: Len Brown <len.brown@intel.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/kernel/acpi/sleep.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c
index d5e0d71..0532f5d 100644
--- a/arch/x86/kernel/acpi/sleep.c
+++ b/arch/x86/kernel/acpi/sleep.c
@@ -69,7 +69,7 @@ int acpi_suspend_lowlevel(void)
 
 #ifndef CONFIG_64BIT
 	header->pmode_entry = (u32)&wakeup_pmode_return;
-	header->pmode_cr3 = (u32)__pa(&initial_page_table);
+	header->pmode_cr3 = (u32)__pa_symbol(initial_page_table);
 	saved_magic = 0x12345678;
 #else /* CONFIG_64BIT */
 #ifdef CONFIG_SMP

[PATCH v3 8/8] x86/lguest: Use __pa_symbol instead of __pa on C visible symbols

[Alexander Duyck]

The function lguest_write_cr3 is using __pa to convert swapper_pg_dir and
initial_page_table from virtual addresses to physical.  The correct function
to use for these values is __pa_symbol since they are C visible symbols.

Cc: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---

 arch/x86/lguest/boot.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/x86/lguest/boot.c b/arch/x86/lguest/boot.c
index df4176c..1cbd89c 100644
--- a/arch/x86/lguest/boot.c
+++ b/arch/x86/lguest/boot.c
@@ -552,7 +552,8 @@ static void lguest_write_cr3(unsigned long cr3)
 	current_cr3 = cr3;
 
 	/* These two page tables are simple, linear, and used during boot */
-	if (cr3 != __pa(swapper_pg_dir) && cr3 != __pa(initial_page_table))
+	if (cr3 != __pa_symbol(swapper_pg_dir) &&
+	    cr3 != __pa_symbol(initial_page_table))
 		cr3_changed = true;
 }
 

Re: [PATCH v3 8/8] x86/lguest: Use __pa_symbol instead of __pa on C visible symbols

[Rusty Russell]

Alexander Duyck <alexander.h.duyck@intel.com> writes:
> The function lguest_write_cr3 is using __pa to convert swapper_pg_dir and
> initial_page_table from virtual addresses to physical.  The correct function
> to use for these values is __pa_symbol since they are C visible symbols.
>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>

Acked-by: Rusty Russell <rusty@rustcorp.com.au>

Cheers,
Rusty.