swiotlb detection should be memory hotplug aware ?

[Alok Kataria <akataria@vmware.com>]

Hi, 

Looking at the current code swiotlb is initialized for 64bit kernels
only when the max_pfn value is greater than 4G (MAX_DMA32_PFN value). 
So in cases when the initial memory is less than 4GB the kernel boots
without enabling swiotlb, when we hotadd memory to such a kernel and go
beyond the 4G limit, swiotlb is still disabled. As a result when any
32bit devices start using this newly added memory beyond 4G, the kernel
starts spitting error messages like below or in some cases it causes
kernel panics.

<3>[ 815.921504] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff
<3>[ 815.944860] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff
<3>[ 815.968808] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff
<3>[ 815.992821] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff
<3>[ 816.016796] nommu_map_sg: overflow 32ffd6000+4096 of device mask ffffffff

For systems which have no HW-IOMMU but are capable of memory hotadd this
can be a potential problem. IMO, there can be few possible solutions to
this.

1. Enable swiotlb for all 64bit kernels which have memory hot-add
support.
2. Instead of checking the max_pfn value in pci_swiotlb_detect, check
for max_hotpluggable_pfn (or some such) value. Though I don't see such a
value readily available. I could parse the SRAT and get hotplug memory
information but that will make swiotlb detection logic a little too
complex. A quick look around srat_xx.c files and the acpi_memhotplug
module didn't find any useful API that could be used directly either.
So was wondering if any of you are aware of an easy way to get such
information ? 

Let me know if you have any other ideas as well.

Thanks in advance, 
Alok