* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Michal Hocko @ 2017-05-04 12:52 UTC (permalink / raw)
To: Balbir Singh
Cc: linux-mm, akpm, khandual, benh, aneesh.kumar, paulmck, srikar,
haren, jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <1493875615.7934.1.camel@gmail.com>
On Thu 04-05-17 15:26:55, Balbir Singh wrote:
> On Tue, 2017-05-02 at 16:36 +0200, Michal Hocko wrote:
> > On Wed 19-04-17 17:52:38, Balbir Singh wrote:
[...]
> > > 2. kswapd reclaim
> >
> > How is the memory reclaim handled then? How are users expected to handle
> > OOM situation?
> >
>
> 1. The fallback node list for coherent memory includes regular memory
> nodes
> 2. Direct reclaim works, I've tested it
But the direct reclaim would be effective only _after_ all other nodes
are full.
I thought that kswapd reclaim is a problem because the HW doesn't
support aging properly but as the direct reclaim works then what is the
actual problem?
> > > The reason for exposing this device memory as NUMA is to simplify
> > > the programming model, where memory allocation via malloc() or
> > > mmap() for example would seamlessly work across both kinds of
> > > memory. Since we expect the size of device memory to be smaller
> > > than system RAM, we would like to control the allocation of such
> > > memory. The proposed mechanism reuses nodemasks and explicit
> > > specification of the coherent node in the nodemask for allocation
> > > from device memory. This implementation also allows for kernel
> > > level allocation via __GFP_THISNODE and existing techniques
> > > such as page migration to work.
> >
> > so it basically resembles isol_cpus except for memory, right. I believe
> > scheduler people are more than unhappy about this interface...
> >
>
> isol_cpus were for an era when timer/interrupts and other scheduler
> infrastructure present today was not around, but I don't mean to digress.
AFAIU, it has been added to _isolate_ some cpus from the scheduling domain
and have them available for the explicit affinity usage. You are
effectivelly proposing the same thing.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Michal Hocko @ 2017-05-04 12:46 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Joonsoo Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
mgorman, Laura Abbott, Minchan Kim, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K . V, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team, Heiko Carstens,
Martin Schwidefsky
In-Reply-To: <398b341c-5fa7-1ad7-0840-752fa1908921@suse.cz>
On Thu 04-05-17 14:33:24, Vlastimil Babka wrote:
> On 05/02/2017 03:03 PM, Michal Hocko wrote:
> > On Tue 02-05-17 10:06:01, Vlastimil Babka wrote:
> >> On 04/27/2017 05:06 PM, Michal Hocko wrote:
> >>> On Tue 25-04-17 12:42:57, Joonsoo Kim wrote:
> >>>> On Mon, Apr 24, 2017 at 03:09:36PM +0200, Michal Hocko wrote:
> >>>>> On Mon 17-04-17 11:02:12, Joonsoo Kim wrote:
> >>>>>> On Thu, Apr 13, 2017 at 01:56:15PM +0200, Michal Hocko wrote:
> >>>>>>> On Wed 12-04-17 10:35:06, Joonsoo Kim wrote:
> >>> [...]
> >>>>> not for free. For most common configurations where we have ZONE_DMA,
> >>>>> ZONE_DMA32, ZONE_NORMAL and ZONE_MOVABLE all the 3 bits are already
> >>>>> consumed so a new zone will need a new one AFAICS.
> >>>>
> >>>> Yes, it requires one more bit for a new zone and it's handled by the patch.
> >>>
> >>> I am pretty sure that you are aware that consuming new page flag bits
> >>> is usually a no-go and something we try to avoid as much as possible
> >>> because we are in a great shortage there. So there really have to be a
> >>> _strong_ reason if we go that way. My current understanding that the
> >>> whole zone concept is more about a more convenient implementation rather
> >>> than a fundamental change which will solve unsolvable problems with the
> >>> current approach. More on that below.
> >>
> >> I don't see it as such a big issue. It's behind a CONFIG option (so we
> >> also don't need the jump labels you suggest later) and enabling it
> >> reduces the number of possible NUMA nodes (not page flags). So either
> >> you are building a kernel for android phone that needs CMA but will have
> >> a single NUMA node, or for a large server with many nodes that won't
> >> have CMA. As long as there won't be large servers that need CMA, we
> >> should be fine (yes, I know some HW vendors can be very creative, but
> >> then it's their problem?).
> >
> > Is this really about Android/UMA systems only? My quick grep seems to disagree
> > $ git grep CONFIG_CMA=y
> > arch/arm/configs/exynos_defconfig:CONFIG_CMA=y
> > arch/arm/configs/imx_v6_v7_defconfig:CONFIG_CMA=y
> > arch/arm/configs/keystone_defconfig:CONFIG_CMA=y
> > arch/arm/configs/multi_v7_defconfig:CONFIG_CMA=y
> > arch/arm/configs/omap2plus_defconfig:CONFIG_CMA=y
> > arch/arm/configs/tegra_defconfig:CONFIG_CMA=y
> > arch/arm/configs/vexpress_defconfig:CONFIG_CMA=y
> > arch/arm64/configs/defconfig:CONFIG_CMA=y
> > arch/mips/configs/ci20_defconfig:CONFIG_CMA=y
> > arch/mips/configs/db1xxx_defconfig:CONFIG_CMA=y
> > arch/s390/configs/default_defconfig:CONFIG_CMA=y
> > arch/s390/configs/gcov_defconfig:CONFIG_CMA=y
> > arch/s390/configs/performance_defconfig:CONFIG_CMA=y
> > arch/s390/defconfig:CONFIG_CMA=y
> >
> > I am pretty sure s390 and ppc support NUMA and aim at supporting really
> > large systems.
>
> I don't see ppc there,
config KVM_BOOK3S_64_HV
tristate "KVM for POWER7 and later using hypervisor mode in host"
depends on KVM_BOOK3S_64 && PPC_POWERNV
select KVM_BOOK3S_HV_POSSIBLE
select MMU_NOTIFIER
select CMA
fa61a4e376d21 tries to explain some more
[...]
> > Are we really ready to add another thing like that? How are distribution
> > kernels going to handle that?
>
> I still hope that generic enterprise/desktop distributions can disable
> it, and it's only used for small devices with custom kernels.
>
> The config burden is already there in any case, it just translates to
> extra migratetype and fastpath hooks, not extra zone and potentially
> less nodes.
AFAIU the extra migrate type costs nothing when there are no cma
reservations. And those hooks can be made noop behind static branch
as well. So distribution kernels do not really have to be afraid of
enabling CMA currently.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v7 0/7] Introduce ZONE_CMA
From: Vlastimil Babka @ 2017-05-04 12:33 UTC (permalink / raw)
To: Michal Hocko
Cc: Joonsoo Kim, Andrew Morton, Rik van Riel, Johannes Weiner,
mgorman, Laura Abbott, Minchan Kim, Marek Szyprowski,
Michal Nazarewicz, Aneesh Kumar K . V, Russell King, Will Deacon,
linux-mm, linux-kernel, kernel-team, Heiko Carstens,
Martin Schwidefsky
In-Reply-To: <20170502130326.GJ14593@dhcp22.suse.cz>
On 05/02/2017 03:03 PM, Michal Hocko wrote:
> On Tue 02-05-17 10:06:01, Vlastimil Babka wrote:
>> On 04/27/2017 05:06 PM, Michal Hocko wrote:
>>> On Tue 25-04-17 12:42:57, Joonsoo Kim wrote:
>>>> On Mon, Apr 24, 2017 at 03:09:36PM +0200, Michal Hocko wrote:
>>>>> On Mon 17-04-17 11:02:12, Joonsoo Kim wrote:
>>>>>> On Thu, Apr 13, 2017 at 01:56:15PM +0200, Michal Hocko wrote:
>>>>>>> On Wed 12-04-17 10:35:06, Joonsoo Kim wrote:
>>> [...]
>>>>> not for free. For most common configurations where we have ZONE_DMA,
>>>>> ZONE_DMA32, ZONE_NORMAL and ZONE_MOVABLE all the 3 bits are already
>>>>> consumed so a new zone will need a new one AFAICS.
>>>>
>>>> Yes, it requires one more bit for a new zone and it's handled by the patch.
>>>
>>> I am pretty sure that you are aware that consuming new page flag bits
>>> is usually a no-go and something we try to avoid as much as possible
>>> because we are in a great shortage there. So there really have to be a
>>> _strong_ reason if we go that way. My current understanding that the
>>> whole zone concept is more about a more convenient implementation rather
>>> than a fundamental change which will solve unsolvable problems with the
>>> current approach. More on that below.
>>
>> I don't see it as such a big issue. It's behind a CONFIG option (so we
>> also don't need the jump labels you suggest later) and enabling it
>> reduces the number of possible NUMA nodes (not page flags). So either
>> you are building a kernel for android phone that needs CMA but will have
>> a single NUMA node, or for a large server with many nodes that won't
>> have CMA. As long as there won't be large servers that need CMA, we
>> should be fine (yes, I know some HW vendors can be very creative, but
>> then it's their problem?).
>
> Is this really about Android/UMA systems only? My quick grep seems to disagree
> $ git grep CONFIG_CMA=y
> arch/arm/configs/exynos_defconfig:CONFIG_CMA=y
> arch/arm/configs/imx_v6_v7_defconfig:CONFIG_CMA=y
> arch/arm/configs/keystone_defconfig:CONFIG_CMA=y
> arch/arm/configs/multi_v7_defconfig:CONFIG_CMA=y
> arch/arm/configs/omap2plus_defconfig:CONFIG_CMA=y
> arch/arm/configs/tegra_defconfig:CONFIG_CMA=y
> arch/arm/configs/vexpress_defconfig:CONFIG_CMA=y
> arch/arm64/configs/defconfig:CONFIG_CMA=y
> arch/mips/configs/ci20_defconfig:CONFIG_CMA=y
> arch/mips/configs/db1xxx_defconfig:CONFIG_CMA=y
> arch/s390/configs/default_defconfig:CONFIG_CMA=y
> arch/s390/configs/gcov_defconfig:CONFIG_CMA=y
> arch/s390/configs/performance_defconfig:CONFIG_CMA=y
> arch/s390/defconfig:CONFIG_CMA=y
>
> I am pretty sure s390 and ppc support NUMA and aim at supporting really
> large systems.
I don't see ppc there, and s390 commit adding CMA as default provides no
info. Heiko/Martin, could you share what does s390 use CMA for? Thanks.
> I can imagine that we could make ZONE_CMA configurable in a way that
> only very well defined use cases would be supported so that we can save
> page flags space. But this alone sounds like a maintainability nightmare
> to me. Especially when I consider ZONE_DMA situation. There is simply
> not an easy way to find out whether my HW really needs DMA zone or
> not. Most probably not but it still is configured and hidden behind
> config ZONE_DMA
> bool "DMA memory allocation support" if EXPERT
> default y
> help
> DMA memory allocation support allows devices with less than 32-bit
> addressing to allocate within the first 16MB of address space.
> Disable if no such devices will be used.
>
> If unsure, say Y.
>
> Are we really ready to add another thing like that? How are distribution
> kernels going to handle that?
I still hope that generic enterprise/desktop distributions can disable
it, and it's only used for small devices with custom kernels.
The config burden is already there in any case, it just translates to
extra migratetype and fastpath hooks, not extra zone and potentially
less nodes.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-04 12:14 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, linux-kernel, Dave Hansen
In-Reply-To: <20170504112159.GC31540@dhcp22.suse.cz>
On 04/05/17 14:21, Michal Hocko wrote:
> On Wed 03-05-17 15:06:36, Igor Stoppa wrote:
[...]
>> * In most, if not all, the cases that could be enhanced, the code will
>> be calling kmalloc/vmalloc, indicating GFP_KERNEL as the desired type of
>> memory.
>
> How do you tell that the seal is active?
The simpler way would be to define the seal as something that is applied
only after late init has concluded.
IOW, if the kernel has already started user-space, the seal is in place.
I do acknowledge that this conflicts with the current implementation of
SE Linux, but it might be possible to extend SE Linux to have a
predefined configuration file that is loaded during kernel init.
In general this is not acceptable, but OTOH IMA does it, so there could
be ground also for advocating similar (optional) behavior for SE Linux.
Should that not be possible,then yes, I should provide some way (ioctl,
sysfs/something else) that can be used to apply the seal.
In such case there should be also some helper function which allows to
confirm that the seal is absent/present.
> I have also asked about the
> life time of these objects in the previous email thread. Do you expect
> those objects get freed one by one or mostly at once? Is this supposed
> to be boot time only or such allocations might happen anytime?
Yes, you did. I didn't mean to ignore the question.
I thought this question would be answered by the current RFC :-(
Alright, here's one more attempt at explaining what I have in mind.
And I might be wrong, so that would explain why it's not clear.
Once the seal is in place, the objects are effectively read-only, so the
lifetime is basically the same as the kernel text.
Since I am after providing the same functionality of
post-init-read-only, but for dynamically allocated data, I would stick
to the same behavior: once the data is read-only, it stays so forever,
or till reset/poweroff, whichever comes first.
I wonder if you are thinking about loadable modules or maybe livepatch.
My proposal, in its current form, is only about what is done when the
kernel initialization is performed. So it would not take those cases
under its umbrella. Actually it might be incompatible with livepatch, if
any of the read-only data is supposed to be updated.
Since it's meant to improve the current level of integrity, I would
prefer to have a progressive approach and address modules/livepatch in a
later phase, if this is not seen as a show stopper.
[...]
> The most immediate suggestion would be to extend SLAB caches with a new
> sealing feature.
Yes, I got few hours ago the same advice also from Dave Hansen (thanks,
btw) [1].
I had just not considered the option.
> Roughly it would mean that once kmem_cache_seal() is
> called on a cache it would changed page tables to used slab pages to RO
> state. This would obviously need some fiddling to make those pages not
> usable for new allocations from sealed pages. It would also mean some
> changes to kfree path but I guess this is doable.
Ok, as it probably has already become evident, I have just started
peeking into the memory subsystem, so this is the sort of guidance I was
hoping I could receive =) - thank you
Question: I see that some pages can be moved around. Would this apply to
the slab-based solution, or can I assume that once I have certain
physical pages sealed, they will not be altered anymore?
>> * While I do not strictly need a new memory zone, memory zones are what
>> kmalloc understands at the moment: AFAIK, it is not possible to tell
>> kmalloc from which memory pool it should fish out the memory, other than
>> having a reference to a memory zone.
>
> As I've said already. I think that a zone is a completely wrong
> approach. How would it help anyway. It is the allocator on top of the
> page allocator which has to do clever things to support sealing.
Ok, as long as there is a way forward that fits my needs and has the
possibility to be merged upstream, I'm fine with it.
I suppose zones are the first thing one meets when reading the code, so
they are probably the first target that comes to mind.
That's what happened to me.
I will probably come back with further questions, but I can then start
putting together some prototype of what you described.
I am fine with providing a generic solution, but I must make sure that
it works with slub. I suppose what you proposed will do it, right?
TBH, from what little I have been reading so far, I find a bit confusing
the fact that there are some header files referring separately to slab,
slub and slob, but then common code still refers to slab (slab.h slab.c
and slab_common.c, for example)
[1] https://marc.info/?l=linux-kernel&m=149388596106305&w=2
---
thanks, igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* mmotm git tree since-4.11 branch created (was: mmotm 2017-05-03-15-16 uploaded)
From: Michal Hocko @ 2017-05-04 11:51 UTC (permalink / raw)
To: akpm
Cc: mm-commits, linux-kernel, linux-mm, linux-fsdevel, linux-next,
sfr, broonie
In-Reply-To: <590a56eb.l+Eu7L7Jdv9KEqSs%akpm@linux-foundation.org>
I have just created since-4.11 branch in mm git tree
(http://git.kernel.org/?p=linux/kernel/git/mhocko/mm.git;a=summary). It
is based on v4.11 tag in Linus tree and mmotm-2017-05-03-15-16.
As usual mmotm trees are tagged with signed tag
(finger print BB43 1E25 7FB8 660F F2F1 D22D 48E2 09A2 B310 E347)
The shortlog says:
Andi Kleen (3):
kref: remove WARN_ON for NULL release functions
drivers/scsi/megaraid: remove expensive inline from megasas_return_cmd
include/linux/uaccess.h: remove expensive WARN_ON in pagefault_disabled_dec
Andrew Morton (4):
mm-page_alloc-count-movable-pages-when-stealing-from-pageblock-fix
drm-use-set_memoryh-header-fix
docs-vm-transhuge-fix-few-trivial-typos-fix
dax-add-tracepoint-to-dax_writeback_one-fix
Andrey Konovalov (9):
kasan: introduce helper functions for determining bug type
kasan: unify report headers
kasan: change allocation and freeing stack traces headers
kasan: simplify address description logic
kasan: change report header
kasan: improve slab object description
kasan: print page description after stacks
kasan: improve double-free report format
kasan: separate report parts by empty lines
Andrey Ryabinin (4):
fs: fix data invalidation in the cleancache during direct IO
fs/block_dev: always invalidate cleancache in invalidate_bdev()
mm/truncate: bail out early from invalidate_inode_pages2_range() if mapping is empty
mm/truncate: avoid pointless cleancache_invalidate_inode() calls.
Andrey Vostrikov (1):
lib/crc-ccitt: add CCITT-FALSE CRC16 variant
Anshuman Khandual (5):
mm/mmap: replace SHM_HUGE_MASK with MAP_HUGE_MASK inside mmap_pgoff
mm/madvise.c: clean up MADV_SOFT_OFFLINE and MADV_HWPOISON
mm/madvise: move up the behavior parameter validation
mm/memory-failure.c: add page flag description in error paths
selftests/vm: add a test for virtual address range mapping
Arnd Bergmann (3):
block, dax: use correct format string in bdev_dax_supported
mm/gup.c: fix access_ok() argument type
drivers/misc: aspeed-lpc-ctrl: fix printk format warning again
Colin Ian King (1):
scripts/spelling.txt: add several more common spelling mistakes
Cyril Bur (2):
drivers/misc: Add Aspeed LPC control driver
drivers/misc: Aspeed LPC control fix compile error and warning
Dan Williams (37):
device-dax: fix cdev leak
acpi, nfit: fix acpi_get_table leak
Merge branch 'for-4.11/libnvdimm' into for-4.12/dax
device-dax: rename 'dax_dev' to 'dev_dax'
dax: refactor dax-fs into a generic provider of 'struct dax_device' instances
Revert "libnvdimm: band aid btt vs clear poison locking"
acpi, nfit: add support for acpi 6.1 dimm state flags
tools/testing/nvdimm: test acpi 6.1 health state flags
acpi, nfit: support "map failed" dimms
acpi, nfit: collate health state flags
acpi, nfit: limit ->flush_probe() to initialization work
tools/testing/nvdimm: fix nfit_test shutdown crash
acpi, nfit: fix module unload vs workqueue shutdown race
dax: add a facility to lookup a dax device by 'host' device name
dax: introduce dax_operations
pmem: add dax_operations support
axon_ram: add dax_operations support
brd: add dax_operations support
dcssblk: add dax_operations support
block: kill bdev_dax_capable()
dax: introduce dax_direct_access()
dm: add dax_device and dax_operations support
libnvdimm, region: fix flush hint detection crash
dm: teach dm-targets to use a dax_device + dax_operations
ext2, ext4, xfs: retrieve dax_device for iomap operations
Revert "block: use DAX for partition table reads"
filesystem-dax: convert to dax_direct_access()
block, dax: convert bdev_dax_supported() to dax_direct_access()
block: remove block_device_operations ->direct_access()
x86, dax, pmem: remove indirection around memcpy_from_pmem()
libnvdimm, region: sysfs trigger for nvdimm_flush()
acpi, nfit: kill ACPI_NFIT_DEBUG
libnvdimm: rework region badblocks clearing
libnvdimm: fix nvdimm_bus_lock() vs device_lock() ordering
libnvdimm: restore "libnvdimm: band aid btt vs clear poison locking"
device-dax: fix sysfs attribute deadlock
Merge branch 'for-4.12/dax' into libnvdimm-for-next
Dave Jiang (5):
libnvdimm: add mechanism to publish badblocks at the region level
libnvdimm: Add 'resource' sysfs attribute to regions
libnvdimm: add support for clear poison list and badblocks for device dax
device-dax, tools/testing/nvdimm: enable device-dax with mock resources
libnvdimm: fix clear poison locking with spinlock and GFP_NOWAIT allocation
David Rientjes (2):
mm, vmstat: print non-populated zones in zoneinfo
mm, vmstat: suppress pcp stats for unpopulated zones in zoneinfo
Deepa Dinamani (2):
fs: f2fs: use ktime_get_real_seconds for sit_info times
trace: make trace_hwlat timestamp y2038 safe
Dinh Nguyen (1):
fpga: fix sparse warnings in fpga-mgr and fpga-bridge
Dmitry Torokhov (2):
rapidio: use is_visible() to hide switch-specific attributes
zorro: stop creating attributes by hand
Florian Fainelli (2):
FPGA: Add TS-7300 FPGA manager
ARM: ep93xx: Register ts73xx-fpga manager driver for TS-7300
Geert Uytterhoeven (5):
auxdisplay: charlcd: Extract character LCD core from misc/panel
auxdisplay: charlcd: Add support for 4-bit interfaces
auxdisplay: charlcd: Add support for displays with more than two lines
dt-bindings: auxdisplay: Add bindings for Hitachi HD44780
auxdisplay: Add HD44780 Character LCD support
Geliang Tang (3):
fs/ocfs2/cluster: use setup_timer
fs/ocfs2/cluster: use offset_in_page() macro
mm/page-writeback.c: use setup_deferrable_timer
Gerald Schaefer (1):
brd: fix uninitialized use of brd->dax_dev
Greg Thelen (1):
slab: avoid IPIs when creating kmem caches
Hao Lee (1):
mm: fix spelling error
Huang Ying (7):
mm, swap: Fix a race in free_swap_and_cache()
mm, swap: fix comment in __read_swap_cache_async
mm, swap: improve readability via make spin_lock/unlock balanced
mm, swap: avoid lock swap_avail_lock when held cluster lock
mm, swap: remove unused function prototype
mm/swapfile.c: fix swap space leak in error path of swap_free_entries()
mm, swap: use kvzalloc to allocate some swap data structures
Joe Perches (1):
drivers/char: Convert remaining use of pr_warning to pr_warn
Joel Holdsworth (2):
Documentation: Add binding document for Lattice iCE40 FPGA manager
fpga: Add support for Lattice iCE40 FPGAs
Johannes Weiner (16):
mm: fix 100% CPU kswapd busyloop on unreclaimable nodes
mm: fix check for reclaimable pages in PF_MEMALLOC reclaim throttling
mm: remove seemingly spurious reclaimability check from laptop_mode gating
mm: remove unnecessary reclaimability check from NUMA balancing target
mm: don't avoid high-priority reclaim on unreclaimable nodes
mm: don't avoid high-priority reclaim on memcg limit reclaim
mm: delete NR_PAGES_SCANNED and pgdat_reclaimable()
Revert "mm, vmscan: account for skipped pages as a partial scan"
mm: remove unnecessary back-off function when retrying page reclaim
mm: memcontrol: provide shmem statistics
mm: page_alloc: __GFP_NOWARN shouldn't suppress stall warnings
mm: vmscan: fix IO/refault regression in cache workingset transition
mm: memcontrol: clean up memory.events counting function
mm: memcontrol: re-use global VM event enum
mm: memcontrol: re-use node VM page state enum
mm: memcontrol: use node page state naming scheme for memcg
Junxiao Bi (1):
ocfs2: o2hb: revert hb threshold to keep compatible
K. Y. Srinivasan (2):
Drivers: hv: Fix a typo
Drivers: hv: Base autoeoi enablement based on hypervisor hints
Kees Cook (2):
mm: remove rodata_test_data export, add pr_fmt
format-security: move static strings to const
Laura Abbott (15):
treewide: move set_memory_* functions away from cacheflush.h
arm: use set_memory.h header
arm64: use set_memory.h header
s390: use set_memory.h header
x86: use set_memory.h header
agp: use set_memory.h header
drm: use set_memory.h header
drivers/hwtracing/intel_th/msu.c: use set_memory.h header
drivers/watchdog/hpwdt.c: use set_memory.h header
include/linux/filter.h: use set_memory.h header
kernel/module.c: use set_memory.h header
kernel/power/snapshot.c: use set_memory.h header
alsa: use set_memory.h header
drivers/misc/sram-exec.c: use set_memory.h header
drivers/video/fbdev/vermilion/vermilion.c: use set_memory.h header
Laurent Dufour (2):
mm: uncharge poisoned pages
mm: skip HWPoisoned pages when onlining pages
Linda Knippers (3):
acpi, nfit: allow override of built-in bitmasks for nvdimm DSMs
acpi, nfit: allow specifying a default DSM family
acpi, nfit: remove unnecessary newline
Logan Gunthorpe (2):
chardev: add helper function to register char devs with a struct device
device-dax: utilize new cdev_device_add helper function
Mariusz Bialonczyk (4):
w1: add missing DS2413 documentation
w1: add support for DS2438 Smart Battery Monitor
w1: add documentation for w1_ds2438
w1: w1_ds2760.h: fix defines indentation
Martyn Welch (2):
docs: Add kernel-doc comments to VME driver API
docs: Update VME documentation to include kerneldoc comments
Masahiro Yamada (1):
blackfin: bf609: let clk_disable() return immediately if clk is NULL
Matt Ranostay (2):
pps: add ioctl_compat function to correct ioctl definitions
pps: fix padding issue with PPS_FETCH for ioctl_compat
Matthew Wilcox (1):
mm: tighten up the fault path a little
Matthias Kaehlcke (1):
hpet: Make cmd parameter of hpet_ioctl_common() unsigned
Mel Gorman (2):
mm, vmscan: only clear pgdat congested/dirty/writeback state when balanced
mm, vmscan: prevent kswapd sleeping prematurely due to mismatched classzone_idx
Michal Hocko (19):
Merge remote-tracking branch 'nvdim/libnvdimm-for-next' into mmotm-since-4.11
lockdep: allow to disable reclaim lockup detection
xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS
mm: introduce memalloc_nofs_{save,restore} API
xfs: use memalloc_nofs_{save,restore} instead of memalloc_noio*
jbd2: mark the transaction context with the scope GFP_NOFS context
jbd2: make the whole kjournald2 kthread NOFS safe
oom: improve oom disable handling
mm: introduce kv[mz]alloc helpers
mm, vmalloc: properly track vmalloc users
mm: support __GFP_REPEAT in kvmalloc_node for >32kB
lib/rhashtable.c: simplify a strange allocation pattern
net/ipv6/ila/ila_xlat.c: simplify a strange allocation pattern
fs/xattr.c: zero out memory copied to userspace in getxattr
treewide: use kv[mz]alloc* rather than opencoded variants
net: use kvmalloc with __GFP_REPEAT rather than open coded variant
drivers/md/dm-ioctl.c: use kvmalloc rather than opencoded variant
drivers/md/bcache/super.c: use kvmalloc
mm, vmalloc: use __GFP_HIGHMEM implicitly
Mike Kravetz (1):
Documentation: vm, add hugetlbfs reservation overview
Mike Rapoport (1):
userfaultfd: selftest: combine all cases into a single executable
Minchan Kim (18):
mm: fix lazyfree BUG_ON check in try_to_unmap_one()
mm: do not use double negation for testing page flags
mm: remove unncessary ret in page_referenced
mm: remove SWAP_DIRTY in ttu
mm: remove SWAP_MLOCK check for SWAP_SUCCESS in ttu
mm: make try_to_munlock() return void
mm: remove SWAP_MLOCK in ttu
mm: remove SWAP_AGAIN in ttu
mm: make ttu's return boolean
mm: make rmap_walk() return void
mm: make rmap_one boolean function
mm: remove SWAP_[SUCCESS|AGAIN|FAIL]
zram: handle multiple pages attached bio's bvec
zram: partial IO refactoring
zram: use zram_slot_lock instead of raw bit_spin_lock op
zram: remove zram_meta structure
zram: introduce zram data accessor
zram: use zram_free_page instead of open-coded
Ming Lei (1):
MAINTAINERS: update firmware loader entry
Moritz Fischer (4):
fpga: Add flag to indicate bitstream needs decrypting
fpga: zynq: Add support for encrypted bitstreams
fpga: region: Add fpga-region property 'encrypted-fpga-config'
fpga: bridge: Replace open-coded list_for_each + list_entry
Naoya Horiguchi (2):
mm: hwpoison: call shake_page() unconditionally
mm: hwpoison: call shake_page() after try_to_unmap() for mlocked page
Nikolay Borisov (1):
lockdep: teach lockdep about memalloc_noio_save
Oliver O'Halloran (3):
device-dax: improve fault handler debug output
mm/huge_memory.c.c: use zap_deposited_table() more
mm/huge_memory.c: deposit a pgtable for DAX PMD faults when required
Pankaj Gupta (1):
lib/dma-debug.c: make locking work for RT
Pavel Tatashin (4):
sparc64: NG4 memset 32 bits overflow
mm: zero hash tables in allocator
mm: update callers to use HASH_ZERO flag
mm: adaptive hash table scaling
Pushkar Jambhlekar (2):
device-dax: fix dax_dev_huge_fault() unknown fault size handling
include/linux/migrate.h: add arg names to prototype
Rob Herring (1):
binder: Add 'hwbinder' to the default devices
Ross Zwisler (7):
dax: add tracepoints to dax_iomap_pte_fault()
dax: add tracepoints to dax_pfn_mkwrite()
dax: add tracepoints to dax_load_hole()
dax: add tracepoints to dax_writeback_mapping_range()
dax: fix regression in dax_writeback_mapping_range()
dax: add tracepoint to dax_writeback_one()
dax: add tracepoint to dax_insert_mapping()
Sangwoo Park (1):
zram: reduce load operation in page_same_filled
SeongJae Park (1):
Documentation/vm/transhuge.txt: fix trivial typos
Shantanu Goel (1):
mm, vmscan: fix zone balance check in prepare_kswapd_sleep
Shaohua Li (6):
mm: delete unnecessary TTU_* flags
mm: don't assume anonymous pages have SwapBacked flag
mm: move MADV_FREE pages into LRU_INACTIVE_FILE list
mm: reclaim MADV_FREE pages
mm: enable MADV_FREE for swapless system
proc: show MADV_FREE pages info in smaps
Stephen Boyd (2):
scripts/spelling.txt: add "memory" pattern and fix typos
scripts/spelling.txt: Add regsiter -> register spelling mistake
Stephen Hemminger (9):
vmbus: only reschedule tasklet if time limit exceeded
hyperv: fix warning about missing prototype
vmbus: remove useless return's
vmbus: remove unnecessary initialization
vmbus: fix spelling errors
hyperv: remove unnecessary return variable
vmbus: make channel_message table constant
vmbus: cleanup header file style
vmbus: expose debug info for drivers
Stephen Rothwell (2):
mm: introduce kv[mz]alloc helpers - f2fs fix up
kprobes/x86: merge fix for set_memory.h decoupling
Tetsuo Handa (2):
mm, page_alloc: remove debug_guardpage_minorder() test in warn_alloc()
fs: semove set but not checked AOP_FLAG_UNINTERRUPTIBLE flag
Tim Chen (1):
mm/swap_slots.c: add warning if swap slots cache failed to initialize
Toshi Kani (3):
libnvdimm: fix phys_addr for nvdimm_clear_poison
libnvdimm, pmem: fix a NULL pointer BUG in nd_pmem_notify
libnvdimm: fix clear length of nvdimm_forget_poison()
Vinayak Menon (2):
mm: enable page poisoning early at boot
mm: vmscan: do not pass reclaimed slab to vmpressure
Vlastimil Babka (14):
mm, compaction: reorder fields in struct compact_control
mm, compaction: remove redundant watermark check in compact_finished()
mm, page_alloc: split smallest stolen page in fallback
mm-page_alloc-split-smallest-stolen-page-in-fallback-fix
mm, page_alloc: count movable pages when stealing from pageblock
mm, compaction: change migrate_async_suitable() to suitable_migration_source()
mm, compaction: add migratetype to compact_control
mm, compaction: restrict async compaction to pageblocks of same migratetype
mm, compaction: finish whole pageblock to reduce fragmentation
mm: prevent potential recursive reclaim due to clearing PF_MEMALLOC
mm: introduce memalloc_noreclaim_{save,restore}
treewide: convert PF_MEMALLOC manipulations to new helpers
treewide-convert-pf_memalloc-manipulations-to-new-helpers-fix
mtd: nand: nandsim: convert to memalloc_noreclaim_*()
Wei Yang (2):
mm/sparse: refine usemap_size() a little
mm/page_alloc: return 0 in case this node has no page within the zone
Xishi Qiu (2):
mm: use is_migrate_highatomic() to simplify the code
mm: use is_migrate_isolate_page() to simplify the code
Yisheng Xie (2):
mm/compaction: ignore block suitable after check large free page
mm/vmscan: more restrictive condition for retry in do_try_to_free_pages
zhong jiang (2):
mm/page_owner: align with pageblock_nr pages
mm/vmstat.c: walk the zone in pageblock_nr_pages steps
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [patch v2] mm, vmscan: avoid thrashing anon lru when free + file is low
From: Michal Hocko @ 2017-05-04 11:43 UTC (permalink / raw)
To: David Rientjes
Cc: Andrew Morton, Minchan Kim, Johannes Weiner, Mel Gorman,
linux-kernel, linux-mm
In-Reply-To: <alpine.DEB.2.10.1705031547360.50439@chino.kir.corp.google.com>
On Wed 03-05-17 15:52:04, David Rientjes wrote:
> On Wed, 3 May 2017, Michal Hocko wrote:
[...]
> > /*
> > - * If there is enough inactive page cache, i.e. if the size of the
> > - * inactive list is greater than that of the active list *and* the
> > - * inactive list actually has some pages to scan on this priority, we
> > - * do not reclaim anything from the anonymous working set right now.
> > - * Without the second condition we could end up never scanning an
> > - * lruvec even if it has plenty of old anonymous pages unless the
> > - * system is under heavy pressure.
> > + * Make sure there are enough pages on the biased LRU before we go
> > + * and do an exclusive reclaim from that list, i.e. if the
> > + * size of the inactive list is greater than that of the active list
> > + * *and* the inactive list actually has some pages to scan on this
> > + * priority.
> > + * Without the second condition we could end up never scanning other
> > + * lruvecs even if they have plenty of old pages unless the system is
> > + * under heavy pressure.
> > */
> > - if (!inactive_list_is_low(lruvec, true, memcg, sc, false) &&
> > - lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, sc->reclaim_idx) >> sc->priority) {
> > - scan_balance = SCAN_FILE;
> > + lru = LRU_INACTIVE_ANON + LRU_FILE * (scan_balance == SCAN_FILE);
>
> This part seems to complicate the logic since it determines the lru under
> test based on the current setting of scan_balance. I think I prefer
> individual heuristics with well written comments, but others may feel
> differently about this.
I do not claim the code would more obvious than before but it gets rid
of the duplication which is usually a good thing. This size check has
the same reasoning regardless of the type of the LRU. But I am not going
to insist...
> > + if (!inactive_list_is_low(lruvec, is_file_lru(lru), memcg, sc, false) &&
> > + lruvec_lru_size(lruvec, lru, sc->reclaim_idx) >> sc->priority)
> > goto out;
> > - }
> >
> > scan_balance = SCAN_FRACT;
> >
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Michal Hocko @ 2017-05-04 11:21 UTC (permalink / raw)
To: Igor Stoppa; +Cc: linux-mm, linux-kernel
In-Reply-To: <9200d87d-33b6-2c70-0095-e974a30639fd@huawei.com>
On Wed 03-05-17 15:06:36, Igor Stoppa wrote:
> Hello,
>
> please review my (longish) line of thoughts, below.
>
> I've restructured them so that they should be easier to follow.
>
>
> Observations
> ------------
>
> * it is currently possible, by using prefix "__read_only", to have the
> linker place a static variable into a special memory region, which will
> become write-protected at the end of the init phase.
>
> * the purpose is to write-protect data which is not expected to change,
> ever, after it has been initialized.
>
> * The mechanism used for locking down the memory region is to program
> the MMU to trap writes to said region. It is fairly efficient and
> HW-backed, so it doesn't introduce any major overhead, but the MMU deals
> only with pages or supersets of pages, hence the need to collect all the
> soon-to-be-read-only data - and only that - into the "special region".
> The "__read_only" modifier is the admission ticket.
>
> * the write-protecting feature helps supporting memory integrity in
> general and can also help spotting rogue writes, whatever their origin
> might be: uninitialized or expired pointers, wrong pointer arithmetic, etc.
I agree that such a feature can be really useful.
> Problem
> -------
>
> The feature is available only for *static* data - it will not work with
> something like a linked list that is put together during init, for example.
>
>
>
> Wish
> ----
>
> My starting point are the policy DB of SE Linux and the LSM Hooks, but
> eventually I would like to extend the protection also to other
> subsystems, in a way that can be merged into mainline.
>
>
>
> Analysis
> --------
>
> * the solution I come up with has to be as little invasive as possible,
> at least for what concerns the various subsystems whose integrity I want
> to enhance.
>
> * In most, if not all, the cases that could be enhanced, the code will
> be calling kmalloc/vmalloc, indicating GFP_KERNEL as the desired type of
> memory.
How do you tell that the seal is active? I have also asked about the
life time of these objects in the previous email thread. Do you expect
those objects get freed one by one or mostly at once? Is this supposed
to be boot time only or such allocations might happen anytime?
> * I suspect/hope that the various maintainer won't object too much if my
> changes are limited to replacing GFP_KERNEL with some other macro, for
> example what I previously called GFP_LOCKABLE, provided I can ensure that:
>
> -1) no penalty is introduced, at least when the extra protection
> feature is not enabled, iow nobody has to suffer from my changes.
> This means that GFP_LOCKABLE should fall back to GFP_KERNEL, when
> it's not enabled.
>
> -2) when the extra protection feature is enabled, the code still
> works as expected, as long as the data identified for this
> enhancement is really unmodified after init.
>
> * In my quest for improved memory integrity, I will deal with very
> different memory size being allocated, so if I start writing my own
> memory allocator, starting from a page-aligned chunk of normal memory,
> at best I will end up with a replica of kmalloc, at worst with something
> buggy. Either way, it will be extremely harder to push other subsystems
> to use it.
> I probably wouldn't like it either, if I was a maintainer.
The most immediate suggestion would be to extend SLAB caches with a new
sealing feature. Roughly it would mean that once kmem_cache_seal() is
called on a cache it would changed page tables to used slab pages to RO
state. This would obviously need some fiddling to make those pages not
usable for new allocations from sealed pages. It would also mean some
changes to kfree path but I guess this is doable.
> * While I do not strictly need a new memory zone, memory zones are what
> kmalloc understands at the moment: AFAIK, it is not possible to tell
> kmalloc from which memory pool it should fish out the memory, other than
> having a reference to a memory zone.
As I've said already. I think that a zone is a completely wrong
approach. How would it help anyway. It is the allocator on top of the
page allocator which has to do clever things to support sealing.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 12/32] x86/mm: Insure that boot memory areas are mapped properly
From: Borislav Petkov @ 2017-05-04 10:16 UTC (permalink / raw)
To: Tom Lendacky
Cc: linux-efi-u79uwXL29TY76Z2rM5mHXA, Brijesh Singh, Toshimitsu Kani,
Radim Krčmář, Matt Fleming,
x86-DgEjT+Ai2ygdnm+yROfE0A, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
Alexander Potapenko, H. Peter Anvin, Larry Woodman,
linux-arch-u79uwXL29TY76Z2rM5mHXA, kvm-u79uwXL29TY76Z2rM5mHXA,
Jonathan Corbet, linux-doc-u79uwXL29TY76Z2rM5mHXA,
kasan-dev-/JYPxA39Uh5TLH3MbocFFw, Ingo Molnar, Andrey Ryabinin,
Dave Young, Rik van Riel, Arnd Bergmann, Andy Lutomirski,
Thomas Gleixner, Dmitry Vyukov,
kexec-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Michael S. Tsirkin
In-Reply-To: <20170418211822.10190.67435.stgit-qCXWGYdRb2BnqfbPTmsdiZQ+2ll4COg0XqFh9Ls21Oc@public.gmane.org>
On Tue, Apr 18, 2017 at 04:18:22PM -0500, Tom Lendacky wrote:
> The boot data and command line data are present in memory in a decrypted
> state and are copied early in the boot process. The early page fault
> support will map these areas as encrypted, so before attempting to copy
> them, add decrypted mappings so the data is accessed properly when copied.
>
> For the initrd, encrypt this data in place. Since the future mapping of the
> initrd area will be mapped as encrypted the data will be accessed properly.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky-5C7GfCeVMHo@public.gmane.org>
> ---
> arch/x86/include/asm/mem_encrypt.h | 11 +++++
> arch/x86/include/asm/pgtable.h | 3 +
> arch/x86/kernel/head64.c | 30 ++++++++++++--
> arch/x86/kernel/setup.c | 10 +++++
> arch/x86/mm/mem_encrypt.c | 77 ++++++++++++++++++++++++++++++++++++
> 5 files changed, 127 insertions(+), 4 deletions(-)
...
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 603a166..a95800b 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -115,6 +115,7 @@
> #include <asm/microcode.h>
> #include <asm/mmu_context.h>
> #include <asm/kaslr.h>
> +#include <asm/mem_encrypt.h>
>
> /*
> * max_low_pfn_mapped: highest direct mapped pfn under 4GB
> @@ -374,6 +375,15 @@ static void __init reserve_initrd(void)
> !ramdisk_image || !ramdisk_size)
> return; /* No initrd provided by bootloader */
>
> + /*
> + * If SME is active, this memory will be marked encrypted by the
> + * kernel when it is accessed (including relocation). However, the
> + * ramdisk image was loaded decrypted by the bootloader, so make
> + * sure that it is encrypted before accessing it.
> + */
> + if (sme_active())
That test is not needed here because __sme_early_enc_dec() already tests
sme_me_mask. There you should change that test to sme_active() instead.
> + sme_early_encrypt(ramdisk_image, ramdisk_end - ramdisk_image);
> +
> initrd_start = 0;
>
> mapped_size = memblock_mem_size(max_pfn_mapped);
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
^ permalink raw reply
* [PATCH v10 6/6] virtio-balloon: VIRTIO_BALLOON_F_MISC_VQ
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
Add a new vq, miscq, to handle miscellaneous requests between the device
and the driver. Only one request is handled in-flight each time.
This patch implements the VIRTIO_BALLOON_MISCQ_CMD_REPORT_UNUSED_PAGES
request sent from the device. Upon receiving the request from the
miscq, the driver offers to the device the guest unused pages.
Tests have shown that skipping the transfer of unused pages of a 32G
idle guest can get the live migration time reduced to 1/8.
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
Signed-off-by: Liang Li <liang.z.li@intel.com>
---
drivers/virtio/virtio_balloon.c | 299 +++++++++++++++++++++++++++++++-----
include/uapi/linux/virtio_balloon.h | 12 ++
2 files changed, 274 insertions(+), 37 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index df16912..4dcee2c 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -57,6 +57,10 @@
/* The maximum number of page_bmap that can be allocated. */
#define VIRTIO_BALLOON_PAGE_BMAP_MAX_NUM 32
+/* Types of pages to chunk */
+#define PAGE_CHUNK_TYPE_BALLOON 0 /* Chunk of inflate/deflate pages */
+#define PAGE_CHUNK_TYPE_UNUSED 1 /* Chunk of unused pages */
+
static int oom_pages = OOM_VBALLOON_DEFAULT_PAGES;
module_param(oom_pages, int, S_IRUSR | S_IWUSR);
MODULE_PARM_DESC(oom_pages, "pages to free on OOM");
@@ -67,16 +71,17 @@ static struct vfsmount *balloon_mnt;
/* Maximum number of page chunks */
#define VIRTIO_BALLOON_MAX_PAGE_CHUNKS ((8 * PAGE_SIZE - \
- sizeof(struct virtio_balloon_page_chunk)) / \
- sizeof(struct virtio_balloon_page_chunk_entry))
+ sizeof(struct virtio_balloon_miscq_msg)) / \
+ sizeof(struct virtio_balloon_page_chunk_entry))
struct virtio_balloon {
struct virtio_device *vdev;
- struct virtqueue *inflate_vq, *deflate_vq, *stats_vq;
+ struct virtqueue *inflate_vq, *deflate_vq, *stats_vq, *miscq;
/* The balloon servicing is delegated to a freezable workqueue. */
struct work_struct update_balloon_stats_work;
struct work_struct update_balloon_size_work;
+ struct work_struct miscq_handle_work;
/* Prevent updating balloon when it is being canceled. */
spinlock_t stop_update_lock;
@@ -98,6 +103,9 @@ struct virtio_balloon {
/* Synchronize access/update to this struct virtio_balloon elements */
struct mutex balloon_lock;
+ /* Miscq msg buffer for the REPORT_UNUSED_PAGES cmd */
+ struct virtio_balloon_miscq_msg *miscq_msg_rup;
+
/* Buffer for chunks of ballooned pages. */
struct virtio_balloon_page_chunk *balloon_page_chunk;
@@ -200,38 +208,85 @@ static void clear_page_bmap(struct virtio_balloon *vb,
memset(vb->page_bmap[i], 0, VIRTIO_BALLOON_PAGE_BMAP_SIZE);
}
-static void send_page_chunks(struct virtio_balloon *vb, struct virtqueue *vq)
+static void send_page_chunks(struct virtio_balloon *vb, struct virtqueue *vq,
+ int type, bool busy_wait)
{
struct scatterlist sg;
struct virtio_balloon_page_chunk *chunk;
- unsigned int len;
+ void *msg_buf;
+ unsigned int msg_len;
+ uint64_t chunk_num = 0;
+
+ switch (type) {
+ case PAGE_CHUNK_TYPE_BALLOON:
+ chunk = vb->balloon_page_chunk;
+ chunk_num = le64_to_cpu(chunk->chunk_num);
+ msg_buf = vb->balloon_page_chunk;
+ msg_len = sizeof(struct virtio_balloon_page_chunk) +
+ sizeof(struct virtio_balloon_page_chunk_entry) *
+ chunk_num;
+ break;
+ case PAGE_CHUNK_TYPE_UNUSED:
+ chunk = &vb->miscq_msg_rup->payload.chunk;
+ chunk_num = le64_to_cpu(chunk->chunk_num);
+ msg_buf = vb->miscq_msg_rup;
+ msg_len = sizeof(struct virtio_balloon_miscq_msg) +
+ sizeof(struct virtio_balloon_page_chunk_entry) *
+ chunk_num;
+ break;
+ default:
+ dev_warn(&vb->vdev->dev, "%s: chunk %d of unknown pages\n",
+ __func__, type);
+ return;
+ }
- chunk = vb->balloon_page_chunk;
- len = sizeof(__le64) +
- le64_to_cpu(chunk->chunk_num) *
- sizeof(struct virtio_balloon_page_chunk_entry);
- sg_init_one(&sg, chunk, len);
+ sg_init_one(&sg, msg_buf, msg_len);
if (!virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL)) {
virtqueue_kick(vq);
- wait_event(vb->acked, virtqueue_get_buf(vq, &len));
+ if (busy_wait)
+ while (!virtqueue_get_buf(vq, &msg_len) &&
+ !virtqueue_is_broken(vq))
+ cpu_relax();
+ else
+ wait_event(vb->acked, virtqueue_get_buf(vq, &msg_len));
+ /*
+ * Now, the chunks have been delivered to the host.
+ * Reset the filed in the structure that records the number of
+ * added chunks, so that new added chunks can be re-counted.
+ */
chunk->chunk_num = 0;
}
}
/* Add a chunk entry to the buffer. */
static void add_one_chunk(struct virtio_balloon *vb, struct virtqueue *vq,
- u64 base, u64 size)
+ int type, u64 base, u64 size)
{
- struct virtio_balloon_page_chunk *chunk = vb->balloon_page_chunk;
+ struct virtio_balloon_page_chunk *chunk;
struct virtio_balloon_page_chunk_entry *entry;
- uint64_t chunk_num = le64_to_cpu(chunk->chunk_num);
-
+ uint64_t chunk_num;
+
+ switch (type) {
+ case PAGE_CHUNK_TYPE_BALLOON:
+ chunk = vb->balloon_page_chunk;
+ chunk_num = le64_to_cpu(vb->balloon_page_chunk->chunk_num);
+ break;
+ case PAGE_CHUNK_TYPE_UNUSED:
+ chunk = &vb->miscq_msg_rup->payload.chunk;
+ chunk_num =
+ le64_to_cpu(vb->miscq_msg_rup->payload.chunk.chunk_num);
+ break;
+ default:
+ dev_warn(&vb->vdev->dev, "%s: chunk %d of unknown pages\n",
+ __func__, type);
+ return;
+ }
entry = &chunk->entry[chunk_num];
entry->base = cpu_to_le64(base << VIRTIO_BALLOON_CHUNK_BASE_SHIFT);
entry->size = cpu_to_le64(size << VIRTIO_BALLOON_CHUNK_SIZE_SHIFT);
chunk->chunk_num = cpu_to_le64(++chunk_num);
if (chunk_num == VIRTIO_BALLOON_MAX_PAGE_CHUNKS)
- send_page_chunks(vb, vq);
+ send_page_chunks(vb, vq, type, 0);
}
static void convert_bmap_to_chunks(struct virtio_balloon *vb,
@@ -259,8 +314,8 @@ static void convert_bmap_to_chunks(struct virtio_balloon *vb,
chunk_size = (next_zero - next_one) *
VIRTIO_BALLOON_PAGES_PER_PAGE;
if (chunk_size) {
- add_one_chunk(vb, vq, pfn_start + next_one,
- chunk_size);
+ add_one_chunk(vb, vq, PAGE_CHUNK_TYPE_BALLOON,
+ pfn_start + next_one, chunk_size);
pos += next_zero + 1;
}
}
@@ -305,7 +360,7 @@ static void tell_host_from_page_bmap(struct virtio_balloon *vb,
pfn_num);
}
if (le64_to_cpu(vb->balloon_page_chunk->chunk_num) > 0)
- send_page_chunks(vb, vq);
+ send_page_chunks(vb, vq, PAGE_CHUNK_TYPE_BALLOON, 0);
}
static void set_page_pfns(struct virtio_balloon *vb,
@@ -679,43 +734,186 @@ static void update_balloon_size_func(struct work_struct *work)
queue_work(system_freezable_wq, work);
}
+/* Add a message buffer for the host to fill in a request */
+static void miscq_msg_inbuf_add(struct virtio_balloon *vb,
+ struct virtio_balloon_miscq_msg *req_buf)
+{
+ struct scatterlist sg_in;
+
+ sg_init_one(&sg_in, req_buf, sizeof(struct virtio_balloon_miscq_msg));
+ if (virtqueue_add_inbuf(vb->miscq, &sg_in, 1, req_buf, GFP_KERNEL)
+ < 0) {
+ __virtio_clear_bit(vb->vdev,
+ VIRTIO_BALLOON_F_MISC_VQ);
+ dev_warn(&vb->vdev->dev, "%s: add miscq msg buf err\n",
+ __func__);
+ return;
+ }
+ virtqueue_kick(vb->miscq);
+}
+
+static void miscq_report_unused_pages(struct virtio_balloon *vb)
+{
+ struct virtio_balloon_miscq_msg *msg = vb->miscq_msg_rup;
+ struct virtqueue *vq = vb->miscq;
+ int ret = 0;
+ unsigned int order = 0, migratetype = 0;
+ struct zone *zone = NULL;
+ struct page *page = NULL;
+ u64 pfn;
+
+ msg->cmd = cpu_to_le32(VIRTIO_BALLOON_MISCQ_CMD_REPORT_UNUSED_PAGES);
+ msg->flags = 0;
+
+ for_each_populated_zone(zone) {
+ for (order = MAX_ORDER - 1; order > 0; order--) {
+ for (migratetype = 0; migratetype < MIGRATE_TYPES;
+ migratetype++) {
+ do {
+ ret = report_unused_page_block(zone,
+ order, migratetype, &page);
+ if (!ret) {
+ pfn = (u64)page_to_pfn(page);
+ add_one_chunk(vb, vq,
+ PAGE_CHUNK_TYPE_UNUSED,
+ pfn,
+ (u64)(1 << order) *
+ VIRTIO_BALLOON_PAGES_PER_PAGE);
+ }
+ } while (!ret);
+ }
+ }
+ }
+ /* Set the cmd completion flag */
+ msg->flags |= cpu_to_le32(VIRTIO_BALLOON_MISCQ_F_COMPLETION);
+ send_page_chunks(vb, vq, PAGE_CHUNK_TYPE_UNUSED, true);
+}
+
+static void miscq_handle_func(struct work_struct *work)
+{
+ struct virtio_balloon *vb;
+ struct virtio_balloon_miscq_msg *msg;
+ unsigned int len;
+
+ vb = container_of(work, struct virtio_balloon,
+ miscq_handle_work);
+ msg = virtqueue_get_buf(vb->miscq, &len);
+ if (!msg || len != sizeof(struct virtio_balloon_miscq_msg)) {
+ dev_warn(&vb->vdev->dev, "%s: invalid miscq msg len\n",
+ __func__);
+ miscq_msg_inbuf_add(vb, vb->miscq_msg_rup);
+ return;
+ }
+ switch (msg->cmd) {
+ case VIRTIO_BALLOON_MISCQ_CMD_REPORT_UNUSED_PAGES:
+ miscq_report_unused_pages(vb);
+ break;
+ default:
+ dev_warn(&vb->vdev->dev, "%s: miscq cmd %d not supported\n",
+ __func__, msg->cmd);
+ }
+ miscq_msg_inbuf_add(vb, vb->miscq_msg_rup);
+}
+
+static void miscq_request(struct virtqueue *vq)
+{
+ struct virtio_balloon *vb = vq->vdev->priv;
+
+ queue_work(system_freezable_wq, &vb->miscq_handle_work);
+}
+
static int init_vqs(struct virtio_balloon *vb)
{
- struct virtqueue *vqs[3];
- vq_callback_t *callbacks[] = { balloon_ack, balloon_ack, stats_request };
- static const char * const names[] = { "inflate", "deflate", "stats" };
- int err, nvqs;
+ struct virtqueue **vqs;
+ vq_callback_t **callbacks;
+ const char **names;
+ int err = -ENOMEM;
+ int i, nvqs;
+
+ /* Inflateq and deflateq are used unconditionally */
+ nvqs = 2;
+
+ if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ))
+ nvqs++;
+ if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_MISC_VQ))
+ nvqs++;
+
+ /* Allocate space for find_vqs parameters */
+ vqs = kcalloc(nvqs, sizeof(*vqs), GFP_KERNEL);
+ if (!vqs)
+ goto err_vq;
+ callbacks = kmalloc_array(nvqs, sizeof(*callbacks), GFP_KERNEL);
+ if (!callbacks)
+ goto err_callback;
+ names = kmalloc_array(nvqs, sizeof(*names), GFP_KERNEL);
+ if (!names)
+ goto err_names;
+
+ callbacks[0] = balloon_ack;
+ names[0] = "inflate";
+ callbacks[1] = balloon_ack;
+ names[1] = "deflate";
+
+ i = 2;
+ if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
+ callbacks[i] = stats_request;
+ names[i] = "stats";
+ i++;
+ }
- /*
- * We expect two virtqueues: inflate and deflate, and
- * optionally stat.
- */
- nvqs = virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ) ? 3 : 2;
- err = vb->vdev->config->find_vqs(vb->vdev, nvqs, vqs, callbacks, names,
- NULL);
+ if (virtio_has_feature(vb->vdev,
+ VIRTIO_BALLOON_F_MISC_VQ)) {
+ callbacks[i] = miscq_request;
+ names[i] = "miscq";
+ }
+
+ err = vb->vdev->config->find_vqs(vb->vdev, nvqs, vqs, callbacks,
+ names, NULL);
if (err)
- return err;
+ goto err_find;
vb->inflate_vq = vqs[0];
vb->deflate_vq = vqs[1];
+ i = 2;
if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_STATS_VQ)) {
struct scatterlist sg;
- unsigned int num_stats;
- vb->stats_vq = vqs[2];
+ vb->stats_vq = vqs[i++];
/*
* Prime this virtqueue with one buffer so the hypervisor can
* use it to signal us later (it can't be broken yet!).
*/
- num_stats = update_balloon_stats(vb);
-
- sg_init_one(&sg, vb->stats, sizeof(vb->stats[0]) * num_stats);
+ sg_init_one(&sg, vb->stats, sizeof(vb->stats));
if (virtqueue_add_outbuf(vb->stats_vq, &sg, 1, vb, GFP_KERNEL)
< 0)
BUG();
virtqueue_kick(vb->stats_vq);
}
+
+ if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_MISC_VQ)) {
+ vb->miscq = vqs[i];
+ /*
+ * Add the msg buf for the REPORT_UNUSED_PAGES request.
+ * The request is handled one in-flight each time. So, just
+ * use the response buffer, msicq_msg_rup, for the host to
+ * fill in a request.
+ */
+ miscq_msg_inbuf_add(vb, vb->miscq_msg_rup);
+ }
+
+ kfree(names);
+ kfree(callbacks);
+ kfree(vqs);
return 0;
+
+err_find:
+ kfree(names);
+err_names:
+ kfree(callbacks);
+err_callback:
+ kfree(vqs);
+err_vq:
+ return err;
}
#ifdef CONFIG_BALLOON_COMPACTION
@@ -723,7 +921,7 @@ static int init_vqs(struct virtio_balloon *vb)
static void tell_host_one_page(struct virtio_balloon *vb,
struct virtqueue *vq, struct page *page)
{
- add_one_chunk(vb, vq, page_to_pfn(page),
+ add_one_chunk(vb, vq, PAGE_CHUNK_TYPE_BALLOON, page_to_pfn(page),
VIRTIO_BALLOON_PAGES_PER_PAGE);
}
@@ -857,6 +1055,22 @@ static int balloon_page_chunk_init(struct virtio_balloon *vb)
return -ENOMEM;
}
+static int miscq_init(struct virtio_balloon *vb)
+{
+ vb->miscq_msg_rup = kmalloc(sizeof(struct virtio_balloon_miscq_msg) +
+ sizeof(struct virtio_balloon_page_chunk_entry) *
+ VIRTIO_BALLOON_MAX_PAGE_CHUNKS, GFP_KERNEL);
+ if (!vb->miscq_msg_rup) {
+ __virtio_clear_bit(vb->vdev, VIRTIO_BALLOON_F_MISC_VQ);
+ dev_warn(&vb->vdev->dev, "%s: failed\n", __func__);
+ return -ENOMEM;
+ }
+
+ INIT_WORK(&vb->miscq_handle_work, miscq_handle_func);
+
+ return 0;
+}
+
static int virtballoon_validate(struct virtio_device *vdev)
{
struct virtio_balloon *vb = NULL;
@@ -874,7 +1088,16 @@ static int virtballoon_validate(struct virtio_device *vdev)
goto err_page_chunk;
}
+ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_MISC_VQ)) {
+ err = miscq_init(vb);
+ if (err < 0)
+ goto err_miscq_rup;
+ }
+
return 0;
+err_miscq_rup:
+ free_page_bmap(vb);
+ kfree(vb->balloon_page_chunk);
err_page_chunk:
kfree(vb);
err_vb:
@@ -971,6 +1194,7 @@ static void virtballoon_remove(struct virtio_device *vdev)
spin_unlock_irq(&vb->stop_update_lock);
cancel_work_sync(&vb->update_balloon_size_work);
cancel_work_sync(&vb->update_balloon_stats_work);
+ cancel_work_sync(&vb->miscq_handle_work);
remove_common(vb);
free_page_bmap(vb);
@@ -1020,6 +1244,7 @@ static unsigned int features[] = {
VIRTIO_BALLOON_F_STATS_VQ,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
VIRTIO_BALLOON_F_PAGE_CHUNKS,
+ VIRTIO_BALLOON_F_MISC_VQ,
};
static struct virtio_driver virtio_balloon_driver = {
diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
index d532ed16..ea83b74 100644
--- a/include/uapi/linux/virtio_balloon.h
+++ b/include/uapi/linux/virtio_balloon.h
@@ -35,6 +35,7 @@
#define VIRTIO_BALLOON_F_STATS_VQ 1 /* Memory Stats virtqueue */
#define VIRTIO_BALLOON_F_DEFLATE_ON_OOM 2 /* Deflate balloon on OOM */
#define VIRTIO_BALLOON_F_PAGE_CHUNKS 3 /* Inflate/Deflate pages in chunks */
+#define VIRTIO_BALLOON_F_MISC_VQ 4 /* Virtqueue for misc. requests */
/* Size of a PFN in the balloon interface. */
#define VIRTIO_BALLOON_PFN_SHIFT 12
@@ -96,4 +97,15 @@ struct virtio_balloon_page_chunk {
struct virtio_balloon_page_chunk_entry entry[];
};
+struct virtio_balloon_miscq_msg {
+#define VIRTIO_BALLOON_MISCQ_CMD_REPORT_UNUSED_PAGES 0
+ __le32 cmd;
+/* Flag to indicate the completion of handling a command */
+#define VIRTIO_BALLOON_MISCQ_F_COMPLETION 1
+ __le32 flags;
+ union {
+ struct virtio_balloon_page_chunk chunk;
+ } payload;
+};
+
#endif /* _LINUX_VIRTIO_BALLOON_H */
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 5/6] mm: export symbol of next_zone and first_online_pgdat
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
This patch enables for_each_zone()/for_each_populated_zone() to be
invoked by a kernel module.
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
---
mm/mmzone.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/mmzone.c b/mm/mmzone.c
index a51c0a6..08a2a3a 100644
--- a/mm/mmzone.c
+++ b/mm/mmzone.c
@@ -13,6 +13,7 @@ struct pglist_data *first_online_pgdat(void)
{
return NODE_DATA(first_online_node);
}
+EXPORT_SYMBOL_GPL(first_online_pgdat);
struct pglist_data *next_online_pgdat(struct pglist_data *pgdat)
{
@@ -41,6 +42,7 @@ struct zone *next_zone(struct zone *zone)
}
return zone;
}
+EXPORT_SYMBOL_GPL(next_zone);
static inline int zref_in_nodemask(struct zoneref *zref, nodemask_t *nodes)
{
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 4/6] mm: function to offer a page block on the free list
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
Add a function to find a page block on the free list specified by the
caller. Pages from the page block may be used immediately after the
function returns. The caller is responsible for detecting or preventing
the use of such pages.
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
Signed-off-by: Liang Li <liang.z.li@intel.com>
---
include/linux/mm.h | 5 +++
mm/page_alloc.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 96 insertions(+)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 5d22e69..82361a6 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1841,6 +1841,11 @@ extern void free_area_init_node(int nid, unsigned long * zones_size,
unsigned long zone_start_pfn, unsigned long *zholes_size);
extern void free_initmem(void);
+#if IS_ENABLED(CONFIG_VIRTIO_BALLOON)
+extern int report_unused_page_block(struct zone *zone, unsigned int order,
+ unsigned int migratetype,
+ struct page **page);
+#endif
/*
* Free reserved pages within range [PAGE_ALIGN(start), end & PAGE_MASK)
* into the buddy system. The freed pages will be poisoned with pattern
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2c25de4..e554ab8 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4615,6 +4615,97 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask)
show_swap_cache_info();
}
+#if IS_ENABLED(CONFIG_VIRTIO_BALLOON)
+
+/**
+ * Heuristically get a page block in the system that is unused.
+ * It is possible that pages from the page block are used immediately after
+ * report_unused_page_block() returns. It is the caller's responsibility
+ * to either detect or prevent the use of such pages.
+ *
+ * The free list to check: zone->free_area[order].free_list[migratetype].
+ *
+ * If the caller supplied page block (i.e. **page) is on the free list, offer
+ * the next page block on the list to the caller. Otherwise, offer the first
+ * page block on the list.
+ *
+ * Return 0 when a page block is found on the caller specified free list.
+ */
+int report_unused_page_block(struct zone *zone, unsigned int order,
+ unsigned int migratetype, struct page **page)
+{
+ struct zone *this_zone;
+ struct list_head *this_list;
+ int ret = 0;
+ unsigned long flags;
+
+ /* Sanity check */
+ if (zone == NULL || page == NULL || order >= MAX_ORDER ||
+ migratetype >= MIGRATE_TYPES)
+ return -EINVAL;
+
+ /* Zone validity check */
+ for_each_populated_zone(this_zone) {
+ if (zone == this_zone)
+ break;
+ }
+
+ /* Got a non-existent zone from the caller? */
+ if (zone != this_zone)
+ return -EINVAL;
+
+ spin_lock_irqsave(&this_zone->lock, flags);
+
+ this_list = &zone->free_area[order].free_list[migratetype];
+ if (list_empty(this_list)) {
+ *page = NULL;
+ ret = 1;
+ goto out;
+ }
+
+ /* The caller is asking for the first free page block on the list */
+ if ((*page) == NULL) {
+ *page = list_first_entry(this_list, struct page, lru);
+ ret = 0;
+ goto out;
+ }
+
+ /**
+ * The page block passed from the caller is not on this free list
+ * anymore (e.g. a 1MB free page block has been split). In this case,
+ * offer the first page block on the free list that the caller is
+ * asking for.
+ */
+ if (PageBuddy(*page) && order != page_order(*page)) {
+ *page = list_first_entry(this_list, struct page, lru);
+ ret = 0;
+ goto out;
+ }
+
+ /**
+ * The page block passed from the caller has been the last page block
+ * on the list.
+ */
+ if ((*page)->lru.next == this_list) {
+ *page = NULL;
+ ret = 1;
+ goto out;
+ }
+
+ /**
+ * Finally, fall into the regular case: the page block passed from the
+ * caller is still on the free list. Offer the next one.
+ */
+ *page = list_next_entry((*page), lru);
+ ret = 0;
+out:
+ spin_unlock_irqrestore(&this_zone->lock, flags);
+ return ret;
+}
+EXPORT_SYMBOL(report_unused_page_block);
+
+#endif
+
static void zoneref_set_zone(struct zone *zone, struct zoneref *zoneref)
{
zoneref->zone = zone;
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 3/6] virtio-balloon: VIRTIO_BALLOON_F_PAGE_CHUNKS
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
Add a new feature, VIRTIO_BALLOON_F_PAGE_CHUNKS, which enables
the transfer of the ballooned (i.e. inflated/deflated) pages in
chunks to the host.
The implementation of the previous virtio-balloon is not very
efficient, because the ballooned pages are transferred to the
host one by one. Here is the breakdown of the time in percentage
spent on each step of the balloon inflating process (inflating
7GB of an 8GB idle guest).
1) allocating pages (6.5%)
2) sending PFNs to host (68.3%)
3) address translation (6.1%)
4) madvise (19%)
It takes about 4126ms for the inflating process to complete.
The above profiling shows that the bottlenecks are stage 2)
and stage 4).
This patch optimizes step 2) by transferring pages to the host in
chunks. A chunk consists of guest physically continuous pages.
When the pages are packed into a chunk, they are converted into
balloon page size (4KB) pages. A chunk is offered to the host
via a base PFN (i.e. the start PFN of those physically continuous
pages) and the size (i.e. the total number of the 4KB balloon size
pages). A chunk is formatted as below:
--------------------------------------------------------
| Base (52 bit) | Rsvd (12 bit) |
--------------------------------------------------------
--------------------------------------------------------
| Size (52 bit) | Rsvd (12 bit) |
--------------------------------------------------------
By doing so, step 4) can also be optimized by doing address
translation and madvise() in chunks rather than page by page.
With this new feature, the above ballooning process takes ~590ms
resulting in an improvement of ~85%.
TODO: optimize stage 1) by allocating/freeing a chunk of pages
instead of a single page each time.
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
Signed-off-by: Liang Li <liang.z.li@intel.com>
Suggested-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/virtio/virtio_balloon.c | 407 +++++++++++++++++++++++++++++++++---
include/uapi/linux/virtio_balloon.h | 14 ++
2 files changed, 396 insertions(+), 25 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index ecb64e9..df16912 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -43,6 +43,20 @@
#define OOM_VBALLOON_DEFAULT_PAGES 256
#define VIRTBALLOON_OOM_NOTIFY_PRIORITY 80
+/* The size of one page_bmap used to record inflated/deflated pages. */
+#define VIRTIO_BALLOON_PAGE_BMAP_SIZE (8 * PAGE_SIZE)
+/*
+ * Callulates how many pfns can a page_bmap record. A bit corresponds to a
+ * page of PAGE_SIZE.
+ */
+#define VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP \
+ (VIRTIO_BALLOON_PAGE_BMAP_SIZE * BITS_PER_BYTE)
+
+/* The number of page_bmap to allocate by default. */
+#define VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM 1
+/* The maximum number of page_bmap that can be allocated. */
+#define VIRTIO_BALLOON_PAGE_BMAP_MAX_NUM 32
+
static int oom_pages = OOM_VBALLOON_DEFAULT_PAGES;
module_param(oom_pages, int, S_IRUSR | S_IWUSR);
MODULE_PARM_DESC(oom_pages, "pages to free on OOM");
@@ -51,6 +65,11 @@ MODULE_PARM_DESC(oom_pages, "pages to free on OOM");
static struct vfsmount *balloon_mnt;
#endif
+/* Maximum number of page chunks */
+#define VIRTIO_BALLOON_MAX_PAGE_CHUNKS ((8 * PAGE_SIZE - \
+ sizeof(struct virtio_balloon_page_chunk)) / \
+ sizeof(struct virtio_balloon_page_chunk_entry))
+
struct virtio_balloon {
struct virtio_device *vdev;
struct virtqueue *inflate_vq, *deflate_vq, *stats_vq;
@@ -79,6 +98,12 @@ struct virtio_balloon {
/* Synchronize access/update to this struct virtio_balloon elements */
struct mutex balloon_lock;
+ /* Buffer for chunks of ballooned pages. */
+ struct virtio_balloon_page_chunk *balloon_page_chunk;
+
+ /* Bitmap used to record pages. */
+ unsigned long *page_bmap[VIRTIO_BALLOON_PAGE_BMAP_MAX_NUM];
+
/* The array of pfns we tell the Host about. */
unsigned int num_pfns;
__virtio32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
@@ -111,6 +136,136 @@ static void balloon_ack(struct virtqueue *vq)
wake_up(&vb->acked);
}
+/* Update pfn_max and pfn_min according to the pfn of page */
+static inline void update_pfn_range(struct virtio_balloon *vb,
+ struct page *page,
+ unsigned long *pfn_min,
+ unsigned long *pfn_max)
+{
+ unsigned long pfn = page_to_pfn(page);
+
+ *pfn_min = min(pfn, *pfn_min);
+ *pfn_max = max(pfn, *pfn_max);
+}
+
+static unsigned int extend_page_bmap_size(struct virtio_balloon *vb,
+ unsigned long pfn_num)
+{
+ unsigned int i, bmap_num, allocated_bmap_num;
+ unsigned long bmap_len;
+
+ allocated_bmap_num = VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM;
+ bmap_len = ALIGN(pfn_num, BITS_PER_LONG) / BITS_PER_BYTE;
+ bmap_len = roundup(bmap_len, VIRTIO_BALLOON_PAGE_BMAP_SIZE);
+ /*
+ * VIRTIO_BALLOON_PAGE_BMAP_SIZE is the size of one page_bmap, so
+ * divide it to calculate how many page_bmap that we need.
+ */
+ bmap_num = (unsigned int)(bmap_len / VIRTIO_BALLOON_PAGE_BMAP_SIZE);
+ /* The number of page_bmap to allocate should not exceed the max */
+ bmap_num = min_t(unsigned int, VIRTIO_BALLOON_PAGE_BMAP_MAX_NUM,
+ bmap_num);
+
+ for (i = VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM; i < bmap_num; i++) {
+ vb->page_bmap[i] = kmalloc(VIRTIO_BALLOON_PAGE_BMAP_SIZE,
+ GFP_KERNEL);
+ if (vb->page_bmap[i])
+ allocated_bmap_num++;
+ else
+ break;
+ }
+
+ return allocated_bmap_num;
+}
+
+static void free_extended_page_bmap(struct virtio_balloon *vb,
+ unsigned int page_bmap_num)
+{
+ unsigned int i;
+
+ for (i = VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM; i < page_bmap_num;
+ i++) {
+ kfree(vb->page_bmap[i]);
+ vb->page_bmap[i] = NULL;
+ page_bmap_num--;
+ }
+}
+
+static void clear_page_bmap(struct virtio_balloon *vb,
+ unsigned int page_bmap_num)
+{
+ int i;
+
+ for (i = 0; i < page_bmap_num; i++)
+ memset(vb->page_bmap[i], 0, VIRTIO_BALLOON_PAGE_BMAP_SIZE);
+}
+
+static void send_page_chunks(struct virtio_balloon *vb, struct virtqueue *vq)
+{
+ struct scatterlist sg;
+ struct virtio_balloon_page_chunk *chunk;
+ unsigned int len;
+
+ chunk = vb->balloon_page_chunk;
+ len = sizeof(__le64) +
+ le64_to_cpu(chunk->chunk_num) *
+ sizeof(struct virtio_balloon_page_chunk_entry);
+ sg_init_one(&sg, chunk, len);
+ if (!virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL)) {
+ virtqueue_kick(vq);
+ wait_event(vb->acked, virtqueue_get_buf(vq, &len));
+ chunk->chunk_num = 0;
+ }
+}
+
+/* Add a chunk entry to the buffer. */
+static void add_one_chunk(struct virtio_balloon *vb, struct virtqueue *vq,
+ u64 base, u64 size)
+{
+ struct virtio_balloon_page_chunk *chunk = vb->balloon_page_chunk;
+ struct virtio_balloon_page_chunk_entry *entry;
+ uint64_t chunk_num = le64_to_cpu(chunk->chunk_num);
+
+ entry = &chunk->entry[chunk_num];
+ entry->base = cpu_to_le64(base << VIRTIO_BALLOON_CHUNK_BASE_SHIFT);
+ entry->size = cpu_to_le64(size << VIRTIO_BALLOON_CHUNK_SIZE_SHIFT);
+ chunk->chunk_num = cpu_to_le64(++chunk_num);
+ if (chunk_num == VIRTIO_BALLOON_MAX_PAGE_CHUNKS)
+ send_page_chunks(vb, vq);
+}
+
+static void convert_bmap_to_chunks(struct virtio_balloon *vb,
+ struct virtqueue *vq,
+ unsigned long *bmap,
+ unsigned long pfn_start,
+ unsigned long size)
+{
+ unsigned long next_one, next_zero, chunk_size, pos = 0;
+
+ while (pos < size) {
+ next_one = find_next_bit(bmap, size, pos);
+ /*
+ * No "1" bit found, which means that there is no pfn
+ * recorded in the rest of this bmap.
+ */
+ if (next_one == size)
+ break;
+ next_zero = find_next_zero_bit(bmap, size, next_one + 1);
+ /*
+ * A bit in page_bmap corresponds to a page of PAGE_SIZE.
+ * Convert it to be pages of 4KB balloon page size when
+ * adding it to a chunk.
+ */
+ chunk_size = (next_zero - next_one) *
+ VIRTIO_BALLOON_PAGES_PER_PAGE;
+ if (chunk_size) {
+ add_one_chunk(vb, vq, pfn_start + next_one,
+ chunk_size);
+ pos += next_zero + 1;
+ }
+ }
+}
+
static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
{
struct scatterlist sg;
@@ -124,7 +279,33 @@ static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)
/* When host has read buffer, this completes via balloon_ack */
wait_event(vb->acked, virtqueue_get_buf(vq, &len));
+}
+
+static void tell_host_from_page_bmap(struct virtio_balloon *vb,
+ struct virtqueue *vq,
+ unsigned long pfn_start,
+ unsigned long pfn_end,
+ unsigned int page_bmap_num)
+{
+ unsigned long i, pfn_num;
+ for (i = 0; i < page_bmap_num; i++) {
+ /*
+ * For the last page_bmap, only the remaining number of pfns
+ * need to be searched rather than the entire page_bmap.
+ */
+ if (i + 1 == page_bmap_num)
+ pfn_num = (pfn_end - pfn_start) %
+ VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP;
+ else
+ pfn_num = VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP;
+
+ convert_bmap_to_chunks(vb, vq, vb->page_bmap[i], pfn_start +
+ i * VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP,
+ pfn_num);
+ }
+ if (le64_to_cpu(vb->balloon_page_chunk->chunk_num) > 0)
+ send_page_chunks(vb, vq);
}
static void set_page_pfns(struct virtio_balloon *vb,
@@ -141,13 +322,88 @@ static void set_page_pfns(struct virtio_balloon *vb,
page_to_balloon_pfn(page) + i);
}
+/*
+ * Send ballooned pages in chunks to host.
+ * The ballooned pages are recorded in page bitmaps. Each bit in a bitmap
+ * corresponds to a page of PAGE_SIZE. The page bitmaps are searched for
+ * continuous "1" bits, which correspond to continuous pages, to chunk.
+ * When packing those continuous pages into chunks, pages are converted into
+ * 4KB balloon pages.
+ *
+ * pfn_max and pfn_min form the range of pfns that need to use page bitmaps to
+ * record. If the range is too large to be recorded into the allocated page
+ * bitmaps, the page bitmaps are used multiple times to record the entire
+ * range of pfns.
+ */
+static void tell_host_page_chunks(struct virtio_balloon *vb,
+ struct list_head *pages,
+ struct virtqueue *vq,
+ unsigned long pfn_max,
+ unsigned long pfn_min)
+{
+ /*
+ * The pfn_start and pfn_end form the range of pfns that the allocated
+ * page_bmap can record in each round.
+ */
+ unsigned long pfn_start, pfn_end;
+ /* Total number of allocated page_bmap */
+ unsigned int page_bmap_num;
+ struct page *page;
+ bool found;
+
+ /*
+ * In the case that one page_bmap is not sufficient to record the pfn
+ * range, page_bmap will be extended by allocating more numbers of
+ * page_bmap.
+ */
+ page_bmap_num = extend_page_bmap_size(vb, pfn_max - pfn_min + 1);
+
+ /* Start from the beginning of the whole pfn range */
+ pfn_start = pfn_min;
+ while (pfn_start < pfn_max) {
+ pfn_end = pfn_start +
+ VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP * page_bmap_num;
+ pfn_end = pfn_end < pfn_max ? pfn_end : pfn_max;
+ clear_page_bmap(vb, page_bmap_num);
+ found = false;
+
+ list_for_each_entry(page, pages, lru) {
+ unsigned long bmap_idx, bmap_pos, this_pfn;
+
+ this_pfn = page_to_pfn(page);
+ if (this_pfn < pfn_start || this_pfn > pfn_end)
+ continue;
+ bmap_idx = (this_pfn - pfn_start) /
+ VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP;
+ bmap_pos = (this_pfn - pfn_start) %
+ VIRTIO_BALLOON_PFNS_PER_PAGE_BMAP;
+ set_bit(bmap_pos, vb->page_bmap[bmap_idx]);
+
+ found = true;
+ }
+ if (found)
+ tell_host_from_page_bmap(vb, vq, pfn_start, pfn_end,
+ page_bmap_num);
+ /*
+ * Start the next round when pfn_start and pfn_end couldn't
+ * cover the whole pfn range given by pfn_max and pfn_min.
+ */
+ pfn_start = pfn_end;
+ }
+ free_extended_page_bmap(vb, page_bmap_num);
+}
+
static unsigned fill_balloon(struct virtio_balloon *vb, size_t num)
{
struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
unsigned num_allocated_pages;
+ bool chunking = virtio_has_feature(vb->vdev,
+ VIRTIO_BALLOON_F_PAGE_CHUNKS);
+ unsigned long pfn_max = 0, pfn_min = ULONG_MAX;
/* We can only do one array worth at a time. */
- num = min(num, ARRAY_SIZE(vb->pfns));
+ if (!chunking)
+ num = min(num, ARRAY_SIZE(vb->pfns));
mutex_lock(&vb->balloon_lock);
for (vb->num_pfns = 0; vb->num_pfns < num;
@@ -162,7 +418,10 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num)
msleep(200);
break;
}
- set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
+ if (chunking)
+ update_pfn_range(vb, page, &pfn_max, &pfn_min);
+ else
+ set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
vb->num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;
if (!virtio_has_feature(vb->vdev,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
@@ -171,8 +430,14 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num)
num_allocated_pages = vb->num_pfns;
/* Did we get any? */
- if (vb->num_pfns != 0)
- tell_host(vb, vb->inflate_vq);
+ if (vb->num_pfns != 0) {
+ if (chunking)
+ tell_host_page_chunks(vb, &vb_dev_info->pages,
+ vb->inflate_vq,
+ pfn_max, pfn_min);
+ else
+ tell_host(vb, vb->inflate_vq);
+ }
mutex_unlock(&vb->balloon_lock);
return num_allocated_pages;
@@ -198,9 +463,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
struct page *page;
struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
LIST_HEAD(pages);
+ bool chunking = virtio_has_feature(vb->vdev,
+ VIRTIO_BALLOON_F_PAGE_CHUNKS);
+ unsigned long pfn_max = 0, pfn_min = ULONG_MAX;
- /* We can only do one array worth at a time. */
- num = min(num, ARRAY_SIZE(vb->pfns));
+ /* Traditionally, we can only do one array worth at a time. */
+ if (!chunking)
+ num = min(num, ARRAY_SIZE(vb->pfns));
mutex_lock(&vb->balloon_lock);
/* We can't release more pages than taken */
@@ -210,7 +479,10 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
page = balloon_page_dequeue(vb_dev_info);
if (!page)
break;
- set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
+ if (chunking)
+ update_pfn_range(vb, page, &pfn_max, &pfn_min);
+ else
+ set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
list_add(&page->lru, &pages);
vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
}
@@ -221,8 +493,13 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
* virtio_has_feature(vdev, VIRTIO_BALLOON_F_MUST_TELL_HOST);
* is true, we *have* to do it in this order
*/
- if (vb->num_pfns != 0)
- tell_host(vb, vb->deflate_vq);
+ if (vb->num_pfns != 0) {
+ if (chunking)
+ tell_host_page_chunks(vb, &pages, vb->deflate_vq,
+ pfn_max, pfn_min);
+ else
+ tell_host(vb, vb->deflate_vq);
+ }
release_pages_balloon(vb, &pages);
mutex_unlock(&vb->balloon_lock);
return num_freed_pages;
@@ -442,6 +719,14 @@ static int init_vqs(struct virtio_balloon *vb)
}
#ifdef CONFIG_BALLOON_COMPACTION
+
+static void tell_host_one_page(struct virtio_balloon *vb,
+ struct virtqueue *vq, struct page *page)
+{
+ add_one_chunk(vb, vq, page_to_pfn(page),
+ VIRTIO_BALLOON_PAGES_PER_PAGE);
+}
+
/*
* virtballoon_migratepage - perform the balloon page migration on behalf of
* a compation thread. (called under page lock)
@@ -465,6 +750,8 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
{
struct virtio_balloon *vb = container_of(vb_dev_info,
struct virtio_balloon, vb_dev_info);
+ bool chunking = virtio_has_feature(vb->vdev,
+ VIRTIO_BALLOON_F_PAGE_CHUNKS);
unsigned long flags;
/*
@@ -486,16 +773,22 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
vb_dev_info->isolated_pages--;
__count_vm_event(BALLOON_MIGRATE);
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
- vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
- set_page_pfns(vb, vb->pfns, newpage);
- tell_host(vb, vb->inflate_vq);
-
+ if (chunking) {
+ tell_host_one_page(vb, vb->inflate_vq, newpage);
+ } else {
+ vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
+ set_page_pfns(vb, vb->pfns, newpage);
+ tell_host(vb, vb->inflate_vq);
+ }
/* balloon's page migration 2nd step -- deflate "page" */
balloon_page_delete(page);
- vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
- set_page_pfns(vb, vb->pfns, page);
- tell_host(vb, vb->deflate_vq);
-
+ if (chunking) {
+ tell_host_one_page(vb, vb->deflate_vq, page);
+ } else {
+ vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
+ set_page_pfns(vb, vb->pfns, page);
+ tell_host(vb, vb->deflate_vq);
+ }
mutex_unlock(&vb->balloon_lock);
put_page(page); /* balloon reference */
@@ -522,9 +815,75 @@ static struct file_system_type balloon_fs = {
#endif /* CONFIG_BALLOON_COMPACTION */
+static void free_page_bmap(struct virtio_balloon *vb)
+{
+ int i;
+
+ for (i = 0; i < VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM; i++) {
+ kfree(vb->page_bmap[i]);
+ vb->page_bmap[i] = NULL;
+ }
+}
+
+static int balloon_page_chunk_init(struct virtio_balloon *vb)
+{
+ int i;
+
+ vb->balloon_page_chunk = kmalloc(sizeof(__le64) +
+ sizeof(struct virtio_balloon_page_chunk_entry) *
+ VIRTIO_BALLOON_MAX_PAGE_CHUNKS, GFP_KERNEL);
+ if (!vb->balloon_page_chunk)
+ goto err_page_chunk;
+
+ /*
+ * The default number of page_bmaps are allocated. More may be
+ * allocated on demand.
+ */
+ for (i = 0; i < VIRTIO_BALLOON_PAGE_BMAP_DEFAULT_NUM; i++) {
+ vb->page_bmap[i] = kmalloc(VIRTIO_BALLOON_PAGE_BMAP_SIZE,
+ GFP_KERNEL);
+ if (!vb->page_bmap[i])
+ goto err_page_bmap;
+ }
+
+ return 0;
+err_page_bmap:
+ free_page_bmap(vb);
+ kfree(vb->balloon_page_chunk);
+ vb->balloon_page_chunk = NULL;
+err_page_chunk:
+ __virtio_clear_bit(vb->vdev, VIRTIO_BALLOON_F_PAGE_CHUNKS);
+ dev_warn(&vb->vdev->dev, "%s: failed\n", __func__);
+ return -ENOMEM;
+}
+
+static int virtballoon_validate(struct virtio_device *vdev)
+{
+ struct virtio_balloon *vb = NULL;
+ int err;
+
+ vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
+ if (!vb) {
+ err = -ENOMEM;
+ goto err_vb;
+ }
+
+ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_CHUNKS)) {
+ err = balloon_page_chunk_init(vb);
+ if (err < 0)
+ goto err_page_chunk;
+ }
+
+ return 0;
+err_page_chunk:
+ kfree(vb);
+err_vb:
+ return err;
+}
+
static int virtballoon_probe(struct virtio_device *vdev)
{
- struct virtio_balloon *vb;
+ struct virtio_balloon *vb = vdev->priv;
int err;
if (!vdev->config->get) {
@@ -533,17 +892,12 @@ static int virtballoon_probe(struct virtio_device *vdev)
return -EINVAL;
}
- vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
- if (!vb) {
- err = -ENOMEM;
- goto out;
- }
-
INIT_WORK(&vb->update_balloon_stats_work, update_balloon_stats_func);
INIT_WORK(&vb->update_balloon_size_work, update_balloon_size_func);
spin_lock_init(&vb->stop_update_lock);
vb->stop_update = false;
vb->num_pages = 0;
+
mutex_init(&vb->balloon_lock);
init_waitqueue_head(&vb->acked);
vb->vdev = vdev;
@@ -590,7 +944,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
vdev->config->del_vqs(vdev);
out_free_vb:
kfree(vb);
-out:
return err;
}
@@ -620,6 +973,8 @@ static void virtballoon_remove(struct virtio_device *vdev)
cancel_work_sync(&vb->update_balloon_stats_work);
remove_common(vb);
+ free_page_bmap(vb);
+ kfree(vb->balloon_page_chunk);
#ifdef CONFIG_BALLOON_COMPACTION
if (vb->vb_dev_info.inode)
iput(vb->vb_dev_info.inode);
@@ -664,6 +1019,7 @@ static unsigned int features[] = {
VIRTIO_BALLOON_F_MUST_TELL_HOST,
VIRTIO_BALLOON_F_STATS_VQ,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM,
+ VIRTIO_BALLOON_F_PAGE_CHUNKS,
};
static struct virtio_driver virtio_balloon_driver = {
@@ -674,6 +1030,7 @@ static struct virtio_driver virtio_balloon_driver = {
.id_table = id_table,
.probe = virtballoon_probe,
.remove = virtballoon_remove,
+ .validate = virtballoon_validate,
.config_changed = virtballoon_changed,
#ifdef CONFIG_PM_SLEEP
.freeze = virtballoon_freeze,
diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virtio_balloon.h
index 343d7dd..d532ed16 100644
--- a/include/uapi/linux/virtio_balloon.h
+++ b/include/uapi/linux/virtio_balloon.h
@@ -34,6 +34,7 @@
#define VIRTIO_BALLOON_F_MUST_TELL_HOST 0 /* Tell before reclaiming pages */
#define VIRTIO_BALLOON_F_STATS_VQ 1 /* Memory Stats virtqueue */
#define VIRTIO_BALLOON_F_DEFLATE_ON_OOM 2 /* Deflate balloon on OOM */
+#define VIRTIO_BALLOON_F_PAGE_CHUNKS 3 /* Inflate/Deflate pages in chunks */
/* Size of a PFN in the balloon interface. */
#define VIRTIO_BALLOON_PFN_SHIFT 12
@@ -82,4 +83,17 @@ struct virtio_balloon_stat {
__virtio64 val;
} __attribute__((packed));
+#define VIRTIO_BALLOON_CHUNK_BASE_SHIFT 12
+#define VIRTIO_BALLOON_CHUNK_SIZE_SHIFT 12
+struct virtio_balloon_page_chunk_entry {
+ __le64 base;
+ __le64 size;
+};
+
+struct virtio_balloon_page_chunk {
+ /* Number of chunks in the payload */
+ __le64 chunk_num;
+ struct virtio_balloon_page_chunk_entry entry[];
+};
+
#endif /* _LINUX_VIRTIO_BALLOON_H */
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 2/6] virtio-balloon: coding format cleanup
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
Clean up the comment format.
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
---
drivers/virtio/virtio_balloon.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 4a9f307..ecb64e9 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -132,8 +132,10 @@ static void set_page_pfns(struct virtio_balloon *vb,
{
unsigned int i;
- /* Set balloon pfns pointing at this page.
- * Note that the first pfn points at start of the page. */
+ /*
+ * Set balloon pfns pointing at this page.
+ * Note that the first pfn points at start of the page.
+ */
for (i = 0; i < VIRTIO_BALLOON_PAGES_PER_PAGE; i++)
pfns[i] = cpu_to_virtio32(vb->vdev,
page_to_balloon_pfn(page) + i);
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 1/6] virtio-balloon: deflate via a page list
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
In-Reply-To: <1493887815-6070-1-git-send-email-wei.w.wang@intel.com>
From: Liang Li <liang.z.li@intel.com>
This patch saves the deflated pages to a list, instead of the PFN array.
Accordingly, the balloon_pfn_to_page() function is removed.
Signed-off-by: Liang Li <liang.z.li@intel.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Wei Wang <wei.w.wang@intel.com>
---
drivers/virtio/virtio_balloon.c | 22 ++++++++--------------
1 file changed, 8 insertions(+), 14 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 34adf9b..4a9f307 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -104,12 +104,6 @@ static u32 page_to_balloon_pfn(struct page *page)
return pfn * VIRTIO_BALLOON_PAGES_PER_PAGE;
}
-static struct page *balloon_pfn_to_page(u32 pfn)
-{
- BUG_ON(pfn % VIRTIO_BALLOON_PAGES_PER_PAGE);
- return pfn_to_page(pfn / VIRTIO_BALLOON_PAGES_PER_PAGE);
-}
-
static void balloon_ack(struct virtqueue *vq)
{
struct virtio_balloon *vb = vq->vdev->priv;
@@ -182,18 +176,16 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num)
return num_allocated_pages;
}
-static void release_pages_balloon(struct virtio_balloon *vb)
+static void release_pages_balloon(struct virtio_balloon *vb,
+ struct list_head *pages)
{
- unsigned int i;
- struct page *page;
+ struct page *page, *next;
- /* Find pfns pointing at start of each page, get pages and free them. */
- for (i = 0; i < vb->num_pfns; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
- page = balloon_pfn_to_page(virtio32_to_cpu(vb->vdev,
- vb->pfns[i]));
+ list_for_each_entry_safe(page, next, pages, lru) {
if (!virtio_has_feature(vb->vdev,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
adjust_managed_page_count(page, 1);
+ list_del(&page->lru);
put_page(page); /* balloon reference */
}
}
@@ -203,6 +195,7 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
unsigned num_freed_pages;
struct page *page;
struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
+ LIST_HEAD(pages);
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@@ -216,6 +209,7 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
if (!page)
break;
set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
+ list_add(&page->lru, &pages);
vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
}
@@ -227,7 +221,7 @@ static unsigned leak_balloon(struct virtio_balloon *vb, size_t num)
*/
if (vb->num_pfns != 0)
tell_host(vb, vb->deflate_vq);
- release_pages_balloon(vb);
+ release_pages_balloon(vb, &pages);
mutex_unlock(&vb->balloon_lock);
return num_freed_pages;
}
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v10 0/6] Extend virtio-balloon for fast (de)inflating & fast live migration
From: Wei Wang @ 2017-05-04 8:50 UTC (permalink / raw)
To: virtio-dev, linux-kernel, qemu-devel, virtualization, kvm,
linux-mm, mst, david, dave.hansen, cornelia.huck, akpm, mgorman,
aarcange, amit.shah, pbonzini, wei.w.wang, liliang.opensource
This patch series implements the follow two things:
1) Optimization of balloon page transfer: instead of transferring balloon pages
to host one by one, the new mechanism transfers them in chunks.
2) A mechanism to report info of guest unused pages: the pages have been unused
at some time between when host sent command and when guest reported them. Host
uses that by tracking memory changes and then discarding changes made to the
pages that it gets from guest before it sent the command.
Changes:
v9->v10:
1) mm: put report_unused_page_block() under CONFIG_VIRTIO_BALLOON;
2) virtio-balloon: add virtballoon_validate();
3) virtio-balloon: msg format change;
4) virtio-balloon: move miscq handling to a task on system_freezable_wq;
5) virtio-balloon: code cleanup.
v8->v9:
1) Split the two new features, VIRTIO_BALLOON_F_BALLOON_CHUNKS and
VIRTIO_BALLOON_F_MISC_VQ, which were mixed together in the previous
implementation;
2) Simpler function to get the free page block.
v7->v8:
1) Use only one chunk format, instead of two.
2) re-write the virtio-balloon implementation patch.
3) commit changes
4) patch re-org
Liang Li (1):
virtio-balloon: deflate via a page list
Wei Wang (5):
virtio-balloon: coding format cleanup
virtio-balloon: VIRTIO_BALLOON_F_PAGE_CHUNKS
mm: function to offer a page block on the free list
mm: export symbol of next_zone and first_online_pgdat
virtio-balloon: VIRTIO_BALLOON_F_MISC_VQ
drivers/virtio/virtio_balloon.c | 696 +++++++++++++++++++++++++++++++++---
include/linux/mm.h | 5 +
include/uapi/linux/virtio_balloon.h | 26 ++
mm/mmzone.c | 2 +
mm/page_alloc.c | 91 +++++
5 files changed, 761 insertions(+), 59 deletions(-)
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-04 8:17 UTC (permalink / raw)
To: Dave Hansen, Michal Hocko; +Cc: linux-mm, linux-kernel
In-Reply-To: <70a9d4db-f374-de45-413b-65b74c59edcb@intel.com>
Hi,
I suspect this was accidentally a Reply-To instead of a Reply-All,
so I'm putting back the CCs that were dropped.
On 03/05/17 21:41, Dave Hansen wrote:
> On 05/03/2017 05:06 AM, Igor Stoppa wrote:
>> My starting point are the policy DB of SE Linux and the LSM Hooks, but
>> eventually I would like to extend the protection also to other
>> subsystems, in a way that can be merged into mainline.
>
> Have you given any thought to just having a set of specialized slabs?
No, the idea of the RFC was to get this sort of comments about options I
might have missed :-)
> Today, for instance, we have a separate set of kmalloc() slabs for DMA:
> dma-kmalloc-{4096,2048,...}. It should be quite possible to have
> another set for your post-init-read-only protected data.
I will definitely investigate it and report back, thanks.
But In the meanwhile I'd appreciate further clarifications.
Please see below ...
> This doesn't take care of vmalloc(), but I have the feeling that
> implementing this for vmalloc() isn't going to be horribly difficult.
ok
>> * The mechanism used for locking down the memory region is to program
>> the MMU to trap writes to said region. It is fairly efficient and
>> HW-backed, so it doesn't introduce any major overhead,
>
> I'd take a bit of an issue with this statement. It *will* fracture
> large pages unless you manage to pack all of these allocations entirely
> within a large page. This is problematic because we use the largest
> size available, and that's 1GB on x86.
I am not sure I fully understand this part.
I am probably missing some point about the way kmalloc works.
I get the problem you describe, but I do not understand why it should
happen.
Going back for a moment to my original idea of the zone, as a physical
address range, why wouldn't it be possible to define it as one large page?
Btw, I do not expect to have much memory occupation, in terms of sheer
size, although there might be many small "variables" scattered across
the code. That's where I hope using kmalloc, instead of a custom made
allocator can make a difference, in terms of optimal occupation.
> IOW, if you scatter these things throughout the address space, you may
> end up fracturing/demoting enough large pages to cause major overhead
> refilling the TLB.
But why would I?
Or, better, what would cause it, unless I take special care?
Or, let me put it differently: my goal is to not fracture more pages
than needed.
It will probably require some profiling to figure out what is the
ballpark of the memory footprint.
I might have overlooked some aspect of this, but the overall goal
is to have a memory range (I won't call it zone, to avoid referring to a
specific implementation) which is as tightly packed as possible, stuffed
with all the data that is expected to become read-only.
> Note that this only applies for kmalloc() allocations, *not* vmalloc()
> since kmalloc() uses the kernel linear map and vmalloc() uses it own,
> separate mappings.
Yes.
---
thanks, igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 7/8] mm, compaction: restrict async compaction to pageblocks of same migratetype
From: Vlastimil Babka @ 2017-05-04 6:12 UTC (permalink / raw)
To: Joonsoo Kim
Cc: Johannes Weiner, Andrew Morton, linux-kernel, linux-mm,
Mel Gorman, David Rientjes, kernel-team, kernel-team
In-Reply-To: <20170407003851.GA17231@js1304-P5Q-DELUXE>
On 04/07/2017 02:38 AM, Joonsoo Kim wrote:
> On Wed, Mar 29, 2017 at 06:06:41PM +0200, Vlastimil Babka wrote:
>> On 03/16/2017 03:14 AM, Joonsoo Kim wrote:
>>> On Tue, Mar 07, 2017 at 02:15:44PM +0100, Vlastimil Babka wrote:
>>>> The migrate scanner in async compaction is currently limited to MIGRATE_MOVABLE
>>>> pageblocks. This is a heuristic intended to reduce latency, based on the
>>>> assumption that non-MOVABLE pageblocks are unlikely to contain movable pages.
>>>>
>>>> However, with the exception of THP's, most high-order allocations are not
>>>> movable. Should the async compaction succeed, this increases the chance that
>>>> the non-MOVABLE allocations will fallback to a MOVABLE pageblock, making the
>>>> long-term fragmentation worse.
>>>
>>> I agree with this idea but have some concerns on this change.
>>>
>>> *ASYNC* compaction is designed for reducing latency and this change
>>> doesn't fit it. If everything works fine, there is a few movable pages
>>> in non-MOVABLE pageblocks as you noted above. Moreover, there is quite
>>> less the number of non-MOVABLE pageblock than MOVABLE one so finding
>>> non-MOVABLE pageblock takes long time. These two factors will increase
>>> the latency of *ASYNC* compaction.
>>
>> Right. I lately started to doubt the whole idea of async compaction (for
>> non-movable allocations). Seems it's one of the compaction heuristics tuned
>> towards the THP usecase. But for non-movable allocations, we just can't have
>> both the low latency and long-term fragmentation avoidance. I see now even my
>> own skip_on_failure mode in isolate_migratepages_block() as a mistake for
>> non-movable allocations.
>
> Why do you think that skip_on_failure mode is a mistake? I think that
> it would lead to reduce the latency and it fits the goal of async
> compaction.
Yes, but the downside is that compaction will create just the single
high-order page that is requested, while previously it would also
migrate away some more lower-order pages. When compacting for
MIGRATE_UNMOVABLE allocation, we then can't steal extra pages, so next
allocation might pollute a different pageblock. It's not a good tradeoff.
>>
>> Ideally I'd like to make async compaction redundant by kcompactd, and direct
>> compaction would mean a serious situation which should warrant sync compaction.
>> Meanwhile I see several options to modify this patch
>> - async compaction for non-movable allocations will stop doing the
>> skip_on_failure mode, and won't restrict the pageblock at all. patch 8/8 will
>> make sure that also this kind of compaction finishes the whole pageblock
>> - non-movable allocations will skip async compaction completely and go for sync
>> compaction immediately
>
> IMO, concept of async compaction is also important for non-movable allocation.
> Non-movable allocation is essential for some workload and they hope
> the low latency.
The low latency should not be at the expense of making long-term
fragmentation worse.
> Thanks.
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Balbir Singh @ 2017-05-04 5:26 UTC (permalink / raw)
To: Michal Hocko
Cc: linux-mm, akpm, khandual, benh, aneesh.kumar, paulmck, srikar,
haren, jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <20170502143608.GM14593@dhcp22.suse.cz>
On Tue, 2017-05-02 at 16:36 +0200, Michal Hocko wrote:
> On Wed 19-04-17 17:52:38, Balbir Singh wrote:
> > This is a request for comments on the discussed approaches
> > for coherent memory at mm-summit (some of the details are at
> > https://lwn.net/Articles/717601/). The latest posted patch
> > series is at https://lwn.net/Articles/713035/. I am reposting
> > this as RFC, Michal Hocko suggested using HMM for CDM, but
> > we believe there are stronger reasons to use the NUMA approach.
> > The earlier patches for Coherent Device memory were implemented
> > and designed by Anshuman Khandual.
> >
> > Jerome posted HMM-CDM at https://lwn.net/Articles/713035/.
> > The patches do a great deal to enable CDM with HMM, but we
> > still believe that HMM with CDM is not a natural way to
> > represent coherent device memory and the mm will need
> > to be audited and enhanced for it to even work.
> >
> > With HMM we'll see ZONE_DEVICE pages mapped into
> > user space and that would mean a thorough audit of all code
> > paths to make sure we are ready for such a use case and enabling
> > those use cases, like with HMM CDM patch 1, which changes
> > move_pages() and migration paths. I've done a quick
> > evaluation to check for features and found limitationd around
> > features like migration (page cache
> > migration), fault handling to the right location
> > (direct page cache allocation in the coherent memory), mlock
> > handling, RSS accounting, memcg enforcement for pages not on LRU, etc.
>
> Are those problems not viable to solve?
Yes, except IIUC the direct page cache allocation one. The reason for calling
them out is to make aware that HMM CDM would require new mm changes/audit
to support ZONE_DEVICE pages across several parts of the mm subsystem.
>
> [...]
> > Introduction
> >
> > CDM device memory is cache coherent with system memory and we would like
> > this to show up as a NUMA node, however there are certain algorithms
> > that might not be currently suitable for N_COHERENT_MEMORY
> >
> > 1. AutoNUMA balancing
>
> OK, I can see a reason for that but theoretically the same applies to
> cpu less numa nodes in general, no?
That is correct. Christoph has shown some interest in isolating some
algorithms as well. I have some ideas that I can send out later.
>
> > 2. kswapd reclaim
>
> How is the memory reclaim handled then? How are users expected to handle
> OOM situation?
>
1. The fallback node list for coherent memory includes regular memory
nodes
2. Direct reclaim works, I've tested it
> > The reason for exposing this device memory as NUMA is to simplify
> > the programming model, where memory allocation via malloc() or
> > mmap() for example would seamlessly work across both kinds of
> > memory. Since we expect the size of device memory to be smaller
> > than system RAM, we would like to control the allocation of such
> > memory. The proposed mechanism reuses nodemasks and explicit
> > specification of the coherent node in the nodemask for allocation
> > from device memory. This implementation also allows for kernel
> > level allocation via __GFP_THISNODE and existing techniques
> > such as page migration to work.
>
> so it basically resembles isol_cpus except for memory, right. I believe
> scheduler people are more than unhappy about this interface...
>
isol_cpus were for an era when timer/interrupts and other scheduler
infrastructure present today was not around, but I don't mean to digress.
> Anyway, I consider CPUless nodes a dirty hack (especially when I see
> them mostly used with poorly configured LPARs where no CPUs are left for
> a particular memory). Now this is trying to extend this concept even
> further to a memory which is not reclaimable by the kernel and requires
Direct reclaim still works
> an explicit and cooperative memory reclaim from userspace. How is this
> going to work? The memory also has a different reliability properties
> from RAM which user space doesn't have any clue about from the NUMA
> properties exported. Or am I misunderstanding it? That all sounds quite
> scary to me.
>
> I very much agree with the last email from Mel and I would really like
> to see how would a real application benefit from these nodes.
>
I see two use cases
1. Aware application/library - allocates from this node and uses this memory
2. Unaware application/library - allocates memory anywhere, but does not use
CDM memory by default, since it is isolated.
Both 1 and 2 can work together and an aware application can use an unaware
library and if required migrate pages between the two. Both 1 and 2
can access each others memory due to coherency, so the final application
level use case is similar to HMM. That is why HMM-CDM and NUMA-CDM are
both equivalent from an application programming model perspective,
except for the limitations mentioned above.
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RESENT PATCH] x86/mem: fix the offset overflow when read/write mem
From: zhong jiang @ 2017-05-04 2:28 UTC (permalink / raw)
To: Rik van Riel
Cc: David Rientjes, Bjorn Helgaas, Yoshinori Sato, Rich Felker,
Andrew Morton, arnd, hannes, kirill, mgorman, hughd, linux-mm,
linux-kernel, Xishi Qiu
In-Reply-To: <1493837167.20270.8.camel@redhat.com>
On 2017/5/4 2:46, Rik van Riel wrote:
> On Tue, 2017-05-02 at 13:54 -0700, David Rientjes wrote:
>
>>> diff --git a/drivers/char/mem.c b/drivers/char/mem.c
>>> index 7e4a9d1..3a765e02 100644
>>> --- a/drivers/char/mem.c
>>> +++ b/drivers/char/mem.c
>>> @@ -55,7 +55,7 @@ static inline int
>> valid_phys_addr_range(phys_addr_t addr, size_t count)
>>>
>>> static inline int valid_mmap_phys_addr_range(unsigned long pfn,
>> size_t size)
>>> {
>>> - return 1;
>>> + return (pfn << PAGE_SHIFT) + size <= __pa(high_memory);
>>> }
>>> #endif
>>>
>> I suppose you are correct that there should be some sanity checking
>> on the
>> size used for the mmap().
> My apologies for not responding earlier. It may
> indeed make sense to have a sanity check here.
>
> However, it is not as easy as simply checking the
> end against __pa(high_memory). Some systems have
> non-contiguous physical memory ranges, with gaps
> of invalid addresses in-between.
The invalid physical address means that it is used as
io mapped. not in system ram region. /dev/mem is not
access to them , is it right?
> You would have to make sure that both the beginning
> and the end are valid, and that there are no gaps of
> invalid pfns in the middle...
If it is limited in system ram, we can walk the resource
to exclude them. or adding pfn_valid further to optimize.
whether other situation should be consider ? I am not sure.
> At that point, is the complexity so much that it no
> longer makes sense to try to protect against root
> crashing the system?
>
your suggestion is to let the issue along without any protection.
just root user know what they are doing.
Thanks
zhongjiang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 1/2] mm: Uncharge poisoned pages
From: Balbir Singh @ 2017-05-04 1:21 UTC (permalink / raw)
To: Michal Hocko, Andi Kleen, Johannes Weiner
Cc: Laurent Dufour, Naoya Horiguchi, linux-kernel, linux-mm, akpm,
Vladimir Davydov
In-Reply-To: <20170502185507.GB19165@dhcp22.suse.cz>
> @@ -5527,7 +5527,7 @@ static void uncharge_list(struct list_head *page_list)
> next = page->lru.next;
>
> VM_BUG_ON_PAGE(PageLRU(page), page);
> - VM_BUG_ON_PAGE(page_count(page), page);
> + VM_BUG_ON_PAGE(!PageHWPoison(page) && page_count(page), page);
>
> if (!page->mem_cgroup)
> continue;
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 8a6bd3a9eb1e..4497d9619bb4 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -541,6 +541,13 @@ static int delete_from_lru_cache(struct page *p)
> */
> ClearPageActive(p);
> ClearPageUnevictable(p);
> +
> + /*
> + * Poisoned page might never drop its ref count to 0 so we have to
> + * uncharge it manually from its memcg.
> + */
> + mem_cgroup_uncharge(p);
> +
Yep, that is the right fix
https://lkml.org/lkml/2017/4/26/133
Reviewed-by: Balbir Singh <bsingharora@gmail.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [mmotm:master 189/272] fs/affs/affs.h:10:22: fatal error: amigaffs.h: No such file or directory
From: kbuild test robot @ 2017-05-04 1:05 UTC (permalink / raw)
To: Andrew Morton
Cc: kbuild-all, Linux Memory Management List, Davidlohr Bueso,
Johannes Weiner
[-- Attachment #1: Type: text/plain, Size: 2403 bytes --]
tree: git://git.cmpxchg.org/linux-mmotm.git master
head: 670376fbea66d7b33412edefc754d4867e26d515
commit: 696cc1169d5609cee552f0e43c322c178a704b60 [189/272] linux-next-git-rejects
config: x86_64-allyesdebian (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
git checkout 696cc1169d5609cee552f0e43c322c178a704b60
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
In file included from fs/affs/super.c:23:0:
>> fs/affs/affs.h:10:22: fatal error: amigaffs.h: No such file or directory
#include "amigaffs.h"
^
compilation terminated.
vim +10 fs/affs/affs.h
0158de12 Fabian Frederick 2014-06-06 1 #ifdef pr_fmt
0158de12 Fabian Frederick 2014-06-06 2 #undef pr_fmt
0158de12 Fabian Frederick 2014-06-06 3 #endif
0158de12 Fabian Frederick 2014-06-06 4
0158de12 Fabian Frederick 2014-06-06 5 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
0158de12 Fabian Frederick 2014-06-06 6
^1da177e Linus Torvalds 2005-04-16 7 #include <linux/types.h>
^1da177e Linus Torvalds 2005-04-16 8 #include <linux/fs.h>
^1da177e Linus Torvalds 2005-04-16 9 #include <linux/buffer_head.h>
c1d6a71a Andrew Morton 2017-05-03 @10 #include "amigaffs.h"
7d135a5d Matthias Kaehlcke 2008-07-25 11 #include <linux/mutex.h>
3dd84782 Artem Bityutskiy 2012-06-06 12 #include <linux/workqueue.h>
^1da177e Linus Torvalds 2005-04-16 13
^1da177e Linus Torvalds 2005-04-16 14 /* Ugly macros make the code more pretty. */
^1da177e Linus Torvalds 2005-04-16 15
^1da177e Linus Torvalds 2005-04-16 16 #define GET_END_PTR(st,p,sz) ((st *)((char *)(p)+((sz)-sizeof(st))))
^1da177e Linus Torvalds 2005-04-16 17 #define AFFS_GET_HASHENTRY(data,hashkey) be32_to_cpu(((struct dir_front *)data)->hashtable[hashkey])
^1da177e Linus Torvalds 2005-04-16 18 #define AFFS_BLOCK(sb, bh, blk) (AFFS_HEAD(bh)->table[AFFS_SB(sb)->s_hashsize-1-(blk)])
:::::: The code at line 10 was first introduced by commit
:::::: c1d6a71a2ed24d4f6576954fc19c7fde5b44696c linux-next
:::::: TO: Andrew Morton <akpm@linux-foundation.org>
:::::: CC: Johannes Weiner <hannes@cmpxchg.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 39323 bytes --]
^ permalink raw reply
* [PATCH] mm, vmstat: Remove spurious WARN() during zoneinfo print
From: Reza Arbab @ 2017-05-03 23:41 UTC (permalink / raw)
To: Andrew Morton
Cc: David Rientjes, Anshuman Khandual, Vlastimil Babka, Mel Gorman,
Johannes Weiner, linux-kernel, linux-mm
In-Reply-To: <alpine.DEB.2.10.1703031451310.98023@chino.kir.corp.google.com>
After "mm, vmstat: print non-populated zones in zoneinfo",
/proc/zoneinfo will show unpopulated zones.
A memoryless node, having no populated zones at all, was previously
ignored, but will now trigger the WARN() in is_zone_first_populated().
Remove this warning, as its only purpose was to warn of a situation that
has since been enabled.
Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
---
Aside: The "per-node stats" are still printed under the first populated
zone, but that's not necessarily the first stanza any more. I'm not sure
which criteria is more important with regard to not breaking parsers, but
it looks a little weird to the eye.
mm/vmstat.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/mm/vmstat.c b/mm/vmstat.c
index f5fa1bd..76f7367 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1359,8 +1359,6 @@ static bool is_zone_first_populated(pg_data_t *pgdat, struct zone *zone)
return zone == compare;
}
- /* The zone must be somewhere! */
- WARN_ON_ONCE(1);
return false;
}
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [patch v2] mm, vmscan: avoid thrashing anon lru when free + file is low
From: David Rientjes @ 2017-05-03 22:52 UTC (permalink / raw)
To: Michal Hocko
Cc: Andrew Morton, Minchan Kim, Johannes Weiner, Mel Gorman,
linux-kernel, linux-mm
In-Reply-To: <20170503084952.GD8836@dhcp22.suse.cz>
On Wed, 3 May 2017, Michal Hocko wrote:
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 24efcc20af91..f3ec8760dc06 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2113,16 +2113,14 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
> u64 denominator = 0; /* gcc */
> struct pglist_data *pgdat = lruvec_pgdat(lruvec);
> unsigned long anon_prio, file_prio;
> - enum scan_balance scan_balance;
> + enum scan_balance scan_balance = SCAN_FILE;
> unsigned long anon, file;
> unsigned long ap, fp;
> enum lru_list lru;
>
> /* If we have no swap space, do not bother scanning anon pages. */
> - if (!sc->may_swap || mem_cgroup_get_nr_swap_pages(memcg) <= 0) {
> - scan_balance = SCAN_FILE;
> + if (!sc->may_swap || mem_cgroup_get_nr_swap_pages(memcg) <= 0)
> goto out;
> - }
>
> /*
> * Global reclaim will swap to prevent OOM even with no
> @@ -2131,10 +2129,8 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
> * using the memory controller's swap limit feature would be
> * too expensive.
> */
> - if (!global_reclaim(sc) && !swappiness) {
> - scan_balance = SCAN_FILE;
> + if (!global_reclaim(sc) && !swappiness)
> goto out;
> - }
>
> /*
> * Do not apply any pressure balancing cleverness when the
Good as a cleanup so far.
> @@ -2147,8 +2143,9 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
> }
>
> /*
> - * Prevent the reclaimer from falling into the cache trap: as
> - * cache pages start out inactive, every cache fault will tip
> + * We usually want to bias page cache reclaim over anonymous
> + * memory. Prevent the reclaimer from falling into the cache trap:
> + * as cache pages start out inactive, every cache fault will tip
> * the scan balance towards the file LRU. And as the file LRU
> * shrinks, so does the window for rotation from references.
> * This means we have a runaway feedback loop where a tiny
I think Minchan made a good point earlier about anon being more likely to
be working set since it is mapped, but this may be a biased opinion coming
from me since I am primarily concerned with malloc.
> @@ -2173,26 +2170,24 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
> total_high_wmark += high_wmark_pages(zone);
> }
>
> - if (unlikely(pgdatfile + pgdatfree <= total_high_wmark)) {
> + if (unlikely(pgdatfile + pgdatfree <= total_high_wmark))
> scan_balance = SCAN_ANON;
> - goto out;
> - }
> }
>
> /*
> - * If there is enough inactive page cache, i.e. if the size of the
> - * inactive list is greater than that of the active list *and* the
> - * inactive list actually has some pages to scan on this priority, we
> - * do not reclaim anything from the anonymous working set right now.
> - * Without the second condition we could end up never scanning an
> - * lruvec even if it has plenty of old anonymous pages unless the
> - * system is under heavy pressure.
> + * Make sure there are enough pages on the biased LRU before we go
> + * and do an exclusive reclaim from that list, i.e. if the
> + * size of the inactive list is greater than that of the active list
> + * *and* the inactive list actually has some pages to scan on this
> + * priority.
> + * Without the second condition we could end up never scanning other
> + * lruvecs even if they have plenty of old pages unless the system is
> + * under heavy pressure.
> */
> - if (!inactive_list_is_low(lruvec, true, memcg, sc, false) &&
> - lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, sc->reclaim_idx) >> sc->priority) {
> - scan_balance = SCAN_FILE;
> + lru = LRU_INACTIVE_ANON + LRU_FILE * (scan_balance == SCAN_FILE);
This part seems to complicate the logic since it determines the lru under
test based on the current setting of scan_balance. I think I prefer
individual heuristics with well written comments, but others may feel
differently about this.
> + if (!inactive_list_is_low(lruvec, is_file_lru(lru), memcg, sc, false) &&
> + lruvec_lru_size(lruvec, lru, sc->reclaim_idx) >> sc->priority)
> goto out;
> - }
>
> scan_balance = SCAN_FRACT;
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* mmotm 2017-05-03-15-16 uploaded
From: akpm @ 2017-05-03 22:17 UTC (permalink / raw)
To: mm-commits, linux-kernel, linux-mm, linux-fsdevel, linux-next,
sfr, mhocko, broonie
The mm-of-the-moment snapshot 2017-05-03-15-16 has been uploaded to
http://www.ozlabs.org/~akpm/mmotm/
mmotm-readme.txt says
README for mm-of-the-moment:
http://www.ozlabs.org/~akpm/mmotm/
This is a snapshot of my -mm patch queue. Uploaded at random hopefully
more than once a week.
You will need quilt to apply these patches to the latest Linus release (4.x
or 4.x-rcY). The series file is in broken-out.tar.gz and is duplicated in
http://ozlabs.org/~akpm/mmotm/series
The file broken-out.tar.gz contains two datestamp files: .DATE and
.DATE-yyyy-mm-dd-hh-mm-ss. Both contain the string yyyy-mm-dd-hh-mm-ss,
followed by the base kernel version against which this patch series is to
be applied.
This tree is partially included in linux-next. To see which patches are
included in linux-next, consult the `series' file. Only the patches
within the #NEXT_PATCHES_START/#NEXT_PATCHES_END markers are included in
linux-next.
A git tree which contains the memory management portion of this tree is
maintained at git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
by Michal Hocko. It contains the patches which are between the
"#NEXT_PATCHES_START mm" and "#NEXT_PATCHES_END" markers, from the series
file, http://www.ozlabs.org/~akpm/mmotm/series.
A full copy of the full kernel tree with the linux-next and mmotm patches
already applied is available through git within an hour of the mmotm
release. Individual mmotm releases are tagged. The master branch always
points to the latest release, so it's constantly rebasing.
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/
To develop on top of mmotm git:
$ git remote add mmotm git://git.kernel.org/pub/scm/linux/kernel/git/mhocko/mm.git
$ git remote update mmotm
$ git checkout -b topic mmotm/master
<make changes, commit>
$ git send-email mmotm/master.. [...]
To rebase a branch with older patches to a new mmotm release:
$ git remote update mmotm
$ git rebase --onto mmotm/master <topic base> topic
The directory http://www.ozlabs.org/~akpm/mmots/ (mm-of-the-second)
contains daily snapshots of the -mm tree. It is updated more frequently
than mmotm, and is untested.
A git copy of this tree is available at
http://git.cmpxchg.org/cgit.cgi/linux-mmots.git/
and use of this tree is similar to
http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/, described above.
This mmotm tree contains the following patches against 4.11:
(patches marked "*" will be included in linux-next)
origin.patch
i-need-old-gcc.patch
* dma-debug-make-locking-to-work-for-rt.patch
* scripts-spellingtxt-add-several-more-common-spelling-mistakes.patch
* blackfin-bf609-let-clk_disable-return-immediately-if-clk-is-null.patch
* fs-ocfs2-cluster-use-setup_timer.patch
* ocfs2-o2hb-revert-hb-threshold-to-keep-compatible.patch
* fs-ocfs2-cluster-use-offset_in_page-macro.patch
* slab-avoid-ipis-when-creating-kmem-caches.patch
* mm-fix-100%-cpu-kswapd-busyloop-on-unreclaimable-nodes.patch
* mm-fix-check-for-reclaimable-pages-in-pf_memalloc-reclaim-throttling.patch
* mm-remove-seemingly-spurious-reclaimability-check-from-laptop_mode-gating.patch
* mm-remove-unnecessary-reclaimability-check-from-numa-balancing-target.patch
* mm-dont-avoid-high-priority-reclaim-on-unreclaimable-nodes.patch
* mm-dont-avoid-high-priority-reclaim-on-memcg-limit-reclaim.patch
* mm-delete-nr_pages_scanned-and-pgdat_reclaimable.patch
* revert-mm-vmscan-account-for-skipped-pages-as-a-partial-scan.patch
* mm-remove-unnecessary-back-off-function-when-retrying-page-reclaim.patch
* writeback-use-setup_deferrable_timer.patch
* mm-delete-unnecessary-ttu_-flags.patch
* mm-dont-assume-anonymous-pages-have-swapbacked-flag.patch
* mm-move-madv_free-pages-into-lru_inactive_file-list.patch
* mm-reclaim-madv_free-pages.patch
* mm-fix-lazyfree-bug-on-check-in-try_to_unmap_one.patch
* mm-enable-madv_free-for-swapless-system.patch
* proc-show-madv_free-pages-info-in-smaps.patch
* mm-memcontrol-provide-shmem-statistics.patch
* mm-swap-fix-a-race-in-free_swap_and_cache.patch
* mm-use-is_migrate_highatomic-to-simplify-the-code.patch
* mm-use-is_migrate_isolate_page-to-simplify-the-code.patch
* mm-vmstat-print-non-populated-zones-in-zoneinfo.patch
* mm-vmstat-suppress-pcp-stats-for-unpopulated-zones-in-zoneinfo.patch
* lockdep-teach-lockdep-about-memalloc_noio_save.patch
* lockdep-allow-to-disable-reclaim-lockup-detection.patch
* xfs-abstract-pf_fstrans-to-pf_memalloc_nofs.patch
* mm-introduce-memalloc_nofs_saverestore-api.patch
* xfs-use-memalloc_nofs_saverestore-instead-of-memalloc_noio.patch
* jbd2-mark-the-transaction-context-with-the-scope-gfp_nofs-context.patch
* jbd2-make-the-whole-kjournald2-kthread-nofs-safe.patch
* mm-tighten-up-the-fault-path-a-little.patch
* mm-remove-rodata_test_data-export-add-pr_fmt.patch
* mm-do-not-use-double-negation-for-testing-page-flags.patch
* mm-vmscan-fix-zone-balance-check-in-prepare_kswapd_sleep.patch
* mm-vmscan-only-clear-pgdat-congested-dirty-writeback-state-when-balanced.patch
* mm-vmscan-prevent-kswapd-sleeping-prematurely-due-to-mismatched-classzone_idx.patch
* mm-page_alloc-__gfp_nowarn-shouldnt-suppress-stall-warnings.patch
* mm-sparse-refine-usemap_size-a-little.patch
* mm-compaction-ignore-block-suitable-after-check-large-free-page.patch
* mm-vmscan-more-restrictive-condition-for-retry-in-do_try_to_free_pages.patch
* mm-remove-unncessary-ret-in-page_referenced.patch
* mm-remove-swap_dirty-in-ttu.patch
* mm-remove-swap_mlock-check-for-swap_success-in-ttu.patch
* mm-make-the-try_to_munlock-void-function.patch
* mm-remove-swap_mlock-in-ttu.patch
* mm-remove-swap_again-in-ttu.patch
* mm-make-ttus-return-boolean.patch
* mm-make-rmap_walk-void-function.patch
* mm-make-rmap_one-boolean-function.patch
* mm-remove-swap_.patch
* mm-swap-fix-comment-in-__read_swap_cache_async.patch
* mm-swap-improve-readability-via-make-spin_lock-unlock-balanced.patch
* mm-swap-avoid-lock-swap_avail_lock-when-held-cluster-lock.patch
* mm-enable-page-poisoning-early-at-boot.patch
* mm-include-linux-migrateh-fixing-checkpatch-warning-regarding-function-definition.patch
* swap-add-warning-if-swap-slots-cache-failed-to-initialize.patch
* mm-fix-spelling-error.patch
* userfaultfd-selftest-combine-all-cases-into-the-single-executable.patch
* oom-improve-oom-disable-handling.patch
* mm-mmap-replace-shm_huge_mask-with-map_huge_mask-inside-mmap_pgoff.patch
* mm-vmscan-fix-io-refault-regression-in-cache-workingset-transition.patch
* mm-memcontrol-clean-up-memoryevents-counting-function.patch
* mm-memcontrol-re-use-global-vm-event-enum.patch
* mm-memcontrol-re-use-node-vm-page-state-enum.patch
* mm-memcontrol-use-node-page-state-naming-scheme-for-memcg.patch
* mm-swap-remove-unused-function-prototype.patch
* documentation-vm-add-hugetlbfs-reservation-overview.patch
* mm-madvise-clean-up-madv_soft_offline-and-madv_hwpoison.patch
* mm-madvise-move-up-the-behavior-parameter-validation.patch
* mm-softoffline-add-page-flag-description-in-error-paths.patch
* mm-page_alloc-remove-debug_guardpage_minorder-test-in-warn_alloc.patch
* zram-handle-multiple-pages-attached-bios-bvec.patch
* zram-partial-io-refactoring.patch
* zram-use-zram_slot_lock-instead-of-raw-bit_spin_lock-op.patch
* zram-remove-zram_meta-structure.patch
* zram-introduce-zram-data-accessor.patch
* zram-use-zram_free_page-instead-of-open-coded.patch
* zram-reduce-load-operation-in-page_same_filled.patch
* fs-fix-data-invalidation-in-the-cleancache-during-direct-io.patch
* fs-block_dev-always-invalidate-cleancache-in-invalidate_bdev.patch
* mm-truncate-bail-out-early-from-invalidate_inode_pages2_range-if-mapping-is-empty.patch
* mm-truncate-avoid-pointless-cleancache_invalidate_inode-calls.patch
* mm-gup-fix-access_ok-argument-type.patch
* mm-swap-fix-swap-space-leak-in-error-path-of-swap_free_entries.patch
* mm-hwpoison-call-shake_page-unconditionally.patch
* mm-hwpoison-call-shake_page-after-try_to_unmap-for-mlocked-page.patch
* kasan-introduce-helper-functions-for-determining-bug-type.patch
* kasan-unify-report-headers.patch
* kasan-change-allocation-and-freeing-stack-traces-headers.patch
* kasan-simplify-address-description-logic.patch
* kasan-change-report-header.patch
* kasan-improve-slab-object-description.patch
* kasan-print-page-description-after-stacks.patch
* kasan-improve-double-free-report-format.patch
* kasan-separate-report-parts-by-empty-lines.patch
* mm-uncharge-poisoned-pages.patch
* mm-skip-hwpoisoned-pages-when-onlining-pages.patch
* arm-arch-arm-include-asm-pageh-needs-personalityh.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called.patch
* ocfs2-old-mle-put-and-release-after-the-function-dlm_add_migration_mle-called-fix.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks.patch
* ocfs2-dlm-optimization-of-code-while-free-dead-node-locks-checkpatch-fixes.patch
* block-restore-proc-partitions-to-not-display-non-partitionable-removable-devices.patch
mm.patch
* mm-compaction-reorder-fields-in-struct-compact_control.patch
* mm-compaction-remove-redundant-watermark-check-in-compact_finished.patch
* mm-page_alloc-split-smallest-stolen-page-in-fallback.patch
* mm-page_alloc-split-smallest-stolen-page-in-fallback-fix.patch
* mm-page_alloc-count-movable-pages-when-stealing-from-pageblock.patch
* mm-page_alloc-count-movable-pages-when-stealing-from-pageblock-fix.patch
* mm-compaction-change-migrate_async_suitable-to-suitable_migration_source.patch
* mm-compaction-add-migratetype-to-compact_control.patch
* mm-compaction-restrict-async-compaction-to-pageblocks-of-same-migratetype.patch
* mm-compaction-finish-whole-pageblock-to-reduce-fragmentation.patch
* mm-page_alloc-return-0-in-case-this-node-has-no-page-within-the-zone.patch
* mm-vmscan-do-not-pass-reclaimed-slab-to-vmpressure.patch
* mm-page_owner-align-with-pageblock_nr-pages.patch
* mm-walk-the-zone-in-pageblock_nr_pages-steps.patch
* proc-remove-cast-from-memory-allocation.patch
* proc-sysctl-fix-the-int-overflow-for-jiffies-conversion.patch
* drivers-virt-use-get_user_pages_unlocked.patch
* jiffiesh-declare-jiffies-and-jiffies_64-with-____cacheline_aligned_in_smp.patch
* make-help-add-tools-help-target.patch
* locking-hung_task-defer-showing-held-locks.patch
* vmci-fix-a-couple-integer-overflow-tests.patch
* c2port-checking-for-null-instead-of-is_err.patch
* revert-lib-test_sortc-make-it-explicitly-non-modular.patch
* lib-add-module-support-to-array-based-sort-tests.patch
* lib-add-module-support-to-linked-list-sorting-tests.patch
* firmware-makefile-force-recompilation-if-makefile-changes.patch
* checkpatch-remove-obsolete-config_experimental-checks.patch
* checkpatch-add-ability-to-find-bad-uses-of-vsprintf-%pfoo-extensions.patch
* checkpatch-add-ability-to-find-bad-uses-of-vsprintf-%pfoo-extensions-fix.patch
* checkpatch-add-ability-to-find-bad-uses-of-vsprintf-%pfoo-extensions-fix-fix.patch
* checkpatch-improve-embedded_function_name-test.patch
* checkpatch-allow-space-leading-blank-lines-in-email-headers.patch
* checkpatch-avoid-suggesting-struct-definitions-should-be-const.patch
* checkpatch-improve-multistatement_macro_use_do_while-test.patch
* checkpatch-clarify-the-embedded_function_name-message.patch
* checkpatch-special-audit-for-revert-commit-line.patch
* checkpatch-improve-kalloc-with-multiplication-and-sizeof-test.patch
* checkpatch-add-typedefsfile.patch
* checkpatch-improve-the-embedded-function-name-test-for-patch-contexts.patch
* checkpatch-improve-the-suspect_code_indent-test.patch
* reiserfs-use-designated-initializers.patch
* fork-free-vmapped-stacks-in-cache-when-cpus-are-offline.patch
* cpumask-make-nr_cpumask_bits-unsigned.patch
* crash-move-crashkernel-parsing-and-vmcore-related-code-under-config_crash_core.patch
* ia64-reuse-append_elf_note-and-final_note-functions.patch
* powerpc-fadump-remove-dependency-with-config_kexec.patch
* powerpc-fadump-reuse-crashkernel-parameter-for-fadump-memory-reservation.patch
* powerpc-fadump-update-documentation-about-crashkernel-parameter-reuse.patch
* kdump-vmcoreinfo-report-actual-value-of-phys_base.patch
* uapi-fix-linux-sysctlh-userspace-compilation-errors.patch
* pidns-disable-pid-allocation-if-pid_ns_prepare_proc-is-failed-in-alloc_pid.patch
* ns-allow-ns_entries-to-have-custom-symlink-content.patch
* pidns-expose-task-pid_ns_for_children-to-userspace.patch
* taskstats-add-e-u-stime-for-tgid-command.patch
* taskstats-add-e-u-stime-for-tgid-command-fix.patch
* taskstats-add-e-u-stime-for-tgid-command-fix-fix.patch
* kcov-simplify-interrupt-check.patch
* scripts-gdb-add-lx-fdtdump-command.patch
* kernel-reboot-add-devm_register_reboot_notifier.patch
* kernel-reboot-add-devm_register_reboot_notifier-fix.patch
* fault-inject-use-correct-check-for-interrupts.patch
* fault-inject-support-systematic-fault-injection.patch
* fault-inject-support-systematic-fault-injection-fix.patch
* fault-inject-automatically-detect-the-number-base-for-fail-nth-write-interface.patch
* fault-inject-parse-as-natural-1-based-value-for-fail-nth-write-interface.patch
* fault-inject-make-fail-nth-read-write-interface-symmetric.patch
* fault-inject-simplify-access-check-for-fail-nth.patch
* fault-inject-simplify-access-check-for-fail-nth-fix.patch
* fault-inject-add-proc-pid-fail-nth.patch
* zlib-inflate-fix-potential-buffer-overflow.patch
* initramfs-provide-a-way-to-ignore-image-provided-by-bootloader.patch
* initramfs-use-vfs_stat-lstat-directly.patch
* ipc-shm-some-shmat-cleanups.patch
* sysvipc-cacheline-align-kern_ipc_perm.patch
linux-next.patch
linux-next-git-rejects.patch
* imx7-fix-kconfig-warning-and-build-errors.patch
* sparc64-ng4-memset-32-bits-overflow.patch
* mm-zeroing-hash-tables-in-allocator.patch
* mm-updated-callers-to-use-hash_zero-flag.patch
* mm-adaptive-hash-table-scaling.patch
* mm-introduce-kvalloc-helpers.patch
* mm-introduce-kvalloc-helpers-fix.patch
* mm-vmalloc-properly-track-vmalloc-users.patch
* mm-support-__gfp_repeat-in-kvmalloc_node-for-32kb.patch
* rhashtable-simplify-a-strange-allocation-pattern.patch
* ila-simplify-a-strange-allocation-pattern.patch
* xattr-zero-out-memory-copied-to-userspace-in-getxattr.patch
* treewide-use-kvalloc-rather-than-opencoded-variants.patch
* net-use-kvmalloc-with-__gfp_repeat-rather-than-open-coded-variant.patch
* md-use-kvmalloc-rather-than-opencoded-variant.patch
* bcache-use-kvmalloc.patch
* mm-swap-use-kvzalloc-to-allocate-some-swap-data-structure.patch
* mm-vmalloc-use-__gfp_highmem-implicitly.patch
* scripts-spellingtxt-add-memory-pattern-and-fix-typos.patch
* scripts-spellingtxt-add-regsiter-register-spelling-mistake.patch
* scripts-spellingtxt-add-intialised-pattern-and-fix-typo-instances.patch
* treewide-correct-diffrent-and-banlance-typos.patch
* treewide-move-set_memory_-functions-away-from-cacheflushh.patch
* arm-use-set_memoryh-header.patch
* arm64-use-set_memoryh-header.patch
* s390-use-set_memoryh-header.patch
* x86-use-set_memoryh-header.patch
* agp-use-set_memoryh-header.patch
* drm-use-set_memoryh-header.patch
* drm-use-set_memoryh-header-fix.patch
* intel_th-use-set_memoryh-header.patch
* watchdog-hpwdt-use-set_memoryh-header.patch
* bpf-use-set_memoryh-header.patch
* module-use-set_memoryh-header.patch
* pm-hibernate-use-set_memoryh-header.patch
* alsa-use-set_memoryh-header.patch
* misc-sram-use-set_memoryh-header.patch
* video-vermilion-use-set_memoryh-header.patch
* drivers-staging-media-atomisp-pci-atomisp2-use-set_memoryh.patch
* treewide-decouple-cacheflushh-and-set_memoryh.patch
* treewide-decouple-cacheflushh-and-set_memoryh-fix.patch
* kref-remove-warn_on-for-null-release-functions.patch
* megasas-remove-expensive-inline-from-megasas_return_cmd.patch
* remove-expensive-warn_on-in-pagefault_disabled_dec.patch
* fs-remove-set-but-not-checked-aop_flag_uninterruptible-flag.patch
* docs-vm-transhuge-fix-few-trivial-typos.patch
* docs-vm-transhuge-fix-few-trivial-typos-fix.patch
* format-security-move-static-strings-to-const.patch
* fs-f2fs-use-ktime_get_real_seconds-for-sit_info-times.patch
* trace-make-trace_hwlat-timestamp-y2038-safe.patch
* fs-cifs-replace-current_time-by-other-appropriate-apis.patch
* fs-ceph-current_time-with-ktime_get_real_ts.patch
* fs-ufs-use-ktime_get_real_ts64-for-birthtime.patch
* fs-ubifs-replace-current_time_sec-with-current_time.patch
* lustre-replace-current_time-macro.patch
* apparmorfs-replace-current_time-with-current_time.patch
* gfs2-replace-current_time-with-current_time.patch
* time-delete-current_time_sec-and-current_time.patch
* time-delete-current_fs_time-function.patch
* mm-huge_memory-use-zap_deposited_table-more.patch
* mm-huge_memory-deposit-a-pgtable-for-dax-pmd-faults-when-required.patch
* mm-prevent-potential-recursive-reclaim-due-to-clearing-pf_memalloc.patch
* mm-introduce-memalloc_noreclaim_saverestore.patch
* treewide-convert-pf_memalloc-manipulations-to-new-helpers.patch
* treewide-convert-pf_memalloc-manipulations-to-new-helpers-fix.patch
* mtd-nand-nandsim-convert-to-memalloc_noreclaim_.patch
* dax-add-tracepoints-to-dax_iomap_pte_fault.patch
* dax-add-tracepoints-to-dax_pfn_mkwrite.patch
* dax-add-tracepoints-to-dax_load_hole.patch
* dax-add-tracepoints-to-dax_writeback_mapping_range.patch
* dax-add-tracepoints-to-dax_writeback_mapping_range-fix.patch
* dax-add-tracepoint-to-dax_writeback_one.patch
* dax-add-tracepoint-to-dax_writeback_one-fix.patch
* dax-add-tracepoint-to-dax_insert_mapping.patch
* selftests-vm-add-a-test-for-virtual-address-range-mapping.patch
* lib-crc-ccitt-add-ccitt-false-crc16-variant.patch
mm-add-strictlimit-knob-v2.patch
make-sure-nobodys-leaking-resources.patch
releasing-resources-with-children.patch
kernel-forkc-export-kernel_thread-to-modules.patch
mutex-subsystem-synchro-test-module.patch
slab-leaks3-default-y.patch
workaround-for-a-pci-restoring-bug.patch
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: Review request: draft ioctl_userfaultfd(2) manual page
From: Andrea Arcangeli @ 2017-05-03 21:46 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Mike Rapoport, lkml, linux-mm@kvack.org, linux-man
In-Reply-To: <e8c5ca4a-0710-7206-b96e-10d171bda218@gmail.com>
On Fri, Apr 21, 2017 at 11:11:18AM +0200, Michael Kerrisk (man-pages) wrote:
> Hello Mike,
> Hello Andrea (we need your help!),
Sorry for not answering sooner! (I had a vacation last week)
>
> On 03/22/2017 02:54 PM, Mike Rapoport wrote:
> >> The various ioctl(2) operations are described below. The UFFDIO_API,
> >> UFFDIO_REGISTER, and UFFDIO_UNREGISTER operations are used to configure
> >> userfaultfd behavior. These operations allow the caller to choose what
> >> features will be enabled and what kinds of events will be delivered to
> >> the application. The remaining operations are range operations. These
> >> operations enable the calling application to resolve page-fault events
> >> in a consistent way.
> >>
> >>
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??Above: What does "consistent" mean? a??
> >> a?? a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >
> > Andrea, can you please help with this one?
>
> Let's see what Andrea has to say.
I think it doesn't mean anything and I see you already removed it, fine!
> So, the thing that was not clear, but now I think I understand:
> 'features' is an input field where one can ask about supported features
> (but none are supported, before Linux 4.11). Is that correct? I've changed
> the text here to read:
>
> Before the call, the features field must be initialized
> to zero. In the future, it is intended that this field can be
> used to ask whether particular features are supported.
>
> Seem okay?
Yes, but in reality nothing has changed. Simply the kernels before
4.11 had no feature support at all.
===
Starting from Linux 4.11, the features field can be used to ask
whether particular features are supported and explicitly enable
userfaultfd features that are disabled by default. The kernel
always reports all the available features in the features
field.
=====
I would prefer if we removed this 4.11 difference here.
We should be able to describe it simply as:
"The features field can be used to ask whether particular features are
supported and explicitly enable userfaultfd features that are disabled
by default. The kernel always reports all the available features in
the features field."
The whole point of this feature flag thing, is so the app at runtime
can check if the feature is available and ask for it. The fact kernels
before 4.11 don't support any feature is a detail.
> > There's a check in uffdio_api call that the user is not trying to enable
> > any other functionality and it asserts that uffdio_api.featurs is zero [1].
> > Starting from 4.11 the features negotiation is different. Now uffdio_call
> > verifies that it can support features the application requested [2].
>
> Okay.
I don't like the differentiation here between 4.11 and before, because
from user point of view nothing has changed.
I think this description is enough " Since Linux 4.11, the
following feature bits may be set: " and no other mention of 4.11 is
needed in the manpage. It looks an unnecessary complication to the reader.
>
> >> The kernel verifies that it can support the requested API version, and
> >> sets the features and ioctls fields to bit masks representing all the
> >> available features and the generic ioctl(2) operations available. Cura??
> >> rently, zero (i.e., no feature bits) is placed in the features field.
> >> The returned ioctls field can contain the following bits:
> >>
> >>
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??This user-space API seems not fully polished. Why a??
> >> a??are there not constants defined for each of the bit- a??
> >> a??mask values listed below? a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >>
> >> 1 << _UFFDIO_API
> >> The UFFDIO_API operation is supported.
> >>
> >> 1 << _UFFDIO_REGISTER
> >> The UFFDIO_REGISTER operation is supported.
> >>
> >> 1 << _UFFDIO_UNREGISTER
> >> The UFFDIO_UNREGISTER operation is supported.
> >
> > Well, I tend to agree. I believe the original intention was to use the
> > OR'ed mask, like UFFD_API_IOCTLS.
> > Andrea, can you add somthing?
>
> Yes, Andrea, please!
I agree it can be polished, but that's not something the manpage can
fix... For now the above is correct.
So about the error retvals I reviewed the final manpage from git which
is the latest version.
> >>
> >> EINVAL The userfaultfd has already been enabled by a previous UFFa??
> >> DIO_API operation.
> >>
> >> EINVAL The API version requested in the api field is not supported by
> >> this kernel, or the features field was not zero.
> >>
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??In the above error case, the returned 'uffdio_api' a??
> >> a??structure zeroed out. Why is this done? This should a??
> >> a??be explained in the manual page. a??
> >> a?? a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >
> > In my understanding the uffdio_api structure is zeroed to allow the caller
> > to distinguish the reasons for -EINVAL.
>
> Andrea, can you please help here?
It is zeroed out just for robustness, it's a slow path. If userland by
mistake won't check -EINVAL but it checks uffdio_api.features to be
set or uffdio_api.ioctls or uffdio_api.api after the UFFDIO_API ioctl
returns, it will have a chance to catch the failure (it won't risk to
parse random uninitialized values at least).
I don't think it should be documented the uffdio_api is zeroed out or
if it is documented, we should say userland shouldn't depend on it and
it's done just for robustness.
The normal correct way to catch an error is to check -EINVAL, after
getting -EINVAL the contents of uffdio_api should be ignored by
userland.
> >> UFFDIO_REGISTER
> >> (Since Linux 4.3.) Register a memory address range with the usera??
> >> faultfd object. The argp argument is a pointer to a uffdio_register
> >> structure, defined as:
> >>
> >> struct uffdio_range {
> >> __u64 start; /* Start of range */
> >> __u64 len; /* Length of rnage (bytes) */
> >> };
> >>
> >> struct uffdio_register {
> >> struct uffdio_range range;
> >> __u64 mode; /* Desired mode of operation (input) */
> >> __u64 ioctls; /* Available ioctl() operations (output) */
> >> };
> >>
> >>
> >> The range field defines a memory range starting at start and continuing
> >> for len bytes that should be handled by the userfaultfd.
> >>
> >> The mode field defines the mode of operation desired for this memory
> >> region. The following values may be bitwise ORed to set the usera??
> >> faultfd mode for the specified range:
> >>
> >> UFFDIO_REGISTER_MODE_MISSING
> >> Track page faults on missing pages.
> >>
> >> UFFDIO_REGISTER_MODE_WP
> >> Track page faults on write-protected pages.
> >>
> >> Currently, the only supported mode is UFFDIO_REGISTER_MODE_MISSING.
> >>
> >> If the operation is successful, the kernel modifies the ioctls bit-mask
> >> field to indicate which ioctl(2) operations are available for the speca??
> >> ified range. This returned bit mask is as for UFFDIO_API.
> >>
> >> This ioctl(2) operation returns 0 on success. On error, -1 is returned
> >> and errno is set to indicate the cause of the error. Possible errors
> >> include:
> >>
> >>
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??Is the following error list correct? a??
> >> a?? a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >
> > Here again it maybe -EFAULT to indicate copy_{from,to}_user failure.
> > And, UFFDIO_REGISTER may return -ENOMEM if the process is exiting and the
> > mm_struct has gone by the time userfault grabs it.
>
> Okay -- added EFAULT. I think I'll skip ENOMEM for the moment, but
> will note the possibility in the page source.
There is also ENOMEM as result of split_vma failing, and it isn't the
cleanest thing that it means both real OOM or out of vmas
(mm->map_count >= sysctl_max_map_count, not real OOM) and that the
process is exiting or there isn't a single vma in the mm.
If there's one vma but it isn't in the range we return -EINVAL so we
could return probably -EINVAL if it's exiting or if there isn't a
single vma in the mm. And leave -ENOMEM for split_vma only.
In general -EINVAL is programmer error of some kind, -ENOMEM is
returned in memory related cases that trigger at runtime, however if
the process is exiting it's debatable if it's programmer error too
which is why I think we could return -EINVAL there.
I'd expect userland to threat -ENOMEM and -EINVAL about the same way.
> >> EINVAL There was no mapping in the specified address range.
> >>
> >> UFFDIO_COPY
> >> (Since Linux 4.3.) Atomically copy a continuous memory chunk into the
> >> userfault registered range and optionally wake up the blocked thread.
> >> The source and destination addresses and the number of bytes to copy
> >> are specified by the src, dst, and len fields of the uffdio_copy struca??
> >> ture pointed to by argp:
> >>
> >> struct uffdio_copy {
> >> __u64 dst; /* Source of copy */
> >> __u64 src; /* Destinate of copy */
> >> __u64 len; /* Number of bytes to copy */
> >> __u64 mode; /* Flags controlling behavior of copy */
> >> __s64 copy; /* Number of bytes copied, or negated error */
> >> };
> >>
> >> The following value may be bitwise ORed in mode to change the behavior
> >> of the UFFDIO_COPY operation:
> >>
> >> UFFDIO_COPY_MODE_DONTWAKE
> >> Do not wake up the thread that waits for page-fault resolution
> >>
> >> The copy field is used by the kernel to return the number of bytes that
> >> was actually copied, or an error (a negated errno-style value).
> >>
> >>
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??Above: Why is the 'copy' field used to return error a??
> >> a??values? This should be explained in the manual a??
> >> a??page. a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >
> > Andrea, can you help with this one, please?
>
> Yes, Andrea, please.
Well not just for error values. copy also tells how much did it copy
if a signal made it return short with -EINTR or some other error
happened in the middle of the copy after we already did some
copying-progress.
Writing any other error into .copy (and not only writing positive
values there) is for robustness in case userland won't check the ioctl
retval but nobody should depend on it. After the ioctl returns an
error userland should not check the uffdio_copy structure at all.
The thing to document is the amount of bytes successfully copied in
uffdio_copy.copy (which may be a short copy and in turn must be
checked... unless one knows the len matches the arch PAGE_SIZE but
it's still safer to check the uffdio_copy.copy field and be
consistent).
One more relevant retvals for UFFDIO_COPY and UFFDIO_ZEROPAGE that I
noticed is missing in the current manpage, is
-EEXIST. UFFDIO_COPY/ZEROPAGE don't behave like mmap/mremap,
UFFDIO_COPY/ZEROPAGE will never teardown any existing established
mapping to guarantee even if the user has race condition in its code,
there's no risk of silent memory corruption, instead a meaningful
error is returned by UFFDIO_COPY/ZEROPAGE. For example if two
UFFDIO_COPY run concurrently on the same page, only the first one will
succeed, the second will return -EEXIST and only the first page will
be copied and no memory corruption can happen this way (furthermore
the programmer can be notified of the race condition in the code which
might even be intentional, but more likely is not).
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??FIXME a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
> >> a??Why is the 'zeropage' field used to return error a??
> >> a??values? This should be explained in the manual a??
> >> a??page. a??
> >> a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??a??
>
> Help is still needed for this FIXME!
Same as uffdio_copy.copy: because we've to return the number of pages
that have been zeroed out in case we run into an error (signal -EINTR
or -EEXIST etc..) after we already succeeded partially on a couple of
pages. So we may as well write the error in the same field if no pages
were zeroed out. This way the program will misbehave more than if that
field is left untouched (and it could even be random in such case as
it can be left uninitialized by userland). Clearly it only makes a
difference if the programmer forgets to check the UFFDIO_ZEROPAGE
ioctl retval and the retval must always be checked, so again, this is
only for robustness.
Awesome manpage! Super useful, it's fundamental to have a manpage
especially when the syscall is not simple and strightforward in
functionality.
Thanks!
Andrea
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox