* [PATCH v11 0/3] xen: better grant v2 support
@ 2017-09-28 10:02 Juergen Gross
2017-09-28 10:02 ` [PATCH v11 1/3] xen: make grant resource limits per domain Juergen Gross
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Juergen Gross @ 2017-09-28 10:02 UTC (permalink / raw)
To: xen-devel
Cc: Juergen Gross, sstabellini, wei.liu2, George.Dunlap,
andrew.cooper3, ian.jackson, tim, julien.grall, jbeulich
Currently Linux has no support for grant v2 as this would reduce the
maximum number of active grants by a factor of 2 compared to v1,
because the number of possible grants are limited by the allowed number
of grant frames and grant entries of v2 need twice as much bytes as
those of v1.
Unfortunately grant v2 is the only way to support either guests with
more than 16TB memory size or PV guests with memory above the 16TB
border, as grant v1 limits the frame number to be 32 bits wide.
In order to remove the disadvantage of grant v2 this patch series
adds support for setting per-domain values regarding grant limits.
Additionally the default limit of grant frames is doubled in case
of hosts with potential memory above the 16TB border.
Changes in V11:
- dropped patches 1-8, as already committed
- patch 1: add comment in include/asm-arm/grant_table.h (Julien Grall)
- patch 2: return max. subleaf number in EAX (Andrew Cooper)
- patch 2: use flsl() instead of generic_flsl() (Jan Beulich)
Changes in V10:
- patch 1: avoid holes in xen_sysctl_physinfo, use uint64_aligned_t
(Jan Beulich)
- patch 7: move locking into grant_table_init() (Jan Beulich)
- patch 9: lower INITIAL_NR_GRANT_FRAMES value to 1
- patch 9: removed no longer needed variables (Jan Beulich)
- patch 9: stream lined messages (Jan Beulich)
- patch 9: corrected setting of gnttab_size on ARM
- patch 10: correct comment in cpuid.h (Jan Beulich)
- added patch 11
Changes in V9:
- dropped the already committed patches 1-3
- merged former patches 4 and 5 (Jan Beulich)
- merged former patches 13 and 14
- patch 1: bump sysctl interface version (Julien Grall)
- patch 1: drop thin common shim of get_upper_mfn_bound() (Jan Beulich)
- patch 1: let get_upper_mfn_bound() return the highest MFN, not the one
following it (Jan Beulich)
- patch 7: allocate initial grant frames from grant_table_init()
(Jan Beulich)
- patch 8: correct and cleanup gnttab_init_arch() for ARM (Julien Grall)
- patch 10: make leaf pv-only (Jan Beulich)
- patch 10: use hex value for mask (Jan Beulich)
- patch 10: guest address width -> machine address width (Jan Beulich)
Changes in V8:
- dropped previous patch 1, as already committed
- patch 1: make gnttab_grow_table() static instead doing so in next
patch (Jan Beulich)
- patch 3: remove stale #if 0, adjust comments (Paul Durrant)
Changes in V7:
- added patches 5, 6, 9, 16
- patch 2: only call gnttab_set_frame_gfn() if no error (Julien Grall)
- patch 10: don't use xc_maximum_ram_page() but max_possible_mfn from
physinfo
- patch 13: re-add #include <asm/grant-table.h> in grant_table.h
(Julien Grall)
- patch 15: add boot parameter documentation changes
Changes in V6:
- several new patches (1, 6, 7, 10, 12)
- order of patches re-arranged to support new hypercall now being
mandatory
- lots of other small changes
Changes in V5:
- patch 6: add set_gnttab_limits to create_domain_common in xen.if
(Daniel De Graaf)
Changes in V4:
- patch 3: make ret more local (Wei Liu)
- patch 7: use domid_t (Wei Liu)
- patch 8: rename configuration items to use max_ prefixes (Wei Liu)
Changes in V3:
- patch 1: update commit message
- patch 3: move call of grant_table_init() from gnttab_setup_table() to
gnttab_grow_table() (Paul Durrant)
- patch 4: correct error message (Paul Durrant)
- patch 6: rename *gnttbl* to *gnttab* (Paul Durrant)
Changes in V2:
- add per-domain grant limits instead of different v1 and v2 limits
- double default limit for huge hosts
Juergen Gross (3):
xen: make grant resource limits per domain
xen: add new Xen cpuid node for max address width info
xen: add some comments in include/public/arch-x86/cpuid.h
docs/misc/xen-command-line.markdown | 25 +++---
xen/arch/arm/domain_build.c | 6 +-
xen/arch/x86/traps.c | 8 ++
xen/common/compat/grant_table.c | 31 ++-----
xen/common/grant_table.c | 156 +++++++++++++++++++++---------------
xen/include/asm-arm/grant_table.h | 20 ++++-
xen/include/asm-x86/grant_table.h | 7 +-
xen/include/public/arch-x86/cpuid.h | 33 +++++++-
xen/include/xen/grant_table.h | 5 +-
9 files changed, 176 insertions(+), 115 deletions(-)
--
2.12.3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v11 1/3] xen: make grant resource limits per domain 2017-09-28 10:02 [PATCH v11 0/3] xen: better grant v2 support Juergen Gross @ 2017-09-28 10:02 ` Juergen Gross 2017-10-03 13:33 ` Julien Grall 2017-09-28 10:02 ` [PATCH v11 2/3] xen: add new Xen cpuid node for max address width info Juergen Gross ` (2 subsequent siblings) 3 siblings, 1 reply; 7+ messages in thread From: Juergen Gross @ 2017-09-28 10:02 UTC (permalink / raw) To: xen-devel Cc: Juergen Gross, sstabellini, wei.liu2, George.Dunlap, andrew.cooper3, ian.jackson, tim, julien.grall, jbeulich Instead of using the same global resource limits of grant tables (max. number of grant frames, max. number of maptrack frames) for all domains make these limits per domain. Set those per-domain limits in grant_table_set_limits(). The global settings are serving as an upper boundary now which must not be exceeded by a per-domain value. The default of max_grant_frames is set to the maximum default xl will use. While updating the semantics of the boot parameters remove the documentation of the no longer existing gnttab_max_nr_frames and correct the default gnttab_max_maptrack_frames uses. Signed-off-by: Juergen Gross <jgross@suse.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> [non-ARM parts] --- V11: - add comment in include/asm-arm/grant_table.h (Julien Grall) V10: - lower INITIAL_NR_GRANT_FRAMES value to 1 - removed no longer needed variables (Jan Beulich) - stream lined messages (Jan Beulich) - corrected setting of gnttab_size on ARM V9: - add caps for per-domain limits (Jan Beulich) - some error messages enhanced (Jan Beulich) - adjusted some types (Jan Beulich) - merge parts of (former) patch 14 into this one - make parameters changeable at runtime - limit size of dom0's grant table on ARM - set default max_grant_frames to 64 V6: - several changes due to new patch order V3: - correct error message (Paul Durrant) --- docs/misc/xen-command-line.markdown | 25 +++--- xen/arch/arm/domain_build.c | 6 +- xen/common/compat/grant_table.c | 31 ++----- xen/common/grant_table.c | 156 +++++++++++++++++++++--------------- xen/include/asm-arm/grant_table.h | 20 ++++- xen/include/asm-x86/grant_table.h | 7 +- xen/include/xen/grant_table.h | 5 +- 7 files changed, 138 insertions(+), 112 deletions(-) diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown index 9797c8db2d..9b6cd8e9d0 100644 --- a/docs/misc/xen-command-line.markdown +++ b/docs/misc/xen-command-line.markdown @@ -875,27 +875,28 @@ Specify which console gdbstub should use. See **console**. ### gnttab\_max\_frames > `= <integer>` -> Default: `32` +> Default: `64` + +> Can be modified at runtime Specify the maximum number of frames which any domain may use as part -of its grant table. +of its grant table. This value is an upper boundary of the per-domain +value settable via Xen tools. + +Dom0 is using this value for sizing its grant table. ### gnttab\_max\_maptrack\_frames > `= <integer>` -> Default: `8 * gnttab_max_frames` - -Specify the maximum number of frames to use as part of a domains -maptrack array. +> Default: `1024` -### gnttab\_max\_nr\_frames -> `= <integer>` +> Can be modified at runtime -*Deprecated* -Use **gnttab\_max\_frames** and **gnttab\_max\_maptrack\_frames** instead. +Specify the maximum number of frames to use as part of a domains +maptrack array. This value is an upper boundary of the per-domain +value settable via Xen tools. -Specify the maximum number of frames per grant table operation and the -maximum number of maptrack frames domain. +Dom0 is using this value for sizing its maptrack table. ### guest\_loglvl > `= <level>[/<rate-limited level>]` where level is `none | error | warning | info | debug | all` diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index c34238ec1b..3723dc3f78 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2095,11 +2095,7 @@ static void __init find_gnttab_region(struct domain *d, * enough space for a large grant table */ kinfo->gnttab_start = __pa(_stext); - kinfo->gnttab_size = (_etext - _stext) & PAGE_MASK; - - /* Make sure the grant table will fit in the region */ - if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames ) - panic("Cannot find a space for the grant table region\n"); + kinfo->gnttab_size = gnttab_dom0_frames() << PAGE_SHIFT; #ifdef CONFIG_ARM_32 /* diff --git a/xen/common/compat/grant_table.c b/xen/common/compat/grant_table.c index cce3ff0b9a..ff1d678f01 100644 --- a/xen/common/compat/grant_table.c +++ b/xen/common/compat/grant_table.c @@ -157,21 +157,14 @@ int compat_grant_table_op(unsigned int cmd, unsigned int max_frame_list_size_in_page = (COMPAT_ARG_XLAT_SIZE - sizeof(*nat.setup)) / sizeof(*nat.setup->frame_list.p); - if ( max_frame_list_size_in_page < max_grant_frames ) - { - gdprintk(XENLOG_WARNING, - "max_grant_frames is too large (%u,%u)\n", - max_grant_frames, max_frame_list_size_in_page); - rc = -EINVAL; - } - else - { + #define XLAT_gnttab_setup_table_HNDL_frame_list(_d_, _s_) \ - set_xen_guest_handle((_d_)->frame_list, (unsigned long *)(nat.setup + 1)) - XLAT_gnttab_setup_table(nat.setup, &cmp.setup); + set_xen_guest_handle((_d_)->frame_list, (unsigned long *)(nat.setup + 1)) + XLAT_gnttab_setup_table(nat.setup, &cmp.setup); #undef XLAT_gnttab_setup_table_HNDL_frame_list - rc = gnttab_setup_table(guest_handle_cast(nat.uop, gnttab_setup_table_t), 1); - } + rc = gnttab_setup_table(guest_handle_cast(nat.uop, + gnttab_setup_table_t), + 1, max_frame_list_size_in_page); } ASSERT(rc <= 0); if ( rc == 0 ) @@ -294,16 +287,6 @@ int compat_grant_table_op(unsigned int cmd, rc = -EFAULT; break; } - if ( max_frame_list_size_in_pages < - grant_to_status_frames(max_grant_frames) ) - { - gdprintk(XENLOG_WARNING, - "grant_to_status_frames(max_grant_frames) is too large (%u,%u)\n", - grant_to_status_frames(max_grant_frames), - max_frame_list_size_in_pages); - rc = -EINVAL; - break; - } #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \ set_xen_guest_handle((_d_)->frame_list, (uint64_t *)(nat.get_status + 1)) @@ -312,7 +295,7 @@ int compat_grant_table_op(unsigned int cmd, rc = gnttab_get_status_frames( guest_handle_cast(nat.uop, gnttab_get_status_frames_t), - count); + count, max_frame_list_size_in_pages); if ( rc >= 0 ) { #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \ diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index 71706f5cba..7c328b0a62 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -54,6 +54,9 @@ struct grant_table { * what version to use yet. */ unsigned int gt_version; + /* Resource limits of the domain. */ + unsigned int max_grant_frames; + unsigned int max_maptrack_frames; /* Table size. Number of frames shared with guest */ unsigned int nr_grant_frames; /* Number of grant status frames shared with guest (for version 2) */ @@ -78,23 +81,18 @@ struct grant_table { #ifndef DEFAULT_MAX_NR_GRANT_FRAMES /* to allow arch to override */ /* Default maximum size of a grant table. [POLICY] */ -#define DEFAULT_MAX_NR_GRANT_FRAMES 32 +#define DEFAULT_MAX_NR_GRANT_FRAMES 64 #endif -unsigned int __read_mostly max_grant_frames; -integer_param("gnttab_max_frames", max_grant_frames); +static unsigned int __read_mostly max_grant_frames = + DEFAULT_MAX_NR_GRANT_FRAMES; +integer_runtime_param("gnttab_max_frames", max_grant_frames); -/* The maximum number of grant mappings is defined as a multiplier of the - * maximum number of grant table entries. This defines the multiplier used. - * Pretty arbitrary. [POLICY] - * As gnttab_max_nr_frames has been deprecated, this multiplier is deprecated too. - * New options allow to set max_maptrack_frames and - * map_grant_table_frames independently. - */ #define DEFAULT_MAX_MAPTRACK_FRAMES 1024 -static unsigned int __read_mostly max_maptrack_frames; -integer_param("gnttab_max_maptrack_frames", max_maptrack_frames); +static unsigned int __read_mostly max_maptrack_frames = + DEFAULT_MAX_MAPTRACK_FRAMES; +integer_runtime_param("gnttab_max_maptrack_frames", max_maptrack_frames); /* * Note that the three values below are effectively part of the ABI, even if @@ -290,8 +288,8 @@ num_act_frames_from_sha_frames(const unsigned int num) return DIV_ROUND_UP(num * sha_per_page, ACGNT_PER_PAGE); } -#define max_nr_active_grant_frames \ - num_act_frames_from_sha_frames(max_grant_frames) +#define max_nr_active_grant_frames(gt) \ + num_act_frames_from_sha_frames((gt)->max_grant_frames) static inline unsigned int nr_active_grant_frames(struct grant_table *gt) @@ -530,7 +528,7 @@ get_maptrack_handle( * out of memory, try stealing an entry from another VCPU (in case the * guest isn't mapping across its VCPUs evenly). */ - if ( nr_maptrack_frames(lgt) < max_maptrack_frames ) + if ( nr_maptrack_frames(lgt) < lgt->max_maptrack_frames ) new_mt = alloc_xenheap_page(); if ( !new_mt ) @@ -1673,7 +1671,7 @@ gnttab_grow_table(struct domain *d, unsigned int req_nr_frames) if ( req_nr_frames < INITIAL_NR_GRANT_FRAMES ) req_nr_frames = INITIAL_NR_GRANT_FRAMES; - ASSERT(req_nr_frames <= max_grant_frames); + ASSERT(req_nr_frames <= gt->max_grant_frames); gdprintk(XENLOG_INFO, "Expanding dom (%d) grant table from (%d) to (%d) frames.\n", @@ -1730,7 +1728,8 @@ active_alloc_failed: } static int -grant_table_init(struct domain *d, struct grant_table *gt) +grant_table_init(struct domain *d, struct grant_table *gt, + unsigned int grant_frames, unsigned int maptrack_frames) { int ret; @@ -1742,25 +1741,31 @@ grant_table_init(struct domain *d, struct grant_table *gt) goto unlock; } + gt->max_grant_frames = grant_frames; + gt->max_maptrack_frames = maptrack_frames; + /* Active grant table. */ gt->active = xzalloc_array(struct active_grant_entry *, - max_nr_active_grant_frames); + max_nr_active_grant_frames(gt)); if ( gt->active == NULL ) goto no_mem; /* Tracking of mapped foreign frames table */ - gt->maptrack = vzalloc(max_maptrack_frames * sizeof(*gt->maptrack)); - if ( gt->maptrack == NULL ) - goto no_mem; + if ( gt->max_maptrack_frames ) + { + gt->maptrack = vzalloc(gt->max_maptrack_frames * sizeof(*gt->maptrack)); + if ( gt->maptrack == NULL ) + goto no_mem; + } /* Shared grant table. */ - gt->shared_raw = xzalloc_array(void *, max_grant_frames); + gt->shared_raw = xzalloc_array(void *, gt->max_grant_frames); if ( gt->shared_raw == NULL ) goto no_mem; /* Status pages for grant table - for version 2 */ gt->status = xzalloc_array(grant_status_t *, - grant_to_status_frames(max_grant_frames)); + grant_to_status_frames(gt->max_grant_frames)); if ( gt->status == NULL ) goto no_mem; @@ -1793,7 +1798,8 @@ grant_table_init(struct domain *d, struct grant_table *gt) static long gnttab_setup_table( - XEN_GUEST_HANDLE_PARAM(gnttab_setup_table_t) uop, unsigned int count) + XEN_GUEST_HANDLE_PARAM(gnttab_setup_table_t) uop, unsigned int count, + unsigned int limit_max) { struct vcpu *curr = current; struct gnttab_setup_table op; @@ -1807,15 +1813,6 @@ gnttab_setup_table( if ( unlikely(copy_from_guest(&op, uop, 1)) ) return -EFAULT; - if ( unlikely(op.nr_frames > max_grant_frames) ) - { - gdprintk(XENLOG_INFO, "Xen only supports up to %d grant-table frames" - " per domain.\n", - max_grant_frames); - op.status = GNTST_general_error; - goto out; - } - if ( !guest_handle_okay(op.frame_list, op.nr_frames) ) return -EFAULT; @@ -1835,6 +1832,21 @@ gnttab_setup_table( gt = d->grant_table; grant_write_lock(gt); + if ( unlikely(op.nr_frames > gt->max_grant_frames) ) + { + gdprintk(XENLOG_INFO, "d%d is limited to %u grant-table frames.\n", + d->domain_id, gt->max_grant_frames); + op.status = GNTST_general_error; + goto unlock; + } + if ( unlikely(limit_max < op.nr_frames) ) + { + gdprintk(XENLOG_WARNING, "nr_frames for d%d is too large (%u,%u)\n", + d->domain_id, op.nr_frames, limit_max); + op.status = GNTST_general_error; + goto unlock; + } + if ( gt->gt_version == 0 ) gt->gt_version = 1; @@ -1844,8 +1856,9 @@ gnttab_setup_table( !gnttab_grow_table(d, op.nr_frames) ) { gdprintk(XENLOG_INFO, - "Expand grant table to %u failed. Current: %u Max: %u\n", - op.nr_frames, nr_grant_frames(gt), max_grant_frames); + "Expand grant table of d%d to %u failed. Current: %u Max: %u\n", + d->domain_id, op.nr_frames, nr_grant_frames(gt), + gt->max_grant_frames); op.status = GNTST_general_error; goto unlock; } @@ -1880,6 +1893,7 @@ gnttab_query_size( { struct gnttab_query_size op; struct domain *d; + struct grant_table *gt; if ( count != 1 ) return -EINVAL; @@ -1900,13 +1914,15 @@ gnttab_query_size( goto out; } - grant_read_lock(d->grant_table); + gt = d->grant_table; + + grant_read_lock(gt); - op.nr_frames = nr_grant_frames(d->grant_table); - op.max_nr_frames = max_grant_frames; + op.nr_frames = nr_grant_frames(gt); + op.max_nr_frames = gt->max_grant_frames; op.status = GNTST_okay; - grant_read_unlock(d->grant_table); + grant_read_unlock(gt); out: if ( d ) @@ -2981,7 +2997,7 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop) static long gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop, - int count) + unsigned int count, unsigned int limit_max) { gnttab_get_status_frames_t op; struct domain *d; @@ -3021,9 +3037,19 @@ gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop, if ( unlikely(op.nr_frames > nr_status_frames(gt)) ) { - gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant status " - "frames, but only %d are available.\n", - op.nr_frames, nr_status_frames(gt)); + gdprintk(XENLOG_INFO, "Requested addresses of d%d for %u grant " + "status frames, but has only %u.\n", + d->domain_id, op.nr_frames, nr_status_frames(gt)); + op.status = GNTST_general_error; + goto unlock; + } + + if ( unlikely(limit_max < grant_to_status_frames(op.nr_frames)) ) + { + gdprintk(XENLOG_WARNING, + "grant_to_status_frames(%u) for d%d is too large (%u,%u)\n", + op.nr_frames, d->domain_id, + grant_to_status_frames(op.nr_frames), limit_max); op.status = GNTST_general_error; goto unlock; } @@ -3336,7 +3362,7 @@ do_grant_table_op( case GNTTABOP_setup_table: rc = gnttab_setup_table( - guest_handle_cast(uop, gnttab_setup_table_t), count); + guest_handle_cast(uop, gnttab_setup_table_t), count, UINT_MAX); ASSERT(rc <= 0); break; @@ -3385,7 +3411,8 @@ do_grant_table_op( case GNTTABOP_get_status_frames: rc = gnttab_get_status_frames( - guest_handle_cast(uop, gnttab_get_status_frames_t), count); + guest_handle_cast(uop, gnttab_get_status_frames_t), count, + UINT_MAX); break; case GNTTABOP_get_version: @@ -3465,7 +3492,7 @@ grant_table_create( if ( d->domain_id == 0 ) { - ret = grant_table_init(d, t); + ret = grant_table_init(d, t, gnttab_dom0_frames(), max_maptrack_frames); } return ret; @@ -3666,11 +3693,15 @@ int grant_table_set_limits(struct domain *d, unsigned int grant_frames, { struct grant_table *gt = d->grant_table; + if ( grant_frames < INITIAL_NR_GRANT_FRAMES || + grant_frames > max_grant_frames || + maptrack_frames > max_maptrack_frames ) + return -EINVAL; if ( !gt ) return -ENOENT; /* Set limits. */ - return grant_table_init(d, gt); + return grant_table_init(d, gt, grant_frames, maptrack_frames); } #ifdef CONFIG_HAS_MEM_SHARING @@ -3742,7 +3773,7 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, } else { - if ( (idx >= nr_grant_frames(gt)) && (idx < max_grant_frames) ) + if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) ) gnttab_grow_table(d, idx + 1); if ( idx < nr_grant_frames(gt) ) @@ -3770,6 +3801,12 @@ static void gnttab_usage_print(struct domain *rd) grant_read_lock(gt); + printk("grant-table for remote d%d (v%u)\n" + " %u frames (%u max), %u maptrack frames (%u max)\n", + rd->domain_id, gt->gt_version, + nr_grant_frames(gt), gt->max_grant_frames, + nr_maptrack_frames(gt), gt->max_maptrack_frames); + for ( ref = 0; ref != nr_grant_entries(gt); ref++ ) { struct active_grant_entry *act; @@ -3797,12 +3834,7 @@ static void gnttab_usage_print(struct domain *rd) status = status_entry(gt, ref); } - if ( first ) - { - printk("grant-table for remote domain:%5d (v%d)\n", - rd->domain_id, gt->gt_version); - first = 0; - } + first = 0; /* [0xXXX] ddddd 0xXXXXXX 0xXXXXXXXX ddddd 0xXXXXXX 0xXX */ printk("[0x%03x] %5d 0x%06lx 0x%08x %5d 0x%06"PRIx64" 0x%02x\n", @@ -3814,8 +3846,7 @@ static void gnttab_usage_print(struct domain *rd) grant_read_unlock(gt); if ( first ) - printk("grant-table for remote domain:%5d ... " - "no active grant table entries\n", rd->domain_id); + printk("no active grant table entries\n"); } static void gnttab_usage_print_all(unsigned char key) @@ -3829,20 +3860,17 @@ static void gnttab_usage_print_all(unsigned char key) static int __init gnttab_usage_init(void) { - BUILD_BUG_ON(DEFAULT_MAX_MAPTRACK_FRAMES < DEFAULT_MAX_NR_GRANT_FRAMES); - - if ( !max_grant_frames ) - max_grant_frames = DEFAULT_MAX_NR_GRANT_FRAMES; - - if ( !max_maptrack_frames ) - max_maptrack_frames = DEFAULT_MAX_MAPTRACK_FRAMES; - register_keyhandler('g', gnttab_usage_print_all, "print grant table usage", 1); return 0; } __initcall(gnttab_usage_init); +unsigned int __init gnttab_dom0_frames(void) +{ + return min(max_grant_frames, gnttab_dom0_max()); +} + /* * Local variables: * mode: C diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h index 30db2d1616..0dfdc5577f 100644 --- a/xen/include/asm-arm/grant_table.h +++ b/xen/include/asm-arm/grant_table.h @@ -2,9 +2,11 @@ #define __ASM_GRANT_TABLE_H__ #include <xen/grant_table.h> +#include <xen/kernel.h> +#include <xen/pfn.h> #include <xen/sched.h> -#define INITIAL_NR_GRANT_FRAMES 4 +#define INITIAL_NR_GRANT_FRAMES 1U struct grant_table_arch { gfn_t *gfn; @@ -26,9 +28,21 @@ static inline int replace_grant_supported(void) return 1; } +/* + * The region used by Xen on the memory will never be mapped in DOM0 + * memory layout. Therefore it can be used for the grant table. + * + * Only use the text section as it's always present and will contain + * enough space for a large grant table + */ +static inline unsigned int gnttab_dom0_max(void) +{ + return PFN_DOWN(_etext - _stext); +} + #define gnttab_init_arch(gt) \ ({ \ - (gt)->arch.gfn = xzalloc_array(gfn_t, max_grant_frames); \ + (gt)->arch.gfn = xzalloc_array(gfn_t, (gt)->max_grant_frames); \ ( (gt)->arch.gfn ? 0 : -ENOMEM ); \ }) @@ -52,7 +66,7 @@ static inline int replace_grant_supported(void) #define gnttab_shared_gmfn(d, t, i) \ ( ((i >= nr_grant_frames(t)) && \ - (i < max_grant_frames)) ? 0 : gfn_x(t->arch.gfn[i])) + (i < (t)->max_grant_frames))? 0 : gfn_x((t)->arch.gfn[i])) #define gnttab_need_iommu_mapping(d) \ (is_domain_direct_mapped(d) && need_iommu(d)) diff --git a/xen/include/asm-x86/grant_table.h b/xen/include/asm-x86/grant_table.h index 1b93c5720d..d9157e4417 100644 --- a/xen/include/asm-x86/grant_table.h +++ b/xen/include/asm-x86/grant_table.h @@ -12,7 +12,7 @@ #include <asm/hvm/grant_table.h> #include <asm/pv/grant_table.h> -#define INITIAL_NR_GRANT_FRAMES 4 +#define INITIAL_NR_GRANT_FRAMES 1U struct grant_table_arch { }; @@ -39,6 +39,11 @@ static inline int replace_grant_host_mapping(uint64_t addr, unsigned long frame, return replace_grant_pv_mapping(addr, frame, new_addr, flags); } +static inline unsigned int gnttab_dom0_max(void) +{ + return UINT_MAX; +} + #define gnttab_init_arch(gt) 0 #define gnttab_destroy_arch(gt) do {} while ( 0 ) #define gnttab_set_frame_gfn(gt, idx, gfn) do {} while ( 0 ) diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h index d2bd2416c4..b3a95fda58 100644 --- a/xen/include/xen/grant_table.h +++ b/xen/include/xen/grant_table.h @@ -31,9 +31,6 @@ struct grant_table; -/* The maximum size of a grant table. */ -extern unsigned int max_grant_frames; - /* Create/destroy per-domain grant table context. */ int grant_table_create( struct domain *d); @@ -59,4 +56,6 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref, int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn); +unsigned int gnttab_dom0_frames(void); + #endif /* __XEN_GRANT_TABLE_H__ */ -- 2.12.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v11 1/3] xen: make grant resource limits per domain 2017-09-28 10:02 ` [PATCH v11 1/3] xen: make grant resource limits per domain Juergen Gross @ 2017-10-03 13:33 ` Julien Grall 0 siblings, 0 replies; 7+ messages in thread From: Julien Grall @ 2017-10-03 13:33 UTC (permalink / raw) To: Juergen Gross, xen-devel Cc: sstabellini, wei.liu2, George.Dunlap, andrew.cooper3, ian.jackson, tim, jbeulich Hi Juergen, On 28/09/17 11:02, Juergen Gross wrote: > Instead of using the same global resource limits of grant tables (max. > number of grant frames, max. number of maptrack frames) for all domains > make these limits per domain. Set those per-domain limits in > grant_table_set_limits(). The global settings are serving as an upper > boundary now which must not be exceeded by a per-domain value. The > default of max_grant_frames is set to the maximum default xl will use. > > While updating the semantics of the boot parameters remove the > documentation of the no longer existing gnttab_max_nr_frames and > correct the default gnttab_max_maptrack_frames uses. > > Signed-off-by: Juergen Gross <jgross@suse.com> > Reviewed-by: Jan Beulich <jbeulich@suse.com> [non-ARM parts] Acked-by: Julien Grall <julien.grall@arm.com> Cheers, > --- > V11: > - add comment in include/asm-arm/grant_table.h (Julien Grall) > > V10: > - lower INITIAL_NR_GRANT_FRAMES value to 1 > - removed no longer needed variables (Jan Beulich) > - stream lined messages (Jan Beulich) > - corrected setting of gnttab_size on ARM > > V9: > - add caps for per-domain limits (Jan Beulich) > - some error messages enhanced (Jan Beulich) > - adjusted some types (Jan Beulich) > - merge parts of (former) patch 14 into this one > - make parameters changeable at runtime > - limit size of dom0's grant table on ARM > - set default max_grant_frames to 64 > > V6: > - several changes due to new patch order > > V3: > - correct error message (Paul Durrant) > --- > docs/misc/xen-command-line.markdown | 25 +++--- > xen/arch/arm/domain_build.c | 6 +- > xen/common/compat/grant_table.c | 31 ++----- > xen/common/grant_table.c | 156 +++++++++++++++++++++--------------- > xen/include/asm-arm/grant_table.h | 20 ++++- > xen/include/asm-x86/grant_table.h | 7 +- > xen/include/xen/grant_table.h | 5 +- > 7 files changed, 138 insertions(+), 112 deletions(-) > > diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown > index 9797c8db2d..9b6cd8e9d0 100644 > --- a/docs/misc/xen-command-line.markdown > +++ b/docs/misc/xen-command-line.markdown > @@ -875,27 +875,28 @@ Specify which console gdbstub should use. See **console**. > ### gnttab\_max\_frames > > `= <integer>` > > -> Default: `32` > +> Default: `64` > + > +> Can be modified at runtime > > Specify the maximum number of frames which any domain may use as part > -of its grant table. > +of its grant table. This value is an upper boundary of the per-domain > +value settable via Xen tools. > + > +Dom0 is using this value for sizing its grant table. > > ### gnttab\_max\_maptrack\_frames > > `= <integer>` > > -> Default: `8 * gnttab_max_frames` > - > -Specify the maximum number of frames to use as part of a domains > -maptrack array. > +> Default: `1024` > > -### gnttab\_max\_nr\_frames > -> `= <integer>` > +> Can be modified at runtime > > -*Deprecated* > -Use **gnttab\_max\_frames** and **gnttab\_max\_maptrack\_frames** instead. > +Specify the maximum number of frames to use as part of a domains > +maptrack array. This value is an upper boundary of the per-domain > +value settable via Xen tools. > > -Specify the maximum number of frames per grant table operation and the > -maximum number of maptrack frames domain. > +Dom0 is using this value for sizing its maptrack table. > > ### guest\_loglvl > > `= <level>[/<rate-limited level>]` where level is `none | error | warning | info | debug | all` > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c > index c34238ec1b..3723dc3f78 100644 > --- a/xen/arch/arm/domain_build.c > +++ b/xen/arch/arm/domain_build.c > @@ -2095,11 +2095,7 @@ static void __init find_gnttab_region(struct domain *d, > * enough space for a large grant table > */ > kinfo->gnttab_start = __pa(_stext); > - kinfo->gnttab_size = (_etext - _stext) & PAGE_MASK; > - > - /* Make sure the grant table will fit in the region */ > - if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames ) > - panic("Cannot find a space for the grant table region\n"); > + kinfo->gnttab_size = gnttab_dom0_frames() << PAGE_SHIFT; > > #ifdef CONFIG_ARM_32 > /* > diff --git a/xen/common/compat/grant_table.c b/xen/common/compat/grant_table.c > index cce3ff0b9a..ff1d678f01 100644 > --- a/xen/common/compat/grant_table.c > +++ b/xen/common/compat/grant_table.c > @@ -157,21 +157,14 @@ int compat_grant_table_op(unsigned int cmd, > unsigned int max_frame_list_size_in_page = > (COMPAT_ARG_XLAT_SIZE - sizeof(*nat.setup)) / > sizeof(*nat.setup->frame_list.p); > - if ( max_frame_list_size_in_page < max_grant_frames ) > - { > - gdprintk(XENLOG_WARNING, > - "max_grant_frames is too large (%u,%u)\n", > - max_grant_frames, max_frame_list_size_in_page); > - rc = -EINVAL; > - } > - else > - { > + > #define XLAT_gnttab_setup_table_HNDL_frame_list(_d_, _s_) \ > - set_xen_guest_handle((_d_)->frame_list, (unsigned long *)(nat.setup + 1)) > - XLAT_gnttab_setup_table(nat.setup, &cmp.setup); > + set_xen_guest_handle((_d_)->frame_list, (unsigned long *)(nat.setup + 1)) > + XLAT_gnttab_setup_table(nat.setup, &cmp.setup); > #undef XLAT_gnttab_setup_table_HNDL_frame_list > - rc = gnttab_setup_table(guest_handle_cast(nat.uop, gnttab_setup_table_t), 1); > - } > + rc = gnttab_setup_table(guest_handle_cast(nat.uop, > + gnttab_setup_table_t), > + 1, max_frame_list_size_in_page); > } > ASSERT(rc <= 0); > if ( rc == 0 ) > @@ -294,16 +287,6 @@ int compat_grant_table_op(unsigned int cmd, > rc = -EFAULT; > break; > } > - if ( max_frame_list_size_in_pages < > - grant_to_status_frames(max_grant_frames) ) > - { > - gdprintk(XENLOG_WARNING, > - "grant_to_status_frames(max_grant_frames) is too large (%u,%u)\n", > - grant_to_status_frames(max_grant_frames), > - max_frame_list_size_in_pages); > - rc = -EINVAL; > - break; > - } > > #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \ > set_xen_guest_handle((_d_)->frame_list, (uint64_t *)(nat.get_status + 1)) > @@ -312,7 +295,7 @@ int compat_grant_table_op(unsigned int cmd, > > rc = gnttab_get_status_frames( > guest_handle_cast(nat.uop, gnttab_get_status_frames_t), > - count); > + count, max_frame_list_size_in_pages); > if ( rc >= 0 ) > { > #define XLAT_gnttab_get_status_frames_HNDL_frame_list(_d_, _s_) \ > diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c > index 71706f5cba..7c328b0a62 100644 > --- a/xen/common/grant_table.c > +++ b/xen/common/grant_table.c > @@ -54,6 +54,9 @@ struct grant_table { > * what version to use yet. > */ > unsigned int gt_version; > + /* Resource limits of the domain. */ > + unsigned int max_grant_frames; > + unsigned int max_maptrack_frames; > /* Table size. Number of frames shared with guest */ > unsigned int nr_grant_frames; > /* Number of grant status frames shared with guest (for version 2) */ > @@ -78,23 +81,18 @@ struct grant_table { > > #ifndef DEFAULT_MAX_NR_GRANT_FRAMES /* to allow arch to override */ > /* Default maximum size of a grant table. [POLICY] */ > -#define DEFAULT_MAX_NR_GRANT_FRAMES 32 > +#define DEFAULT_MAX_NR_GRANT_FRAMES 64 > #endif > > -unsigned int __read_mostly max_grant_frames; > -integer_param("gnttab_max_frames", max_grant_frames); > +static unsigned int __read_mostly max_grant_frames = > + DEFAULT_MAX_NR_GRANT_FRAMES; > +integer_runtime_param("gnttab_max_frames", max_grant_frames); > > -/* The maximum number of grant mappings is defined as a multiplier of the > - * maximum number of grant table entries. This defines the multiplier used. > - * Pretty arbitrary. [POLICY] > - * As gnttab_max_nr_frames has been deprecated, this multiplier is deprecated too. > - * New options allow to set max_maptrack_frames and > - * map_grant_table_frames independently. > - */ > #define DEFAULT_MAX_MAPTRACK_FRAMES 1024 > > -static unsigned int __read_mostly max_maptrack_frames; > -integer_param("gnttab_max_maptrack_frames", max_maptrack_frames); > +static unsigned int __read_mostly max_maptrack_frames = > + DEFAULT_MAX_MAPTRACK_FRAMES; > +integer_runtime_param("gnttab_max_maptrack_frames", max_maptrack_frames); > > /* > * Note that the three values below are effectively part of the ABI, even if > @@ -290,8 +288,8 @@ num_act_frames_from_sha_frames(const unsigned int num) > return DIV_ROUND_UP(num * sha_per_page, ACGNT_PER_PAGE); > } > > -#define max_nr_active_grant_frames \ > - num_act_frames_from_sha_frames(max_grant_frames) > +#define max_nr_active_grant_frames(gt) \ > + num_act_frames_from_sha_frames((gt)->max_grant_frames) > > static inline unsigned int > nr_active_grant_frames(struct grant_table *gt) > @@ -530,7 +528,7 @@ get_maptrack_handle( > * out of memory, try stealing an entry from another VCPU (in case the > * guest isn't mapping across its VCPUs evenly). > */ > - if ( nr_maptrack_frames(lgt) < max_maptrack_frames ) > + if ( nr_maptrack_frames(lgt) < lgt->max_maptrack_frames ) > new_mt = alloc_xenheap_page(); > > if ( !new_mt ) > @@ -1673,7 +1671,7 @@ gnttab_grow_table(struct domain *d, unsigned int req_nr_frames) > > if ( req_nr_frames < INITIAL_NR_GRANT_FRAMES ) > req_nr_frames = INITIAL_NR_GRANT_FRAMES; > - ASSERT(req_nr_frames <= max_grant_frames); > + ASSERT(req_nr_frames <= gt->max_grant_frames); > > gdprintk(XENLOG_INFO, > "Expanding dom (%d) grant table from (%d) to (%d) frames.\n", > @@ -1730,7 +1728,8 @@ active_alloc_failed: > } > > static int > -grant_table_init(struct domain *d, struct grant_table *gt) > +grant_table_init(struct domain *d, struct grant_table *gt, > + unsigned int grant_frames, unsigned int maptrack_frames) > { > int ret; > > @@ -1742,25 +1741,31 @@ grant_table_init(struct domain *d, struct grant_table *gt) > goto unlock; > } > > + gt->max_grant_frames = grant_frames; > + gt->max_maptrack_frames = maptrack_frames; > + > /* Active grant table. */ > gt->active = xzalloc_array(struct active_grant_entry *, > - max_nr_active_grant_frames); > + max_nr_active_grant_frames(gt)); > if ( gt->active == NULL ) > goto no_mem; > > /* Tracking of mapped foreign frames table */ > - gt->maptrack = vzalloc(max_maptrack_frames * sizeof(*gt->maptrack)); > - if ( gt->maptrack == NULL ) > - goto no_mem; > + if ( gt->max_maptrack_frames ) > + { > + gt->maptrack = vzalloc(gt->max_maptrack_frames * sizeof(*gt->maptrack)); > + if ( gt->maptrack == NULL ) > + goto no_mem; > + } > > /* Shared grant table. */ > - gt->shared_raw = xzalloc_array(void *, max_grant_frames); > + gt->shared_raw = xzalloc_array(void *, gt->max_grant_frames); > if ( gt->shared_raw == NULL ) > goto no_mem; > > /* Status pages for grant table - for version 2 */ > gt->status = xzalloc_array(grant_status_t *, > - grant_to_status_frames(max_grant_frames)); > + grant_to_status_frames(gt->max_grant_frames)); > if ( gt->status == NULL ) > goto no_mem; > > @@ -1793,7 +1798,8 @@ grant_table_init(struct domain *d, struct grant_table *gt) > > static long > gnttab_setup_table( > - XEN_GUEST_HANDLE_PARAM(gnttab_setup_table_t) uop, unsigned int count) > + XEN_GUEST_HANDLE_PARAM(gnttab_setup_table_t) uop, unsigned int count, > + unsigned int limit_max) > { > struct vcpu *curr = current; > struct gnttab_setup_table op; > @@ -1807,15 +1813,6 @@ gnttab_setup_table( > if ( unlikely(copy_from_guest(&op, uop, 1)) ) > return -EFAULT; > > - if ( unlikely(op.nr_frames > max_grant_frames) ) > - { > - gdprintk(XENLOG_INFO, "Xen only supports up to %d grant-table frames" > - " per domain.\n", > - max_grant_frames); > - op.status = GNTST_general_error; > - goto out; > - } > - > if ( !guest_handle_okay(op.frame_list, op.nr_frames) ) > return -EFAULT; > > @@ -1835,6 +1832,21 @@ gnttab_setup_table( > gt = d->grant_table; > grant_write_lock(gt); > > + if ( unlikely(op.nr_frames > gt->max_grant_frames) ) > + { > + gdprintk(XENLOG_INFO, "d%d is limited to %u grant-table frames.\n", > + d->domain_id, gt->max_grant_frames); > + op.status = GNTST_general_error; > + goto unlock; > + } > + if ( unlikely(limit_max < op.nr_frames) ) > + { > + gdprintk(XENLOG_WARNING, "nr_frames for d%d is too large (%u,%u)\n", > + d->domain_id, op.nr_frames, limit_max); > + op.status = GNTST_general_error; > + goto unlock; > + } > + > if ( gt->gt_version == 0 ) > gt->gt_version = 1; > > @@ -1844,8 +1856,9 @@ gnttab_setup_table( > !gnttab_grow_table(d, op.nr_frames) ) > { > gdprintk(XENLOG_INFO, > - "Expand grant table to %u failed. Current: %u Max: %u\n", > - op.nr_frames, nr_grant_frames(gt), max_grant_frames); > + "Expand grant table of d%d to %u failed. Current: %u Max: %u\n", > + d->domain_id, op.nr_frames, nr_grant_frames(gt), > + gt->max_grant_frames); > op.status = GNTST_general_error; > goto unlock; > } > @@ -1880,6 +1893,7 @@ gnttab_query_size( > { > struct gnttab_query_size op; > struct domain *d; > + struct grant_table *gt; > > if ( count != 1 ) > return -EINVAL; > @@ -1900,13 +1914,15 @@ gnttab_query_size( > goto out; > } > > - grant_read_lock(d->grant_table); > + gt = d->grant_table; > + > + grant_read_lock(gt); > > - op.nr_frames = nr_grant_frames(d->grant_table); > - op.max_nr_frames = max_grant_frames; > + op.nr_frames = nr_grant_frames(gt); > + op.max_nr_frames = gt->max_grant_frames; > op.status = GNTST_okay; > > - grant_read_unlock(d->grant_table); > + grant_read_unlock(gt); > > out: > if ( d ) > @@ -2981,7 +2997,7 @@ gnttab_set_version(XEN_GUEST_HANDLE_PARAM(gnttab_set_version_t) uop) > > static long > gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop, > - int count) > + unsigned int count, unsigned int limit_max) > { > gnttab_get_status_frames_t op; > struct domain *d; > @@ -3021,9 +3037,19 @@ gnttab_get_status_frames(XEN_GUEST_HANDLE_PARAM(gnttab_get_status_frames_t) uop, > > if ( unlikely(op.nr_frames > nr_status_frames(gt)) ) > { > - gdprintk(XENLOG_INFO, "Guest requested addresses for %d grant status " > - "frames, but only %d are available.\n", > - op.nr_frames, nr_status_frames(gt)); > + gdprintk(XENLOG_INFO, "Requested addresses of d%d for %u grant " > + "status frames, but has only %u.\n", > + d->domain_id, op.nr_frames, nr_status_frames(gt)); > + op.status = GNTST_general_error; > + goto unlock; > + } > + > + if ( unlikely(limit_max < grant_to_status_frames(op.nr_frames)) ) > + { > + gdprintk(XENLOG_WARNING, > + "grant_to_status_frames(%u) for d%d is too large (%u,%u)\n", > + op.nr_frames, d->domain_id, > + grant_to_status_frames(op.nr_frames), limit_max); > op.status = GNTST_general_error; > goto unlock; > } > @@ -3336,7 +3362,7 @@ do_grant_table_op( > > case GNTTABOP_setup_table: > rc = gnttab_setup_table( > - guest_handle_cast(uop, gnttab_setup_table_t), count); > + guest_handle_cast(uop, gnttab_setup_table_t), count, UINT_MAX); > ASSERT(rc <= 0); > break; > > @@ -3385,7 +3411,8 @@ do_grant_table_op( > > case GNTTABOP_get_status_frames: > rc = gnttab_get_status_frames( > - guest_handle_cast(uop, gnttab_get_status_frames_t), count); > + guest_handle_cast(uop, gnttab_get_status_frames_t), count, > + UINT_MAX); > break; > > case GNTTABOP_get_version: > @@ -3465,7 +3492,7 @@ grant_table_create( > > if ( d->domain_id == 0 ) > { > - ret = grant_table_init(d, t); > + ret = grant_table_init(d, t, gnttab_dom0_frames(), max_maptrack_frames); > } > > return ret; > @@ -3666,11 +3693,15 @@ int grant_table_set_limits(struct domain *d, unsigned int grant_frames, > { > struct grant_table *gt = d->grant_table; > > + if ( grant_frames < INITIAL_NR_GRANT_FRAMES || > + grant_frames > max_grant_frames || > + maptrack_frames > max_maptrack_frames ) > + return -EINVAL; > if ( !gt ) > return -ENOENT; > > /* Set limits. */ > - return grant_table_init(d, gt); > + return grant_table_init(d, gt, grant_frames, maptrack_frames); > } > > #ifdef CONFIG_HAS_MEM_SHARING > @@ -3742,7 +3773,7 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, > } > else > { > - if ( (idx >= nr_grant_frames(gt)) && (idx < max_grant_frames) ) > + if ( (idx >= nr_grant_frames(gt)) && (idx < gt->max_grant_frames) ) > gnttab_grow_table(d, idx + 1); > > if ( idx < nr_grant_frames(gt) ) > @@ -3770,6 +3801,12 @@ static void gnttab_usage_print(struct domain *rd) > > grant_read_lock(gt); > > + printk("grant-table for remote d%d (v%u)\n" > + " %u frames (%u max), %u maptrack frames (%u max)\n", > + rd->domain_id, gt->gt_version, > + nr_grant_frames(gt), gt->max_grant_frames, > + nr_maptrack_frames(gt), gt->max_maptrack_frames); > + > for ( ref = 0; ref != nr_grant_entries(gt); ref++ ) > { > struct active_grant_entry *act; > @@ -3797,12 +3834,7 @@ static void gnttab_usage_print(struct domain *rd) > status = status_entry(gt, ref); > } > > - if ( first ) > - { > - printk("grant-table for remote domain:%5d (v%d)\n", > - rd->domain_id, gt->gt_version); > - first = 0; > - } > + first = 0; > > /* [0xXXX] ddddd 0xXXXXXX 0xXXXXXXXX ddddd 0xXXXXXX 0xXX */ > printk("[0x%03x] %5d 0x%06lx 0x%08x %5d 0x%06"PRIx64" 0x%02x\n", > @@ -3814,8 +3846,7 @@ static void gnttab_usage_print(struct domain *rd) > grant_read_unlock(gt); > > if ( first ) > - printk("grant-table for remote domain:%5d ... " > - "no active grant table entries\n", rd->domain_id); > + printk("no active grant table entries\n"); > } > > static void gnttab_usage_print_all(unsigned char key) > @@ -3829,20 +3860,17 @@ static void gnttab_usage_print_all(unsigned char key) > > static int __init gnttab_usage_init(void) > { > - BUILD_BUG_ON(DEFAULT_MAX_MAPTRACK_FRAMES < DEFAULT_MAX_NR_GRANT_FRAMES); > - > - if ( !max_grant_frames ) > - max_grant_frames = DEFAULT_MAX_NR_GRANT_FRAMES; > - > - if ( !max_maptrack_frames ) > - max_maptrack_frames = DEFAULT_MAX_MAPTRACK_FRAMES; > - > register_keyhandler('g', gnttab_usage_print_all, > "print grant table usage", 1); > return 0; > } > __initcall(gnttab_usage_init); > > +unsigned int __init gnttab_dom0_frames(void) > +{ > + return min(max_grant_frames, gnttab_dom0_max()); > +} > + > /* > * Local variables: > * mode: C > diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h > index 30db2d1616..0dfdc5577f 100644 > --- a/xen/include/asm-arm/grant_table.h > +++ b/xen/include/asm-arm/grant_table.h > @@ -2,9 +2,11 @@ > #define __ASM_GRANT_TABLE_H__ > > #include <xen/grant_table.h> > +#include <xen/kernel.h> > +#include <xen/pfn.h> > #include <xen/sched.h> > > -#define INITIAL_NR_GRANT_FRAMES 4 > +#define INITIAL_NR_GRANT_FRAMES 1U > > struct grant_table_arch { > gfn_t *gfn; > @@ -26,9 +28,21 @@ static inline int replace_grant_supported(void) > return 1; > } > > +/* > + * The region used by Xen on the memory will never be mapped in DOM0 > + * memory layout. Therefore it can be used for the grant table. > + * > + * Only use the text section as it's always present and will contain > + * enough space for a large grant table > + */ > +static inline unsigned int gnttab_dom0_max(void) > +{ > + return PFN_DOWN(_etext - _stext); > +} > + > #define gnttab_init_arch(gt) \ > ({ \ > - (gt)->arch.gfn = xzalloc_array(gfn_t, max_grant_frames); \ > + (gt)->arch.gfn = xzalloc_array(gfn_t, (gt)->max_grant_frames); \ > ( (gt)->arch.gfn ? 0 : -ENOMEM ); \ > }) > > @@ -52,7 +66,7 @@ static inline int replace_grant_supported(void) > > #define gnttab_shared_gmfn(d, t, i) \ > ( ((i >= nr_grant_frames(t)) && \ > - (i < max_grant_frames)) ? 0 : gfn_x(t->arch.gfn[i])) > + (i < (t)->max_grant_frames))? 0 : gfn_x((t)->arch.gfn[i])) > > #define gnttab_need_iommu_mapping(d) \ > (is_domain_direct_mapped(d) && need_iommu(d)) > diff --git a/xen/include/asm-x86/grant_table.h b/xen/include/asm-x86/grant_table.h > index 1b93c5720d..d9157e4417 100644 > --- a/xen/include/asm-x86/grant_table.h > +++ b/xen/include/asm-x86/grant_table.h > @@ -12,7 +12,7 @@ > #include <asm/hvm/grant_table.h> > #include <asm/pv/grant_table.h> > > -#define INITIAL_NR_GRANT_FRAMES 4 > +#define INITIAL_NR_GRANT_FRAMES 1U > > struct grant_table_arch { > }; > @@ -39,6 +39,11 @@ static inline int replace_grant_host_mapping(uint64_t addr, unsigned long frame, > return replace_grant_pv_mapping(addr, frame, new_addr, flags); > } > > +static inline unsigned int gnttab_dom0_max(void) > +{ > + return UINT_MAX; > +} > + > #define gnttab_init_arch(gt) 0 > #define gnttab_destroy_arch(gt) do {} while ( 0 ) > #define gnttab_set_frame_gfn(gt, idx, gfn) do {} while ( 0 ) > diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h > index d2bd2416c4..b3a95fda58 100644 > --- a/xen/include/xen/grant_table.h > +++ b/xen/include/xen/grant_table.h > @@ -31,9 +31,6 @@ > > struct grant_table; > > -/* The maximum size of a grant table. */ > -extern unsigned int max_grant_frames; > - > /* Create/destroy per-domain grant table context. */ > int grant_table_create( > struct domain *d); > @@ -59,4 +56,6 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref, > int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, > mfn_t *mfn); > > +unsigned int gnttab_dom0_frames(void); > + > #endif /* __XEN_GRANT_TABLE_H__ */ > -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v11 2/3] xen: add new Xen cpuid node for max address width info 2017-09-28 10:02 [PATCH v11 0/3] xen: better grant v2 support Juergen Gross 2017-09-28 10:02 ` [PATCH v11 1/3] xen: make grant resource limits per domain Juergen Gross @ 2017-09-28 10:02 ` Juergen Gross 2017-09-28 10:02 ` [PATCH v11 3/3] xen: add some comments in include/public/arch-x86/cpuid.h Juergen Gross 2017-09-28 19:02 ` [PATCH v11 0/3] xen: better grant v2 support Andrew Cooper 3 siblings, 0 replies; 7+ messages in thread From: Juergen Gross @ 2017-09-28 10:02 UTC (permalink / raw) To: xen-devel Cc: Juergen Gross, sstabellini, wei.liu2, George.Dunlap, andrew.cooper3, ian.jackson, tim, julien.grall, jbeulich On very large hosts a pv-guest needs to know whether it will have to handle frame numbers larger than 32 bits in order to select the appropriate grant interface version. Add a new Xen specific CPUID node to contain the maximum machine address width similar to the x86 CPUID node 0x80000008 containing the maximum physical address width. The maximum frame width needs to take memory hotplug into account. Signed-off-by: Juergen Gross <jgross@suse.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> --- V11: - return max. subleaf number in EAX (Andrew Cooper) - use flsl() instead of generic_flsl() (Jan Beulich) V10: - correct comment in cpuid.h (Jan Beulich) V9: - make leaf pv-only (Jan Beulich) - use hex value for mask (Jan Beulich) - guest address width -> machine address width (Jan Beulich) --- xen/arch/x86/traps.c | 8 ++++++++ xen/include/public/arch-x86/cpuid.h | 13 ++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index d8feef2942..05b21c57cb 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -930,6 +930,14 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf, res->b = v->vcpu_id; break; + case 5: /* PV-specific parameters */ + if ( is_hvm_domain(d) || subleaf > XEN_CPUID_MAX_NUM_SUBLEAVES_5 ) + break; + + res->a = XEN_CPUID_MAX_NUM_SUBLEAVES_5; + res->b = flsl(get_upper_mfn_bound()) + PAGE_SHIFT; + break; + default: ASSERT_UNREACHABLE(); } diff --git a/xen/include/public/arch-x86/cpuid.h b/xen/include/public/arch-x86/cpuid.h index d709340f18..3d88c15598 100644 --- a/xen/include/public/arch-x86/cpuid.h +++ b/xen/include/public/arch-x86/cpuid.h @@ -85,6 +85,17 @@ #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) #define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */ -#define XEN_CPUID_MAX_NUM_LEAVES 4 +/* + * Leaf 6 (0x40000x05) + * PV-specific parameters + * Sub-leaf 0: EAX: max. sub-leaf number + * EBX: bits 0-7: max machine address width + */ +#define XEN_CPUID_MAX_NUM_SUBLEAVES_5 0 + +/* Max. address width in bits taking memory hotplug into account. */ +#define XEN_CPUID_MACHINE_ADDRESS_WIDTH_MASK (0xffu << 0) + +#define XEN_CPUID_MAX_NUM_LEAVES 5 #endif /* __XEN_PUBLIC_ARCH_X86_CPUID_H__ */ -- 2.12.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v11 3/3] xen: add some comments in include/public/arch-x86/cpuid.h 2017-09-28 10:02 [PATCH v11 0/3] xen: better grant v2 support Juergen Gross 2017-09-28 10:02 ` [PATCH v11 1/3] xen: make grant resource limits per domain Juergen Gross 2017-09-28 10:02 ` [PATCH v11 2/3] xen: add new Xen cpuid node for max address width info Juergen Gross @ 2017-09-28 10:02 ` Juergen Gross 2017-09-28 19:02 ` [PATCH v11 0/3] xen: better grant v2 support Andrew Cooper 3 siblings, 0 replies; 7+ messages in thread From: Juergen Gross @ 2017-09-28 10:02 UTC (permalink / raw) To: xen-devel Cc: Juergen Gross, sstabellini, wei.liu2, George.Dunlap, andrew.cooper3, ian.jackson, tim, julien.grall, jbeulich Leaf 4 of the Xen-specific CPUID leaves isn't mentioned at all in include/public/arch-x86/cpuid.h, the comments for leaf 5 don't tell anything about the sub-leaf semantics. Add comments to clarify the interface. Signed-off-by: Juergen Gross <jgross@suse.com> Acked-by: Jan Beulich <jbeulich@suse.com> --- xen/include/public/arch-x86/cpuid.h | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/xen/include/public/arch-x86/cpuid.h b/xen/include/public/arch-x86/cpuid.h index 3d88c15598..f7569d58b0 100644 --- a/xen/include/public/arch-x86/cpuid.h +++ b/xen/include/public/arch-x86/cpuid.h @@ -74,10 +74,26 @@ #define XEN_CPUID_FEAT1_MMU_PT_UPDATE_PRESERVE_AD (1u<<0) /* + * Leaf 4 (0x40000x03) + * Sub-leaf 0: EAX: bit 0: emulated tsc + * bit 1: host tsc is known to be reliable + * bit 2: RDTSCP instruction available + * EBX: tsc_mode: 0=default (emulate if necessary), 1=emulate, + * 2=no emulation, 3=no emulation + TSC_AUX support + * ECX: guest tsc frequency in kHz + * EDX: guest tsc incarnation (migration count) + * Sub-leaf 1: EAX: tsc offset low part + * EBX: tsc offset high part + * ECX: multiplicator for tsc->ns conversion + * EDX: shift amount for tsc->ns conversion + * Sub-leaf 2: EAX: host tsc frequency in kHz + */ + +/* * Leaf 5 (0x40000x04) * HVM-specific features - * EAX: Features - * EBX: vcpu id (iff EAX has XEN_HVM_CPUID_VCPU_ID_PRESENT flag) + * Sub-leaf 0: EAX: Features + * Sub-leaf 0: EBX: vcpu id (iff EAX has XEN_HVM_CPUID_VCPU_ID_PRESENT flag) */ #define XEN_HVM_CPUID_APIC_ACCESS_VIRT (1u << 0) /* Virtualized APIC registers */ #define XEN_HVM_CPUID_X2APIC_VIRT (1u << 1) /* Virtualized x2APIC accesses */ -- 2.12.3 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v11 0/3] xen: better grant v2 support 2017-09-28 10:02 [PATCH v11 0/3] xen: better grant v2 support Juergen Gross ` (2 preceding siblings ...) 2017-09-28 10:02 ` [PATCH v11 3/3] xen: add some comments in include/public/arch-x86/cpuid.h Juergen Gross @ 2017-09-28 19:02 ` Andrew Cooper 2017-09-29 4:33 ` Juergen Gross 3 siblings, 1 reply; 7+ messages in thread From: Andrew Cooper @ 2017-09-28 19:02 UTC (permalink / raw) To: Juergen Gross, xen-devel Cc: sstabellini, wei.liu2, George.Dunlap, tim, ian.jackson, julien.grall, jbeulich On 28/09/17 11:02, Juergen Gross wrote: > Currently Linux has no support for grant v2 as this would reduce the > maximum number of active grants by a factor of 2 compared to v1, > because the number of possible grants are limited by the allowed number > of grant frames and grant entries of v2 need twice as much bytes as > those of v1. > > Unfortunately grant v2 is the only way to support either guests with > more than 16TB memory size or PV guests with memory above the 16TB > border, as grant v1 limits the frame number to be 32 bits wide. > > In order to remove the disadvantage of grant v2 this patch series > adds support for setting per-domain values regarding grant limits. > Additionally the default limit of grant frames is doubled in case > of hosts with potential memory above the 16TB border. > > Changes in V11: > - dropped patches 1-8, as already committed > - patch 1: add comment in include/asm-arm/grant_table.h (Julien Grall) > - patch 2: return max. subleaf number in EAX (Andrew Cooper) > - patch 2: use flsl() instead of generic_flsl() (Jan Beulich) Jan: You appear to have committed v10 of this series, which amongst other things, is missing the max subleaf assignment. Juergen: Would you mind rebasing v11 over staging, just to see what else got missed? ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v11 0/3] xen: better grant v2 support 2017-09-28 19:02 ` [PATCH v11 0/3] xen: better grant v2 support Andrew Cooper @ 2017-09-29 4:33 ` Juergen Gross 0 siblings, 0 replies; 7+ messages in thread From: Juergen Gross @ 2017-09-29 4:33 UTC (permalink / raw) To: Andrew Cooper, xen-devel Cc: sstabellini, wei.liu2, George.Dunlap, tim, ian.jackson, julien.grall, jbeulich On 28/09/17 21:02, Andrew Cooper wrote: > On 28/09/17 11:02, Juergen Gross wrote: >> Currently Linux has no support for grant v2 as this would reduce the >> maximum number of active grants by a factor of 2 compared to v1, >> because the number of possible grants are limited by the allowed number >> of grant frames and grant entries of v2 need twice as much bytes as >> those of v1. >> >> Unfortunately grant v2 is the only way to support either guests with >> more than 16TB memory size or PV guests with memory above the 16TB >> border, as grant v1 limits the frame number to be 32 bits wide. >> >> In order to remove the disadvantage of grant v2 this patch series >> adds support for setting per-domain values regarding grant limits. >> Additionally the default limit of grant frames is doubled in case >> of hosts with potential memory above the 16TB border. >> >> Changes in V11: >> - dropped patches 1-8, as already committed >> - patch 1: add comment in include/asm-arm/grant_table.h (Julien Grall) >> - patch 2: return max. subleaf number in EAX (Andrew Cooper) >> - patch 2: use flsl() instead of generic_flsl() (Jan Beulich) > > Jan: You appear to have committed v10 of this series, which amongst > other things, is missing the max subleaf assignment. While not stating it explicitly, the subleaf stuff is okay: res->a isn't touched, so it will be 0. The address width is put into res->b. > Juergen: Would you mind rebasing v11 over staging, just to see what else > got missed? As this was the only modification in v11 regarding the applied patches we are fine. Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-10-03 13:33 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-09-28 10:02 [PATCH v11 0/3] xen: better grant v2 support Juergen Gross 2017-09-28 10:02 ` [PATCH v11 1/3] xen: make grant resource limits per domain Juergen Gross 2017-10-03 13:33 ` Julien Grall 2017-09-28 10:02 ` [PATCH v11 2/3] xen: add new Xen cpuid node for max address width info Juergen Gross 2017-09-28 10:02 ` [PATCH v11 3/3] xen: add some comments in include/public/arch-x86/cpuid.h Juergen Gross 2017-09-28 19:02 ` [PATCH v11 0/3] xen: better grant v2 support Andrew Cooper 2017-09-29 4:33 ` Juergen Gross
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).