* [PATCH v2 1/2] xen: credit2: enable per cpu runqueue creation
2017-03-30 19:19 [PATCH v2 0/2] xen: credit2: flexible configuration for runqueues Praveen Kumar
@ 2017-03-30 19:19 ` Praveen Kumar
2017-03-30 19:19 ` [PATCH v2 2/2] xen: credit2: provide custom option to create Praveen Kumar
1 sibling, 0 replies; 3+ messages in thread
From: Praveen Kumar @ 2017-03-30 19:19 UTC (permalink / raw)
To: george.dunlap, dario.faggioli; +Cc: Praveen Kumar, xen-devel
The patch introduces a new command line option 'cpu' that when used will
create runqueue per logical pCPU.
Signed-off-by: Praveen Kumar <kpraveen.lkml@gmail.com>
---
docs/misc/xen-command-line.markdown | 3 ++-
xen/common/sched_credit2.c | 15 +++++++++++----
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 9eb85d68b5..c245cfa471 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -525,7 +525,7 @@ also slow in responding to load changes.
The default value of `1 sec` is rather long.
### credit2\_runqueue
-> `= core | socket | node | all`
+> `= cpu | core | socket | node | all`
> Default: `socket`
@@ -536,6 +536,7 @@ balancing (for instance, it will deal better with hyperthreading),
but also more overhead.
Available alternatives, with their meaning, are:
+* `cpu`: one runqueue per each logical pCPUs of the host;
* `core`: one runqueue per each physical core of the host;
* `socket`: one runqueue per each physical socket (which often,
but not always, matches a NUMA node) of the host;
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index bb1c657e76..ee7b443f9e 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -301,6 +301,9 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
* want that to happen basing on topology. At the moment, it is possible
* to choose to arrange runqueues to be:
*
+ * - per-cpu: meaning that there will be one runqueue per logical cpu. This
+ * will happen when if the opt_runqueue parameter is set to 'cpu'.
+ *
* - per-core: meaning that there will be one runqueue per each physical
* core of the host. This will happen if the opt_runqueue
* parameter is set to 'core';
@@ -322,11 +325,13 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
* either the same physical core, the same physical socket, the same NUMA
* node, or just all of them, will be put together to form runqueues.
*/
-#define OPT_RUNQUEUE_CORE 0
-#define OPT_RUNQUEUE_SOCKET 1
-#define OPT_RUNQUEUE_NODE 2
-#define OPT_RUNQUEUE_ALL 3
+#define OPT_RUNQUEUE_CPU 0
+#define OPT_RUNQUEUE_CORE 1
+#define OPT_RUNQUEUE_SOCKET 2
+#define OPT_RUNQUEUE_NODE 3
+#define OPT_RUNQUEUE_ALL 4
static const char *const opt_runqueue_str[] = {
+ [OPT_RUNQUEUE_CPU] = "cpu",
[OPT_RUNQUEUE_CORE] = "core",
[OPT_RUNQUEUE_SOCKET] = "socket",
[OPT_RUNQUEUE_NODE] = "node",
@@ -682,6 +687,8 @@ cpu_to_runqueue(struct csched2_private *prv, unsigned int cpu)
BUG_ON(cpu_to_socket(cpu) == XEN_INVALID_SOCKET_ID ||
cpu_to_socket(peer_cpu) == XEN_INVALID_SOCKET_ID);
+ if (opt_runqueue == OPT_RUNQUEUE_CPU)
+ continue;
if ( opt_runqueue == OPT_RUNQUEUE_ALL ||
(opt_runqueue == OPT_RUNQUEUE_CORE && same_core(peer_cpu, cpu)) ||
(opt_runqueue == OPT_RUNQUEUE_SOCKET && same_socket(peer_cpu, cpu)) ||
--
2.12.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 3+ messages in thread* [PATCH v2 2/2] xen: credit2: provide custom option to create
2017-03-30 19:19 [PATCH v2 0/2] xen: credit2: flexible configuration for runqueues Praveen Kumar
2017-03-30 19:19 ` [PATCH v2 1/2] xen: credit2: enable per cpu runqueue creation Praveen Kumar
@ 2017-03-30 19:19 ` Praveen Kumar
1 sibling, 0 replies; 3+ messages in thread
From: Praveen Kumar @ 2017-03-30 19:19 UTC (permalink / raw)
To: george.dunlap, dario.faggioli; +Cc: Praveen Kumar, xen-devel
The patch introduces a new command line option 'custom' that when used will
create runqueue based upon the pCPU subset provide during bootup.
Signed-off-by: Praveen Kumar <kpraveen.lkml@gmail.com>
---
docs/misc/xen-command-line.markdown | 8 +-
xen/common/sched_credit2.c | 170 +++++++++++++++++++++++++++++++++++-
2 files changed, 176 insertions(+), 2 deletions(-)
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index c245cfa471..e65056b3d0 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -525,7 +525,7 @@ also slow in responding to load changes.
The default value of `1 sec` is rather long.
### credit2\_runqueue
-> `= cpu | core | socket | node | all`
+> `= cpu | core | socket | node | all | custom`
> Default: `socket`
@@ -543,6 +543,12 @@ Available alternatives, with their meaning, are:
* `node`: one runqueue per each NUMA node of the host;
* `all`: just one runqueue shared by all the logical pCPUs of
the host
+* `custom`: one runqueue per subset. Example:
+ credit2_runqueue=[[0,1,4,5][2,3,6,7][8,9,12,13][10,11,14,15]]
+ - pCPUs 0, 1, 4 and 5 belong to runqueue 0
+ - pCPUs 2, 3, 6 and 7 belong to runqueue 1
+ - pCPUs 8, 9, 12 and 13 belong to runqueue 2
+ - pCPUs 10, 11, 14 and 15 belong to runqueue 3
### dbgp
> `= ehci[ <integer> | @pci<bus>:<slot>.<func> ]`
diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index ee7b443f9e..9234b023cb 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -321,6 +321,15 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
* (logical) processors of the host belong. This will happen if
* the opt_runqueue parameter is set to 'all'.
*
+ * - custom: meaning that there will be one runqueue per subset being passed as
+ * parameter to credit2_runqueue as shown in below example.
+ * Example:
+ * credit2_runqueue=[[cpu0,cpu1][cpu3][cpu4,cpu5]]
+ * The example mentioned states :
+ * cpu0 and cpu1 belongs to runqueue 0
+ * cpu3 belongs to runqueue 1
+ * cpu4 and cpu5 belongs to runqueue 2
+ *
* Depending on the value of opt_runqueue, therefore, cpus that are part of
* either the same physical core, the same physical socket, the same NUMA
* node, or just all of them, will be put together to form runqueues.
@@ -330,18 +339,60 @@ integer_param("credit2_balance_over", opt_overload_balance_tolerance);
#define OPT_RUNQUEUE_SOCKET 2
#define OPT_RUNQUEUE_NODE 3
#define OPT_RUNQUEUE_ALL 4
+#define OPT_RUNQUEUE_CUSTOM 5
static const char *const opt_runqueue_str[] = {
[OPT_RUNQUEUE_CPU] = "cpu",
[OPT_RUNQUEUE_CORE] = "core",
[OPT_RUNQUEUE_SOCKET] = "socket",
[OPT_RUNQUEUE_NODE] = "node",
- [OPT_RUNQUEUE_ALL] = "all"
+ [OPT_RUNQUEUE_ALL] = "all",
+ [OPT_RUNQUEUE_CUSTOM] = "custom"
};
static int __read_mostly opt_runqueue = OPT_RUNQUEUE_SOCKET;
+static int __read_mostly custom_cpu_runqueue[NR_CPUS];
+
+#define GETTOKEN( token, len, start, end ) \
+{ \
+ char _tmp[len+1]; \
+ int _i; \
+ safe_strcpy(_tmp, start); \
+ _tmp[len] = '\0'; \
+ for ( _i = 0; _tmp[_i] != '\0'; _i++ ) \
+ token = ( ( token * 10 ) + ( _tmp[_i] - '0' ) ); \
+}
+
+static inline int trim(const char *c, char *t, char elem)
+{
+ int l = strlen(c);
+ const char *x = c ;
+ int i = 0;
+ if ( !c || !t )
+ return -1;
+ while ( *x != '\0' && i < l )
+ {
+ if ( *x != elem )
+ t[i++] = *x;
+ x++;
+ }
+ t[i] = '\0';
+ return 0;
+}
+
+static inline int getlen(char *start, char *end)
+{
+ if ( ( start ) && ( end ) && ( end > start ) )
+ return end-start;
+ else
+ return -1;
+}
+
static void parse_credit2_runqueue(const char *s)
{
unsigned int i;
+ const char *s_end = NULL;
+ char m[strlen(s)];
+ char *_s = NULL;
for ( i = 0; i < ARRAY_SIZE(opt_runqueue_str); i++ )
{
@@ -351,7 +402,115 @@ static void parse_credit2_runqueue(const char *s)
return;
}
}
+/*
+ * At this stage we are either unknown value of credit2_runqueue or we can
+ * consider it to be custom cpu. Lets try parsing the same.
+ * Resetting the custom_cpu_runqueue for future use. Only the non-negative
+ * entries will be valid. The index 'i' in custom_cpu_runqueue will store
+ * the specific runqueue it belongs to.
+ * Example:
+ * If custom_cpu_runqueue[3] == 2
+ * Then, it means that cpu 3 belong to runqueue 2.
+ * If custom_cpu_runqueue[4] == -1
+ * Then, it means that cpu 4 doesn't belong to any runqueue.
+ */
+ for ( i = 0; i < nr_cpu_ids; i++ )
+ custom_cpu_runqueue[i] = -1;
+
+ /*
+ * Format [[0,1,4,5][2,3,6,7][8,9,12,13][10,11,14,15]]
+ */
+ i = 0;
+
+ /* In case user have spaces included in the input */
+ if ( trim(s, m, ' ') != 0 )
+ {
+ printk( "WARNING : %s[%d] trim failed.\n", __func__, __LINE__ );
+ goto errReturn;
+ }
+ /* Starting to parse and get the cpu information on trimmed data */
+ _s = m;
+ s_end = _s + strlen(_s);
+
+ /* The start and should always be in format of '[..]' */
+ if ( ( '[' == *_s ) && ( ']' == *(s_end-1)) )
+ {
+ char *start = NULL, *end = NULL;
+ int cpu_added_to_runqueue = 0;
+ _s++;
+ while ( ( _s != NULL ) && ( _s < s_end ) )
+ {
+ char *token_sub_str = NULL;
+ start = strstr(_s, "[");
+ end = strstr(_s, "]");
+ /* Validation checks for '[' and ']' to properly parse the entries
+ */
+ if ( ( !start && !end ) || ( ( end == ( s_end -1 ) ) && start ) )
+ goto errReturn;
+ /* Check for last entry */
+ else if ( !start && ( end == ( s_end -1 ) ) )
+ goto nextSet;
+
+ /* If we have [] as entry, move to nextSet */
+ if ( getlen ( start, end ) < 1 )
+ goto nextSet;
+ /* Start to parse the actual value */
+ start++;
+ /*
+ * find token within the subset
+ */
+ do
+ {
+ int token = 0;
+ int len = 0;
+ /* Get cpu ids separated by ',' within each set */
+ token_sub_str = strpbrk(start, ",");
+ if ( ( !token_sub_str && start < end ) ||
+ ( token_sub_str > end && token_sub_str > start ) )
+ len = getlen(start, end);
+ else
+ len = getlen(start, token_sub_str);
+
+ if ( len <= 0 )
+ continue;
+ /* GETTOKEN will get return the parse and populate the cpu in
+ * token
+ */
+ GETTOKEN(token, len, start , end );
+
+ if ( token >= nr_cpu_ids)
+ goto errReturn;
+ /* If not set already */
+ if ( custom_cpu_runqueue[token] == -1 )
+ {
+ custom_cpu_runqueue[token] = i;
+ cpu_added_to_runqueue = 1;
+ }
+ else
+ goto errReturn;
+
+ if ( !token_sub_str || token_sub_str > end )
+ goto nextSet;
+
+ start = ++token_sub_str;
+ } while ( start < end );
+nextSet:
+ if ( cpu_added_to_runqueue )
+ {
+ i++;
+ cpu_added_to_runqueue = 0;
+ }
+ _s = ++end;
+ }
+ opt_runqueue = OPT_RUNQUEUE_CUSTOM;
+ return;
+ }
+errReturn:
+ /* Resetting in case of failure, so that we don't mess-up during any failure
+ * due to wrong or spurious pattern passed by user.
+ */
+ opt_runqueue = OPT_RUNQUEUE_SOCKET;
printk("WARNING, unrecognized value of credit2_runqueue option!\n");
}
custom_param("credit2_runqueue", parse_credit2_runqueue);
@@ -661,6 +820,15 @@ cpu_to_runqueue(struct csched2_private *prv, unsigned int cpu)
struct csched2_runqueue_data *rqd;
unsigned int rqi;
+ if ( opt_runqueue == OPT_RUNQUEUE_CUSTOM )
+ {
+ if ( custom_cpu_runqueue[cpu] != -1 )
+ {
+ BUG_ON(custom_cpu_runqueue[cpu] >= nr_cpu_ids);
+ return custom_cpu_runqueue[cpu];
+ }
+ }
+
for ( rqi = 0; rqi < nr_cpu_ids; rqi++ )
{
unsigned int peer_cpu;
--
2.12.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 3+ messages in thread