* [PATCH] KVM: Add Kconfig option to signal cross-endian guests [not found] <20150707195012-mutt-send-email-mst@redhat.com> @ 2015-07-09 7:49 ` Thomas Huth 2015-07-09 9:48 ` Laurent Vivier ` (3 more replies) 0 siblings, 4 replies; 8+ messages in thread From: Thomas Huth @ 2015-07-09 7:49 UTC (permalink / raw) To: linux-arm-kernel The option for supporting cross-endianness legacy guests in the vhost and tun code should only be available on systems that support cross-endian guests. Signed-off-by: Thomas Huth <thuth@redhat.com> --- arch/arm/kvm/Kconfig | 1 + arch/arm64/kvm/Kconfig | 1 + arch/powerpc/kvm/Kconfig | 1 + drivers/net/Kconfig | 1 + drivers/vhost/Kconfig | 1 + virt/kvm/Kconfig | 3 +++ 6 files changed, 8 insertions(+) diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig index bfb915d..9d8f363 100644 --- a/arch/arm/kvm/Kconfig +++ b/arch/arm/kvm/Kconfig @@ -31,6 +31,7 @@ config KVM select KVM_VFIO select HAVE_KVM_EVENTFD select HAVE_KVM_IRQFD + select KVM_CROSS_ENDIAN_GUESTS depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER ---help--- Support hosting virtualized guest machines. diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index bfffe8f..9af39fe 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -31,6 +31,7 @@ config KVM select KVM_VFIO select HAVE_KVM_EVENTFD select HAVE_KVM_IRQFD + select KVM_CROSS_ENDIAN_GUESTS ---help--- Support hosting virtualized guest machines. diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig index 3caec2c..e028710 100644 --- a/arch/powerpc/kvm/Kconfig +++ b/arch/powerpc/kvm/Kconfig @@ -79,6 +79,7 @@ config KVM_BOOK3S_64_HV select KVM_BOOK3S_HV_POSSIBLE select MMU_NOTIFIER select CMA + select KVM_CROSS_ENDIAN_GUESTS ---help--- Support running unmodified book3s_64 guest kernels in virtual machines on POWER7 and PPC970 processors that have diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig index c18f9e6..0c4ce47 100644 --- a/drivers/net/Kconfig +++ b/drivers/net/Kconfig @@ -261,6 +261,7 @@ config TUN config TUN_VNET_CROSS_LE bool "Support for cross-endian vnet headers on little-endian kernels" default n + depends on KVM_CROSS_ENDIAN_GUESTS ---help--- This option allows TUN/TAP and MACVTAP device drivers in a little-endian kernel to parse vnet headers that come from a diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig index 533eaf0..4d8ae6b 100644 --- a/drivers/vhost/Kconfig +++ b/drivers/vhost/Kconfig @@ -35,6 +35,7 @@ config VHOST config VHOST_CROSS_ENDIAN_LEGACY bool "Cross-endian support for vhost" + depends on KVM_CROSS_ENDIAN_GUESTS default n ---help--- This option allows vhost to support guests with a different byte diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig index e2c876d..cc7b28a 100644 --- a/virt/kvm/Kconfig +++ b/virt/kvm/Kconfig @@ -47,3 +47,6 @@ config KVM_GENERIC_DIRTYLOG_READ_PROTECT config KVM_COMPAT def_bool y depends on COMPAT && !S390 + +config KVM_CROSS_ENDIAN_GUESTS + bool -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 7:49 ` [PATCH] KVM: Add Kconfig option to signal cross-endian guests Thomas Huth @ 2015-07-09 9:48 ` Laurent Vivier 2015-07-09 12:57 ` Paolo Bonzini 2015-07-09 10:02 ` Christoffer Dall ` (2 subsequent siblings) 3 siblings, 1 reply; 8+ messages in thread From: Laurent Vivier @ 2015-07-09 9:48 UTC (permalink / raw) To: linux-arm-kernel On 09/07/2015 09:49, Thomas Huth wrote: > The option for supporting cross-endianness legacy guests in > the vhost and tun code should only be available on systems > that support cross-endian guests. I'm sure I misunderstand something, but what happens if we use QEMU with TCG instead of KVM, i.e. a big endian powerpc kernel guest on x86_64 little endian host ? Do you forbid the use of vhost in this case ? > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > arch/arm/kvm/Kconfig | 1 + > arch/arm64/kvm/Kconfig | 1 + > arch/powerpc/kvm/Kconfig | 1 + > drivers/net/Kconfig | 1 + > drivers/vhost/Kconfig | 1 + > virt/kvm/Kconfig | 3 +++ > 6 files changed, 8 insertions(+) > > diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig > index bfb915d..9d8f363 100644 > --- a/arch/arm/kvm/Kconfig > +++ b/arch/arm/kvm/Kconfig > @@ -31,6 +31,7 @@ config KVM > select KVM_VFIO > select HAVE_KVM_EVENTFD > select HAVE_KVM_IRQFD > + select KVM_CROSS_ENDIAN_GUESTS > depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER > ---help--- > Support hosting virtualized guest machines. > diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig > index bfffe8f..9af39fe 100644 > --- a/arch/arm64/kvm/Kconfig > +++ b/arch/arm64/kvm/Kconfig > @@ -31,6 +31,7 @@ config KVM > select KVM_VFIO > select HAVE_KVM_EVENTFD > select HAVE_KVM_IRQFD > + select KVM_CROSS_ENDIAN_GUESTS > ---help--- > Support hosting virtualized guest machines. > > diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig > index 3caec2c..e028710 100644 > --- a/arch/powerpc/kvm/Kconfig > +++ b/arch/powerpc/kvm/Kconfig > @@ -79,6 +79,7 @@ config KVM_BOOK3S_64_HV > select KVM_BOOK3S_HV_POSSIBLE > select MMU_NOTIFIER > select CMA > + select KVM_CROSS_ENDIAN_GUESTS > ---help--- > Support running unmodified book3s_64 guest kernels in > virtual machines on POWER7 and PPC970 processors that have > diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig > index c18f9e6..0c4ce47 100644 > --- a/drivers/net/Kconfig > +++ b/drivers/net/Kconfig > @@ -261,6 +261,7 @@ config TUN > config TUN_VNET_CROSS_LE > bool "Support for cross-endian vnet headers on little-endian kernels" > default n > + depends on KVM_CROSS_ENDIAN_GUESTS > ---help--- > This option allows TUN/TAP and MACVTAP device drivers in a > little-endian kernel to parse vnet headers that come from a > diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig > index 533eaf0..4d8ae6b 100644 > --- a/drivers/vhost/Kconfig > +++ b/drivers/vhost/Kconfig > @@ -35,6 +35,7 @@ config VHOST > > config VHOST_CROSS_ENDIAN_LEGACY > bool "Cross-endian support for vhost" > + depends on KVM_CROSS_ENDIAN_GUESTS > default n > ---help--- > This option allows vhost to support guests with a different byte > diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig > index e2c876d..cc7b28a 100644 > --- a/virt/kvm/Kconfig > +++ b/virt/kvm/Kconfig > @@ -47,3 +47,6 @@ config KVM_GENERIC_DIRTYLOG_READ_PROTECT > config KVM_COMPAT > def_bool y > depends on COMPAT && !S390 > + > +config KVM_CROSS_ENDIAN_GUESTS > + bool > ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 9:48 ` Laurent Vivier @ 2015-07-09 12:57 ` Paolo Bonzini 2015-07-09 13:07 ` Michael S. Tsirkin 0 siblings, 1 reply; 8+ messages in thread From: Paolo Bonzini @ 2015-07-09 12:57 UTC (permalink / raw) To: linux-arm-kernel On 09/07/2015 11:48, Laurent Vivier wrote: > > > On 09/07/2015 09:49, Thomas Huth wrote: >> The option for supporting cross-endianness legacy guests in >> the vhost and tun code should only be available on systems >> that support cross-endian guests. > > I'm sure I misunderstand something, but what happens if we use QEMU with > TCG instead of KVM, i.e. a big endian powerpc kernel guest on x86_64 > little endian host ? TCG does not yet support irqfd/ioeventfd, so it cannot be used with vhost. Paolo > Do you forbid the use of vhost in this case ? > >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> arch/arm/kvm/Kconfig | 1 + >> arch/arm64/kvm/Kconfig | 1 + >> arch/powerpc/kvm/Kconfig | 1 + >> drivers/net/Kconfig | 1 + >> drivers/vhost/Kconfig | 1 + >> virt/kvm/Kconfig | 3 +++ >> 6 files changed, 8 insertions(+) >> >> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig >> index bfb915d..9d8f363 100644 >> --- a/arch/arm/kvm/Kconfig >> +++ b/arch/arm/kvm/Kconfig >> @@ -31,6 +31,7 @@ config KVM >> select KVM_VFIO >> select HAVE_KVM_EVENTFD >> select HAVE_KVM_IRQFD >> + select KVM_CROSS_ENDIAN_GUESTS >> depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER >> ---help--- >> Support hosting virtualized guest machines. >> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig >> index bfffe8f..9af39fe 100644 >> --- a/arch/arm64/kvm/Kconfig >> +++ b/arch/arm64/kvm/Kconfig >> @@ -31,6 +31,7 @@ config KVM >> select KVM_VFIO >> select HAVE_KVM_EVENTFD >> select HAVE_KVM_IRQFD >> + select KVM_CROSS_ENDIAN_GUESTS >> ---help--- >> Support hosting virtualized guest machines. >> >> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig >> index 3caec2c..e028710 100644 >> --- a/arch/powerpc/kvm/Kconfig >> +++ b/arch/powerpc/kvm/Kconfig >> @@ -79,6 +79,7 @@ config KVM_BOOK3S_64_HV >> select KVM_BOOK3S_HV_POSSIBLE >> select MMU_NOTIFIER >> select CMA >> + select KVM_CROSS_ENDIAN_GUESTS >> ---help--- >> Support running unmodified book3s_64 guest kernels in >> virtual machines on POWER7 and PPC970 processors that have >> diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig >> index c18f9e6..0c4ce47 100644 >> --- a/drivers/net/Kconfig >> +++ b/drivers/net/Kconfig >> @@ -261,6 +261,7 @@ config TUN >> config TUN_VNET_CROSS_LE >> bool "Support for cross-endian vnet headers on little-endian kernels" >> default n >> + depends on KVM_CROSS_ENDIAN_GUESTS >> ---help--- >> This option allows TUN/TAP and MACVTAP device drivers in a >> little-endian kernel to parse vnet headers that come from a >> diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig >> index 533eaf0..4d8ae6b 100644 >> --- a/drivers/vhost/Kconfig >> +++ b/drivers/vhost/Kconfig >> @@ -35,6 +35,7 @@ config VHOST >> >> config VHOST_CROSS_ENDIAN_LEGACY >> bool "Cross-endian support for vhost" >> + depends on KVM_CROSS_ENDIAN_GUESTS >> default n >> ---help--- >> This option allows vhost to support guests with a different byte >> diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig >> index e2c876d..cc7b28a 100644 >> --- a/virt/kvm/Kconfig >> +++ b/virt/kvm/Kconfig >> @@ -47,3 +47,6 @@ config KVM_GENERIC_DIRTYLOG_READ_PROTECT >> config KVM_COMPAT >> def_bool y >> depends on COMPAT && !S390 >> + >> +config KVM_CROSS_ENDIAN_GUESTS >> + bool >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 12:57 ` Paolo Bonzini @ 2015-07-09 13:07 ` Michael S. Tsirkin 2015-07-09 18:59 ` Thomas Huth 0 siblings, 1 reply; 8+ messages in thread From: Michael S. Tsirkin @ 2015-07-09 13:07 UTC (permalink / raw) To: linux-arm-kernel On Thu, Jul 09, 2015 at 02:57:33PM +0200, Paolo Bonzini wrote: > > > On 09/07/2015 11:48, Laurent Vivier wrote: > > > > > > On 09/07/2015 09:49, Thomas Huth wrote: > >> The option for supporting cross-endianness legacy guests in > >> the vhost and tun code should only be available on systems > >> that support cross-endian guests. > > > > I'm sure I misunderstand something, but what happens if we use QEMU with > > TCG instead of KVM, i.e. a big endian powerpc kernel guest on x86_64 > > little endian host ? > > TCG does not yet support irqfd/ioeventfd, so it cannot be used with vhost. > > Paolo vhost does not require irqfd anymore. I think ioeventfd actually works fine though I didn't try, it would be easy to support. > > Do you forbid the use of vhost in this case ? > > > >> Signed-off-by: Thomas Huth <thuth@redhat.com> > >> --- > >> arch/arm/kvm/Kconfig | 1 + > >> arch/arm64/kvm/Kconfig | 1 + > >> arch/powerpc/kvm/Kconfig | 1 + > >> drivers/net/Kconfig | 1 + > >> drivers/vhost/Kconfig | 1 + > >> virt/kvm/Kconfig | 3 +++ > >> 6 files changed, 8 insertions(+) > >> > >> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig > >> index bfb915d..9d8f363 100644 > >> --- a/arch/arm/kvm/Kconfig > >> +++ b/arch/arm/kvm/Kconfig > >> @@ -31,6 +31,7 @@ config KVM > >> select KVM_VFIO > >> select HAVE_KVM_EVENTFD > >> select HAVE_KVM_IRQFD > >> + select KVM_CROSS_ENDIAN_GUESTS > >> depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER > >> ---help--- > >> Support hosting virtualized guest machines. > >> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig > >> index bfffe8f..9af39fe 100644 > >> --- a/arch/arm64/kvm/Kconfig > >> +++ b/arch/arm64/kvm/Kconfig > >> @@ -31,6 +31,7 @@ config KVM > >> select KVM_VFIO > >> select HAVE_KVM_EVENTFD > >> select HAVE_KVM_IRQFD > >> + select KVM_CROSS_ENDIAN_GUESTS > >> ---help--- > >> Support hosting virtualized guest machines. > >> > >> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig > >> index 3caec2c..e028710 100644 > >> --- a/arch/powerpc/kvm/Kconfig > >> +++ b/arch/powerpc/kvm/Kconfig > >> @@ -79,6 +79,7 @@ config KVM_BOOK3S_64_HV > >> select KVM_BOOK3S_HV_POSSIBLE > >> select MMU_NOTIFIER > >> select CMA > >> + select KVM_CROSS_ENDIAN_GUESTS > >> ---help--- > >> Support running unmodified book3s_64 guest kernels in > >> virtual machines on POWER7 and PPC970 processors that have > >> diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig > >> index c18f9e6..0c4ce47 100644 > >> --- a/drivers/net/Kconfig > >> +++ b/drivers/net/Kconfig > >> @@ -261,6 +261,7 @@ config TUN > >> config TUN_VNET_CROSS_LE > >> bool "Support for cross-endian vnet headers on little-endian kernels" > >> default n > >> + depends on KVM_CROSS_ENDIAN_GUESTS > >> ---help--- > >> This option allows TUN/TAP and MACVTAP device drivers in a > >> little-endian kernel to parse vnet headers that come from a > >> diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig > >> index 533eaf0..4d8ae6b 100644 > >> --- a/drivers/vhost/Kconfig > >> +++ b/drivers/vhost/Kconfig > >> @@ -35,6 +35,7 @@ config VHOST > >> > >> config VHOST_CROSS_ENDIAN_LEGACY > >> bool "Cross-endian support for vhost" > >> + depends on KVM_CROSS_ENDIAN_GUESTS > >> default n > >> ---help--- > >> This option allows vhost to support guests with a different byte > >> diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig > >> index e2c876d..cc7b28a 100644 > >> --- a/virt/kvm/Kconfig > >> +++ b/virt/kvm/Kconfig > >> @@ -47,3 +47,6 @@ config KVM_GENERIC_DIRTYLOG_READ_PROTECT > >> config KVM_COMPAT > >> def_bool y > >> depends on COMPAT && !S390 > >> + > >> +config KVM_CROSS_ENDIAN_GUESTS > >> + bool > >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 13:07 ` Michael S. Tsirkin @ 2015-07-09 18:59 ` Thomas Huth 0 siblings, 0 replies; 8+ messages in thread From: Thomas Huth @ 2015-07-09 18:59 UTC (permalink / raw) To: linux-arm-kernel On Thu, 9 Jul 2015 16:07:47 +0300 "Michael S. Tsirkin" <mst@redhat.com> wrote: > On Thu, Jul 09, 2015 at 02:57:33PM +0200, Paolo Bonzini wrote: > > > > > > On 09/07/2015 11:48, Laurent Vivier wrote: > > > > > > > > > On 09/07/2015 09:49, Thomas Huth wrote: > > >> The option for supporting cross-endianness legacy guests in > > >> the vhost and tun code should only be available on systems > > >> that support cross-endian guests. > > > > > > I'm sure I misunderstand something, but what happens if we use QEMU with > > > TCG instead of KVM, i.e. a big endian powerpc kernel guest on x86_64 > > > little endian host ? > > > > TCG does not yet support irqfd/ioeventfd, so it cannot be used with vhost. > > > > Paolo > > vhost does not require irqfd anymore. I think ioeventfd actually works > fine though I didn't try, it would be easy to support. That's an interesting issue, thanks for pointing this out, Laurent! So do we now rather want to leave everything as it currently is, in case somebody wants to use vhost-net with a cross-endian TCG guest one day? Or do we assume that either a) TCG is so slow anyway that nobody wants to accelerate it with vhost or b) TCG vhost likely won't happen that soon so we hope that everybody will already be using virtio 1.0 at that point in time (with a fixed endianness) ? ... then I think we should go on and include this patch. Thomas ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 7:49 ` [PATCH] KVM: Add Kconfig option to signal cross-endian guests Thomas Huth 2015-07-09 9:48 ` Laurent Vivier @ 2015-07-09 10:02 ` Christoffer Dall 2015-07-09 12:21 ` Cornelia Huck 2015-07-13 9:24 ` Greg Kurz 3 siblings, 0 replies; 8+ messages in thread From: Christoffer Dall @ 2015-07-09 10:02 UTC (permalink / raw) To: linux-arm-kernel On Thu, Jul 09, 2015 at 09:49:05AM +0200, Thomas Huth wrote: > The option for supporting cross-endianness legacy guests in > the vhost and tun code should only be available on systems > that support cross-endian guests. > > Signed-off-by: Thomas Huth <thuth@redhat.com> Acked-by: Christoffer Dall <christoffer.dall@linaro.org> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 7:49 ` [PATCH] KVM: Add Kconfig option to signal cross-endian guests Thomas Huth 2015-07-09 9:48 ` Laurent Vivier 2015-07-09 10:02 ` Christoffer Dall @ 2015-07-09 12:21 ` Cornelia Huck 2015-07-13 9:24 ` Greg Kurz 3 siblings, 0 replies; 8+ messages in thread From: Cornelia Huck @ 2015-07-09 12:21 UTC (permalink / raw) To: linux-arm-kernel On Thu, 9 Jul 2015 09:49:05 +0200 Thomas Huth <thuth@redhat.com> wrote: > The option for supporting cross-endianness legacy guests in s/cross-endianness/cross-endian/ ? > the vhost and tun code should only be available on systems > that support cross-endian guests. > > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > arch/arm/kvm/Kconfig | 1 + > arch/arm64/kvm/Kconfig | 1 + > arch/powerpc/kvm/Kconfig | 1 + > drivers/net/Kconfig | 1 + > drivers/vhost/Kconfig | 1 + > virt/kvm/Kconfig | 3 +++ > 6 files changed, 8 insertions(+) Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] KVM: Add Kconfig option to signal cross-endian guests 2015-07-09 7:49 ` [PATCH] KVM: Add Kconfig option to signal cross-endian guests Thomas Huth ` (2 preceding siblings ...) 2015-07-09 12:21 ` Cornelia Huck @ 2015-07-13 9:24 ` Greg Kurz 3 siblings, 0 replies; 8+ messages in thread From: Greg Kurz @ 2015-07-13 9:24 UTC (permalink / raw) To: linux-arm-kernel On Thu, 9 Jul 2015 09:49:05 +0200 Thomas Huth <thuth@redhat.com> wrote: > The option for supporting cross-endianness legacy guests in > the vhost and tun code should only be available on systems > that support cross-endian guests. > > Signed-off-by: Thomas Huth <thuth@redhat.com> Acked-by: Greg Kurz <gkurz@linux.vnet.ibm.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2015-07-13 9:24 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20150707195012-mutt-send-email-mst@redhat.com>
2015-07-09 7:49 ` [PATCH] KVM: Add Kconfig option to signal cross-endian guests Thomas Huth
2015-07-09 9:48 ` Laurent Vivier
2015-07-09 12:57 ` Paolo Bonzini
2015-07-09 13:07 ` Michael S. Tsirkin
2015-07-09 18:59 ` Thomas Huth
2015-07-09 10:02 ` Christoffer Dall
2015-07-09 12:21 ` Cornelia Huck
2015-07-13 9:24 ` Greg Kurz
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).