* [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
@ 2015-11-05 11:42 Peter Maydell
2015-11-05 11:49 ` Paolo Bonzini
2015-11-05 12:13 ` Gerd Hoffmann
0 siblings, 2 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 11:42 UTC (permalink / raw)
To: QEMU Developers, Gerd Hoffmann, Paolo Bonzini
I tried to do a linux headers update against kvm/next (commit
197a4f4b063e4) with update-linux-headers.sh, but the result gives
a virtio-gpu that doesn't compile:
make: Entering directory
`/home/petmay01/linaro/qemu-from-laptop/qemu/build/all-debug'
CC aarch64-softmmu/hw/display/virtio-gpu.o
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
In function ‘virtio_gpu_get_features’:
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27:
error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this
function)
features |= (1 << VIRTIO_GPU_FEATURE_VIRGL);
^
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27:
note: each undeclared identifier is reported only once for each
function it appears in
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
In function ‘virtio_gpu_set_features’:
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:156:41:
error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this
function)
static const uint32_t virgl = (1 << VIRTIO_GPU_FEATURE_VIRGL);
^
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
In function ‘virtio_gpu_device_realize’:
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:914:25:
error: ‘struct virtio_gpu_config’ has no member named ‘num_capsets’
g->virtio_config.num_capsets = 1;
^
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
At top level:
/home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:1035:46:
error: invalid application of ‘sizeof’ to incomplete type ‘struct
virtio_gpu_transfer_host_3d’
QEMU_BUILD_BUG_ON(sizeof(struct virtio_gpu_transfer_host_3d) != 72);
^
etc, because all the virtio_gpu definitions disappear from
include/standard-headers/linux/virtio_gpu.h.
What's happening here?
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell
@ 2015-11-05 11:49 ` Paolo Bonzini
2015-11-05 12:13 ` Gerd Hoffmann
1 sibling, 0 replies; 26+ messages in thread
From: Paolo Bonzini @ 2015-11-05 11:49 UTC (permalink / raw)
To: Peter Maydell, QEMU Developers, Gerd Hoffmann
On 05/11/2015 12:42, Peter Maydell wrote:
> I tried to do a linux headers update against kvm/next (commit
> 197a4f4b063e4) with update-linux-headers.sh, but the result gives
> a virtio-gpu that doesn't compile:
>
> make: Entering directory
> `/home/petmay01/linaro/qemu-from-laptop/qemu/build/all-debug'
> CC aarch64-softmmu/hw/display/virtio-gpu.o
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
> In function ‘virtio_gpu_get_features’:
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27:
> error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this
> function)
> features |= (1 << VIRTIO_GPU_FEATURE_VIRGL);
> ^
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27:
> note: each undeclared identifier is reported only once for each
> function it appears in
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
> In function ‘virtio_gpu_set_features’:
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:156:41:
> error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this
> function)
> static const uint32_t virgl = (1 << VIRTIO_GPU_FEATURE_VIRGL);
> ^
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
> In function ‘virtio_gpu_device_realize’:
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:914:25:
> error: ‘struct virtio_gpu_config’ has no member named ‘num_capsets’
> g->virtio_config.num_capsets = 1;
> ^
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:
> At top level:
> /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:1035:46:
> error: invalid application of ‘sizeof’ to incomplete type ‘struct
> virtio_gpu_transfer_host_3d’
> QEMU_BUILD_BUG_ON(sizeof(struct virtio_gpu_transfer_host_3d) != 72);
> ^
>
> etc, because all the virtio_gpu definitions disappear from
> include/standard-headers/linux/virtio_gpu.h.
>
> What's happening here?
My guess: these will get into Linus's tree in a few days, and the update
was done on a maintainer tree.
Paolo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell
2015-11-05 11:49 ` Paolo Bonzini
@ 2015-11-05 12:13 ` Gerd Hoffmann
2015-11-05 12:32 ` Peter Maydell
1 sibling, 1 reply; 26+ messages in thread
From: Gerd Hoffmann @ 2015-11-05 12:13 UTC (permalink / raw)
To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers
Hi,
> etc, because all the virtio_gpu definitions disappear from
> include/standard-headers/linux/virtio_gpu.h.
Updates not yet in mainline, they are sitting in drm-next and should
land during the merge window (i.e. 4.4-rc1 should have them).
I'd suggest to exclude virtio_gpu.h changes when updating linux headers
for the time being.
cheers,
Gerd
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 12:13 ` Gerd Hoffmann
@ 2015-11-05 12:32 ` Peter Maydell
2015-11-05 13:23 ` Christian Borntraeger
` (2 more replies)
0 siblings, 3 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 12:32 UTC (permalink / raw)
To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers
On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> etc, because all the virtio_gpu definitions disappear from
>> include/standard-headers/linux/virtio_gpu.h.
>
> Updates not yet in mainline, they are sitting in drm-next and should
> land during the merge window (i.e. 4.4-rc1 should have them).
>
> I'd suggest to exclude virtio_gpu.h changes when updating linux headers
> for the time being.
I would strongly prefer it if we could get to a point where
we can say "kernel headers must only be updated from this tree"
and be guaranteed that it always works. This used to be true
with the tree in question being kvm/next, but it doesn't seem
to be so now. If it's going to be common that we have header
changes that don't go via kvm/next, maybe we need to coordinate
a tree that merges together the abi-guaranteed-stable changes
from different places before they hit mainline?
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 12:32 ` Peter Maydell
@ 2015-11-05 13:23 ` Christian Borntraeger
2015-11-05 13:44 ` Peter Maydell
2015-11-05 13:48 ` Laszlo Ersek
2015-11-05 14:42 ` Gerd Hoffmann
2 siblings, 1 reply; 26+ messages in thread
From: Christian Borntraeger @ 2015-11-05 13:23 UTC (permalink / raw)
To: Peter Maydell, Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers
Am 05.11.2015 um 13:32 schrieb Peter Maydell:
> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>> etc, because all the virtio_gpu definitions disappear from
>>> include/standard-headers/linux/virtio_gpu.h.
>>
>> Updates not yet in mainline, they are sitting in drm-next and should
>> land during the merge window (i.e. 4.4-rc1 should have them).
>>
>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers
>> for the time being.
>
> I would strongly prefer it if we could get to a point where
> we can say "kernel headers must only be updated from this tree"
> and be guaranteed that it always works. This used to be true
> with the tree in question being kvm/next, but it doesn't seem
> to be so now. If it's going to be common that we have header
> changes that don't go via kvm/next, maybe we need to coordinate
> a tree that merges together the abi-guaranteed-stable changes
> from different places before they hit mainline?
Yes I agree. In the end this might imply to only use Linus tree to
merge things in qemu. This is probably fine regarding the Linux tree,
as it has a 63/70 day cadence and we only synced from kvm/next after
rc4 or so. So this might defer for 4,5, or 6 weeks.
One problem why people won't like that is that the QEMU releases are
not often enough, though, as missing the deadline will defer the feature
for 4 or 5 month.
2014-04-17 Tag v2.0.0
2014-08-01 Tag v2.1.0
2014-12-09 Tag v2.2.0
2015-04-24 Tag v2.3.0
2015-08-11 Tag v2.4.0
2015-12-10 Tag v2.5.0
Maybe we should try to aim for a 3 month cadence for QEMU to reduce the
pain of missing soft/hard freeze. -> maybe a topic for the next QEMU
summit?
Christian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 13:23 ` Christian Borntraeger
@ 2015-11-05 13:44 ` Peter Maydell
2015-11-05 13:46 ` Christian Borntraeger
2015-11-05 14:01 ` Paolo Bonzini
0 siblings, 2 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 13:44 UTC (permalink / raw)
To: Christian Borntraeger; +Cc: Paolo Bonzini, Gerd Hoffmann, QEMU Developers
On 5 November 2015 at 13:23, Christian Borntraeger
<borntraeger@de.ibm.com> wrote:
> Maybe we should try to aim for a 3 month cadence for QEMU to reduce the
> pain of missing soft/hard freeze. -> maybe a topic for the next QEMU
> summit?
We used to have 3 month releases, but I think we found that these
were too short: If you have about a month hardfreeze and a month
softfreeze then you only have a month of open feature development,
which seems pretty short to me.
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 13:44 ` Peter Maydell
@ 2015-11-05 13:46 ` Christian Borntraeger
2015-11-05 14:01 ` Paolo Bonzini
1 sibling, 0 replies; 26+ messages in thread
From: Christian Borntraeger @ 2015-11-05 13:46 UTC (permalink / raw)
To: Peter Maydell; +Cc: Paolo Bonzini, Gerd Hoffmann, QEMU Developers
Am 05.11.2015 um 14:44 schrieb Peter Maydell:
> On 5 November 2015 at 13:23, Christian Borntraeger
> <borntraeger@de.ibm.com> wrote:
>> Maybe we should try to aim for a 3 month cadence for QEMU to reduce the
>> pain of missing soft/hard freeze. -> maybe a topic for the next QEMU
>> summit?
>
> We used to have 3 month releases, but I think we found that these
> were too short: If you have about a month hardfreeze and a month
> softfreeze then you only have a month of open feature development,
> which seems pretty short to me.
Yes, the overall process has to change for a 3 month cadence.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 12:32 ` Peter Maydell
2015-11-05 13:23 ` Christian Borntraeger
@ 2015-11-05 13:48 ` Laszlo Ersek
2015-11-05 15:52 ` Alex Bennée
2015-11-05 14:42 ` Gerd Hoffmann
2 siblings, 1 reply; 26+ messages in thread
From: Laszlo Ersek @ 2015-11-05 13:48 UTC (permalink / raw)
To: Peter Maydell, Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers
On 11/05/15 13:32, Peter Maydell wrote:
> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>> etc, because all the virtio_gpu definitions disappear from
>>> include/standard-headers/linux/virtio_gpu.h.
>>
>> Updates not yet in mainline, they are sitting in drm-next and should
>> land during the merge window (i.e. 4.4-rc1 should have them).
>>
>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers
>> for the time being.
>
> I would strongly prefer it if we could get to a point where
> we can say "kernel headers must only be updated from this tree"
> and be guaranteed that it always works. This used to be true
> with the tree in question being kvm/next, but it doesn't seem
> to be so now. If it's going to be common that we have header
> changes that don't go via kvm/next, maybe we need to coordinate
> a tree that merges together the abi-guaranteed-stable changes
> from different places before they hit mainline?
I've always frowned upon importing headers from one project to another
project. First, they can have different coding styles. (Case in point.)
Second, not everything that needs to be defined for the original project
is useful to the receiving project, and I find such cruft in the
receiving project very annoying. Third, in some cases it might even
raise licensing questions.
If it is an ABI, it should be specified in text format somewhere, and
then the projects can have their independent type definitions, macros
etc that implement the spec.
Laszlo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 13:44 ` Peter Maydell
2015-11-05 13:46 ` Christian Borntraeger
@ 2015-11-05 14:01 ` Paolo Bonzini
2015-11-05 14:30 ` Peter Maydell
1 sibling, 1 reply; 26+ messages in thread
From: Paolo Bonzini @ 2015-11-05 14:01 UTC (permalink / raw)
To: Peter Maydell, Christian Borntraeger; +Cc: Gerd Hoffmann, QEMU Developers
On 05/11/2015 14:44, Peter Maydell wrote:
> On 5 November 2015 at 13:23, Christian Borntraeger
> <borntraeger@de.ibm.com> wrote:
>> Maybe we should try to aim for a 3 month cadence for QEMU to reduce the
>> pain of missing soft/hard freeze. -> maybe a topic for the next QEMU
>> summit?
>
> We used to have 3 month releases, but I think we found that these
> were too short: If you have about a month hardfreeze and a month
> softfreeze then you only have a month of open feature development,
> which seems pretty short to me.
It's short unless you assume that development goes on in private
maintainer trees during hard freeze. It's always happened at least for
the block layer, and probably for everything else. About 400 commits in
2.5 were authored before the 2.4 release.
The main issue is that shorter cycles may mean fewer and bigger pull
requests. It also means more awareness of conflicts is needed. We
definitely lack the continuous integration infrastructure that is needed
for that.
Paolo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:01 ` Paolo Bonzini
@ 2015-11-05 14:30 ` Peter Maydell
2015-11-05 14:52 ` Peter Maydell
2015-11-05 14:58 ` Paolo Bonzini
0 siblings, 2 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 14:30 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers
On 5 November 2015 at 14:01, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 05/11/2015 14:44, Peter Maydell wrote:
>> We used to have 3 month releases, but I think we found that these
>> were too short: If you have about a month hardfreeze and a month
>> softfreeze then you only have a month of open feature development,
>> which seems pretty short to me.
>
> It's short unless you assume that development goes on in private
> maintainer trees during hard freeze. It's always happened at least for
> the block layer, and probably for everything else. About 400 commits in
> 2.5 were authored before the 2.4 release.
Mostly I don't have a for-the-next-release tree for target-arm
during freeze unless I completely can't avoid it, because it's
a pain to maintain both. I suspect at least some of the other
subsystems work the same way. People can develop during freeze,
and post patches to the list, but whether they get reviewed or
picked up into submaintainer trees is a different question.
> The main issue is that shorter cycles may mean fewer and bigger pull
> requests. It also means more awareness of conflicts is needed. We
> definitely lack the continuous integration infrastructure that is needed
> for that.
I think it works for the kernel because the different subsystems
have large communities of their own and the subtrees get a
reasonable amount of testing as a result, plus there are
efforts like linux-next to pre-check subtrees for conflicts
before an actual merge attempt happens. I don't think the QEMU
community is big enough for the kernel's dev practices to be
reasonably applicable to us.
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 12:32 ` Peter Maydell
2015-11-05 13:23 ` Christian Borntraeger
2015-11-05 13:48 ` Laszlo Ersek
@ 2015-11-05 14:42 ` Gerd Hoffmann
2015-11-05 14:45 ` Peter Maydell
2 siblings, 1 reply; 26+ messages in thread
From: Gerd Hoffmann @ 2015-11-05 14:42 UTC (permalink / raw)
To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers
On Do, 2015-11-05 at 12:32 +0000, Peter Maydell wrote:
> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
> >> etc, because all the virtio_gpu definitions disappear from
> >> include/standard-headers/linux/virtio_gpu.h.
> >
> > Updates not yet in mainline, they are sitting in drm-next and should
> > land during the merge window (i.e. 4.4-rc1 should have them).
> >
> > I'd suggest to exclude virtio_gpu.h changes when updating linux headers
> > for the time being.
>
> I would strongly prefer it if we could get to a point where
> we can say "kernel headers must only be updated from this tree"
> and be guaranteed that it always works. This used to be true
> with the tree in question being kvm/next, but it doesn't seem
> to be so now. If it's going to be common that we have header
> changes that don't go via kvm/next, maybe we need to coordinate
> a tree that merges together the abi-guaranteed-stable changes
> from different places before they hit mainline?
Chicken & egg issue in that case because airlied (linux kernel drm
maintainer) asked to have the qemu changes merged before taking the
virtio-gpu pull request. So I had no other chance than creating the
patches with not-yet upstream virtio header changes ...
cheers,
Gerd
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:42 ` Gerd Hoffmann
@ 2015-11-05 14:45 ` Peter Maydell
2015-11-05 14:58 ` Gerd Hoffmann
0 siblings, 1 reply; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 14:45 UTC (permalink / raw)
To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers
On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
> Chicken & egg issue in that case because airlied (linux kernel drm
> maintainer) asked to have the qemu changes merged before taking the
> virtio-gpu pull request. So I had no other chance than creating the
> patches with not-yet upstream virtio header changes ...
Hmm. If I'd realised that at the time I'd have pushed back on it.
We should never take code that relies on upstream kernel
ABI that hasn't been accepted by the maintainer yet.
thanks
- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:30 ` Peter Maydell
@ 2015-11-05 14:52 ` Peter Maydell
2015-11-05 15:03 ` Paolo Bonzini
2015-11-05 14:58 ` Paolo Bonzini
1 sibling, 1 reply; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 14:52 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers
On 5 November 2015 at 14:30, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 5 November 2015 at 14:01, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> On 05/11/2015 14:44, Peter Maydell wrote:
>>> We used to have 3 month releases, but I think we found that these
>>> were too short: If you have about a month hardfreeze and a month
>>> softfreeze then you only have a month of open feature development,
>>> which seems pretty short to me.
>>
>> It's short unless you assume that development goes on in private
>> maintainer trees during hard freeze. It's always happened at least for
>> the block layer, and probably for everything else. About 400 commits in
>> 2.5 were authored before the 2.4 release.
>
> Mostly I don't have a for-the-next-release tree for target-arm
> during freeze unless I completely can't avoid it, because it's
> a pain to maintain both.
...also speaking as the person who does the wrangling of freezes and
rc candidate tags and so on, three times a year is quite enough :-)
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:30 ` Peter Maydell
2015-11-05 14:52 ` Peter Maydell
@ 2015-11-05 14:58 ` Paolo Bonzini
2015-11-05 18:56 ` Peter Maydell
1 sibling, 1 reply; 26+ messages in thread
From: Paolo Bonzini @ 2015-11-05 14:58 UTC (permalink / raw)
To: Peter Maydell; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers
On 05/11/2015 15:30, Peter Maydell wrote:
> I suspect at least some of the other subsystems work the same way.
I try not to have a for-this-release tree during hard freeze. :)
> > The main issue is that shorter cycles may mean fewer and bigger pull
> > requests. It also means more awareness of conflicts is needed. We
> > definitely lack the continuous integration infrastructure that is needed
> > for that.
>
> I think it works for the kernel because the different subsystems
> have large communities of their own and the subtrees get a
> reasonable amount of testing as a result, plus there are
> efforts like linux-next to pre-check subtrees for conflicts
> before an actual merge attempt happens.
Yes, that's what I meant for continuous integration.
My hunch is that conflicts outside .json or trace-events are rare. It
would be an interesting experiment if someone was willing to prepare a
daily or bi-weekly (Mon/Thu) "qemu-next" branch for a few weeks during
the 2.6 development.
> I don't think the QEMU
> community is big enough for the kernel's dev practices to be
> reasonably applicable to us.
On the other hand, something like "qemu-next" would be much easier to
use daily than linux-next.
I think the main issue is that right now we have a very long freeze
period. It would be nice to know why (e.g. what kind of bugs are fixed?
are they release blockers only?) and whether a shorter development
period could also lead to a shorter hard freeze period.
Perhaps even 2-ish months, for example it could be 1 month development
(4.5 weeks) + 2 weeks to rc0 + 3.5 weeks to final (i.e. aim for final
equal to -rc3).
Or even change soft freeze to "time to respin pending pull requests if
they fail" (i.e. *pull requests* must be on the list, not patches!) and
shorten it to 1 week. That would give 4.5 weeks development + 1 week to
rc0 + 3.5 weeks to final. This is still very different from the
kernel's merge window model. OTOH with such a 2 months cadence we could
probably get rid of stable releases altogether, limiting them to
security issues.
So, there's room for experimenting. That said, we probably agree that 4
months and staying on time is better than saying officially 3 months and
delaying every release.
Paolo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:45 ` Peter Maydell
@ 2015-11-05 14:58 ` Gerd Hoffmann
2015-11-05 15:11 ` Peter Maydell
0 siblings, 1 reply; 26+ messages in thread
From: Gerd Hoffmann @ 2015-11-05 14:58 UTC (permalink / raw)
To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers
On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
> > Chicken & egg issue in that case because airlied (linux kernel drm
> > maintainer) asked to have the qemu changes merged before taking the
> > virtio-gpu pull request. So I had no other chance than creating the
> > patches with not-yet upstream virtio header changes ...
>
> Hmm. If I'd realised that at the time I'd have pushed back on it.
> We should never take code that relies on upstream kernel
> ABI that hasn't been accepted by the maintainer yet.
The reason airlied asked for qemu being upstream first is to avoid
having code in the kernel tree not accepted by qemu yet ...
So, one of the two has to go first ;)
When I did the pull requests patches had passed review on both
qemu-devel (qemu side) and dri-devel (kernel side), so this was really
just a formal "wait for qemu" thing with a very low risk of ABI changes,
and airlied pulled it into drm-next only days after it landed in
qemu/master.
cheers,
Gerd
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:52 ` Peter Maydell
@ 2015-11-05 15:03 ` Paolo Bonzini
0 siblings, 0 replies; 26+ messages in thread
From: Paolo Bonzini @ 2015-11-05 15:03 UTC (permalink / raw)
To: Peter Maydell; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers
On 05/11/2015 15:52, Peter Maydell wrote:
> ...also speaking as the person who does the wrangling of freezes and
> rc candidate tags and so on, three times a year is quite enough :-)
That's actually a good reason. On the other hand, we could look for a
new policy that makes your work shorter during freezes. We could also
find a second committer when you're on vacation, though that would be
harder.
4 months (120 days = 17 weeks) does have an advantage, in that it's
pretty much synced with the kernel's 9 week cycle if we insert a "leap
week" here and there. That theoretically means there's no reason to
sync headers from anything other than mainline.
Paolo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:58 ` Gerd Hoffmann
@ 2015-11-05 15:11 ` Peter Maydell
2015-11-05 17:15 ` Laszlo Ersek
0 siblings, 1 reply; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 15:11 UTC (permalink / raw)
To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers
On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote:
> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> > Chicken & egg issue in that case because airlied (linux kernel drm
>> > maintainer) asked to have the qemu changes merged before taking the
>> > virtio-gpu pull request. So I had no other chance than creating the
>> > patches with not-yet upstream virtio header changes ...
>>
>> Hmm. If I'd realised that at the time I'd have pushed back on it.
>> We should never take code that relies on upstream kernel
>> ABI that hasn't been accepted by the maintainer yet.
>
> The reason airlied asked for qemu being upstream first is to avoid
> having code in the kernel tree not accepted by qemu yet ...
>
> So, one of the two has to go first ;)
Right, but this isn't a symmetrical arrangement. If on the
kernel side the ABI is changed before it's finally accepted
into mainline, that means that any QEMU that got released with
changes made on the basis of not-yet-upstream kernel patches
will be broken. But if the kernel accepts code which needs a
not-yet-in-qemu feature to be useful, that doesn't break the
kernel, because the kernel isn't actually dependent on the
QEMU changes. That's why I think the kernel side of the ABI
always needs to go first (the kernel provides the ABI, QEMU
uses it, never the other way around).
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 13:48 ` Laszlo Ersek
@ 2015-11-05 15:52 ` Alex Bennée
2015-11-05 17:05 ` Laszlo Ersek
0 siblings, 1 reply; 26+ messages in thread
From: Alex Bennée @ 2015-11-05 15:52 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: QEMU Developers, Peter Maydell, Gerd Hoffmann, Paolo Bonzini
Laszlo Ersek <lersek@redhat.com> writes:
> On 11/05/15 13:32, Peter Maydell wrote:
>> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>> etc, because all the virtio_gpu definitions disappear from
>>>> include/standard-headers/linux/virtio_gpu.h.
>>>
>>> Updates not yet in mainline, they are sitting in drm-next and should
>>> land during the merge window (i.e. 4.4-rc1 should have them).
>>>
>>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers
>>> for the time being.
>>
>> I would strongly prefer it if we could get to a point where
>> we can say "kernel headers must only be updated from this tree"
>> and be guaranteed that it always works. This used to be true
>> with the tree in question being kvm/next, but it doesn't seem
>> to be so now. If it's going to be common that we have header
>> changes that don't go via kvm/next, maybe we need to coordinate
>> a tree that merges together the abi-guaranteed-stable changes
>> from different places before they hit mainline?
>
> I've always frowned upon importing headers from one project to another
> project. First, they can have different coding styles. (Case in point.)
> Second, not everything that needs to be defined for the original project
> is useful to the receiving project, and I find such cruft in the
> receiving project very annoying. Third, in some cases it might even
> raise licensing questions.
>
> If it is an ABI, it should be specified in text format somewhere, and
> then the projects can have their independent type definitions, macros
> etc that implement the spec.
Surely the kernel uapi includes is that textual format. Once stuff is
accepted into the master kernel tree there is your stable API.
FWIW the few bits I've done that involve syncing headers I basically
carry a [DEV] patch in my QEMU tree until the feature is finally merged
in the kernel and then I do my final re-base against the official tree
state.
--
Alex Bennée
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 15:52 ` Alex Bennée
@ 2015-11-05 17:05 ` Laszlo Ersek
2015-11-05 17:09 ` Paolo Bonzini
0 siblings, 1 reply; 26+ messages in thread
From: Laszlo Ersek @ 2015-11-05 17:05 UTC (permalink / raw)
To: Alex Bennée
Cc: Peter Maydell, Paolo Bonzini, QEMU Developers, Gerd Hoffmann
On 11/05/15 16:52, Alex Bennée wrote:
>
> Laszlo Ersek <lersek@redhat.com> writes:
>
>> On 11/05/15 13:32, Peter Maydell wrote:
>>> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>>> etc, because all the virtio_gpu definitions disappear from
>>>>> include/standard-headers/linux/virtio_gpu.h.
>>>>
>>>> Updates not yet in mainline, they are sitting in drm-next and should
>>>> land during the merge window (i.e. 4.4-rc1 should have them).
>>>>
>>>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers
>>>> for the time being.
>>>
>>> I would strongly prefer it if we could get to a point where
>>> we can say "kernel headers must only be updated from this tree"
>>> and be guaranteed that it always works. This used to be true
>>> with the tree in question being kvm/next, but it doesn't seem
>>> to be so now. If it's going to be common that we have header
>>> changes that don't go via kvm/next, maybe we need to coordinate
>>> a tree that merges together the abi-guaranteed-stable changes
>>> from different places before they hit mainline?
>>
>> I've always frowned upon importing headers from one project to another
>> project. First, they can have different coding styles. (Case in point.)
>> Second, not everything that needs to be defined for the original project
>> is useful to the receiving project, and I find such cruft in the
>> receiving project very annoying. Third, in some cases it might even
>> raise licensing questions.
>>
>> If it is an ABI, it should be specified in text format somewhere, and
>> then the projects can have their independent type definitions, macros
>> etc that implement the spec.
>
> Surely the kernel uapi includes is that textual format. Once stuff is
> accepted into the master kernel tree there is your stable API.
What is uapi for? If it is for QEMU (the userspace process) to consume
the host kernel's services, then I agree.
If uapi is for the guest kernel to consume (= drive) QEMU's virtual
hardware, then I strongly disagree. In that case Linux is just one of
the possible guests that can drive that hardware.
(Side point: and QEMU is just one of the emulators / hypervisors that
can provide that hardware. Which is why the actual hardware description
should exist independently of both.)
I'll elaborate elsewhere in the thread.
Thanks
Laszlo
> FWIW the few bits I've done that involve syncing headers I basically
> carry a [DEV] patch in my QEMU tree until the feature is finally merged
> in the kernel and then I do my final re-base against the official tree
> state.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 17:05 ` Laszlo Ersek
@ 2015-11-05 17:09 ` Paolo Bonzini
0 siblings, 0 replies; 26+ messages in thread
From: Paolo Bonzini @ 2015-11-05 17:09 UTC (permalink / raw)
To: Laszlo Ersek, Alex Bennée
Cc: Peter Maydell, QEMU Developers, Gerd Hoffmann
On 05/11/2015 18:05, Laszlo Ersek wrote:
>> > Surely the kernel uapi includes is that textual format. Once stuff is
>> > accepted into the master kernel tree there is your stable API.
> What is uapi for? If it is for QEMU (the userspace process) to consume
> the host kernel's services, then I agree.
uapi = Userspace API.
> If uapi is for the guest kernel to consume (= drive) QEMU's virtual
> hardware, then I strongly disagree. In that case Linux is just one of
> the possible guests that can drive that hardware.
Kernel-only headers can and will include uapi headers, though not the
other way round. So there can be indeed a case where a typo in uapi
headers causes both QEMU and the guest kernel to deviate from the spec.
> (Side point: and QEMU is just one of the emulators / hypervisors that
> can provide that hardware. Which is why the actual hardware description
> should exist independently of both.)
Typically it exists as an industry standard (e.g. the virtio
specification) or a datasheet.
Paolo
> I'll elaborate elsewhere in the thread.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 15:11 ` Peter Maydell
@ 2015-11-05 17:15 ` Laszlo Ersek
2015-11-05 18:13 ` Cornelia Huck
` (2 more replies)
0 siblings, 3 replies; 26+ messages in thread
From: Laszlo Ersek @ 2015-11-05 17:15 UTC (permalink / raw)
To: Peter Maydell, Gerd Hoffmann
Cc: Paolo Bonzini, Alex Bennée, QEMU Developers
On 11/05/15 16:11, Peter Maydell wrote:
> On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
>>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>> Chicken & egg issue in that case because airlied (linux kernel drm
>>>> maintainer) asked to have the qemu changes merged before taking the
>>>> virtio-gpu pull request. So I had no other chance than creating the
>>>> patches with not-yet upstream virtio header changes ...
>>>
>>> Hmm. If I'd realised that at the time I'd have pushed back on it.
>>> We should never take code that relies on upstream kernel
>>> ABI that hasn't been accepted by the maintainer yet.
>>
>> The reason airlied asked for qemu being upstream first is to avoid
>> having code in the kernel tree not accepted by qemu yet ...
>>
>> So, one of the two has to go first ;)
>
> Right, but this isn't a symmetrical arrangement. If on the
> kernel side the ABI is changed before it's finally accepted
> into mainline, that means that any QEMU that got released with
> changes made on the basis of not-yet-upstream kernel patches
> will be broken. But if the kernel accepts code which needs a
> not-yet-in-qemu feature to be useful, that doesn't break the
> kernel, because the kernel isn't actually dependent on the
> QEMU changes. That's why I think the kernel side of the ABI
> always needs to go first (the kernel provides the ABI, QEMU
> uses it, never the other way around).
I could be confused about what ABI means, but in case of *guest* kernel
drivers (for virtual devices provided by QEMU), the dependency seems to
be the opposite.
Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't
know what "uapi" means here. Are those headers there to be relied upon
by userspace processes that need services from the kernel? That would
match your above argument.
However, I found the following files there:
include/uapi/linux/virtio_console.h
include/uapi/linux/virtio_input.h
include/uapi/linux/virtio_net.h
include/uapi/linux/virtio_types.h
include/uapi/linux/virtio_9p.h
include/uapi/linux/virtio_balloon.h
include/uapi/linux/virtio_scsi.h
include/uapi/linux/virtio_rng.h
include/uapi/linux/virtio_ids.h
include/uapi/linux/virtio_ring.h
include/uapi/linux/virtio_pci.h
include/uapi/linux/virtio_gpu.h
include/uapi/linux/virtio_config.h
include/uapi/linux/virtio_blk.h
Which kinda confuses me. I cannot imagine that a userspace process
should depend on these, for the purpose of consuming kernel services.
Instead, QEMU should *definitely* dictate in this case, because it
provides the service (the hardware), and Linux has drivers that consume
that service.
Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too.
Should QEMU include their header files as well? I think not.
In other words, I agree with your argument, and the "QEMU depends on the
kernel" statement, as far as QEMU plays the role of a userspace process
that consumes the *host* kernel's services (VFIO, KVM, ...).
However, when QEMU provides the *hardware* (which is hopefully described
by an industry standard, or at least by some kind of independent
documentation), then the *guest* kernel plays the consumer role (as do
other non-Linux-kernel guests), and in that case the QEMU changes should
go in first (hopefully after testing them with the under-development
guest code, of course).
Hence, I'm leaning to think that the above virtio header files should
not be under uapi *at all*. They should be under "include/virtio", and
should be private to the guest drivers.
--*--
I think we had the same discussion when Wei was working on SMBIOS 3.0
for ARM. In
<http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>,
you asked
"Is support for this all in the mainline kernel yet?"
I didn't really understand that question -- it didn't matter. QEMU as a
*platform* was learning how to provide a service that was governed by an
industry standard. The status of *guest* kernel support (and of guest
utility support) was irrelevant, in my opinion.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 17:15 ` Laszlo Ersek
@ 2015-11-05 18:13 ` Cornelia Huck
2015-11-05 18:51 ` Laszlo Ersek
2015-11-06 16:34 ` Alex Bennée
2015-11-06 16:43 ` Peter Maydell
2 siblings, 1 reply; 26+ messages in thread
From: Cornelia Huck @ 2015-11-05 18:13 UTC (permalink / raw)
To: Laszlo Ersek
Cc: QEMU Developers, Peter Maydell, Alex Bennée, Gerd Hoffmann,
Paolo Bonzini
On Thu, 5 Nov 2015 18:15:04 +0100
Laszlo Ersek <lersek@redhat.com> wrote:
> On 11/05/15 16:11, Peter Maydell wrote:
> > On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote:
> >> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
> >>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
> >>>> Chicken & egg issue in that case because airlied (linux kernel drm
> >>>> maintainer) asked to have the qemu changes merged before taking the
> >>>> virtio-gpu pull request. So I had no other chance than creating the
> >>>> patches with not-yet upstream virtio header changes ...
> >>>
> >>> Hmm. If I'd realised that at the time I'd have pushed back on it.
> >>> We should never take code that relies on upstream kernel
> >>> ABI that hasn't been accepted by the maintainer yet.
> >>
> >> The reason airlied asked for qemu being upstream first is to avoid
> >> having code in the kernel tree not accepted by qemu yet ...
> >>
> >> So, one of the two has to go first ;)
> >
> > Right, but this isn't a symmetrical arrangement. If on the
> > kernel side the ABI is changed before it's finally accepted
> > into mainline, that means that any QEMU that got released with
> > changes made on the basis of not-yet-upstream kernel patches
> > will be broken. But if the kernel accepts code which needs a
> > not-yet-in-qemu feature to be useful, that doesn't break the
> > kernel, because the kernel isn't actually dependent on the
> > QEMU changes. That's why I think the kernel side of the ABI
> > always needs to go first (the kernel provides the ABI, QEMU
> > uses it, never the other way around).
>
> I could be confused about what ABI means, but in case of *guest* kernel
> drivers (for virtual devices provided by QEMU), the dependency seems to
> be the opposite.
>
> Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't
> know what "uapi" means here. Are those headers there to be relied upon
> by userspace processes that need services from the kernel? That would
> match your above argument.
>
> However, I found the following files there:
>
> include/uapi/linux/virtio_console.h
> include/uapi/linux/virtio_input.h
> include/uapi/linux/virtio_net.h
> include/uapi/linux/virtio_types.h
> include/uapi/linux/virtio_9p.h
> include/uapi/linux/virtio_balloon.h
> include/uapi/linux/virtio_scsi.h
> include/uapi/linux/virtio_rng.h
> include/uapi/linux/virtio_ids.h
> include/uapi/linux/virtio_ring.h
> include/uapi/linux/virtio_pci.h
> include/uapi/linux/virtio_gpu.h
> include/uapi/linux/virtio_config.h
> include/uapi/linux/virtio_blk.h
>
> Which kinda confuses me. I cannot imagine that a userspace process
> should depend on these, for the purpose of consuming kernel services.
> Instead, QEMU should *definitely* dictate in this case, because it
> provides the service (the hardware), and Linux has drivers that consume
> that service.
>
> Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too.
> Should QEMU include their header files as well? I think not.
>
> In other words, I agree with your argument, and the "QEMU depends on the
> kernel" statement, as far as QEMU plays the role of a userspace process
> that consumes the *host* kernel's services (VFIO, KVM, ...).
>
> However, when QEMU provides the *hardware* (which is hopefully described
> by an industry standard, or at least by some kind of independent
> documentation), then the *guest* kernel plays the consumer role (as do
> other non-Linux-kernel guests), and in that case the QEMU changes should
> go in first (hopefully after testing them with the under-development
> guest code, of course).
>
> Hence, I'm leaning to think that the above virtio header files should
> not be under uapi *at all*. They should be under "include/virtio", and
> should be private to the guest drivers.
The virtio case is more of a convenience issue.
The *canonical* definitions for virtio implementations are neither in
the kernel nor in qemu, but only in the standard. We can't conveniently
pull headers (for Linux and qemu) from a third place, so we make sure
that the Linux headers implement the standard and have qemu pull them
(this is probably also a corollary of mostly the same people working on
the Linux and qemu implementations).
IOW, I think in the virtio case *both* Linux and qemu are consumers.
The setup is as-is as it seems to be most convenient for the developers
involved.
(There has been discussion on this before, which I'm too lazy to dig up
right now.)
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 18:13 ` Cornelia Huck
@ 2015-11-05 18:51 ` Laszlo Ersek
0 siblings, 0 replies; 26+ messages in thread
From: Laszlo Ersek @ 2015-11-05 18:51 UTC (permalink / raw)
To: Cornelia Huck
Cc: QEMU Developers, Peter Maydell, Alex Bennée, Gerd Hoffmann,
Paolo Bonzini
On 11/05/15 19:13, Cornelia Huck wrote:
> On Thu, 5 Nov 2015 18:15:04 +0100
> Laszlo Ersek <lersek@redhat.com> wrote:
>
>> On 11/05/15 16:11, Peter Maydell wrote:
>>> On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
>>>>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>>>> Chicken & egg issue in that case because airlied (linux kernel drm
>>>>>> maintainer) asked to have the qemu changes merged before taking the
>>>>>> virtio-gpu pull request. So I had no other chance than creating the
>>>>>> patches with not-yet upstream virtio header changes ...
>>>>>
>>>>> Hmm. If I'd realised that at the time I'd have pushed back on it.
>>>>> We should never take code that relies on upstream kernel
>>>>> ABI that hasn't been accepted by the maintainer yet.
>>>>
>>>> The reason airlied asked for qemu being upstream first is to avoid
>>>> having code in the kernel tree not accepted by qemu yet ...
>>>>
>>>> So, one of the two has to go first ;)
>>>
>>> Right, but this isn't a symmetrical arrangement. If on the
>>> kernel side the ABI is changed before it's finally accepted
>>> into mainline, that means that any QEMU that got released with
>>> changes made on the basis of not-yet-upstream kernel patches
>>> will be broken. But if the kernel accepts code which needs a
>>> not-yet-in-qemu feature to be useful, that doesn't break the
>>> kernel, because the kernel isn't actually dependent on the
>>> QEMU changes. That's why I think the kernel side of the ABI
>>> always needs to go first (the kernel provides the ABI, QEMU
>>> uses it, never the other way around).
>>
>> I could be confused about what ABI means, but in case of *guest* kernel
>> drivers (for virtual devices provided by QEMU), the dependency seems to
>> be the opposite.
>>
>> Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't
>> know what "uapi" means here. Are those headers there to be relied upon
>> by userspace processes that need services from the kernel? That would
>> match your above argument.
>>
>> However, I found the following files there:
>>
>> include/uapi/linux/virtio_console.h
>> include/uapi/linux/virtio_input.h
>> include/uapi/linux/virtio_net.h
>> include/uapi/linux/virtio_types.h
>> include/uapi/linux/virtio_9p.h
>> include/uapi/linux/virtio_balloon.h
>> include/uapi/linux/virtio_scsi.h
>> include/uapi/linux/virtio_rng.h
>> include/uapi/linux/virtio_ids.h
>> include/uapi/linux/virtio_ring.h
>> include/uapi/linux/virtio_pci.h
>> include/uapi/linux/virtio_gpu.h
>> include/uapi/linux/virtio_config.h
>> include/uapi/linux/virtio_blk.h
>>
>> Which kinda confuses me. I cannot imagine that a userspace process
>> should depend on these, for the purpose of consuming kernel services.
>> Instead, QEMU should *definitely* dictate in this case, because it
>> provides the service (the hardware), and Linux has drivers that consume
>> that service.
>>
>> Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too.
>> Should QEMU include their header files as well? I think not.
>>
>> In other words, I agree with your argument, and the "QEMU depends on the
>> kernel" statement, as far as QEMU plays the role of a userspace process
>> that consumes the *host* kernel's services (VFIO, KVM, ...).
>>
>> However, when QEMU provides the *hardware* (which is hopefully described
>> by an industry standard, or at least by some kind of independent
>> documentation), then the *guest* kernel plays the consumer role (as do
>> other non-Linux-kernel guests), and in that case the QEMU changes should
>> go in first (hopefully after testing them with the under-development
>> guest code, of course).
>>
>> Hence, I'm leaning to think that the above virtio header files should
>> not be under uapi *at all*. They should be under "include/virtio", and
>> should be private to the guest drivers.
>
> The virtio case is more of a convenience issue.
>
> The *canonical* definitions for virtio implementations are neither in
> the kernel nor in qemu, but only in the standard. We can't conveniently
> pull headers (for Linux and qemu) from a third place, so we make sure
> that the Linux headers implement the standard and have qemu pull them
My argument is that this convenience is a bad thing, because it
introduces cross-project dependencies (and misleading uses of uapi per
se) that are not warranted for by the roles the projects play.
The convenience the current workflow promises is actually a pain. These
(non-)dependency arguments surface every time something new is
implemented that requires both host and guest support.
QEMU and the Linux kernel (and all other guests) should maintain their
own independent sets of definitions.
> (this is probably also a corollary of mostly the same people working on
> the Linux and qemu implementations).
The convenience sync also allows contributors to conveniently forget
about other guest code beside Linux. "It's in Linux and in QEMU, I'm done."
That is alright, unless the contributor's primary guest code actually
has a (not so obvious) dependency on that other guest code.
> IOW, I think in the virtio case *both* Linux and qemu are consumers.
In the actual service consumption sense, I don't think so; but as far as
"supporting the virtio standard" is concerned, I agree fully.
The point is, cross-project dependencies should only be dictated by the
former kind -- actual service consumption.
Given that QEMU and Linux are separate, independent "consumers" of the
virtio spec (in the latter sense), each should have an independent set
of definitions for compliance with the spec.
> The setup is as-is as it seems to be most convenient for the developers
> involved.
I agree. Until Peter asks if the header changes are in Linux yet, and
the contributor responds that the kernel maintainer wanted to see the
changes in QEMU first.
> (There has been discussion on this before, which I'm too lazy to dig up
> right now.)
>
I apologize for dredging it up then. I've not been aware of the earlier
discussion. But, it does show that the problem remains unsolved, and a pain.
... I guess my main reason for obsessing about this is because I
manually code up all industry standard-related macros and types in edk2,
from the revelant specs directly. It gives me incredible freedom; the
code conforms to the coding style; the definitions are as minimal as
possible; and I regularly gain insight from typing them and laying them
out. Sometimes they even help me find undocumented restrictions /
limitations in the interfacing project.
Yes, with the separate definitions, new features require more typing.
That's a good thing.
Anyway, thanks for replying; I guess I've annoyed most people following
this thread quite a bit by now. I shall shut up.
Cheers
Laszlo
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 14:58 ` Paolo Bonzini
@ 2015-11-05 18:56 ` Peter Maydell
0 siblings, 0 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-05 18:56 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers
On 5 November 2015 at 14:58, Paolo Bonzini <pbonzini@redhat.com> wrote:
> I think the main issue is that right now we have a very long freeze
> period. It would be nice to know why (e.g. what kind of bugs are fixed?
> are they release blockers only?) and whether a shorter development
> period could also lead to a shorter hard freeze period.
My gut feeling is that what typically happens is this:
* at or just after softfreeze deadline, we apply a bunch of
pull requests for big features
* during softfreeze there's usually one big thing that hits
middling-to-late, and so in the last week or so we need to
scrabble around fixing up the stuff that broke when these
features hit so that rc0 isn't too awful
[this cycle, that's looking like it might be record-and-replay]
* hardfreeze is when other people than the core dev community
think about testing stuff, and we get the resulting bug reports
* there's always one or two last release-critical issues that don't
get fixed until very late. This is why hardfreeze always slips
by an extra week or so (and the last week of hardfreeze is always
"no changes between final rc and real release").
> Perhaps even 2-ish months, for example it could be 1 month development
> (4.5 weeks) + 2 weeks to rc0 + 3.5 weeks to final (i.e. aim for final
> equal to -rc3).
>
> Or even change soft freeze to "time to respin pending pull requests if
> they fail" (i.e. *pull requests* must be on the list, not patches!) and
> shorten it to 1 week. That would give 4.5 weeks development + 1 week to
> rc0 + 3.5 weeks to final.
I would be sceptical that we could get a non-broken rc0 if we
put it that close to the thundering-herd of last minute feature pulls.
thanks
- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 17:15 ` Laszlo Ersek
2015-11-05 18:13 ` Cornelia Huck
@ 2015-11-06 16:34 ` Alex Bennée
2015-11-06 16:43 ` Peter Maydell
2 siblings, 0 replies; 26+ messages in thread
From: Alex Bennée @ 2015-11-06 16:34 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: QEMU Developers, Peter Maydell, Gerd Hoffmann, Paolo Bonzini
Laszlo Ersek <lersek@redhat.com> writes:
> On 11/05/15 16:11, Peter Maydell wrote:
>> On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote:
>>>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>>>> Chicken & egg issue in that case because airlied (linux kernel drm
>>>>> maintainer) asked to have the qemu changes merged before taking the
>>>>> virtio-gpu pull request. So I had no other chance than creating the
>>>>> patches with not-yet upstream virtio header changes ...
>>>>
>>>> Hmm. If I'd realised that at the time I'd have pushed back on it.
>>>> We should never take code that relies on upstream kernel
>>>> ABI that hasn't been accepted by the maintainer yet.
>>>
>>> The reason airlied asked for qemu being upstream first is to avoid
>>> having code in the kernel tree not accepted by qemu yet ...
>>>
>>> So, one of the two has to go first ;)
>>
>> Right, but this isn't a symmetrical arrangement. If on the
>> kernel side the ABI is changed before it's finally accepted
>> into mainline, that means that any QEMU that got released with
>> changes made on the basis of not-yet-upstream kernel patches
>> will be broken. But if the kernel accepts code which needs a
>> not-yet-in-qemu feature to be useful, that doesn't break the
>> kernel, because the kernel isn't actually dependent on the
>> QEMU changes. That's why I think the kernel side of the ABI
>> always needs to go first (the kernel provides the ABI, QEMU
>> uses it, never the other way around).
>
> I could be confused about what ABI means, but in case of *guest* kernel
> drivers (for virtual devices provided by QEMU), the dependency seems to
> be the opposite.
Well sort of, for guests using virtio devices both the kernel and QEMU
should be following the spec defined API (so guests being run under any
random virtio hypervisor work the same).
> Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't
> know what "uapi" means here. Are those headers there to be relied upon
> by userspace processes that need services from the kernel? That would
> match your above argument.
In this case the Userspace API the kernel provides to userspace which
QEMU is one user (in fact in the KVM kernel code anything that controls
the hypervisor is just referred to as userspace).
My bias is showing of course because I'm not considering guests here,
QEMU is the user of the kernel API. I'm thinking of things like the
KVM_RUN structure and the various ioctls and knobs for interaction
between KVM and QEMU.
>
> However, I found the following files there:
>
> include/uapi/linux/virtio_console.h
> include/uapi/linux/virtio_input.h
> include/uapi/linux/virtio_net.h
> include/uapi/linux/virtio_types.h
> include/uapi/linux/virtio_9p.h
> include/uapi/linux/virtio_balloon.h
> include/uapi/linux/virtio_scsi.h
> include/uapi/linux/virtio_rng.h
> include/uapi/linux/virtio_ids.h
> include/uapi/linux/virtio_ring.h
> include/uapi/linux/virtio_pci.h
> include/uapi/linux/virtio_gpu.h
> include/uapi/linux/virtio_config.h
> include/uapi/linux/virtio_blk.h
>
> Which kinda confuses me. I cannot imagine that a userspace process
> should depend on these, for the purpose of consuming kernel services.
> Instead, QEMU should *definitely* dictate in this case, because it
> provides the service (the hardware), and Linux has drivers that consume
> that service.
>
> Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too.
> Should QEMU include their header files as well? I think not.
>
> In other words, I agree with your argument, and the "QEMU depends on the
> kernel" statement, as far as QEMU plays the role of a userspace process
> that consumes the *host* kernel's services (VFIO, KVM, ...).
>
> However, when QEMU provides the *hardware* (which is hopefully described
> by an industry standard, or at least by some kind of independent
> documentation), then the *guest* kernel plays the consumer role (as do
> other non-Linux-kernel guests), and in that case the QEMU changes should
> go in first (hopefully after testing them with the under-development
> guest code, of course).
>
> Hence, I'm leaning to think that the above virtio header files should
> not be under uapi *at all*. They should be under "include/virtio", and
> should be private to the guest drivers.
That's an interesting point. I suspect in this case QEMU is using the
Linux defined headers as a handy implementation of the virtio specs.
>
> --*--
>
> I think we had the same discussion when Wei was working on SMBIOS 3.0
> for ARM. In
> <http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>,
> you asked
>
> "Is support for this all in the mainline kernel yet?"
>
> I didn't really understand that question -- it didn't matter. QEMU as a
> *platform* was learning how to provide a service that was governed by an
> industry standard. The status of *guest* kernel support (and of guest
> utility support) was irrelevant, in my opinion.
No there are plenty of examples, for example in my case guest debug
support requires a new kernel API which is only declared stable once
it passes review and gets merged into mainline.
--
Alex Bennée
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next
2015-11-05 17:15 ` Laszlo Ersek
2015-11-05 18:13 ` Cornelia Huck
2015-11-06 16:34 ` Alex Bennée
@ 2015-11-06 16:43 ` Peter Maydell
2 siblings, 0 replies; 26+ messages in thread
From: Peter Maydell @ 2015-11-06 16:43 UTC (permalink / raw)
To: Laszlo Ersek
Cc: Paolo Bonzini, Alex Bennée, Gerd Hoffmann, QEMU Developers
On 5 November 2015 at 17:15, Laszlo Ersek <lersek@redhat.com> wrote:
> --*--
>
> I think we had the same discussion when Wei was working on SMBIOS 3.0
> for ARM. In
> <http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>,
> you asked
>
> "Is support for this all in the mainline kernel yet?"
>
> I didn't really understand that question -- it didn't matter. QEMU as a
> *platform* was learning how to provide a service that was governed by an
> industry standard. The status of *guest* kernel support (and of guest
> utility support) was irrelevant, in my opinion.
The reason for the question in that case was slightly different.
As you say, it would be OK to implement support in either the
kernel or QEMU first. I asked because:
(1) I felt I had no understanding of the relevant spec to be able
to correctly review the QEMU code changes in detail
(2) I did trust that people on the kernel mailing list would be
able to provide review at that level of detail
(3) I was therefore trying to "outsource" the review to the
kernel folks so I could rely more on "QEMU works with the kernel"
as a check on the QEMU patches without falling into the trap
of "both sets of patches have complementary bugs that cancel each
other out".
with a side order of
(4) if a spec isn't important enough to get support in Linux then
it's at least worth asking whether it's important enough to support
in QEMU.
thanks
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2015-11-06 16:43 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell
2015-11-05 11:49 ` Paolo Bonzini
2015-11-05 12:13 ` Gerd Hoffmann
2015-11-05 12:32 ` Peter Maydell
2015-11-05 13:23 ` Christian Borntraeger
2015-11-05 13:44 ` Peter Maydell
2015-11-05 13:46 ` Christian Borntraeger
2015-11-05 14:01 ` Paolo Bonzini
2015-11-05 14:30 ` Peter Maydell
2015-11-05 14:52 ` Peter Maydell
2015-11-05 15:03 ` Paolo Bonzini
2015-11-05 14:58 ` Paolo Bonzini
2015-11-05 18:56 ` Peter Maydell
2015-11-05 13:48 ` Laszlo Ersek
2015-11-05 15:52 ` Alex Bennée
2015-11-05 17:05 ` Laszlo Ersek
2015-11-05 17:09 ` Paolo Bonzini
2015-11-05 14:42 ` Gerd Hoffmann
2015-11-05 14:45 ` Peter Maydell
2015-11-05 14:58 ` Gerd Hoffmann
2015-11-05 15:11 ` Peter Maydell
2015-11-05 17:15 ` Laszlo Ersek
2015-11-05 18:13 ` Cornelia Huck
2015-11-05 18:51 ` Laszlo Ersek
2015-11-06 16:34 ` Alex Bennée
2015-11-06 16:43 ` Peter Maydell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).