* [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next @ 2015-11-05 11:42 Peter Maydell 2015-11-05 11:49 ` Paolo Bonzini 2015-11-05 12:13 ` Gerd Hoffmann 0 siblings, 2 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-05 11:42 UTC (permalink / raw) To: QEMU Developers, Gerd Hoffmann, Paolo Bonzini I tried to do a linux headers update against kvm/next (commit 197a4f4b063e4) with update-linux-headers.sh, but the result gives a virtio-gpu that doesn't compile: make: Entering directory `/home/petmay01/linaro/qemu-from-laptop/qemu/build/all-debug' CC aarch64-softmmu/hw/display/virtio-gpu.o /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: In function ‘virtio_gpu_get_features’: /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27: error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this function) features |= (1 << VIRTIO_GPU_FEATURE_VIRGL); ^ /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27: note: each undeclared identifier is reported only once for each function it appears in /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: In function ‘virtio_gpu_set_features’: /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:156:41: error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this function) static const uint32_t virgl = (1 << VIRTIO_GPU_FEATURE_VIRGL); ^ /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: In function ‘virtio_gpu_device_realize’: /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:914:25: error: ‘struct virtio_gpu_config’ has no member named ‘num_capsets’ g->virtio_config.num_capsets = 1; ^ /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: At top level: /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:1035:46: error: invalid application of ‘sizeof’ to incomplete type ‘struct virtio_gpu_transfer_host_3d’ QEMU_BUILD_BUG_ON(sizeof(struct virtio_gpu_transfer_host_3d) != 72); ^ etc, because all the virtio_gpu definitions disappear from include/standard-headers/linux/virtio_gpu.h. What's happening here? thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell @ 2015-11-05 11:49 ` Paolo Bonzini 2015-11-05 12:13 ` Gerd Hoffmann 1 sibling, 0 replies; 26+ messages in thread From: Paolo Bonzini @ 2015-11-05 11:49 UTC (permalink / raw) To: Peter Maydell, QEMU Developers, Gerd Hoffmann On 05/11/2015 12:42, Peter Maydell wrote: > I tried to do a linux headers update against kvm/next (commit > 197a4f4b063e4) with update-linux-headers.sh, but the result gives > a virtio-gpu that doesn't compile: > > make: Entering directory > `/home/petmay01/linaro/qemu-from-laptop/qemu/build/all-debug' > CC aarch64-softmmu/hw/display/virtio-gpu.o > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: > In function ‘virtio_gpu_get_features’: > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27: > error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this > function) > features |= (1 << VIRTIO_GPU_FEATURE_VIRGL); > ^ > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:149:27: > note: each undeclared identifier is reported only once for each > function it appears in > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: > In function ‘virtio_gpu_set_features’: > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:156:41: > error: ‘VIRTIO_GPU_FEATURE_VIRGL’ undeclared (first use in this > function) > static const uint32_t virgl = (1 << VIRTIO_GPU_FEATURE_VIRGL); > ^ > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: > In function ‘virtio_gpu_device_realize’: > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:914:25: > error: ‘struct virtio_gpu_config’ has no member named ‘num_capsets’ > g->virtio_config.num_capsets = 1; > ^ > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c: > At top level: > /home/petmay01/linaro/qemu-from-laptop/qemu/hw/display/virtio-gpu.c:1035:46: > error: invalid application of ‘sizeof’ to incomplete type ‘struct > virtio_gpu_transfer_host_3d’ > QEMU_BUILD_BUG_ON(sizeof(struct virtio_gpu_transfer_host_3d) != 72); > ^ > > etc, because all the virtio_gpu definitions disappear from > include/standard-headers/linux/virtio_gpu.h. > > What's happening here? My guess: these will get into Linus's tree in a few days, and the update was done on a maintainer tree. Paolo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell 2015-11-05 11:49 ` Paolo Bonzini @ 2015-11-05 12:13 ` Gerd Hoffmann 2015-11-05 12:32 ` Peter Maydell 1 sibling, 1 reply; 26+ messages in thread From: Gerd Hoffmann @ 2015-11-05 12:13 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers Hi, > etc, because all the virtio_gpu definitions disappear from > include/standard-headers/linux/virtio_gpu.h. Updates not yet in mainline, they are sitting in drm-next and should land during the merge window (i.e. 4.4-rc1 should have them). I'd suggest to exclude virtio_gpu.h changes when updating linux headers for the time being. cheers, Gerd ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 12:13 ` Gerd Hoffmann @ 2015-11-05 12:32 ` Peter Maydell 2015-11-05 13:23 ` Christian Borntraeger ` (2 more replies) 0 siblings, 3 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-05 12:32 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: >> etc, because all the virtio_gpu definitions disappear from >> include/standard-headers/linux/virtio_gpu.h. > > Updates not yet in mainline, they are sitting in drm-next and should > land during the merge window (i.e. 4.4-rc1 should have them). > > I'd suggest to exclude virtio_gpu.h changes when updating linux headers > for the time being. I would strongly prefer it if we could get to a point where we can say "kernel headers must only be updated from this tree" and be guaranteed that it always works. This used to be true with the tree in question being kvm/next, but it doesn't seem to be so now. If it's going to be common that we have header changes that don't go via kvm/next, maybe we need to coordinate a tree that merges together the abi-guaranteed-stable changes from different places before they hit mainline? thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 12:32 ` Peter Maydell @ 2015-11-05 13:23 ` Christian Borntraeger 2015-11-05 13:44 ` Peter Maydell 2015-11-05 13:48 ` Laszlo Ersek 2015-11-05 14:42 ` Gerd Hoffmann 2 siblings, 1 reply; 26+ messages in thread From: Christian Borntraeger @ 2015-11-05 13:23 UTC (permalink / raw) To: Peter Maydell, Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers Am 05.11.2015 um 13:32 schrieb Peter Maydell: > On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: >>> etc, because all the virtio_gpu definitions disappear from >>> include/standard-headers/linux/virtio_gpu.h. >> >> Updates not yet in mainline, they are sitting in drm-next and should >> land during the merge window (i.e. 4.4-rc1 should have them). >> >> I'd suggest to exclude virtio_gpu.h changes when updating linux headers >> for the time being. > > I would strongly prefer it if we could get to a point where > we can say "kernel headers must only be updated from this tree" > and be guaranteed that it always works. This used to be true > with the tree in question being kvm/next, but it doesn't seem > to be so now. If it's going to be common that we have header > changes that don't go via kvm/next, maybe we need to coordinate > a tree that merges together the abi-guaranteed-stable changes > from different places before they hit mainline? Yes I agree. In the end this might imply to only use Linus tree to merge things in qemu. This is probably fine regarding the Linux tree, as it has a 63/70 day cadence and we only synced from kvm/next after rc4 or so. So this might defer for 4,5, or 6 weeks. One problem why people won't like that is that the QEMU releases are not often enough, though, as missing the deadline will defer the feature for 4 or 5 month. 2014-04-17 Tag v2.0.0 2014-08-01 Tag v2.1.0 2014-12-09 Tag v2.2.0 2015-04-24 Tag v2.3.0 2015-08-11 Tag v2.4.0 2015-12-10 Tag v2.5.0 Maybe we should try to aim for a 3 month cadence for QEMU to reduce the pain of missing soft/hard freeze. -> maybe a topic for the next QEMU summit? Christian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 13:23 ` Christian Borntraeger @ 2015-11-05 13:44 ` Peter Maydell 2015-11-05 13:46 ` Christian Borntraeger 2015-11-05 14:01 ` Paolo Bonzini 0 siblings, 2 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-05 13:44 UTC (permalink / raw) To: Christian Borntraeger; +Cc: Paolo Bonzini, Gerd Hoffmann, QEMU Developers On 5 November 2015 at 13:23, Christian Borntraeger <borntraeger@de.ibm.com> wrote: > Maybe we should try to aim for a 3 month cadence for QEMU to reduce the > pain of missing soft/hard freeze. -> maybe a topic for the next QEMU > summit? We used to have 3 month releases, but I think we found that these were too short: If you have about a month hardfreeze and a month softfreeze then you only have a month of open feature development, which seems pretty short to me. thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 13:44 ` Peter Maydell @ 2015-11-05 13:46 ` Christian Borntraeger 2015-11-05 14:01 ` Paolo Bonzini 1 sibling, 0 replies; 26+ messages in thread From: Christian Borntraeger @ 2015-11-05 13:46 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, Gerd Hoffmann, QEMU Developers Am 05.11.2015 um 14:44 schrieb Peter Maydell: > On 5 November 2015 at 13:23, Christian Borntraeger > <borntraeger@de.ibm.com> wrote: >> Maybe we should try to aim for a 3 month cadence for QEMU to reduce the >> pain of missing soft/hard freeze. -> maybe a topic for the next QEMU >> summit? > > We used to have 3 month releases, but I think we found that these > were too short: If you have about a month hardfreeze and a month > softfreeze then you only have a month of open feature development, > which seems pretty short to me. Yes, the overall process has to change for a 3 month cadence. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 13:44 ` Peter Maydell 2015-11-05 13:46 ` Christian Borntraeger @ 2015-11-05 14:01 ` Paolo Bonzini 2015-11-05 14:30 ` Peter Maydell 1 sibling, 1 reply; 26+ messages in thread From: Paolo Bonzini @ 2015-11-05 14:01 UTC (permalink / raw) To: Peter Maydell, Christian Borntraeger; +Cc: Gerd Hoffmann, QEMU Developers On 05/11/2015 14:44, Peter Maydell wrote: > On 5 November 2015 at 13:23, Christian Borntraeger > <borntraeger@de.ibm.com> wrote: >> Maybe we should try to aim for a 3 month cadence for QEMU to reduce the >> pain of missing soft/hard freeze. -> maybe a topic for the next QEMU >> summit? > > We used to have 3 month releases, but I think we found that these > were too short: If you have about a month hardfreeze and a month > softfreeze then you only have a month of open feature development, > which seems pretty short to me. It's short unless you assume that development goes on in private maintainer trees during hard freeze. It's always happened at least for the block layer, and probably for everything else. About 400 commits in 2.5 were authored before the 2.4 release. The main issue is that shorter cycles may mean fewer and bigger pull requests. It also means more awareness of conflicts is needed. We definitely lack the continuous integration infrastructure that is needed for that. Paolo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:01 ` Paolo Bonzini @ 2015-11-05 14:30 ` Peter Maydell 2015-11-05 14:52 ` Peter Maydell 2015-11-05 14:58 ` Paolo Bonzini 0 siblings, 2 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-05 14:30 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers On 5 November 2015 at 14:01, Paolo Bonzini <pbonzini@redhat.com> wrote: > On 05/11/2015 14:44, Peter Maydell wrote: >> We used to have 3 month releases, but I think we found that these >> were too short: If you have about a month hardfreeze and a month >> softfreeze then you only have a month of open feature development, >> which seems pretty short to me. > > It's short unless you assume that development goes on in private > maintainer trees during hard freeze. It's always happened at least for > the block layer, and probably for everything else. About 400 commits in > 2.5 were authored before the 2.4 release. Mostly I don't have a for-the-next-release tree for target-arm during freeze unless I completely can't avoid it, because it's a pain to maintain both. I suspect at least some of the other subsystems work the same way. People can develop during freeze, and post patches to the list, but whether they get reviewed or picked up into submaintainer trees is a different question. > The main issue is that shorter cycles may mean fewer and bigger pull > requests. It also means more awareness of conflicts is needed. We > definitely lack the continuous integration infrastructure that is needed > for that. I think it works for the kernel because the different subsystems have large communities of their own and the subtrees get a reasonable amount of testing as a result, plus there are efforts like linux-next to pre-check subtrees for conflicts before an actual merge attempt happens. I don't think the QEMU community is big enough for the kernel's dev practices to be reasonably applicable to us. thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:30 ` Peter Maydell @ 2015-11-05 14:52 ` Peter Maydell 2015-11-05 15:03 ` Paolo Bonzini 2015-11-05 14:58 ` Paolo Bonzini 1 sibling, 1 reply; 26+ messages in thread From: Peter Maydell @ 2015-11-05 14:52 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers On 5 November 2015 at 14:30, Peter Maydell <peter.maydell@linaro.org> wrote: > On 5 November 2015 at 14:01, Paolo Bonzini <pbonzini@redhat.com> wrote: >> On 05/11/2015 14:44, Peter Maydell wrote: >>> We used to have 3 month releases, but I think we found that these >>> were too short: If you have about a month hardfreeze and a month >>> softfreeze then you only have a month of open feature development, >>> which seems pretty short to me. >> >> It's short unless you assume that development goes on in private >> maintainer trees during hard freeze. It's always happened at least for >> the block layer, and probably for everything else. About 400 commits in >> 2.5 were authored before the 2.4 release. > > Mostly I don't have a for-the-next-release tree for target-arm > during freeze unless I completely can't avoid it, because it's > a pain to maintain both. ...also speaking as the person who does the wrangling of freezes and rc candidate tags and so on, three times a year is quite enough :-) thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:52 ` Peter Maydell @ 2015-11-05 15:03 ` Paolo Bonzini 0 siblings, 0 replies; 26+ messages in thread From: Paolo Bonzini @ 2015-11-05 15:03 UTC (permalink / raw) To: Peter Maydell; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers On 05/11/2015 15:52, Peter Maydell wrote: > ...also speaking as the person who does the wrangling of freezes and > rc candidate tags and so on, three times a year is quite enough :-) That's actually a good reason. On the other hand, we could look for a new policy that makes your work shorter during freezes. We could also find a second committer when you're on vacation, though that would be harder. 4 months (120 days = 17 weeks) does have an advantage, in that it's pretty much synced with the kernel's 9 week cycle if we insert a "leap week" here and there. That theoretically means there's no reason to sync headers from anything other than mainline. Paolo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:30 ` Peter Maydell 2015-11-05 14:52 ` Peter Maydell @ 2015-11-05 14:58 ` Paolo Bonzini 2015-11-05 18:56 ` Peter Maydell 1 sibling, 1 reply; 26+ messages in thread From: Paolo Bonzini @ 2015-11-05 14:58 UTC (permalink / raw) To: Peter Maydell; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers On 05/11/2015 15:30, Peter Maydell wrote: > I suspect at least some of the other subsystems work the same way. I try not to have a for-this-release tree during hard freeze. :) > > The main issue is that shorter cycles may mean fewer and bigger pull > > requests. It also means more awareness of conflicts is needed. We > > definitely lack the continuous integration infrastructure that is needed > > for that. > > I think it works for the kernel because the different subsystems > have large communities of their own and the subtrees get a > reasonable amount of testing as a result, plus there are > efforts like linux-next to pre-check subtrees for conflicts > before an actual merge attempt happens. Yes, that's what I meant for continuous integration. My hunch is that conflicts outside .json or trace-events are rare. It would be an interesting experiment if someone was willing to prepare a daily or bi-weekly (Mon/Thu) "qemu-next" branch for a few weeks during the 2.6 development. > I don't think the QEMU > community is big enough for the kernel's dev practices to be > reasonably applicable to us. On the other hand, something like "qemu-next" would be much easier to use daily than linux-next. I think the main issue is that right now we have a very long freeze period. It would be nice to know why (e.g. what kind of bugs are fixed? are they release blockers only?) and whether a shorter development period could also lead to a shorter hard freeze period. Perhaps even 2-ish months, for example it could be 1 month development (4.5 weeks) + 2 weeks to rc0 + 3.5 weeks to final (i.e. aim for final equal to -rc3). Or even change soft freeze to "time to respin pending pull requests if they fail" (i.e. *pull requests* must be on the list, not patches!) and shorten it to 1 week. That would give 4.5 weeks development + 1 week to rc0 + 3.5 weeks to final. This is still very different from the kernel's merge window model. OTOH with such a 2 months cadence we could probably get rid of stable releases altogether, limiting them to security issues. So, there's room for experimenting. That said, we probably agree that 4 months and staying on time is better than saying officially 3 months and delaying every release. Paolo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:58 ` Paolo Bonzini @ 2015-11-05 18:56 ` Peter Maydell 0 siblings, 0 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-05 18:56 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Christian Borntraeger, Gerd Hoffmann, QEMU Developers On 5 November 2015 at 14:58, Paolo Bonzini <pbonzini@redhat.com> wrote: > I think the main issue is that right now we have a very long freeze > period. It would be nice to know why (e.g. what kind of bugs are fixed? > are they release blockers only?) and whether a shorter development > period could also lead to a shorter hard freeze period. My gut feeling is that what typically happens is this: * at or just after softfreeze deadline, we apply a bunch of pull requests for big features * during softfreeze there's usually one big thing that hits middling-to-late, and so in the last week or so we need to scrabble around fixing up the stuff that broke when these features hit so that rc0 isn't too awful [this cycle, that's looking like it might be record-and-replay] * hardfreeze is when other people than the core dev community think about testing stuff, and we get the resulting bug reports * there's always one or two last release-critical issues that don't get fixed until very late. This is why hardfreeze always slips by an extra week or so (and the last week of hardfreeze is always "no changes between final rc and real release"). > Perhaps even 2-ish months, for example it could be 1 month development > (4.5 weeks) + 2 weeks to rc0 + 3.5 weeks to final (i.e. aim for final > equal to -rc3). > > Or even change soft freeze to "time to respin pending pull requests if > they fail" (i.e. *pull requests* must be on the list, not patches!) and > shorten it to 1 week. That would give 4.5 weeks development + 1 week to > rc0 + 3.5 weeks to final. I would be sceptical that we could get a non-broken rc0 if we put it that close to the thundering-herd of last minute feature pulls. thanks - PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 12:32 ` Peter Maydell 2015-11-05 13:23 ` Christian Borntraeger @ 2015-11-05 13:48 ` Laszlo Ersek 2015-11-05 15:52 ` Alex Bennée 2015-11-05 14:42 ` Gerd Hoffmann 2 siblings, 1 reply; 26+ messages in thread From: Laszlo Ersek @ 2015-11-05 13:48 UTC (permalink / raw) To: Peter Maydell, Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers On 11/05/15 13:32, Peter Maydell wrote: > On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: >>> etc, because all the virtio_gpu definitions disappear from >>> include/standard-headers/linux/virtio_gpu.h. >> >> Updates not yet in mainline, they are sitting in drm-next and should >> land during the merge window (i.e. 4.4-rc1 should have them). >> >> I'd suggest to exclude virtio_gpu.h changes when updating linux headers >> for the time being. > > I would strongly prefer it if we could get to a point where > we can say "kernel headers must only be updated from this tree" > and be guaranteed that it always works. This used to be true > with the tree in question being kvm/next, but it doesn't seem > to be so now. If it's going to be common that we have header > changes that don't go via kvm/next, maybe we need to coordinate > a tree that merges together the abi-guaranteed-stable changes > from different places before they hit mainline? I've always frowned upon importing headers from one project to another project. First, they can have different coding styles. (Case in point.) Second, not everything that needs to be defined for the original project is useful to the receiving project, and I find such cruft in the receiving project very annoying. Third, in some cases it might even raise licensing questions. If it is an ABI, it should be specified in text format somewhere, and then the projects can have their independent type definitions, macros etc that implement the spec. Laszlo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 13:48 ` Laszlo Ersek @ 2015-11-05 15:52 ` Alex Bennée 2015-11-05 17:05 ` Laszlo Ersek 0 siblings, 1 reply; 26+ messages in thread From: Alex Bennée @ 2015-11-05 15:52 UTC (permalink / raw) To: Laszlo Ersek; +Cc: QEMU Developers, Peter Maydell, Gerd Hoffmann, Paolo Bonzini Laszlo Ersek <lersek@redhat.com> writes: > On 11/05/15 13:32, Peter Maydell wrote: >> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>> etc, because all the virtio_gpu definitions disappear from >>>> include/standard-headers/linux/virtio_gpu.h. >>> >>> Updates not yet in mainline, they are sitting in drm-next and should >>> land during the merge window (i.e. 4.4-rc1 should have them). >>> >>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers >>> for the time being. >> >> I would strongly prefer it if we could get to a point where >> we can say "kernel headers must only be updated from this tree" >> and be guaranteed that it always works. This used to be true >> with the tree in question being kvm/next, but it doesn't seem >> to be so now. If it's going to be common that we have header >> changes that don't go via kvm/next, maybe we need to coordinate >> a tree that merges together the abi-guaranteed-stable changes >> from different places before they hit mainline? > > I've always frowned upon importing headers from one project to another > project. First, they can have different coding styles. (Case in point.) > Second, not everything that needs to be defined for the original project > is useful to the receiving project, and I find such cruft in the > receiving project very annoying. Third, in some cases it might even > raise licensing questions. > > If it is an ABI, it should be specified in text format somewhere, and > then the projects can have their independent type definitions, macros > etc that implement the spec. Surely the kernel uapi includes is that textual format. Once stuff is accepted into the master kernel tree there is your stable API. FWIW the few bits I've done that involve syncing headers I basically carry a [DEV] patch in my QEMU tree until the feature is finally merged in the kernel and then I do my final re-base against the official tree state. -- Alex Bennée ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 15:52 ` Alex Bennée @ 2015-11-05 17:05 ` Laszlo Ersek 2015-11-05 17:09 ` Paolo Bonzini 0 siblings, 1 reply; 26+ messages in thread From: Laszlo Ersek @ 2015-11-05 17:05 UTC (permalink / raw) To: Alex Bennée Cc: Peter Maydell, Paolo Bonzini, QEMU Developers, Gerd Hoffmann On 11/05/15 16:52, Alex Bennée wrote: > > Laszlo Ersek <lersek@redhat.com> writes: > >> On 11/05/15 13:32, Peter Maydell wrote: >>> On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>>> etc, because all the virtio_gpu definitions disappear from >>>>> include/standard-headers/linux/virtio_gpu.h. >>>> >>>> Updates not yet in mainline, they are sitting in drm-next and should >>>> land during the merge window (i.e. 4.4-rc1 should have them). >>>> >>>> I'd suggest to exclude virtio_gpu.h changes when updating linux headers >>>> for the time being. >>> >>> I would strongly prefer it if we could get to a point where >>> we can say "kernel headers must only be updated from this tree" >>> and be guaranteed that it always works. This used to be true >>> with the tree in question being kvm/next, but it doesn't seem >>> to be so now. If it's going to be common that we have header >>> changes that don't go via kvm/next, maybe we need to coordinate >>> a tree that merges together the abi-guaranteed-stable changes >>> from different places before they hit mainline? >> >> I've always frowned upon importing headers from one project to another >> project. First, they can have different coding styles. (Case in point.) >> Second, not everything that needs to be defined for the original project >> is useful to the receiving project, and I find such cruft in the >> receiving project very annoying. Third, in some cases it might even >> raise licensing questions. >> >> If it is an ABI, it should be specified in text format somewhere, and >> then the projects can have their independent type definitions, macros >> etc that implement the spec. > > Surely the kernel uapi includes is that textual format. Once stuff is > accepted into the master kernel tree there is your stable API. What is uapi for? If it is for QEMU (the userspace process) to consume the host kernel's services, then I agree. If uapi is for the guest kernel to consume (= drive) QEMU's virtual hardware, then I strongly disagree. In that case Linux is just one of the possible guests that can drive that hardware. (Side point: and QEMU is just one of the emulators / hypervisors that can provide that hardware. Which is why the actual hardware description should exist independently of both.) I'll elaborate elsewhere in the thread. Thanks Laszlo > FWIW the few bits I've done that involve syncing headers I basically > carry a [DEV] patch in my QEMU tree until the feature is finally merged > in the kernel and then I do my final re-base against the official tree > state. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 17:05 ` Laszlo Ersek @ 2015-11-05 17:09 ` Paolo Bonzini 0 siblings, 0 replies; 26+ messages in thread From: Paolo Bonzini @ 2015-11-05 17:09 UTC (permalink / raw) To: Laszlo Ersek, Alex Bennée Cc: Peter Maydell, QEMU Developers, Gerd Hoffmann On 05/11/2015 18:05, Laszlo Ersek wrote: >> > Surely the kernel uapi includes is that textual format. Once stuff is >> > accepted into the master kernel tree there is your stable API. > What is uapi for? If it is for QEMU (the userspace process) to consume > the host kernel's services, then I agree. uapi = Userspace API. > If uapi is for the guest kernel to consume (= drive) QEMU's virtual > hardware, then I strongly disagree. In that case Linux is just one of > the possible guests that can drive that hardware. Kernel-only headers can and will include uapi headers, though not the other way round. So there can be indeed a case where a typo in uapi headers causes both QEMU and the guest kernel to deviate from the spec. > (Side point: and QEMU is just one of the emulators / hypervisors that > can provide that hardware. Which is why the actual hardware description > should exist independently of both.) Typically it exists as an industry standard (e.g. the virtio specification) or a datasheet. Paolo > I'll elaborate elsewhere in the thread. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 12:32 ` Peter Maydell 2015-11-05 13:23 ` Christian Borntraeger 2015-11-05 13:48 ` Laszlo Ersek @ 2015-11-05 14:42 ` Gerd Hoffmann 2015-11-05 14:45 ` Peter Maydell 2 siblings, 1 reply; 26+ messages in thread From: Gerd Hoffmann @ 2015-11-05 14:42 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers On Do, 2015-11-05 at 12:32 +0000, Peter Maydell wrote: > On 5 November 2015 at 12:13, Gerd Hoffmann <kraxel@redhat.com> wrote: > >> etc, because all the virtio_gpu definitions disappear from > >> include/standard-headers/linux/virtio_gpu.h. > > > > Updates not yet in mainline, they are sitting in drm-next and should > > land during the merge window (i.e. 4.4-rc1 should have them). > > > > I'd suggest to exclude virtio_gpu.h changes when updating linux headers > > for the time being. > > I would strongly prefer it if we could get to a point where > we can say "kernel headers must only be updated from this tree" > and be guaranteed that it always works. This used to be true > with the tree in question being kvm/next, but it doesn't seem > to be so now. If it's going to be common that we have header > changes that don't go via kvm/next, maybe we need to coordinate > a tree that merges together the abi-guaranteed-stable changes > from different places before they hit mainline? Chicken & egg issue in that case because airlied (linux kernel drm maintainer) asked to have the qemu changes merged before taking the virtio-gpu pull request. So I had no other chance than creating the patches with not-yet upstream virtio header changes ... cheers, Gerd ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:42 ` Gerd Hoffmann @ 2015-11-05 14:45 ` Peter Maydell 2015-11-05 14:58 ` Gerd Hoffmann 0 siblings, 1 reply; 26+ messages in thread From: Peter Maydell @ 2015-11-05 14:45 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: > Chicken & egg issue in that case because airlied (linux kernel drm > maintainer) asked to have the qemu changes merged before taking the > virtio-gpu pull request. So I had no other chance than creating the > patches with not-yet upstream virtio header changes ... Hmm. If I'd realised that at the time I'd have pushed back on it. We should never take code that relies on upstream kernel ABI that hasn't been accepted by the maintainer yet. thanks - PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:45 ` Peter Maydell @ 2015-11-05 14:58 ` Gerd Hoffmann 2015-11-05 15:11 ` Peter Maydell 0 siblings, 1 reply; 26+ messages in thread From: Gerd Hoffmann @ 2015-11-05 14:58 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: > On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: > > Chicken & egg issue in that case because airlied (linux kernel drm > > maintainer) asked to have the qemu changes merged before taking the > > virtio-gpu pull request. So I had no other chance than creating the > > patches with not-yet upstream virtio header changes ... > > Hmm. If I'd realised that at the time I'd have pushed back on it. > We should never take code that relies on upstream kernel > ABI that hasn't been accepted by the maintainer yet. The reason airlied asked for qemu being upstream first is to avoid having code in the kernel tree not accepted by qemu yet ... So, one of the two has to go first ;) When I did the pull requests patches had passed review on both qemu-devel (qemu side) and dri-devel (kernel side), so this was really just a formal "wait for qemu" thing with a very low risk of ABI changes, and airlied pulled it into drm-next only days after it landed in qemu/master. cheers, Gerd ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 14:58 ` Gerd Hoffmann @ 2015-11-05 15:11 ` Peter Maydell 2015-11-05 17:15 ` Laszlo Ersek 0 siblings, 1 reply; 26+ messages in thread From: Peter Maydell @ 2015-11-05 15:11 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: Paolo Bonzini, QEMU Developers On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote: > On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: >> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: >> > Chicken & egg issue in that case because airlied (linux kernel drm >> > maintainer) asked to have the qemu changes merged before taking the >> > virtio-gpu pull request. So I had no other chance than creating the >> > patches with not-yet upstream virtio header changes ... >> >> Hmm. If I'd realised that at the time I'd have pushed back on it. >> We should never take code that relies on upstream kernel >> ABI that hasn't been accepted by the maintainer yet. > > The reason airlied asked for qemu being upstream first is to avoid > having code in the kernel tree not accepted by qemu yet ... > > So, one of the two has to go first ;) Right, but this isn't a symmetrical arrangement. If on the kernel side the ABI is changed before it's finally accepted into mainline, that means that any QEMU that got released with changes made on the basis of not-yet-upstream kernel patches will be broken. But if the kernel accepts code which needs a not-yet-in-qemu feature to be useful, that doesn't break the kernel, because the kernel isn't actually dependent on the QEMU changes. That's why I think the kernel side of the ABI always needs to go first (the kernel provides the ABI, QEMU uses it, never the other way around). thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 15:11 ` Peter Maydell @ 2015-11-05 17:15 ` Laszlo Ersek 2015-11-05 18:13 ` Cornelia Huck ` (2 more replies) 0 siblings, 3 replies; 26+ messages in thread From: Laszlo Ersek @ 2015-11-05 17:15 UTC (permalink / raw) To: Peter Maydell, Gerd Hoffmann Cc: Paolo Bonzini, Alex Bennée, QEMU Developers On 11/05/15 16:11, Peter Maydell wrote: > On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote: >> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: >>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>> Chicken & egg issue in that case because airlied (linux kernel drm >>>> maintainer) asked to have the qemu changes merged before taking the >>>> virtio-gpu pull request. So I had no other chance than creating the >>>> patches with not-yet upstream virtio header changes ... >>> >>> Hmm. If I'd realised that at the time I'd have pushed back on it. >>> We should never take code that relies on upstream kernel >>> ABI that hasn't been accepted by the maintainer yet. >> >> The reason airlied asked for qemu being upstream first is to avoid >> having code in the kernel tree not accepted by qemu yet ... >> >> So, one of the two has to go first ;) > > Right, but this isn't a symmetrical arrangement. If on the > kernel side the ABI is changed before it's finally accepted > into mainline, that means that any QEMU that got released with > changes made on the basis of not-yet-upstream kernel patches > will be broken. But if the kernel accepts code which needs a > not-yet-in-qemu feature to be useful, that doesn't break the > kernel, because the kernel isn't actually dependent on the > QEMU changes. That's why I think the kernel side of the ABI > always needs to go first (the kernel provides the ABI, QEMU > uses it, never the other way around). I could be confused about what ABI means, but in case of *guest* kernel drivers (for virtual devices provided by QEMU), the dependency seems to be the opposite. Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't know what "uapi" means here. Are those headers there to be relied upon by userspace processes that need services from the kernel? That would match your above argument. However, I found the following files there: include/uapi/linux/virtio_console.h include/uapi/linux/virtio_input.h include/uapi/linux/virtio_net.h include/uapi/linux/virtio_types.h include/uapi/linux/virtio_9p.h include/uapi/linux/virtio_balloon.h include/uapi/linux/virtio_scsi.h include/uapi/linux/virtio_rng.h include/uapi/linux/virtio_ids.h include/uapi/linux/virtio_ring.h include/uapi/linux/virtio_pci.h include/uapi/linux/virtio_gpu.h include/uapi/linux/virtio_config.h include/uapi/linux/virtio_blk.h Which kinda confuses me. I cannot imagine that a userspace process should depend on these, for the purpose of consuming kernel services. Instead, QEMU should *definitely* dictate in this case, because it provides the service (the hardware), and Linux has drivers that consume that service. Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too. Should QEMU include their header files as well? I think not. In other words, I agree with your argument, and the "QEMU depends on the kernel" statement, as far as QEMU plays the role of a userspace process that consumes the *host* kernel's services (VFIO, KVM, ...). However, when QEMU provides the *hardware* (which is hopefully described by an industry standard, or at least by some kind of independent documentation), then the *guest* kernel plays the consumer role (as do other non-Linux-kernel guests), and in that case the QEMU changes should go in first (hopefully after testing them with the under-development guest code, of course). Hence, I'm leaning to think that the above virtio header files should not be under uapi *at all*. They should be under "include/virtio", and should be private to the guest drivers. --*-- I think we had the same discussion when Wei was working on SMBIOS 3.0 for ARM. In <http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>, you asked "Is support for this all in the mainline kernel yet?" I didn't really understand that question -- it didn't matter. QEMU as a *platform* was learning how to provide a service that was governed by an industry standard. The status of *guest* kernel support (and of guest utility support) was irrelevant, in my opinion. Thanks Laszlo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 17:15 ` Laszlo Ersek @ 2015-11-05 18:13 ` Cornelia Huck 2015-11-05 18:51 ` Laszlo Ersek 2015-11-06 16:34 ` Alex Bennée 2015-11-06 16:43 ` Peter Maydell 2 siblings, 1 reply; 26+ messages in thread From: Cornelia Huck @ 2015-11-05 18:13 UTC (permalink / raw) To: Laszlo Ersek Cc: QEMU Developers, Peter Maydell, Alex Bennée, Gerd Hoffmann, Paolo Bonzini On Thu, 5 Nov 2015 18:15:04 +0100 Laszlo Ersek <lersek@redhat.com> wrote: > On 11/05/15 16:11, Peter Maydell wrote: > > On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote: > >> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: > >>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: > >>>> Chicken & egg issue in that case because airlied (linux kernel drm > >>>> maintainer) asked to have the qemu changes merged before taking the > >>>> virtio-gpu pull request. So I had no other chance than creating the > >>>> patches with not-yet upstream virtio header changes ... > >>> > >>> Hmm. If I'd realised that at the time I'd have pushed back on it. > >>> We should never take code that relies on upstream kernel > >>> ABI that hasn't been accepted by the maintainer yet. > >> > >> The reason airlied asked for qemu being upstream first is to avoid > >> having code in the kernel tree not accepted by qemu yet ... > >> > >> So, one of the two has to go first ;) > > > > Right, but this isn't a symmetrical arrangement. If on the > > kernel side the ABI is changed before it's finally accepted > > into mainline, that means that any QEMU that got released with > > changes made on the basis of not-yet-upstream kernel patches > > will be broken. But if the kernel accepts code which needs a > > not-yet-in-qemu feature to be useful, that doesn't break the > > kernel, because the kernel isn't actually dependent on the > > QEMU changes. That's why I think the kernel side of the ABI > > always needs to go first (the kernel provides the ABI, QEMU > > uses it, never the other way around). > > I could be confused about what ABI means, but in case of *guest* kernel > drivers (for virtual devices provided by QEMU), the dependency seems to > be the opposite. > > Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't > know what "uapi" means here. Are those headers there to be relied upon > by userspace processes that need services from the kernel? That would > match your above argument. > > However, I found the following files there: > > include/uapi/linux/virtio_console.h > include/uapi/linux/virtio_input.h > include/uapi/linux/virtio_net.h > include/uapi/linux/virtio_types.h > include/uapi/linux/virtio_9p.h > include/uapi/linux/virtio_balloon.h > include/uapi/linux/virtio_scsi.h > include/uapi/linux/virtio_rng.h > include/uapi/linux/virtio_ids.h > include/uapi/linux/virtio_ring.h > include/uapi/linux/virtio_pci.h > include/uapi/linux/virtio_gpu.h > include/uapi/linux/virtio_config.h > include/uapi/linux/virtio_blk.h > > Which kinda confuses me. I cannot imagine that a userspace process > should depend on these, for the purpose of consuming kernel services. > Instead, QEMU should *definitely* dictate in this case, because it > provides the service (the hardware), and Linux has drivers that consume > that service. > > Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too. > Should QEMU include their header files as well? I think not. > > In other words, I agree with your argument, and the "QEMU depends on the > kernel" statement, as far as QEMU plays the role of a userspace process > that consumes the *host* kernel's services (VFIO, KVM, ...). > > However, when QEMU provides the *hardware* (which is hopefully described > by an industry standard, or at least by some kind of independent > documentation), then the *guest* kernel plays the consumer role (as do > other non-Linux-kernel guests), and in that case the QEMU changes should > go in first (hopefully after testing them with the under-development > guest code, of course). > > Hence, I'm leaning to think that the above virtio header files should > not be under uapi *at all*. They should be under "include/virtio", and > should be private to the guest drivers. The virtio case is more of a convenience issue. The *canonical* definitions for virtio implementations are neither in the kernel nor in qemu, but only in the standard. We can't conveniently pull headers (for Linux and qemu) from a third place, so we make sure that the Linux headers implement the standard and have qemu pull them (this is probably also a corollary of mostly the same people working on the Linux and qemu implementations). IOW, I think in the virtio case *both* Linux and qemu are consumers. The setup is as-is as it seems to be most convenient for the developers involved. (There has been discussion on this before, which I'm too lazy to dig up right now.) ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 18:13 ` Cornelia Huck @ 2015-11-05 18:51 ` Laszlo Ersek 0 siblings, 0 replies; 26+ messages in thread From: Laszlo Ersek @ 2015-11-05 18:51 UTC (permalink / raw) To: Cornelia Huck Cc: QEMU Developers, Peter Maydell, Alex Bennée, Gerd Hoffmann, Paolo Bonzini On 11/05/15 19:13, Cornelia Huck wrote: > On Thu, 5 Nov 2015 18:15:04 +0100 > Laszlo Ersek <lersek@redhat.com> wrote: > >> On 11/05/15 16:11, Peter Maydell wrote: >>> On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: >>>>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>>>> Chicken & egg issue in that case because airlied (linux kernel drm >>>>>> maintainer) asked to have the qemu changes merged before taking the >>>>>> virtio-gpu pull request. So I had no other chance than creating the >>>>>> patches with not-yet upstream virtio header changes ... >>>>> >>>>> Hmm. If I'd realised that at the time I'd have pushed back on it. >>>>> We should never take code that relies on upstream kernel >>>>> ABI that hasn't been accepted by the maintainer yet. >>>> >>>> The reason airlied asked for qemu being upstream first is to avoid >>>> having code in the kernel tree not accepted by qemu yet ... >>>> >>>> So, one of the two has to go first ;) >>> >>> Right, but this isn't a symmetrical arrangement. If on the >>> kernel side the ABI is changed before it's finally accepted >>> into mainline, that means that any QEMU that got released with >>> changes made on the basis of not-yet-upstream kernel patches >>> will be broken. But if the kernel accepts code which needs a >>> not-yet-in-qemu feature to be useful, that doesn't break the >>> kernel, because the kernel isn't actually dependent on the >>> QEMU changes. That's why I think the kernel side of the ABI >>> always needs to go first (the kernel provides the ABI, QEMU >>> uses it, never the other way around). >> >> I could be confused about what ABI means, but in case of *guest* kernel >> drivers (for virtual devices provided by QEMU), the dependency seems to >> be the opposite. >> >> Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't >> know what "uapi" means here. Are those headers there to be relied upon >> by userspace processes that need services from the kernel? That would >> match your above argument. >> >> However, I found the following files there: >> >> include/uapi/linux/virtio_console.h >> include/uapi/linux/virtio_input.h >> include/uapi/linux/virtio_net.h >> include/uapi/linux/virtio_types.h >> include/uapi/linux/virtio_9p.h >> include/uapi/linux/virtio_balloon.h >> include/uapi/linux/virtio_scsi.h >> include/uapi/linux/virtio_rng.h >> include/uapi/linux/virtio_ids.h >> include/uapi/linux/virtio_ring.h >> include/uapi/linux/virtio_pci.h >> include/uapi/linux/virtio_gpu.h >> include/uapi/linux/virtio_config.h >> include/uapi/linux/virtio_blk.h >> >> Which kinda confuses me. I cannot imagine that a userspace process >> should depend on these, for the purpose of consuming kernel services. >> Instead, QEMU should *definitely* dictate in this case, because it >> provides the service (the hardware), and Linux has drivers that consume >> that service. >> >> Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too. >> Should QEMU include their header files as well? I think not. >> >> In other words, I agree with your argument, and the "QEMU depends on the >> kernel" statement, as far as QEMU plays the role of a userspace process >> that consumes the *host* kernel's services (VFIO, KVM, ...). >> >> However, when QEMU provides the *hardware* (which is hopefully described >> by an industry standard, or at least by some kind of independent >> documentation), then the *guest* kernel plays the consumer role (as do >> other non-Linux-kernel guests), and in that case the QEMU changes should >> go in first (hopefully after testing them with the under-development >> guest code, of course). >> >> Hence, I'm leaning to think that the above virtio header files should >> not be under uapi *at all*. They should be under "include/virtio", and >> should be private to the guest drivers. > > The virtio case is more of a convenience issue. > > The *canonical* definitions for virtio implementations are neither in > the kernel nor in qemu, but only in the standard. We can't conveniently > pull headers (for Linux and qemu) from a third place, so we make sure > that the Linux headers implement the standard and have qemu pull them My argument is that this convenience is a bad thing, because it introduces cross-project dependencies (and misleading uses of uapi per se) that are not warranted for by the roles the projects play. The convenience the current workflow promises is actually a pain. These (non-)dependency arguments surface every time something new is implemented that requires both host and guest support. QEMU and the Linux kernel (and all other guests) should maintain their own independent sets of definitions. > (this is probably also a corollary of mostly the same people working on > the Linux and qemu implementations). The convenience sync also allows contributors to conveniently forget about other guest code beside Linux. "It's in Linux and in QEMU, I'm done." That is alright, unless the contributor's primary guest code actually has a (not so obvious) dependency on that other guest code. > IOW, I think in the virtio case *both* Linux and qemu are consumers. In the actual service consumption sense, I don't think so; but as far as "supporting the virtio standard" is concerned, I agree fully. The point is, cross-project dependencies should only be dictated by the former kind -- actual service consumption. Given that QEMU and Linux are separate, independent "consumers" of the virtio spec (in the latter sense), each should have an independent set of definitions for compliance with the spec. > The setup is as-is as it seems to be most convenient for the developers > involved. I agree. Until Peter asks if the header changes are in Linux yet, and the contributor responds that the kernel maintainer wanted to see the changes in QEMU first. > (There has been discussion on this before, which I'm too lazy to dig up > right now.) > I apologize for dredging it up then. I've not been aware of the earlier discussion. But, it does show that the problem remains unsolved, and a pain. ... I guess my main reason for obsessing about this is because I manually code up all industry standard-related macros and types in edk2, from the revelant specs directly. It gives me incredible freedom; the code conforms to the coding style; the definitions are as minimal as possible; and I regularly gain insight from typing them and laying them out. Sometimes they even help me find undocumented restrictions / limitations in the interfacing project. Yes, with the separate definitions, new features require more typing. That's a good thing. Anyway, thanks for replying; I guess I've annoyed most people following this thread quite a bit by now. I shall shut up. Cheers Laszlo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 17:15 ` Laszlo Ersek 2015-11-05 18:13 ` Cornelia Huck @ 2015-11-06 16:34 ` Alex Bennée 2015-11-06 16:43 ` Peter Maydell 2 siblings, 0 replies; 26+ messages in thread From: Alex Bennée @ 2015-11-06 16:34 UTC (permalink / raw) To: Laszlo Ersek; +Cc: QEMU Developers, Peter Maydell, Gerd Hoffmann, Paolo Bonzini Laszlo Ersek <lersek@redhat.com> writes: > On 11/05/15 16:11, Peter Maydell wrote: >> On 5 November 2015 at 14:58, Gerd Hoffmann <kraxel@redhat.com> wrote: >>> On Do, 2015-11-05 at 14:45 +0000, Peter Maydell wrote: >>>> On 5 November 2015 at 14:42, Gerd Hoffmann <kraxel@redhat.com> wrote: >>>>> Chicken & egg issue in that case because airlied (linux kernel drm >>>>> maintainer) asked to have the qemu changes merged before taking the >>>>> virtio-gpu pull request. So I had no other chance than creating the >>>>> patches with not-yet upstream virtio header changes ... >>>> >>>> Hmm. If I'd realised that at the time I'd have pushed back on it. >>>> We should never take code that relies on upstream kernel >>>> ABI that hasn't been accepted by the maintainer yet. >>> >>> The reason airlied asked for qemu being upstream first is to avoid >>> having code in the kernel tree not accepted by qemu yet ... >>> >>> So, one of the two has to go first ;) >> >> Right, but this isn't a symmetrical arrangement. If on the >> kernel side the ABI is changed before it's finally accepted >> into mainline, that means that any QEMU that got released with >> changes made on the basis of not-yet-upstream kernel patches >> will be broken. But if the kernel accepts code which needs a >> not-yet-in-qemu feature to be useful, that doesn't break the >> kernel, because the kernel isn't actually dependent on the >> QEMU changes. That's why I think the kernel side of the ABI >> always needs to go first (the kernel provides the ABI, QEMU >> uses it, never the other way around). > > I could be confused about what ABI means, but in case of *guest* kernel > drivers (for virtual devices provided by QEMU), the dependency seems to > be the opposite. Well sort of, for guests using virtio devices both the kernel and QEMU should be following the spec defined API (so guests being run under any random virtio hypervisor work the same). > Alex mentioned "kernel uapi includes" up-thread. To be honest, I don't > know what "uapi" means here. Are those headers there to be relied upon > by userspace processes that need services from the kernel? That would > match your above argument. In this case the Userspace API the kernel provides to userspace which QEMU is one user (in fact in the KVM kernel code anything that controls the hypervisor is just referred to as userspace). My bias is showing of course because I'm not considering guests here, QEMU is the user of the kernel API. I'm thinking of things like the KVM_RUN structure and the various ioctls and knobs for interaction between KVM and QEMU. > > However, I found the following files there: > > include/uapi/linux/virtio_console.h > include/uapi/linux/virtio_input.h > include/uapi/linux/virtio_net.h > include/uapi/linux/virtio_types.h > include/uapi/linux/virtio_9p.h > include/uapi/linux/virtio_balloon.h > include/uapi/linux/virtio_scsi.h > include/uapi/linux/virtio_rng.h > include/uapi/linux/virtio_ids.h > include/uapi/linux/virtio_ring.h > include/uapi/linux/virtio_pci.h > include/uapi/linux/virtio_gpu.h > include/uapi/linux/virtio_config.h > include/uapi/linux/virtio_blk.h > > Which kinda confuses me. I cannot imagine that a userspace process > should depend on these, for the purpose of consuming kernel services. > Instead, QEMU should *definitely* dictate in this case, because it > provides the service (the hardware), and Linux has drivers that consume > that service. > > Consider: there are virtio drivers in SeaBIOS, OVMF, and Windows too. > Should QEMU include their header files as well? I think not. > > In other words, I agree with your argument, and the "QEMU depends on the > kernel" statement, as far as QEMU plays the role of a userspace process > that consumes the *host* kernel's services (VFIO, KVM, ...). > > However, when QEMU provides the *hardware* (which is hopefully described > by an industry standard, or at least by some kind of independent > documentation), then the *guest* kernel plays the consumer role (as do > other non-Linux-kernel guests), and in that case the QEMU changes should > go in first (hopefully after testing them with the under-development > guest code, of course). > > Hence, I'm leaning to think that the above virtio header files should > not be under uapi *at all*. They should be under "include/virtio", and > should be private to the guest drivers. That's an interesting point. I suspect in this case QEMU is using the Linux defined headers as a handy implementation of the virtio specs. > > --*-- > > I think we had the same discussion when Wei was working on SMBIOS 3.0 > for ARM. In > <http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>, > you asked > > "Is support for this all in the mainline kernel yet?" > > I didn't really understand that question -- it didn't matter. QEMU as a > *platform* was learning how to provide a service that was governed by an > industry standard. The status of *guest* kernel support (and of guest > utility support) was irrelevant, in my opinion. No there are plenty of examples, for example in my case guest debug support requires a new kernel API which is only declared stable once it passes review and gets merged into mainline. -- Alex Bennée ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next 2015-11-05 17:15 ` Laszlo Ersek 2015-11-05 18:13 ` Cornelia Huck 2015-11-06 16:34 ` Alex Bennée @ 2015-11-06 16:43 ` Peter Maydell 2 siblings, 0 replies; 26+ messages in thread From: Peter Maydell @ 2015-11-06 16:43 UTC (permalink / raw) To: Laszlo Ersek Cc: Paolo Bonzini, Alex Bennée, Gerd Hoffmann, QEMU Developers On 5 November 2015 at 17:15, Laszlo Ersek <lersek@redhat.com> wrote: > --*-- > > I think we had the same discussion when Wei was working on SMBIOS 3.0 > for ARM. In > <http://thread.gmane.org/gmane.comp.emulators.qemu/353282/focus=354806>, > you asked > > "Is support for this all in the mainline kernel yet?" > > I didn't really understand that question -- it didn't matter. QEMU as a > *platform* was learning how to provide a service that was governed by an > industry standard. The status of *guest* kernel support (and of guest > utility support) was irrelevant, in my opinion. The reason for the question in that case was slightly different. As you say, it would be OK to implement support in either the kernel or QEMU first. I asked because: (1) I felt I had no understanding of the relevant spec to be able to correctly review the QEMU code changes in detail (2) I did trust that people on the kernel mailing list would be able to provide review at that level of detail (3) I was therefore trying to "outsource" the review to the kernel folks so I could rely more on "QEMU works with the kernel" as a check on the QEMU patches without falling into the trap of "both sets of patches have complementary bugs that cancel each other out". with a side order of (4) if a spec isn't important enough to get support in Linux then it's at least worth asking whether it's important enough to support in QEMU. thanks -- PMM ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2015-11-06 16:43 UTC | newest] Thread overview: 26+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-11-05 11:42 [Qemu-devel] virtio-gpu doesn't build if you do a linux-headers update from kvm/next Peter Maydell 2015-11-05 11:49 ` Paolo Bonzini 2015-11-05 12:13 ` Gerd Hoffmann 2015-11-05 12:32 ` Peter Maydell 2015-11-05 13:23 ` Christian Borntraeger 2015-11-05 13:44 ` Peter Maydell 2015-11-05 13:46 ` Christian Borntraeger 2015-11-05 14:01 ` Paolo Bonzini 2015-11-05 14:30 ` Peter Maydell 2015-11-05 14:52 ` Peter Maydell 2015-11-05 15:03 ` Paolo Bonzini 2015-11-05 14:58 ` Paolo Bonzini 2015-11-05 18:56 ` Peter Maydell 2015-11-05 13:48 ` Laszlo Ersek 2015-11-05 15:52 ` Alex Bennée 2015-11-05 17:05 ` Laszlo Ersek 2015-11-05 17:09 ` Paolo Bonzini 2015-11-05 14:42 ` Gerd Hoffmann 2015-11-05 14:45 ` Peter Maydell 2015-11-05 14:58 ` Gerd Hoffmann 2015-11-05 15:11 ` Peter Maydell 2015-11-05 17:15 ` Laszlo Ersek 2015-11-05 18:13 ` Cornelia Huck 2015-11-05 18:51 ` Laszlo Ersek 2015-11-06 16:34 ` Alex Bennée 2015-11-06 16:43 ` Peter Maydell
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).