Qemu first time contribution

Qemu first time contribution

[Harshavardhan Unnibhavi]

Hi,

I am looking to contribute code to QEMU, a software I have enjoyed
using for my own projects. I want to contribute to developing a
specific component in the project. I was looking for possible project
ideas and came across this page:
https://wiki.qemu.org/Google_Summer_of_Code_2020 . Is it ok to start
working on a project I pick from here? For example I am interested in
the " TCG Plugin Cache Modelling"  and "NVMe Emulation Performance
Optimization".

If not do you maintain a list of components(or issues) that require
some work or new features to be added?

Looking forward to your replies.

Best,
Harsha

Re: Qemu first time contribution

[Thomas Huth]

On 07/11/2020 20.18, Harshavardhan Unnibhavi wrote:
> Hi,
> 
> I am looking to contribute code to QEMU, a software I have enjoyed
> using for my own projects. I want to contribute to developing a
> specific component in the project. I was looking for possible project
> ideas and came across this page:
> https://wiki.qemu.org/Google_Summer_of_Code_2020 . Is it ok to start
> working on a project I pick from here? For example I am interested in
> the " TCG Plugin Cache Modelling"  and "NVMe Emulation Performance
> Optimization".
> 
> If not do you maintain a list of components(or issues) that require
> some work or new features to be added?
> 
> Looking forward to your replies.

 Hi,

thanks for your interest in QEMU! However, GSoC 2020 is already over, and
the projects for 2021 haven't been defined yet (as far as I know). But if
you want to have a try with a small task first, we keep a list of rather
small and easy tasks here:

https://wiki.qemu.org/Contribute/BiteSizedTasks

 HTH,
  Thomas

Re: Qemu first time contribution

[Harshavardhan Unnibhavi]

Hi,

Thank you for the reply! Yes, I understand that gsoc is over for 2020,
and projects for 2021 will come up next year. I was thinking of
contributing outside of gsoc(for which I won't be eligible anyways for
next year). Anyway, I will work on some of the bite sized tasks, and
get back to you for some other concrete project ideas that require
somebody to work on, in qemu.

Best,
Harsha

On Sun, Nov 8, 2020 at 8:44 AM Thomas Huth <thuth@redhat.com> wrote:
>
> On 07/11/2020 20.18, Harshavardhan Unnibhavi wrote:
> > Hi,
> >
> > I am looking to contribute code to QEMU, a software I have enjoyed
> > using for my own projects. I want to contribute to developing a
> > specific component in the project. I was looking for possible project
> > ideas and came across this page:
> > https://wiki.qemu.org/Google_Summer_of_Code_2020 . Is it ok to start
> > working on a project I pick from here? For example I am interested in
> > the " TCG Plugin Cache Modelling"  and "NVMe Emulation Performance
> > Optimization".
> >
> > If not do you maintain a list of components(or issues) that require
> > some work or new features to be added?
> >
> > Looking forward to your replies.
>
>  Hi,
>
> thanks for your interest in QEMU! However, GSoC 2020 is already over, and
> the projects for 2021 haven't been defined yet (as far as I know). But if
> you want to have a try with a small task first, we keep a list of rather
> small and easy tasks here:
>
> https://wiki.qemu.org/Contribute/BiteSizedTasks
>
>  HTH,
>   Thomas
>

Re: Qemu first time contribution

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]

On Sun, Nov 08, 2020 at 12:21:33PM +0000, Harshavardhan Unnibhavi wrote:
> Thank you for the reply! Yes, I understand that gsoc is over for 2020,
> and projects for 2021 will come up next year. I was thinking of
> contributing outside of gsoc(for which I won't be eligible anyways for
> next year). Anyway, I will work on some of the bite sized tasks, and
> get back to you for some other concrete project ideas that require
> somebody to work on, in qemu.

Hi Harsha,
Here is an idea you could explore:

The Linux AIO API was extended to support fsync(2)/fdatasync(2) in the
following commit from 2018:

  commit a3c0d439e4d92411c2b4b21a526a4de720d0806b
  Author: Christoph Hellwig <hch@lst.de>
  Date:   Tue Mar 27 19:18:57 2018 +0200

      aio: implement IOCB_CMD_FSYNC and IOCB_CMD_FDSYNC

QEMU's Linux AIO code does not take advantage of this feature yet.
Instead it invokes the traditional fdatasync(2) system call from a
thread pool because it assumes the Linux AIO API doesn't support the
operation. The function where this happens is
block/file-posix.c:raw_co_flush_to_disk().

The goal is to implement IO_CMD_FDSYNC support in block/linux-aio.c
using io_prep_fdsync() and update
block/file-posix.c:raw_co_flush_to_disk() to use this when the feature
is available. See <libaio.h> for the Linux AIO library API.

Keep in mind that old host kernels may not support IO_CMD_FDSYNC. In
that case QEMU should continue to use the thread pool.

Taking advantage of the Linux AIO API means QEMU will spawn fewer
worker threads and disk flush performance may improve. You can benchmark
performance using the fio(1) tool. Configure it with ioengine=pvsync2
rw=randwrite direct=1 fdatasync=1 bs=4k to measure the peformance of 4
KB writes followed by fdatasync. For more information about disk I/O
benchmarking, including example fio jobs, see:
https://blog.vmsplice.net/2017/11/common-disk-benchmarking-mistakes.html

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Qemu first time contribution

[Harshavardhan Unnibhavi]

Hi Stefan,

This looks interesting, let me take a look at it. Thank you!

Best,
Harsha

On Mon, Nov 9, 2020 at 10:17 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Sun, Nov 08, 2020 at 12:21:33PM +0000, Harshavardhan Unnibhavi wrote:
> > Thank you for the reply! Yes, I understand that gsoc is over for 2020,
> > and projects for 2021 will come up next year. I was thinking of
> > contributing outside of gsoc(for which I won't be eligible anyways for
> > next year). Anyway, I will work on some of the bite sized tasks, and
> > get back to you for some other concrete project ideas that require
> > somebody to work on, in qemu.
>
> Hi Harsha,
> Here is an idea you could explore:
>
> The Linux AIO API was extended to support fsync(2)/fdatasync(2) in the
> following commit from 2018:
>
>   commit a3c0d439e4d92411c2b4b21a526a4de720d0806b
>   Author: Christoph Hellwig <hch@lst.de>
>   Date:   Tue Mar 27 19:18:57 2018 +0200
>
>       aio: implement IOCB_CMD_FSYNC and IOCB_CMD_FDSYNC
>
> QEMU's Linux AIO code does not take advantage of this feature yet.
> Instead it invokes the traditional fdatasync(2) system call from a
> thread pool because it assumes the Linux AIO API doesn't support the
> operation. The function where this happens is
> block/file-posix.c:raw_co_flush_to_disk().
>
> The goal is to implement IO_CMD_FDSYNC support in block/linux-aio.c
> using io_prep_fdsync() and update
> block/file-posix.c:raw_co_flush_to_disk() to use this when the feature
> is available. See <libaio.h> for the Linux AIO library API.
>
> Keep in mind that old host kernels may not support IO_CMD_FDSYNC. In
> that case QEMU should continue to use the thread pool.
>
> Taking advantage of the Linux AIO API means QEMU will spawn fewer
> worker threads and disk flush performance may improve. You can benchmark
> performance using the fio(1) tool. Configure it with ioengine=pvsync2
> rw=randwrite direct=1 fdatasync=1 bs=4k to measure the peformance of 4
> KB writes followed by fdatasync. For more information about disk I/O
> benchmarking, including example fio jobs, see:
> https://blog.vmsplice.net/2017/11/common-disk-benchmarking-mistakes.html
>
> Stefan

Re: Qemu first time contribution

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

On Tue, Nov 10, 2020 at 10:07:44AM +0000, Harshavardhan Unnibhavi wrote:
> This looks interesting, let me take a look at it. Thank you!

Okay, great. Feel free to email qemu-devel and CC me if you have
questions. We can also chat on #qemu irc.oftc.net (my nick is
"stefanha").

You can configure QEMU to use Linux AIO (block/linux-aio.c) like this:

  $ qemu-system-x86_64 -M accel=kvm -m 1G -cpu host \
        -drive if=virtio,file=test.img,format=raw,aio=native,cache=none

This will boot from the test.img image file using an emulated virtio-blk
device.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]