RE: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

RE: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Dong, Eddie]

Hi Anthony:
	I think you made misunderstanding to this patch. Current Qemu in
Xen is already DMA enabled. If I remembered correctly, it happens since
we change DM from Bochs to Qemu.
	Without this patch, guest IO operation that trigger DMA (like
port 0xc000 write) will wait in Qemu till the DMA operation is
completed, that is original single thread IDE device model mean. 
	With this patch, a seperate thread will service the dma
operation started by IO operation, and interrupt target processor when
it is completed, while the main thread can rapidly return to guest (like
0xc000 write).
Thanks,eddie

Anthony Liguori wrote:
> Hi Eddie,
> 
> There was a patch floating around on qemu-devel recently to make IDE
> DMA concurrent.  Fabrice is planning to include it in QEMU as long as
> there are no regressions.  It may already be in CVS.
> 
> See
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html 
> 
> The reported performance improvement IO is up to 20% so it's
> definitely worth applying...
> 
> Regards,
> 
> Anthony Liguori
> 

Re: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Anthony Liguori]

Dong, Eddie wrote:

>Hi Anthony:
>	I think you made misunderstanding to this patch. Current Qemu in
>Xen is already DMA enabled. If I remembered correctly, it happens since
>we change DM from Bochs to Qemu.
>	Without this patch, guest IO operation that trigger DMA (like
>port 0xc000 write) will wait in Qemu till the DMA operation is
>completed, that is original single thread IDE device model mean. 
>	With this patch, a seperate thread will service the dma
>operation started by IO operation, and interrupt target processor when
>it is completed, while the main thread can rapidly return to guest (like
>0xc000 write).
>  
>
Yup, the site I linked to has two patches: a DMA patch and a concurrent 
IO patch.

Here's a direct link: 
http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_concurrent_io.patch

It's using the same basic approach as your patch (another thread waits 
for completion of IO event).  I'm pointing it out though in case there's 
a desire to stay closer to QEMU upstream when possible.  Might enable 
more code sharing in the future.

Regards,

Anthony Liguori

>Thanks,eddie
>
>Anthony Liguori wrote:
>  
>
>>Hi Eddie,
>>
>>There was a patch floating around on qemu-devel recently to make IDE
>>DMA concurrent.  Fabrice is planning to include it in QEMU as long as
>>there are no regressions.  It may already be in CVS.
>>
>>See
>>http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html 
>>
>>The reported performance improvement IO is up to 20% so it's
>>definitely worth applying...
>>
>>Regards,
>>
>>Anthony Liguori
>>
>>    
>>
>
>  
>

RE: [PATCH][VT] Multithread IDE device model ( was: RE:[Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Nakajima, Jun]

Keir Fraser wrote:
> On 27 Oct 2005, at 15:18, Dong, Eddie wrote:
> 
>> 	So, I think we can stick with current situation now, and revist
>> later when we rebase Qemu.
> 
> I think any damage is now done with regards to keeping tools/ioemu in
> sync with qemu, so we may as well take this patch.
This particular patch is very small and cleanly contained by #ifdef
DMA_MULTI_THREAD. In a sense it's disposable, and it won't stand out as
deviation from qemu.

> 
> At some point there will be a big restructuring of the device model
> code (moving out of dom0) and rebasing to qemu then is probably a good
> plan. If some of the changes we've made could have been pushed up to
> qemu that would have been good -- perhaps we can enforce something
> like that after a rebase.
> 
>   -- Keir
> 
I agree. 

Jun
---
Intel Open Source Technology Center

RE: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Dong, Eddie]

[-- Attachment #1: Type: text/plain, Size: 2843 bytes --]

Anthony:
	Thanks for your information.
	The DMA patch in the qemu site  is to extend the "Windows 2000
Install Hack" to work for DMA, the current qemu in Xen is already DMA
enabled. The non blocking IO patch there is similar with the one we sent
out for Xen. Yes maybe in future we need to rebase Xen/Qemu to latest
Qemu version, but it is dangerous to do now as Xen/3.0 release is
coming. So after discussing with Jun, we think it is better to keep
those rebase to be done after Xen 3.0.
	For the performance data, the patch listed in Qemu site
mentioned about 20% improvement in a situation where a CPU-intensive
thread was competing with an IO-intensive thread. Formal test with our
IDE multithread patch show:
	"cp -a linux-2.6.12 backup"  get 30% up
	"tar xjf linux-2.6.12.tar.bz2" get 47% up
	"hdparam -t /dev/hda1" get 48% up.
	"Kernel build" get 8-14% up
	So, I think we can stick with current situation now, and revist
later when we rebase Qemu.
Keir:
	I just reattach the patch.
Thanks
	
Signed-off-by: Ke Yu <ke.yu@intel.com>
Signed-off-by: Xiaowei Yang <xiaowei.yang@intel.com>

Anthony Liguori wrote:
> Dong, Eddie wrote:
> 
>> Hi Anthony:
>> 	I think you made misunderstanding to this patch. Current Qemu in
>> Xen is already DMA enabled. If I remembered correctly, it happens
>> since we change DM from Bochs to Qemu.
>> 	Without this patch, guest IO operation that trigger DMA (like
>> port 0xc000 write) will wait in Qemu till the DMA operation is
>> completed, that is original single thread IDE device model mean.
>> 	With this patch, a seperate thread will service the dma
>> operation started by IO operation, and interrupt target processor
>> when it is completed, while the main thread can rapidly return to
>> guest (like 0xc000 write). 
>> 
>> 
> Yup, the site I linked to has two patches: a DMA patch and a
> concurrent IO patch.
> 
> Here's a direct link:
>
http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_concurrent_io.patch
> 
> It's using the same basic approach as your patch (another thread waits
> for completion of IO event).  I'm pointing it out though in case
> there's a desire to stay closer to QEMU upstream when possible. 
> Might enable more code sharing in the future.
> 
> Regards,
> 
> Anthony Liguori
> 
>> Thanks,eddie
>> 
>> Anthony Liguori wrote:
>> 
>> 
>>> Hi Eddie,
>>> 
>>> There was a patch floating around on qemu-devel recently to make IDE
>>> DMA concurrent.  Fabrice is planning to include it in QEMU as long
>>> as there are no regressions.  It may already be in CVS.
>>> 
>>> See
>>> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
>>> 
>>> The reported performance improvement IO is up to 20% so it's
>>> definitely worth applying... 
>>> 
>>> Regards,
>>> 
>>> Anthony Liguori


[-- Attachment #2: ide_mt.patch --]
[-- Type: application/octet-stream, Size: 2387 bytes --]

diff -r ff3244aec25e tools/ioemu/hw/ide.c
--- a/tools/ioemu/hw/ide.c	Mon Oct 17 17:45:18 2005
+++ b/tools/ioemu/hw/ide.c	Wed Oct 26 16:59:34 2005
@@ -22,6 +22,7 @@
  * THE SOFTWARE.
  */
 #include "vl.h"
+#include <pthread.h>
 
 /* debug IDE devices */
 //#define DEBUG_IDE
@@ -359,6 +360,48 @@
     IDEState ide_if[4];
     BMDMAState bmdma[2];
 } PCIIDEState;
+
+#define DMA_MULTI_THREAD
+
+#ifdef DMA_MULTI_THREAD
+
+static int file_pipes[2];
+
+static void ide_dma_loop(BMDMAState *bm);
+static void dma_thread_loop(BMDMAState *bm);
+
+static void *dma_thread_func(void* opaque)
+{
+    BMDMAState* req;
+
+    while (read(file_pipes[0], &req, sizeof(req))) {
+        dma_thread_loop(req);
+    }
+
+    return NULL;
+}
+
+static void dma_create_thread()
+{
+    pthread_t tid;
+    int rt;
+
+    if (pipe(file_pipes) != 0){
+        fprintf(stderr, "create pipe failed\n");
+        exit(1);
+    }
+
+    if ( (rt = pthread_create(&tid, NULL, dma_thread_func, NULL)) ) {
+        fprintf(stderr, "Oops, dma thread creation failed, errno=%d\n", rt);
+        exit(1);
+    }
+
+    if ( (rt = pthread_detach(tid)) ) {
+        fprintf(stderr, "Oops, dma thread detachment failed, errno=%d\n", rt);
+        exit(1);
+    }
+}
+#endif //DMA_MULTI_THREAD
 
 static void ide_dma_start(IDEState *s, IDEDMAFunc *dma_cb);
 
@@ -1978,7 +2021,15 @@
 
 /* XXX: full callback usage to prepare non blocking I/Os support -
    error handling */
+#ifdef DMA_MULTI_THREAD
 static void ide_dma_loop(BMDMAState *bm)
+{
+    write(file_pipes[1], &bm, sizeof(bm));
+}
+static void dma_thread_loop(BMDMAState *bm)
+#else 
+static void ide_dma_loop(BMDMAState *bm)
+#endif //DMA_MULTI_THREAD
 {
     struct {
         uint32_t addr;
@@ -2166,6 +2217,9 @@
         d->ide_if[i].pci_dev = (PCIDevice *)d;
     ide_init2(&d->ide_if[0], 16, hd_table[0], hd_table[1]);
     ide_init2(&d->ide_if[2], 16, hd_table[2], hd_table[3]);
+#ifdef DMA_MULTI_THREAD    
+    dma_create_thread();
+#endif //DMA_MULTI_THREAD    
 }
 
 /* hd_table must contain 4 block drivers */
@@ -2196,6 +2250,9 @@
     ide_init2(&d->ide_if[2], 15, hd_table[2], hd_table[3]);
     ide_init_ioport(&d->ide_if[0], 0x1f0, 0x3f6);
     ide_init_ioport(&d->ide_if[2], 0x170, 0x376);
+#ifdef DMA_MULTI_THREAD    
+    dma_create_thread();
+#endif //DMA_MULTI_THREAD    
 }
 
 /***********************************************************/

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Keir Fraser]

On 27 Oct 2005, at 15:18, Dong, Eddie wrote:

> 	So, I think we can stick with current situation now, and revist
> later when we rebase Qemu.

I think any damage is now done with regards to keeping tools/ioemu in 
sync with qemu, so we may as well take this patch.

At some point there will be a big restructuring of the device model 
code (moving out of dom0) and rebasing to qemu then is probably a good 
plan. If some of the changes we've made could have been pushed up to 
qemu that would have been good -- perhaps we can enforce something like 
that after a rebase.

  -- Keir

[PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Dong, Eddie]

Keir:
	This is to to make the IDE device model multithreading so that
the VMX domain IO access completion (triggering DMA operation) can be
asynchronize with the completion of DMA operation. With this patch we
get 8%--14% performance gain for kernel build.
	Thanks, 
eddie

Yang, Xiaowei wrote:
> Originally in qemu when a IDE dma transfer is started which is
> triggered by access to 0xc000 port in VMX, qemu will blocked there
> until it finished. And during that block, there are extra dom
> switches between dom0 and idle domain, and VMX can't be resumed. By
> making real IDE dma tranfer run in another thread, qemu will return
> and VMX resume on time and there are fewer cycle wasted.
> 
> Signed-off-by: Ke Yu <ke.yu@intel.com>
> Signed-off-by: Xiaowei Yang <xiaowei.yang@intel.com>
> 
> -Xiaowei

Re: [PATCH][VT] Multithread IDE device model ( was: RE: [Xen-devel] [PATCH]Make IDE dma tranfer run in another thread inqemu)

[Anthony Liguori]

Dong, Eddie wrote:

>Keir:
>	This is to to make the IDE device model multithreading so that
>the VMX domain IO access completion (triggering DMA operation) can be
>asynchronize with the completion of DMA operation. With this patch we
>get 8%--14% performance gain for kernel build.
>	Thanks, 
>eddie
>  
>
Hi Eddie,

There was a patch floating around on qemu-devel recently to make IDE DMA 
concurrent.  Fabrice is planning to include it in QEMU as long as there 
are no regressions.  It may already be in CVS.

See http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html

The reported performance improvement IO is up to 20% so it's definitely 
worth applying...

Regards,

Anthony Liguori

>Yang, Xiaowei wrote:
>  
>
>>Originally in qemu when a IDE dma transfer is started which is
>>triggered by access to 0xc000 port in VMX, qemu will blocked there
>>until it finished. And during that block, there are extra dom
>>switches between dom0 and idle domain, and VMX can't be resumed. By
>>making real IDE dma tranfer run in another thread, qemu will return
>>and VMX resume on time and there are fewer cycle wasted.
>>
>>Signed-off-by: Ke Yu <ke.yu@intel.com>
>>Signed-off-by: Xiaowei Yang <xiaowei.yang@intel.com>
>>
>>-Xiaowei
>>    
>>
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
>  
>