DPDK testpmd, Virtual Disk IO limitation

DPDK testpmd, Virtual Disk IO limitation

[Cheng Kevin]

Hi all,

   I am a beginner of DPDK. Recently, i am interest in DPDK vHost app -
testpmd.

   And i have been tracing on testpmd.c and iofwd.c for a while.

   Also add some code inside iofwd.c for storing the payload of packets.

   Everything goes fine, and the performance is great as expected.

   But when i use fwrite to store the payload into a file,

   the performance decrease from 800mbps to 3mbps (input stream is 1 Gbps).

   Is is caused by the limitation of Virtual Disk IO? How can i solve it?

   I have tried to search the answer, some people say "pthread" might solve
the problem.

   Can someone give me some hint, i really appreciate for your help.


Best Regard,

Kevin Cheng

Re: DPDK testpmd, Virtual Disk IO limitation

[Bruce Richardson]

On Wed, Mar 25, 2015 at 10:06:48PM +0800, Cheng Kevin wrote:
> Hi all,
> 
>    I am a beginner of DPDK. Recently, i am interest in DPDK vHost app -
> testpmd.
> 
>    And i have been tracing on testpmd.c and iofwd.c for a while.
> 
>    Also add some code inside iofwd.c for storing the payload of packets.
> 
>    Everything goes fine, and the performance is great as expected.
> 
>    But when i use fwrite to store the payload into a file,
> 
>    the performance decrease from 800mbps to 3mbps (input stream is 1 Gbps).
> 
>    Is is caused by the limitation of Virtual Disk IO? How can i solve it?
> 
>    I have tried to search the answer, some people say "pthread" might solve
> the problem.
> 
>    Can someone give me some hint, i really appreciate for your help.
> 
> 
> Best Regard,
> 
> Kevin Cheng

Two general issues you will hit writing to disk:
1) IO, including disk IO, is slow
2) System calls are slow.

You are probably hitting both bottlenecks.

/Bruce

Re: DPDK testpmd, Virtual Disk IO limitation

[Cheng Kevin]

Mr. Bruce Richardson

Yes, you are right. This really bother me.

Is there any way to get rid of system call? Maybe some DPDK threading API?
Maybe i should use a extra nic card for posting the data out through
internet, instead of writing on the disk - ex. fwrite.

Or you have some better advises??

Thanks
Kevin


2015-03-25 23:01 GMT+08:00 Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>:

> On Wed, Mar 25, 2015 at 10:06:48PM +0800, Cheng Kevin wrote:
> > Hi all,
> >
> >    I am a beginner of DPDK. Recently, i am interest in DPDK vHost app -
> > testpmd.
> >
> >    And i have been tracing on testpmd.c and iofwd.c for a while.
> >
> >    Also add some code inside iofwd.c for storing the payload of packets.
> >
> >    Everything goes fine, and the performance is great as expected.
> >
> >    But when i use fwrite to store the payload into a file,
> >
> >    the performance decrease from 800mbps to 3mbps (input stream is 1
> Gbps).
> >
> >    Is is caused by the limitation of Virtual Disk IO? How can i solve it?
> >
> >    I have tried to search the answer, some people say "pthread" might
> solve
> > the problem.
> >
> >    Can someone give me some hint, i really appreciate for your help.
> >
> >
> > Best Regard,
> >
> > Kevin Cheng
>
> Two general issues you will hit writing to disk:
> 1) IO, including disk IO, is slow
> 2) System calls are slow.
>
> You are probably hitting both bottlenecks.
>
> /Bruce
>

Re: DPDK testpmd, Virtual Disk IO limitation

[Bruce Richardson]

On Thu, Mar 26, 2015 at 12:20:42AM +0800, Cheng Kevin wrote:
> Mr. Bruce Richardson
> 
> Yes, you are right. This really bother me.
> 
> Is there any way to get rid of system call? Maybe some DPDK threading API?
> Maybe i should use a extra nic card for posting the data out through
> internet, instead of writing on the disk - ex. fwrite.
> 
> Or you have some better advises??
> 
> Thanks
> Kevin
> 

Hi Kevin,

what is your end-goal that you are trying to get to? 

/Bruce

> 
> 2015-03-25 23:01 GMT+08:00 Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>:
> 
> > On Wed, Mar 25, 2015 at 10:06:48PM +0800, Cheng Kevin wrote:
> > > Hi all,
> > >
> > >    I am a beginner of DPDK. Recently, i am interest in DPDK vHost app -
> > > testpmd.
> > >
> > >    And i have been tracing on testpmd.c and iofwd.c for a while.
> > >
> > >    Also add some code inside iofwd.c for storing the payload of packets.
> > >
> > >    Everything goes fine, and the performance is great as expected.
> > >
> > >    But when i use fwrite to store the payload into a file,
> > >
> > >    the performance decrease from 800mbps to 3mbps (input stream is 1
> > Gbps).
> > >
> > >    Is is caused by the limitation of Virtual Disk IO? How can i solve it?
> > >
> > >    I have tried to search the answer, some people say "pthread" might
> > solve
> > > the problem.
> > >
> > >    Can someone give me some hint, i really appreciate for your help.
> > >
> > >
> > > Best Regard,
> > >
> > > Kevin Cheng
> >
> > Two general issues you will hit writing to disk:
> > 1) IO, including disk IO, is slow
> > 2) System calls are slow.
> >
> > You are probably hitting both bottlenecks.
> >
> > /Bruce
> >

Re: DPDK testpmd, Virtual Disk IO limitation

[Cheng Kevin]

Hi Mr.Bruce Richardson

    I expect the testpmd is able to store the payload of the coming packets
and that's all.
    However, as you can see, my program hits the bottlenecks.
    The performance is unable to reach 800mbps anymore.

    My end-goal is simple, finding a way to store the payload without
decreasing the speed of packet forwarding.

Thanks
Kevin



2015-03-26 1:32 GMT+08:00 Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>:

> On Thu, Mar 26, 2015 at 12:20:42AM +0800, Cheng Kevin wrote:
> > Mr. Bruce Richardson
> >
> > Yes, you are right. This really bother me.
> >
> > Is there any way to get rid of system call? Maybe some DPDK threading
> API?
> > Maybe i should use a extra nic card for posting the data out through
> > internet, instead of writing on the disk - ex. fwrite.
> >
> > Or you have some better advises??
> >
> > Thanks
> > Kevin
> >
>
> Hi Kevin,
>
> what is your end-goal that you are trying to get to?
>
> /Bruce
>
> >
> > 2015-03-25 23:01 GMT+08:00 Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
> >:
> >
> > > On Wed, Mar 25, 2015 at 10:06:48PM +0800, Cheng Kevin wrote:
> > > > Hi all,
> > > >
> > > >    I am a beginner of DPDK. Recently, i am interest in DPDK vHost
> app -
> > > > testpmd.
> > > >
> > > >    And i have been tracing on testpmd.c and iofwd.c for a while.
> > > >
> > > >    Also add some code inside iofwd.c for storing the payload of
> packets.
> > > >
> > > >    Everything goes fine, and the performance is great as expected.
> > > >
> > > >    But when i use fwrite to store the payload into a file,
> > > >
> > > >    the performance decrease from 800mbps to 3mbps (input stream is 1
> > > Gbps).
> > > >
> > > >    Is is caused by the limitation of Virtual Disk IO? How can i
> solve it?
> > > >
> > > >    I have tried to search the answer, some people say "pthread" might
> > > solve
> > > > the problem.
> > > >
> > > >    Can someone give me some hint, i really appreciate for your help.
> > > >
> > > >
> > > > Best Regard,
> > > >
> > > > Kevin Cheng
> > >
> > > Two general issues you will hit writing to disk:
> > > 1) IO, including disk IO, is slow
> > > 2) System calls are slow.
> > >
> > > You are probably hitting both bottlenecks.
> > >
> > > /Bruce
> > >
>