[Re: paging behavior in Linux]

[Re: paging behavior in Linux]

[Neelam Saboo]

hi,

After I installed a newer version of Kernel (2.4.2) and enable DMA option in
hardware configuration, the behavior changes.
I can see performance improvements when another thread is used. Also, i can
see timing overlaps between two threads. i.e. when one thread is blocked on a
page fault, other thread keeps working.
Now, how can this behavior be explained , given the earlier argument.
Is it that, a newer version of kernel has fixed the problem of the semaphore
?

thanks
neelam

> That's a known problem:
> 
> The paging io for a process is controlled with a per-process semaphore.
> The semaphore is held while waiting for the actual io. Thus the paging
> in multi threaded applications is single threaded.
> Probably your prefetch thread is waiting for disk io, and the worker
> thread causes a minor pagefault --> worker thread sleeps until the disk
> io is completed.
> 
> --
> 	Manfred


____________________________________________________________________
Get free email and a permanent address at http://www.netaddress.com/?N=1

Re: [Re: paging behavior in Linux]

[Manfred Spraul]

Neelam Saboo wrote:
> 
> hi,
> 
> After I installed a newer version of Kernel (2.4.2) and enable DMA option in
> hardware configuration, the behavior changes.
> I can see performance improvements when another thread is used. Also, i can
> see timing overlaps between two threads. i.e. when one thread is blocked on a
> page fault, other thread keeps working.
> Now, how can this behavior be explained , given the earlier argument.
> Is it that, a newer version of kernel has fixed the problem of the semaphore
> ?
>
No, that change won't happen until 2.5

I can only guess:
the other thread keeps working until it causes a page fault - with both
2.4.1 and 2.4.2.

I haven't followed the threads about the mm changes closely, but I
assume that the swapout behaviour changed, and that your worker thread
now runs without causing page faults.

--
	Manfred

Re: [Re: paging behavior in Linux]

[Jeremy Jackson]

Manfred Spraul wrote:

> Neelam Saboo wrote:
> >
> > hi,
> >
> > After I installed a newer version of Kernel (2.4.2) and enable DMA option in

Ah hah!  There's a huge difference in performance (in my experience) with
DMA.  also, using hdparm utility, *most* drives work fine with dma,
irq unmasking, multisector transfers, and 32bit access.
hdparm -i /dev/hda or such will tell you maximum multisector supported.
the only reason these aren't enabled AFAIK is that SOME drives don't,
and when there's a problem it could cause data loss.

The worker thread may just have been overtaking the prefetcher
because dma was off and disk was slow.

>
> > hardware configuration, the behavior changes.
> > I can see performance improvements when another thread is used. Also, i can
> > see timing overlaps between two threads. i.e. when one thread is blocked on a
> > page fault, other thread keeps working.
> > Now, how can this behavior be explained , given the earlier argument.
> > Is it that, a newer version of kernel has fixed the problem of the semaphore
> > ?
> >
> No, that change won't happen until 2.5
>
> I can only guess:
> the other thread keeps working until it causes a page fault - with both
> 2.4.1 and 2.4.2.
>
> I haven't followed the threads about the mm changes closely, but I
> assume that the swapout behaviour changed, and that your worker thread
> now runs without causing page faults.

Re: [Re: paging behavior in Linux]

[Neelam Saboo]

Another observation. I have two independent programs. One program incurring
page faults and another program just doing some work.
When work program run undependently it takes ~19 seconds of CPU time, but when
it is run along with page faulting program on the same machine, it takes ~32
seconds of CPU time. Doesnt this indicate that page faults in a program slows
down all the program on the machine and not only threads in the same process
?

neelam

Andrew Morton <andrewm@uow.edu.au> wrote:
> David Mansfield wrote:
> > 
> > Manfred Spraul wrote:
> > >
> > > >
> > > > When I run my program on a readhat linux machine, I dont get results
as
> > > > expected, work thread seems to be stuck when prefetch thread is
waiting on
> > > > a page fault
> > > >
> > > That's a known problem:
> > >
> > > The paging io for a process is controlled with a per-process semaphore.
> > > The semaphore is held while waiting for the actual io. Thus the paging
> > > in multi threaded applications is single threaded.
> > > Probably your prefetch thread is waiting for disk io, and the worker
> > > thread causes a minor pagefault --> worker thread sleeps until the disk
> > > io is completed.
> > 
> > This behavior is actually pretty annoying.  There can be cases where a
> > process wakes up from a page fault, does some work, goes back to sleep
> > on a page fault, thereby keeping it's mmap_sem locked at all times (i.e.
> > vmstat, top, ps unusable) on a UP system.  I posted this complaint a
> > while ago, it was discussed by Linus and Andrew Morton about how it also
> > boiled down to semaphore wakeup unfairness (and bugs?).  The current
> > semaphore was determined to be too ugly to even look at.  So it was
> > dropped.
> > 
> > Is there any way that the mmap_sem could be dropped during the blocking
> > on I/O, and reclaimed after the handle_mm_fault?  Probably not, or it'd
> > be done.
> > 
> > It can be a real DOS though, a 'well-written' clobbering program can
> > make ps/vmstat useless.  (it's actually /proc/pid/stat that's the
> > killer, IIRC).
> 
> Did the `goto inside' trick in the semaphore code actually
> fix this unfairness issue?
> 
> -


____________________________________________________________________
Get free email and a permanent address at http://www.netaddress.com/?N=1

Re: [Re: paging behavior in Linux]

[Neelam Saboo]

Another observation. I have two independent programs. One program incurring
page faults and another program just doing some work.
When work program run undependently it takes ~19 seconds of CPU time, but when
it is run along with page faulting program on the same machine, it takes ~32
seconds of CPU time. Doesnt this indicate that page faults in a program slows
down all the program on the machine and not only threads in the same process
?

neelam


David Mansfield <lkml@dm.ultramaster.com> wrote:
> Manfred Spraul wrote:
> > 
> > The paging io for a process is controlled with a per-process semaphore.
> > The semaphore is held while waiting for the actual io. Thus the paging
> > in multi threaded applications is single threaded.
> > Probably your prefetch thread is waiting for disk io, and the worker
> > thread causes a minor pagefault --> worker thread sleeps until the disk
> > io is completed.
> 
> This behavior is actually pretty annoying.  There can be cases where a
> process wakes up from a page fault, does some work, goes back to sleep
> on a page fault, thereby keeping it's mmap_sem locked at all times (i.e.
> vmstat, top, ps unusable) on a UP system.  I posted this complaint a
> while ago, it was discussed by Linus and Andrew Morton about how it also
> boiled down to semaphore wakeup unfairness (and bugs?).  The current
> semaphore was determined to be too ugly to even look at.  So it was
> dropped.
> 
> Is there any way that the mmap_sem could be dropped during the blocking
> on I/O, and reclaimed after the handle_mm_fault?  Probably not, or it'd
> be done.
> 
> It can be a real DOS though, a 'well-written' clobbering program can
> make ps/vmstat useless.  (it's actually /proc/pid/stat that's the
> killer, IIRC).
> 
> David
> 
> -- 
> David Mansfield                                           (718) 963-2020
> david@ultramaster.com
> Ultramaster Group, LLC                               www.ultramaster.com


____________________________________________________________________
Get free email and a permanent address at http://www.netaddress.com/?N=1

Re: [Re: paging behavior in Linux]

[Helge Hafting]

Neelam Saboo wrote:
> 
> Another observation. I have two independent programs. One program incurring
> page faults and another program just doing some work.
> When work program run undependently it takes ~19 seconds of CPU time, but when
> it is run along with page faulting program on the same machine, it takes ~32
> seconds of CPU time. Doesnt this indicate that page faults in a program slows
> down all the program on the machine and not only threads in the same process
> ?
Is this really CPU time or merely wall clock time?  The latter will
obviously increase as running two programs takes more time than running
one.

Helge Hafting