Re: paging behavior in Linux

Re: paging behavior in Linux

[Manfred Spraul]

> 
> When I run my program on a readhat linux machine, I dont get results as 
> expected, work thread seems to be stuck when prefetch thread is waiting on 
> a page fault
>
That's a known problem:

The paging io for a process is controlled with a per-process semaphore.
The semaphore is held while waiting for the actual io. Thus the paging
in multi threaded applications is single threaded.
Probably your prefetch thread is waiting for disk io, and the worker
thread causes a minor pagefault --> worker thread sleeps until the disk
io is completed.

--
	Manfred

Re: paging behavior in Linux

[David Mansfield]

Manfred Spraul wrote:
> 
> >
> > When I run my program on a readhat linux machine, I dont get results as
> > expected, work thread seems to be stuck when prefetch thread is waiting on
> > a page fault
> >
> That's a known problem:
> 
> The paging io for a process is controlled with a per-process semaphore.
> The semaphore is held while waiting for the actual io. Thus the paging
> in multi threaded applications is single threaded.
> Probably your prefetch thread is waiting for disk io, and the worker
> thread causes a minor pagefault --> worker thread sleeps until the disk
> io is completed.

This behavior is actually pretty annoying.  There can be cases where a
process wakes up from a page fault, does some work, goes back to sleep
on a page fault, thereby keeping it's mmap_sem locked at all times (i.e.
vmstat, top, ps unusable) on a UP system.  I posted this complaint a
while ago, it was discussed by Linus and Andrew Morton about how it also
boiled down to semaphore wakeup unfairness (and bugs?).  The current
semaphore was determined to be too ugly to even look at.  So it was
dropped.

Is there any way that the mmap_sem could be dropped during the blocking
on I/O, and reclaimed after the handle_mm_fault?  Probably not, or it'd
be done.

It can be a real DOS though, a 'well-written' clobbering program can
make ps/vmstat useless.  (it's actually /proc/pid/stat that's the
killer, IIRC).

David

-- 
David Mansfield                                           (718) 963-2020
david@ultramaster.com
Ultramaster Group, LLC                               www.ultramaster.com

Re: paging behavior in Linux

[Andrew Morton]

David Mansfield wrote:
> 
> Manfred Spraul wrote:
> >
> > >
> > > When I run my program on a readhat linux machine, I dont get results as
> > > expected, work thread seems to be stuck when prefetch thread is waiting on
> > > a page fault
> > >
> > That's a known problem:
> >
> > The paging io for a process is controlled with a per-process semaphore.
> > The semaphore is held while waiting for the actual io. Thus the paging
> > in multi threaded applications is single threaded.
> > Probably your prefetch thread is waiting for disk io, and the worker
> > thread causes a minor pagefault --> worker thread sleeps until the disk
> > io is completed.
> 
> This behavior is actually pretty annoying.  There can be cases where a
> process wakes up from a page fault, does some work, goes back to sleep
> on a page fault, thereby keeping it's mmap_sem locked at all times (i.e.
> vmstat, top, ps unusable) on a UP system.  I posted this complaint a
> while ago, it was discussed by Linus and Andrew Morton about how it also
> boiled down to semaphore wakeup unfairness (and bugs?).  The current
> semaphore was determined to be too ugly to even look at.  So it was
> dropped.
> 
> Is there any way that the mmap_sem could be dropped during the blocking
> on I/O, and reclaimed after the handle_mm_fault?  Probably not, or it'd
> be done.
> 
> It can be a real DOS though, a 'well-written' clobbering program can
> make ps/vmstat useless.  (it's actually /proc/pid/stat that's the
> killer, IIRC).

Did the `goto inside' trick in the semaphore code actually
fix this unfairness issue?

-

paging behavior in Linux

[Neelam Saboo]

Hi,

I am a graduate student at Univ Of Illinois at Urbana-Champaign.
Regarding my thesis, I need some help with running a multithreaded program on
red hat linux. I am experimenting with paging behavior. I use a machine where
swap space is large compared to available memory. I run two threads, worker
thread & prefetch thread.
prefetch thread prefetches data in memory by accessing it which worker
thread will use later. Expected result is prefetch thread should take
most of the page faults leaving worker thread time to do actual work.
So, paging time in prefetch thread and work time in worker thread should
overlap and total time taken should be less.
In a PThreads book I found out that when one thread is waiting on a page
fault, other thread continues to work.

When I run my program on a readhat linux machine, I dont get results as
expected, work thread seems to be stuck when prefetch thread is waiting on
a page fault. Is it because, when one thread waits on a page fault, page
table is locked for the other thread also. This is linux specific, as I
get expected results on solaris. 

Can you please provide some explanations ?

Thanks
Neelam  
 


____________________________________________________________________
Get free email and a permanent address at http://www.netaddress.com/?N=1