Re: OSD deadlock with cephfs client and OSD on same machine

[Amon Ott <a.ott@m-privacy.de>]

On Tuesday 29 May 2012 wrote Tommi Virtanen:
> On Tue, May 29, 2012 at 12:44 AM, Amon Ott <a.ott@m-privacy.de> wrote:
> > On Linux, if you run OSD on ext4 filesystem, have a cephfs kernel client
> > mount on the same system and no syncfs system call (as to be expected
> > with libc6 < 2.14 or kernel < 2.6.39), OSD deadlocks in sys_sync(). Only
> > reboot recovers the system.
>
> This is the classic issue of memory pressure needing free memory to be
> relieved. While syncfs(2) may make the hang less common, I do not
> think having syncfs(2) is enough; nothing sort of having a reserved
> memory pool guaranteed to be big enough to handle the request will,
> and maintaining that solution is hideously complex.

AFAIR, when the deadlocks came, there were some GB of the 12 GB RAM still 
unused, not even for caching. But it might be a problem with low memory, 
because we are running with 32 Bit.

Would it be possible to preallocate a significant amount of RAM for the 
purpose of syncing? I would not mind reserving a few 100 MB for that, but 
deadlocks must not happen in any case. Can the size of the journal give a 
hint on how much is needed?

Amon Ott
-- 
Dr. Amon Ott
m-privacy GmbH           Tel: +49 30 24342334
Am Köllnischen Park 1    Fax: +49 30 24342336
10179 Berlin             http://www.m-privacy.de

Amtsgericht Charlottenburg, HRB 84946

Geschäftsführer:
 Dipl.-Kfm. Holger Maczkowsky,
 Roman Maczkowsky

GnuPG-Key-ID: 0x2DD3A649
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html