From: Mark Nelson <mark.nelson@inktank.com>
To: John Wright <jwright@psfc.mit.edu>
Cc: ceph-devel@vger.kernel.org
Subject: Re: MPI applications on ceph fs
Date: Sun, 26 Aug 2012 19:37:59 -0500 [thread overview]
Message-ID: <503AC167.1060502@inktank.com> (raw)
In-Reply-To: <36202107-A075-42E6-8B4B-AF6FEB86E2AD@psfc.mit.edu>
Hi John,
Is the MPI application writing to one file from multiple ranks? Any
idea what the application was doing when the lookup happens?
Mark
On 08/26/2012 09:03 AM, John Wright wrote:
> Hi All,
> We're running ceph 0.48 on small three node test cluster. We've had good stability with I/O using dd and iozone especially after upgrading to 0.48. However, we're running into a repeatable lockup of the linux ceph client ( 3.3.5-2.fc16.x86_64 ) when running an mpi program that has simple I/O on a ceph mount. This is an mpi program running processes on two nodes. It is the remote node on which the ceph client locks up. The cient becomes immediately unresponsive and any attempt to access the mounted volume produces a process with status 'D'. I can see no indication in the server logs that it is ever contacted. Regular serial processes run fine on the volume. MPI runs on the nodes work fine when not using the ceph volume.
>
> So any suggestions on where to look? Any one have an experience testing parallel programs on ceph?
>
> thanks,
> -john
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2012-08-27 0:38 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-08-26 14:03 MPI applications on ceph fs John Wright
2012-08-27 0:37 ` Mark Nelson [this message]
2012-08-27 4:06 ` Sage Weil
2012-08-28 15:54 ` John Wright
2012-08-31 4:40 ` John C. Wright
2012-08-31 20:03 ` Sage Weil
2012-09-02 14:42 ` John Wright
2012-09-17 16:16 ` John Wright
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=503AC167.1060502@inktank.com \
--to=mark.nelson@inktank.com \
--cc=ceph-devel@vger.kernel.org \
--cc=jwright@psfc.mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.