From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sj-iport-3.cisco.com (sj-iport-3-in.cisco.com [171.71.176.72]) by ozlabs.org (Postfix) with ESMTP id 011AEDDE39 for ; Tue, 11 Dec 2007 08:47:40 +1100 (EST) To: Joachim Fenkes Subject: Re: [PATCH] IB/ehca: Serialize HCA-related hCalls on POWER5 References: <200712061607.20004.fenkes@de.ibm.com> <200712071058.38416.arnd@arndb.de> <200712101841.30010.fenkes@de.ibm.com> From: Roland Dreier Date: Mon, 10 Dec 2007 13:47:37 -0800 In-Reply-To: <200712101841.30010.fenkes@de.ibm.com> (Joachim Fenkes's message of "Mon, 10 Dec 2007 18:41:29 +0100") Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Arnd Bergmann , LKML , OF-EWG , linuxppc-dev@ozlabs.org, Christoph Raisch , Marcus Eder , OF-General , Stefan Roscher List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , > > map_phys_fmr > > In fact, we do use hCalls there. Our hardware doesn't actually support FMRs, > so we translate a "map FMR" into a "reallocate PMR", which doesn't work > without hCalls. What's more, the hCalls involved (e.g. H_FREE_RESOURCE) > might well return H_LONG_BUSY, so the whole operation might sleep; no way > around it. It's a big problem. If you cannot implement FMRs in such a way that you can handling having map_phys_fmr being called in a context that can't sleep, then I think the only option is to remove your FMR support. It's an optional device feature, so this should be OK (although the iSER driver currently seems to depend on a device supporting FMRs, which is probably going to be a problem with iWARP support in the future anyway). The fact that consumers can map FMRs from interrupt context, while holding locks, etc, is pretty fundamental to the use of FMRs so I don't see any way around the requirement that map_phys_fmr never sleep. - R. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754747AbXLJVrt (ORCPT ); Mon, 10 Dec 2007 16:47:49 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751214AbXLJVrk (ORCPT ); Mon, 10 Dec 2007 16:47:40 -0500 Received: from sj-iport-3-in.cisco.com ([171.71.176.72]:64083 "EHLO sj-iport-3.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751198AbXLJVrj (ORCPT ); Mon, 10 Dec 2007 16:47:39 -0500 To: Joachim Fenkes Cc: Arnd Bergmann , linuxppc-dev@ozlabs.org, LKML , OF-EWG , Christoph Raisch , Marcus Eder , OF-General , Stefan Roscher Subject: Re: [PATCH] IB/ehca: Serialize HCA-related hCalls on POWER5 X-Message-Flag: Warning: May contain useful information References: <200712061607.20004.fenkes@de.ibm.com> <200712071058.38416.arnd@arndb.de> <200712101841.30010.fenkes@de.ibm.com> From: Roland Dreier Date: Mon, 10 Dec 2007 13:47:37 -0800 In-Reply-To: <200712101841.30010.fenkes@de.ibm.com> (Joachim Fenkes's message of "Mon, 10 Dec 2007 18:41:29 +0100") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.4.20 (linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-OriginalArrivalTime: 10 Dec 2007 21:47:38.0200 (UTC) FILETIME=[481B7D80:01C83B76] Authentication-Results: sj-dkim-3; header.From=rdreier@cisco.com; dkim=pass ( sig from cisco.com/sjdkim3002 verified; ); Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > map_phys_fmr > > In fact, we do use hCalls there. Our hardware doesn't actually support FMRs, > so we translate a "map FMR" into a "reallocate PMR", which doesn't work > without hCalls. What's more, the hCalls involved (e.g. H_FREE_RESOURCE) > might well return H_LONG_BUSY, so the whole operation might sleep; no way > around it. It's a big problem. If you cannot implement FMRs in such a way that you can handling having map_phys_fmr being called in a context that can't sleep, then I think the only option is to remove your FMR support. It's an optional device feature, so this should be OK (although the iSER driver currently seems to depend on a device supporting FMRs, which is probably going to be a problem with iWARP support in the future anyway). The fact that consumers can map FMRs from interrupt context, while holding locks, etc, is pretty fundamental to the use of FMRs so I don't see any way around the requirement that map_phys_fmr never sleep. - R.