From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7FC1ECAAA1 for ; Thu, 27 Oct 2022 21:26:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooAMR-0003Ae-2q; Thu, 27 Oct 2022 17:24:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooAMP-00037H-G1; Thu, 27 Oct 2022 17:24:21 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5] helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooAM2-0004nz-29; Thu, 27 Oct 2022 17:24:21 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29RKtNja001910; Thu, 27 Oct 2022 21:23:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=SS/rsLZL7oOPbAXV7spcuF2OjDWykQN4AfL+jAuUPlo=; b=JUEHNkqbHj7dN+mR0GXxPQ6wiP1ZFiMtVUXCd/OZ0UigNOeQoAy0atLadikfZe7MP1YW iZfa+IKErcJ+9LxrfpDbchymMGOOiyQjlBTruFdmtSKVbquyWzz+bxRY8QSrow5PuIdZ QlZb2+rW9xoTeaJ6MfUhcvb8Hrp9gnHGE/IkEwPBfH5skG4IM42RM7a5fzge1kCXJPeq HGzb9ZvKgRdBLesQs1kvbjlK8vt+5FIX7fmu+ihsfzMOo9ovs3OcABRgJYwqIAJOgVQy /OeqSMHi5F4uNEeUKuF94Ge5/EZyynKMf963/hVsm3nvs02JXMFToejr3NoVR7CplFPI Yg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3kg1b6rxq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Oct 2022 21:23:55 +0000 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 29RKo1TT011848; Thu, 27 Oct 2022 21:23:55 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3kg1b6rxpa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Oct 2022 21:23:55 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 29RLLMmx026783; Thu, 27 Oct 2022 21:23:53 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma06ams.nl.ibm.com with ESMTP id 3kfahu2qsq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Oct 2022 21:23:53 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 29RLNoRa4981334 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 27 Oct 2022 21:23:50 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 144D511C04C; Thu, 27 Oct 2022 21:23:50 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EE34911C04A; Thu, 27 Oct 2022 21:23:49 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Thu, 27 Oct 2022 21:23:49 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55807) id AD95EE0139; Thu, 27 Oct 2022 23:23:49 +0200 (CEST) From: Peter Jin To: pasic@linux.ibm.com, borntraeger@linux.ibm.com, farman@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, cohuck@redhat.com, thuth@redhat.com, mjrosato@linux.ibm.com Cc: peter@peterjin.org, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, Peter Jin Subject: [PATCH v2] s390x/css: revert SCSW ctrl/flag bits on error Date: Thu, 27 Oct 2022 23:23:41 +0200 Message-Id: <20221027212341.2904795-1-pjin@linux.ibm.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: DayGgwFPxaItYYquqR8gSu6J9t6x96qq X-Proofpoint-ORIG-GUID: zN0vR-KMHa-04PksXQedzIHoAenpoGff X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-27_07,2022-10-27_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 malwarescore=0 priorityscore=1501 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2210270118 Received-SPF: pass client-ip=148.163.158.5; envelope-from=Peter.Jin@ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Revert the control and flag bits in the subchannel status word in case the SSCH operation fails with non-zero CC (ditto for CSCH and HSCH). According to POPS, the control and flag bits are only changed if SSCH, CSCH, and HSCH return CC 0, and no other action should be taken otherwise. In order to simulate that after the fact, the bits need to be reverted on non-zero CC. This change is necessary due to the fact that the pwrite() in vfio-ccw which triggers the SSCH can fail at any time. Previously, there was only virtio-ccw, whose do_subchannel_work function was only able to return CC0. However, once vfio-ccw went into the mix, it has become necessary to handle errors in code paths that were previously assumed to always return success. In our case, we found that in case of pwrite() failure (which was discovered by strace injection), the subchannel could be stuck in start pending state, which could be problematic if the pwrite() call returns CC2. Experimentation shows that the guest tries to retry the SSCH call as normal for CC2, but it actually continously fails due to the fact that the subchannel is stuck in start pending state even though no start function is actually taking place. Signed-off-by: Peter Jin --- hw/s390x/css.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 3 deletions(-) diff --git a/hw/s390x/css.c b/hw/s390x/css.c index 7d9523f811..95d1b3a3ce 100644 --- a/hw/s390x/css.c +++ b/hw/s390x/css.c @@ -1522,21 +1522,37 @@ IOInstEnding css_do_xsch(SubchDev *sch) IOInstEnding css_do_csch(SubchDev *sch) { SCHIB *schib = &sch->curr_status; + uint16_t old_scsw_ctrl; + IOInstEnding ccode; if (~(schib->pmcw.flags) & (PMCW_FLAGS_MASK_DNV | PMCW_FLAGS_MASK_ENA)) { return IOINST_CC_NOT_OPERATIONAL; } + /* + * Save the current scsw.ctrl in case CSCH fails and we need + * to revert the scsw to the status quo ante. + */ + old_scsw_ctrl = schib->scsw.ctrl; + /* Trigger the clear function. */ schib->scsw.ctrl &= ~(SCSW_CTRL_MASK_FCTL | SCSW_CTRL_MASK_ACTL); schib->scsw.ctrl |= SCSW_FCTL_CLEAR_FUNC | SCSW_ACTL_CLEAR_PEND; - return do_subchannel_work(sch); + ccode = do_subchannel_work(sch); + + if (ccode != IOINST_CC_EXPECTED) { + schib->scsw.ctrl = old_scsw_ctrl; + } + + return ccode; } IOInstEnding css_do_hsch(SubchDev *sch) { SCHIB *schib = &sch->curr_status; + uint16_t old_scsw_ctrl; + IOInstEnding ccode; if (~(schib->pmcw.flags) & (PMCW_FLAGS_MASK_DNV | PMCW_FLAGS_MASK_ENA)) { return IOINST_CC_NOT_OPERATIONAL; @@ -1553,6 +1569,12 @@ IOInstEnding css_do_hsch(SubchDev *sch) return IOINST_CC_BUSY; } + /* + * Save the current scsw.ctrl in case HSCH fails and we need + * to revert the scsw to the status quo ante. + */ + old_scsw_ctrl = schib->scsw.ctrl; + /* Trigger the halt function. */ schib->scsw.ctrl |= SCSW_FCTL_HALT_FUNC; schib->scsw.ctrl &= ~SCSW_FCTL_START_FUNC; @@ -1564,7 +1586,13 @@ IOInstEnding css_do_hsch(SubchDev *sch) } schib->scsw.ctrl |= SCSW_ACTL_HALT_PEND; - return do_subchannel_work(sch); + ccode = do_subchannel_work(sch); + + if (ccode != IOINST_CC_EXPECTED) { + schib->scsw.ctrl = old_scsw_ctrl; + } + + return ccode; } static void css_update_chnmon(SubchDev *sch) @@ -1605,6 +1633,8 @@ static void css_update_chnmon(SubchDev *sch) IOInstEnding css_do_ssch(SubchDev *sch, ORB *orb) { SCHIB *schib = &sch->curr_status; + uint16_t old_scsw_ctrl, old_scsw_flags; + IOInstEnding ccode; if (~(schib->pmcw.flags) & (PMCW_FLAGS_MASK_DNV | PMCW_FLAGS_MASK_ENA)) { return IOINST_CC_NOT_OPERATIONAL; @@ -1626,11 +1656,26 @@ IOInstEnding css_do_ssch(SubchDev *sch, ORB *orb) } sch->orb = *orb; sch->channel_prog = orb->cpa; + + /* + * Save the current scsw.ctrl and scsw.flags in case SSCH fails and we need + * to revert the scsw to the status quo ante. + */ + old_scsw_ctrl = schib->scsw.ctrl; + old_scsw_flags = schib->scsw.flags; + /* Trigger the start function. */ schib->scsw.ctrl |= (SCSW_FCTL_START_FUNC | SCSW_ACTL_START_PEND); schib->scsw.flags &= ~SCSW_FLAGS_MASK_PNO; - return do_subchannel_work(sch); + ccode = do_subchannel_work(sch); + + if (ccode != IOINST_CC_EXPECTED) { + schib->scsw.ctrl = old_scsw_ctrl; + schib->scsw.flags = old_scsw_flags; + } + + return ccode; } static void copy_irb_to_guest(IRB *dest, const IRB *src, const PMCW *pmcw, -- 2.37.3