From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F775C54755 for ; Thu, 15 May 2025 15:49:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uFam8-0007MX-LZ; Thu, 15 May 2025 11:45:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uFalZ-0006Ke-Jb for qemu-devel@nongnu.org; Thu, 15 May 2025 11:45:04 -0400 Received: from mx0b-002c1b01.pphosted.com ([148.163.155.12]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uFalU-0006E7-SB for qemu-devel@nongnu.org; Thu, 15 May 2025 11:45:01 -0400 Received: from pps.filterd (m0127841.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 54FBPdHZ007971; Thu, 15 May 2025 08:44:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=SvrRLBoBv0l7V9QrW/Mozo4syme+X9WO96yGjCo48 nM=; b=Re3icmJdAksTTBBwvNPxnWkuYdJwM0SZ57G2bm1pQ6GYV0xgHpsVSx4zb 36XQAv59iFRQEfiOwxU8xTDqBZXemqlyi/qgFRNOQoytmus8Hg694rxO+XNpnC9M rl1HrK1bcLOc3KduMqGk9+F9JI754pJhU1t64HWyF44U0iXw3Qa/YA9JpVTSaUZ2 0tSQTrPoimZcWNLNLU9kOx4Tsfyu53WJSazPupG9/LcqT2pjd7IjMDAsLoG2sLv3 gxucCLcW0ZIgZN+kL91keJAPEx0MqAXGjyF0k66Byf9/JqhdLITPnfRNCPqsNW8v qN4zpPYq76aTrSqDNUYprIAJJvvew== Received: from sn4pr2101cu001.outbound.protection.outlook.com (mail-southcentralusazlp17012010.outbound.protection.outlook.com [40.93.14.10]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 46mbd0we20-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 May 2025 08:44:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=unt7qa6OmggvDvA//VN08Z6Bu7IScGDAlq0yA8r1+X0imEJOtN2SNGYaTefzVLuI0+WTWc+T3xyCv8atGuHS4Ia1qsL4E1vKVmWhBfY+Zbk0kYFcALjqs5tjWFa7jT28aHq8CILZoJ/DeR2EdCoN1xyobErSOvM91Ve92avbXVrgVW4pITXuZPWAFQ5MIzYExhZCIntmGgwivglPvoEQt2b3giPTHFo4NtJX/kaCdW1tAZJgkilaJZIf29lWOWLmdy4/+/hCyCIPIhpirNF12G4YUoWboMnzLSYHPmyOsrNN0fRpWt/p68R4e+j6Qg1kR0peesa3x1fCWHOaac3DaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SvrRLBoBv0l7V9QrW/Mozo4syme+X9WO96yGjCo48nM=; b=JVEX6Y31Z+frkTC9R78fjYMmZFK9TM3HoHuzhvW2Ii0lonEgFAeipCwlP1qO5+TPVdm1HHEuYhJYdDCdt+N/lCWsOPnSZ/pM0JGAdkghhpM+s5CY2SlAhvY1RXrxG9G672hy8goluXgAtYSnmmCCY9u0CaH4thGnD/y1tmBlcTuxIBKJ0+FyYfK5fpJtY9dwyOhL7vsQeBjI/Oykv8PtQuYyA5ZDoH2vF5eVob9YjSu9xUKY7nzHCionK+7W59SV8zXsMH4OSqpFIuwkmXtfRhvYCg+x0OVClKNrqMa7c2Um6vPn3RW+sfwoBJja0oKNr4/cNlqOSL2WaY1J0PysZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SvrRLBoBv0l7V9QrW/Mozo4syme+X9WO96yGjCo48nM=; b=yHVuh21ARnorqXxD7BZAnKchwTJUh0la4gyCmvOj9elktY40S6TZy13VjvDKsLEwRmUJnWhMu37ABwpSEDEPwDilKwKwE+x0fbMp0bJMzNqK33B3fdgwP0Tnvbaxp1X6sSUSJsQ5LfEkuIqott7G8n60ygnwbLdfaFJYP+EHeDcx8NKX5fklwKwUah3uNE6kh2AriEOMKYuH8xyon2Q7TW2ARtuAl1LHaBA3T/o3CTNo5LasdYpnqCm1V5mP2znVAEVL8NpIt62w2WGVczTuBYZ8Tn0N5IzStGE5tJ8sTMKL+H9RvVFjjlUN/JzEKi4lGydLK3fovBYQ6M0a9cexUg== Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by BY1PR02MB10459.namprd02.prod.outlook.com (2603:10b6:a03:5a8::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8722.21; Thu, 15 May 2025 15:44:51 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%6]) with mapi id 15.20.8722.024; Thu, 15 May 2025 15:44:51 +0000 From: John Levon To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Thanos Makatos , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Paolo Bonzini , Peter Xu , David Hildenbrand , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Stefano Garzarella , "Michael S. Tsirkin" , Alex Williamson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , John Levon , John Johnson , Jagannathan Raman , Elena Ufimtseva Subject: [PATCH 13/27] vfio-user: implement message send infrastructure Date: Thu, 15 May 2025 16:43:58 +0100 Message-ID: <20250515154413.210315-14-john.levon@nutanix.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250515154413.210315-1-john.levon@nutanix.com> References: <20250515154413.210315-1-john.levon@nutanix.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: AM8P190CA0010.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::15) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|BY1PR02MB10459:EE_ X-MS-Office365-Filtering-Correlation-Id: f7a65b94-7f87-413e-7f84-08dd93c76e27 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ZXIN2Ezh7z0QoCrU1JIxRDZUtvvjMw+ATixvpQJ+YqXA8jbsmLPbG3H058jg?= =?us-ascii?Q?kejwXQB6RYIIsbgH0QAQQQvSzY9jiHJNg87UkEIsLSPXVyBnv6wzg1sR0IVd?= =?us-ascii?Q?LpVBo12Z30JWizyvSRiKV3T3NTKR4s3UJkEslK8ufY04S7fM21usyhIIC/OK?= =?us-ascii?Q?wtA+LNLE6/ccXOzJWrjJBL7qS9oiBm19JriMCNsjv1egZlykTYkkUetEaNhv?= =?us-ascii?Q?xfhj/CmivneWNFsfa1b0hqienoYq8uRNWaZYj85kdoGUL0UsH4wlrb+domuR?= =?us-ascii?Q?zVKd9zB7Ps9Dsdl629yxmxhKo0pOcAeaRBD1wVt5lkIVNFkmfdmHzBVLUxdi?= =?us-ascii?Q?+xKVE1sJYZKrLAW+xjBtul8MseqwEl/KgIHyYVH88L7SUvmduPr1bL6I+trd?= =?us-ascii?Q?Nh7duZX78BAZuShFW+NZpzjcQqr+kd/Bn6SkkDVGlQwjaWi8/vuqbESkp30P?= =?us-ascii?Q?eNK3k4oPlH3i1j2JJbLNgyHZmCE1cg0Znh60uHNggFuNMBJFfB4UB8xaOBtt?= =?us-ascii?Q?AZP6c+zFasDOgyqPM3QjZysBXbOTBzfMrPa05pECbm0YBr0iyepaKK/dATwm?= =?us-ascii?Q?F6p5c4XuZmZ1bCVT5kOMSvrQqMwQ8TQdPHFeLxA91Aau51fdiwWqCLiFP2j2?= =?us-ascii?Q?4VkXEB0vPfjNP1DNJIk2tRAvhm5uGRMkcw9HL9LsaWTZRbLZzjapGs2caNu8?= =?us-ascii?Q?LwKm1VtT3U+MaaRn/MKpivIqv0RKqpTIIPstzBK7+IH2XXTY/ba47CHitlU+?= =?us-ascii?Q?nq4CL44m39FEQe3fsYCOYSc1rxDFEUVHo+OPgubQ6cmzXW0yL7K6BbAeFCfu?= =?us-ascii?Q?yhhRYpJEZG8tKhGeB4X8KQtkDTshlt3K/cIb3DLv7k/YEIH56hRhKn+bomm8?= =?us-ascii?Q?vUe42m5cftG+lNPmG6kLvkoy4rPG9pltuKJFkw4jwD9MsoE/zpL9oZNFO1a6?= =?us-ascii?Q?zw0/oLZIkLYUBsYU3dHp9lHDbcpr9SpuPUO0DlcXGveL7xVBLBq7A1DOdn4X?= =?us-ascii?Q?pYOk72ZcGbORu++X1lgn30UN7xteXeMCpivYUrCmlJeD+xQug1egth2miUPV?= =?us-ascii?Q?0FjJpgggw/vyOBkwmwXuiLhrZoX9xk7HsSDvJWeQiF/9X12Qda0YNB0hxy2B?= =?us-ascii?Q?DCRz7VwByB5ZdovHxUKdwLUzRheaCfmmqYS3cfWqcQL6+GIdZgiO0Cz0rC2b?= =?us-ascii?Q?P60T4IbpBbL9UffxTqSYZYVL3UcY6HFfzafpMyfQLPb09XHWDJiO76EbbUtA?= =?us-ascii?Q?U+3ABHtuzzcpm+SAYalHqpGrZyrE35dNUjv9fcwD5sBe9ntjqzMCeFfaALWV?= =?us-ascii?Q?tMXhoj6SDv5JR+g12H0UHzSoJlqwU+8e0z7G8naYM/CZHsNglIWG3crK/RKU?= =?us-ascii?Q?/L5iVTb5olM8qYDVzsWoD0WYKM7Sv7q4IR9ovw4gjXmbAu5TXVP3Uce3HZAM?= =?us-ascii?Q?Df8H7aq2uAE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(7416014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RXIrhKTcAmdHFujo5G2KmoUgjl89+ET0yybd2Cc8WhNLXuy686IEI8ebZ03f?= =?us-ascii?Q?N6GJOpc4GBOSMZbqLR9CObacIZyVqTaJxLRE8mYYaearYGhQ+aqb4EEO+jGQ?= =?us-ascii?Q?F1wgDd6OerJ7nvSxH97RkHG541TpVJkZYUbo1nBYGs37nCUakCc9pTLSgj0l?= =?us-ascii?Q?K4UphYjl4HcWmEDNCnFVE8PsdS49/m4p5d2exERetXWg94K9HwUyXU+bYnWr?= =?us-ascii?Q?a2jIuBgDNqAOWckFGqLOBlOCeup8MxvnJA8JEOBqob/DgRd0N2z/WGyFOEDn?= =?us-ascii?Q?sCQG0GHIED1kA6BGDfHCuTOmUaKefHQ2wrnwU1Rkqsa4ZXk4uPEHDPqAvaP4?= =?us-ascii?Q?SnS/dQAGduUkGgBfRBgxXjilnqtpS61uhJR0J0qb6ga6OCIHaznuMm96PpNs?= =?us-ascii?Q?1I2PPcOqZlTqsZCTz+czga9wj5XW1zNxvcxVX6HEeDo/bGTtAqV2+bm6f3Dj?= =?us-ascii?Q?ZPXN0vrM6pwBkINQF4WaA2E8YScGLstfPS3me1ZzDglrPa8N9a3EAlp2HcAn?= =?us-ascii?Q?nfJf2JT9q44I0Oyng5JfJpeUf8U0JSgaQMPtfHWNxGlelIkfRLb6K6wMPMkp?= =?us-ascii?Q?8Bm9IVMk3V3QDPxyPcUfLypn5AWFD86/TErlarzZPYhLZMDZiIc+88KRXQPp?= =?us-ascii?Q?T724oMfhhpQ5ZRDEz+TAIfICP72Lo2VSWjQTPZKLeaWOJzLXWFPdcCd+QZEK?= =?us-ascii?Q?cYbemOhND0fqemHOwO6NVdmPx89vyZe0YpX8bfQOlDOFkX94ZsJRlwp5AfLW?= =?us-ascii?Q?Ssv1RvElnVnPhMUsdmUcXpq0+y5zRnQ7pRmRtmrUSU3YEVCFBUGnsG6IHXLT?= =?us-ascii?Q?6R3BC+dcvJBKXzjZCs56zh7MDp03KLG4Fr+GOT8kR8Q/ijWQK6glSRTC0vUY?= =?us-ascii?Q?hdG4G6/HKrOgivx0x4DmAfkx5PIoKt/FpS2ugxdAJdkS6M8w2BJtVt0kjQP0?= =?us-ascii?Q?rV9Ewk9VvJfkMa45CnD8E0BadP5DI4UDONwrQCSmWRgLs/YQRP9XlaJII5jx?= =?us-ascii?Q?Ak4WTXfzofWtFqKdKQLHEI76r9SCw46U5Hbf+N8KpHW+ktouikGsKxLX4C+r?= =?us-ascii?Q?NfdlpqDEY3DQKhp23Riq0q7WxAiw7AS8sfujcx3JuamfTRGlrtWI/kImCWP7?= =?us-ascii?Q?4pq+hUggV5ZdkGuomGZkfWCe/+dLBu2sfWphv1E0tkdHh80WykVElolKYZ/z?= =?us-ascii?Q?9paRweNsBX7TmBEo06vDCF8vBUTj8TXzgWAsBDAlSij/W6GGa49ROXGB4kBC?= =?us-ascii?Q?zxbRuM8FvutIjsX3AYj08iN33xmkgwgmAvpvXYqffO0GB73aX2RaDkrar4on?= =?us-ascii?Q?6Eh3ehCBHoOx/VRLCmilr73S52OvOStRP8y1FpJX+IK7CblRMnmJN8iv16yK?= =?us-ascii?Q?aHEBWLzyWlv7TqY0BF4w0LOIFOXl8lbGjI5+4Xlr2qZIkeC6cJv18QyQKVPF?= =?us-ascii?Q?mcufj6qVKuzxzYr7FXof1rU2vjVK89ASbVbiC0I/1tkVbqV/wVV1WsTLC0+6?= =?us-ascii?Q?FX8B6W8rkzPxyLyGFKxaKuckBplmiRCYcQPo3zfA2C6f6EXXoJHxP4CN2h5a?= =?us-ascii?Q?6/bvP2TC0Qw7Ekj+D5TZvD4IqQsIvSQIWSANuPxL?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: f7a65b94-7f87-413e-7f84-08dd93c76e27 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2025 15:44:51.4654 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S0oXhECPhQiqSYxwL4wHsx6nc2Jhif7q/klD4bQnoIlzpYq52/tNzhQyv9G7u3bRx63aF+XBmfWyswIRpKhz8w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR02MB10459 X-Proofpoint-ORIG-GUID: TIOaWvPc9Gvg68N4_AEq23gMuBk9Fx2b X-Proofpoint-GUID: TIOaWvPc9Gvg68N4_AEq23gMuBk9Fx2b X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNTE1MDE1NiBTYWx0ZWRfX56GEmQYUiR0e af2GShivaOhiqTCSaQ2L3GpN49bbMA/8AEbFbZ5j38oIGvwDApVxG/VUnNbjUzcITVvMuPR7nYJ EKGdz5LtK8hkDvgsQgBnGR066xZU1AX93n7CFP0Lj9XBHpjP/j9JElmns+BpuqPvObTIvhB3r/F xr58GQPGiWI34+NO5sWcAZqmqABUZUZJfJtlaKJI62O0B3Cbub3v+0JEDIC6uAhplt3cN+Go42R /+7Ijrmmj+MdjfafLPnpMUoQkjGqZcQ4/Bz242jPfjyw9JlUS4KKC/wGxGVOGLKJ26sPU1lISzB y2bQtu5hRi+8DDTC2Pwof/ExAl8vxIVyZfjUiP8wuGCZBsurDEY/8Me9gsuOFrxiT5y0ReFa1jl tXTMYhD8cZF4TlNadcNlN8mWZkUReSsToEFTiHfTbgif90igfx2077sdyKhmE3G6quFq7Zov X-Authority-Analysis: v=2.4 cv=KKlaDEFo c=1 sm=1 tr=0 ts=68260bf6 cx=c_pps a=5L1ZokDb34JZJib1CqSWiA==:117 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=dt9VzEwgFbYA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=UnscCpd6-9sbi9RN3AUA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.0.736,FMLib:17.12.80.40 definitions=2025-05-15_06,2025-05-15_01,2025-03-28_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass client-ip=148.163.155.12; envelope-from=john.levon@nutanix.com; helo=mx0b-002c1b01.pphosted.com X-Spam_score_int: -44 X-Spam_score: -4.5 X-Spam_bar: ---- X-Spam_report: (-4.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.686, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add plumbing for sending vfio-user messages on the control socket. Add initial version negotation on connection. Originally-by: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John Levon --- hw/vfio-user/protocol.h | 62 +++++ hw/vfio-user/proxy.h | 9 + hw/vfio-user/pci.c | 18 +- hw/vfio-user/proxy.c | 494 ++++++++++++++++++++++++++++++++++++++ hw/vfio-user/trace-events | 2 + 5 files changed, 583 insertions(+), 2 deletions(-) diff --git a/hw/vfio-user/protocol.h b/hw/vfio-user/protocol.h index 6300c681e7..74ea2a94e6 100644 --- a/hw/vfio-user/protocol.h +++ b/hw/vfio-user/protocol.h @@ -53,4 +53,66 @@ enum vfio_user_command { #define VFIO_USER_NO_REPLY 0x10 #define VFIO_USER_ERROR 0x20 + +/* + * VFIO_USER_VERSION + */ +typedef struct { + VFIOUserHdr hdr; + uint16_t major; + uint16_t minor; + char capabilities[]; +} VFIOUserVersion; + +#define VFIO_USER_MAJOR_VER 0 +#define VFIO_USER_MINOR_VER 0 + +#define VFIO_USER_CAP "capabilities" + +/* "capabilities" members */ +#define VFIO_USER_CAP_MAX_FDS "max_msg_fds" +#define VFIO_USER_CAP_MAX_XFER "max_data_xfer_size" +#define VFIO_USER_CAP_PGSIZES "pgsizes" +#define VFIO_USER_CAP_MAP_MAX "max_dma_maps" +#define VFIO_USER_CAP_MIGR "migration" + +/* "migration" members */ +#define VFIO_USER_CAP_PGSIZE "pgsize" +#define VFIO_USER_CAP_MAX_BITMAP "max_bitmap_size" + +/* + * Max FDs mainly comes into play when a device supports multiple interrupts + * where each ones uses an eventfd to inject it into the guest. + * It is clamped by the the number of FDs the qio channel supports in a + * single message. + */ +#define VFIO_USER_DEF_MAX_FDS 8 +#define VFIO_USER_MAX_MAX_FDS 16 + +/* + * Max transfer limits the amount of data in region and DMA messages. + * Region R/W will be very small (limited by how much a single instruction + * can process) so just use a reasonable limit here. + */ +#define VFIO_USER_DEF_MAX_XFER (1024 * 1024) +#define VFIO_USER_MAX_MAX_XFER (64 * 1024 * 1024) + +/* + * Default pagesizes supported is 4k. + */ +#define VFIO_USER_DEF_PGSIZE 4096 + +/* + * Default max number of DMA mappings is stolen from the + * linux kernel "dma_entry_limit" + */ +#define VFIO_USER_DEF_MAP_MAX 65535 + +/* + * Default max bitmap size is also take from the linux kernel, + * where usage of signed ints limits the VA range to 2^31 bytes. + * Dividing that by the number of bits per byte yields 256MB + */ +#define VFIO_USER_DEF_MAX_BITMAP (256 * 1024 * 1024) + #endif /* VFIO_USER_PROTOCOL_H */ diff --git a/hw/vfio-user/proxy.h b/hw/vfio-user/proxy.h index 0105479d02..33e8a4d83b 100644 --- a/hw/vfio-user/proxy.h +++ b/hw/vfio-user/proxy.h @@ -40,6 +40,7 @@ typedef struct VFIOUserMsg { uint32_t id; QemuCond cv; bool complete; + bool pending; enum msg_type type; } VFIOUserMsg; @@ -59,6 +60,12 @@ typedef struct VFIOUserProxy { struct QIOChannel *ioc; void (*request)(void *opaque, VFIOUserMsg *msg); void *req_arg; + uint64_t max_xfer_size; + uint64_t max_send_fds; + uint64_t max_dma; + uint64_t dma_pgsizes; + uint64_t max_bitmap; + uint64_t migr_pgsize; int flags; QemuCond close_cv; AioContext *ctx; @@ -81,6 +88,7 @@ typedef struct VFIOUserProxy { /* VFIOProxy flags */ #define VFIO_PROXY_CLIENT 0x1 +#define VFIO_PROXY_FORCE_QUEUED 0x4 typedef struct VFIODevice VFIODevice; @@ -89,5 +97,6 @@ void vfio_user_disconnect(VFIOUserProxy *proxy); void vfio_user_set_handler(VFIODevice *vbasedev, void (*handler)(void *opaque, VFIOUserMsg *msg), void *reqarg); +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp); #endif /* VFIO_USER_PROXY_H */ diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 53fbc218f3..ee343023f3 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -22,6 +22,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VFIOUserPCIDevice, VFIO_USER_PCI) struct VFIOUserPCIDevice { VFIOPCIDevice device; char *sock_name; + bool send_queued; /* all sends are queued */ }; /* @@ -81,6 +82,14 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) vbasedev->proxy = proxy; vfio_user_set_handler(vbasedev, vfio_user_pci_process_req, vdev); + if (udev->send_queued) { + proxy->flags |= VFIO_PROXY_FORCE_QUEUED; + } + + if (!vfio_user_validate_version(proxy, errp)) { + goto error; + } + vbasedev->name = g_strdup_printf("VFIO user <%s>", udev->sock_name); /* @@ -92,9 +101,13 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, as, errp)) { - error_prepend(errp, VFIO_MSG_PREFIX, vbasedev->name); - return; + goto error; } + + return; + +error: + error_prepend(errp, VFIO_MSG_PREFIX, vdev->vbasedev.name); } static void vfio_user_instance_init(Object *obj) @@ -145,6 +158,7 @@ static const Property vfio_user_pci_dev_properties[] = { DEFINE_PROP_UINT32("x-pci-sub-device-id", VFIOPCIDevice, sub_device_id, PCI_ANY_ID), DEFINE_PROP_STRING("socket", VFIOUserPCIDevice, sock_name), + DEFINE_PROP_BOOL("x-send-queued", VFIOUserPCIDevice, send_queued, false), }; static void vfio_user_pci_dev_class_init(ObjectClass *klass, const void *data) diff --git a/hw/vfio-user/proxy.c b/hw/vfio-user/proxy.c index 4a654d7597..0bc12da865 100644 --- a/hw/vfio-user/proxy.c +++ b/hw/vfio-user/proxy.c @@ -16,14 +16,19 @@ #include "hw/vfio-user/proxy.h" #include "hw/vfio-user/trace.h" #include "qapi/error.h" +#include "qobject/qdict.h" +#include "qobject/qjson.h" +#include "qobject/qnum.h" #include "qemu/error-report.h" #include "qemu/lockable.h" #include "qemu/main-loop.h" #include "system/iothread.h" +static int wait_time = 5000; /* wait up to 5 sec for busy servers */ static IOThread *vfio_user_iothread; static void vfio_user_shutdown(VFIOUserProxy *proxy); +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg); static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds); static VFIOUserFDs *vfio_user_getfds(int numfds); @@ -31,9 +36,16 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg); static void vfio_user_recv(void *opaque); static int vfio_user_recv_one(VFIOUserProxy *proxy); +static void vfio_user_send(void *opaque); +static int vfio_user_send_one(VFIOUserProxy *proxy); static void vfio_user_cb(void *opaque); static void vfio_user_request(void *opaque); +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg); +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize); +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags); static inline void vfio_user_set_error(VFIOUserHdr *hdr, uint32_t err) { @@ -52,6 +64,35 @@ static void vfio_user_shutdown(VFIOUserProxy *proxy) proxy->ctx, NULL, NULL); } +static int vfio_user_send_qio(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + VFIOUserFDs *fds = msg->fds; + struct iovec iov = { + .iov_base = msg->hdr, + .iov_len = msg->hdr->size, + }; + size_t numfds = 0; + int ret, *fdp = NULL; + Error *local_err = NULL; + + if (fds != NULL && fds->send_fds != 0) { + numfds = fds->send_fds; + fdp = fds->fds; + } + + ret = qio_channel_writev_full(proxy->ioc, &iov, 1, fdp, numfds, 0, + &local_err); + + if (ret == -1) { + vfio_user_set_error(msg->hdr, EIO); + vfio_user_shutdown(proxy); + error_report_err(local_err); + } + trace_vfio_user_send_write(msg->hdr->id, ret); + + return ret; +} + static VFIOUserMsg *vfio_user_getmsg(VFIOUserProxy *proxy, VFIOUserHdr *hdr, VFIOUserFDs *fds) { @@ -92,6 +133,7 @@ static void vfio_user_recycle(VFIOUserProxy *proxy, VFIOUserMsg *msg) msg->hdr = NULL; msg->fds = NULL; msg->complete = false; + msg->pending = false; QTAILQ_INSERT_HEAD(&proxy->free, msg, next); } @@ -386,6 +428,54 @@ err: return -1; } +/* + * Send messages from outgoing queue when the socket buffer has space. + * If we deplete 'outgoing', remove ourselves from the poll list. + */ +static void vfio_user_send(void *opaque) +{ + VFIOUserProxy *proxy = opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + if (proxy->state == VFIO_PROXY_CONNECTED) { + while (!QTAILQ_EMPTY(&proxy->outgoing)) { + if (vfio_user_send_one(proxy) < 0) { + return; + } + } + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, NULL, NULL, proxy); + } +} + +/* + * Send a single message. + * + * Sent async messages are freed, others are moved to pending queue. + */ +static int vfio_user_send_one(VFIOUserProxy *proxy) +{ + VFIOUserMsg *msg; + int ret; + + msg = QTAILQ_FIRST(&proxy->outgoing); + ret = vfio_user_send_qio(proxy, msg); + if (ret < 0) { + return ret; + } + + QTAILQ_REMOVE(&proxy->outgoing, msg, next); + if (msg->type == VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending = true; + } + + return 0; +} + static void vfio_user_cb(void *opaque) { VFIOUserProxy *proxy = opaque; @@ -447,6 +537,119 @@ static void vfio_user_request(void *opaque) } } +/* + * Messages are queued onto the proxy's outgoing list. + * + * It handles 3 types of messages: + * + * async messages - replies and posted writes + * + * There will be no reply from the server, so message + * buffers are freed after they're sent. + * + * nowait messages - map/unmap during address space transactions + * + * These are also sent async, but a reply is expected so that + * vfio_wait_reqs() can wait for the youngest nowait request. + * They transition from the outgoing list to the pending list + * when sent, and are freed when the reply is received. + * + * wait messages - all other requests + * + * The reply to these messages is waited for by their caller. + * They also transition from outgoing to pending when sent, but + * the message buffer is returned to the caller with the reply + * contents. The caller is responsible for freeing these messages. + * + * As an optimization, if the outgoing list and the socket send + * buffer are empty, the message is sent inline instead of being + * added to the outgoing list. The rest of the transitions are + * unchanged. + * + * returns 0 if the message was sent or queued + * returns -1 on send error + */ +static int vfio_user_send_queued(VFIOUserProxy *proxy, VFIOUserMsg *msg) +{ + int ret; + + /* + * Unsent outgoing msgs - add to tail + */ + if (!QTAILQ_EMPTY(&proxy->outgoing)) { + QTAILQ_INSERT_TAIL(&proxy->outgoing, msg, next); + return 0; + } + + /* + * Try inline - if blocked, queue it and kick send poller + */ + if (proxy->flags & VFIO_PROXY_FORCE_QUEUED) { + ret = QIO_CHANNEL_ERR_BLOCK; + } else { + ret = vfio_user_send_qio(proxy, msg); + } + if (ret == QIO_CHANNEL_ERR_BLOCK) { + QTAILQ_INSERT_HEAD(&proxy->outgoing, msg, next); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, + vfio_user_recv, proxy->ctx, + vfio_user_send, proxy); + return 0; + } + if (ret == -1) { + return ret; + } + + /* + * Sent - free async, add others to pending + */ + if (msg->type == VFIO_MSG_ASYNC) { + vfio_user_recycle(proxy, msg); + } else { + QTAILQ_INSERT_TAIL(&proxy->pending, msg, next); + msg->pending = true; + } + + return 0; +} + +static void vfio_user_send_wait(VFIOUserProxy *proxy, VFIOUserHdr *hdr, + VFIOUserFDs *fds, int rsize) +{ + VFIOUserMsg *msg; + int ret; + + if (hdr->flags & VFIO_USER_NO_REPLY) { + error_printf("vfio_user_send_wait on async message\n"); + vfio_user_set_error(hdr, EINVAL); + return; + } + + qemu_mutex_lock(&proxy->lock); + + msg = vfio_user_getmsg(proxy, hdr, fds); + msg->id = hdr->id; + msg->rsize = rsize ? rsize : hdr->size; + msg->type = VFIO_MSG_WAIT; + + ret = vfio_user_send_queued(proxy, msg); + + if (ret == 0) { + while (!msg->complete) { + if (!qemu_cond_timedwait(&msg->cv, &proxy->lock, wait_time)) { + VFIOUserMsgQ *list; + + list = msg->pending ? &proxy->pending : &proxy->outgoing; + QTAILQ_REMOVE(list, msg, next); + vfio_user_set_error(hdr, ETIMEDOUT); + break; + } + } + } + vfio_user_recycle(proxy, msg); + + qemu_mutex_unlock(&proxy->lock); +} static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets = QLIST_HEAD_INITIALIZER(vfio_user_sockets); @@ -475,6 +678,15 @@ VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) proxy = g_malloc0(sizeof(VFIOUserProxy)); proxy->sockname = g_strdup_printf("unix:%s", sockname); proxy->ioc = ioc; + + /* init defaults */ + proxy->max_xfer_size = VFIO_USER_DEF_MAX_XFER; + proxy->max_send_fds = VFIO_USER_DEF_MAX_FDS; + proxy->max_dma = VFIO_USER_DEF_MAP_MAX; + proxy->dma_pgsizes = VFIO_USER_DEF_PGSIZE; + proxy->max_bitmap = VFIO_USER_DEF_MAX_BITMAP; + proxy->migr_pgsize = VFIO_USER_DEF_PGSIZE; + proxy->flags = VFIO_PROXY_CLIENT; proxy->state = VFIO_PROXY_CONNECTED; @@ -572,3 +784,285 @@ void vfio_user_disconnect(VFIOUserProxy *proxy) g_free(proxy->sockname); g_free(proxy); } + +static void vfio_user_request_msg(VFIOUserHdr *hdr, uint16_t cmd, + uint32_t size, uint32_t flags) +{ + static uint16_t next_id; + + hdr->id = qatomic_fetch_inc(&next_id); + hdr->command = cmd; + hdr->size = size; + hdr->flags = (flags & ~VFIO_USER_TYPE) | VFIO_USER_REQUEST; + hdr->error_reply = 0; +} + +struct cap_entry { + const char *name; + bool (*check)(VFIOUserProxy *proxy, QObject *qobj, Error **errp); +}; + +static bool caps_parse(VFIOUserProxy *proxy, QDict *qdict, + struct cap_entry caps[], Error **errp) +{ + QObject *qobj; + struct cap_entry *p; + + for (p = caps; p->name != NULL; p++) { + qobj = qdict_get(qdict, p->name); + if (qobj != NULL) { + if (!p->check(proxy, qobj, errp)) { + return false; + } + qdict_del(qdict, p->name); + } + } + + /* warning, for now */ + if (qdict_size(qdict) != 0) { + warn_report("spurious capabilities"); + } + return true; +} + +static bool check_migr_pgsize(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t pgsize; + + if (qn == NULL || !qnum_get_try_uint(qn, &pgsize)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZE); + return false; + } + + /* must be larger than default */ + if (pgsize & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsize); + return false; + } + + proxy->migr_pgsize = pgsize; + return true; +} + +static bool check_bitmap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t bitmap_size; + + if (qn == NULL || !qnum_get_try_uint(qn, &bitmap_size)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + /* can only lower it */ + if (bitmap_size > VFIO_USER_DEF_MAX_BITMAP) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAX_BITMAP); + return false; + } + + proxy->max_bitmap = bitmap_size; + return true; +} + +static struct cap_entry caps_migr[] = { + { VFIO_USER_CAP_PGSIZE, check_migr_pgsize }, + { VFIO_USER_CAP_MAX_BITMAP, check_bitmap }, + { NULL } +}; + +static bool check_max_fds(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t max_send_fds; + + if (qn == NULL || !qnum_get_try_uint(qn, &max_send_fds) || + max_send_fds > VFIO_USER_MAX_MAX_FDS) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return false; + } + proxy->max_send_fds = max_send_fds; + return true; +} + +static bool check_max_xfer(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t max_xfer_size; + + if (qn == NULL || !qnum_get_try_uint(qn, &max_xfer_size) || + max_xfer_size > VFIO_USER_MAX_MAX_XFER) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_XFER); + return false; + } + proxy->max_xfer_size = max_xfer_size; + return true; +} + +static bool check_pgsizes(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t pgsizes; + + if (qn == NULL || !qnum_get_try_uint(qn, &pgsizes)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_PGSIZES); + return false; + } + + /* must be larger than default */ + if (pgsizes & (VFIO_USER_DEF_PGSIZE - 1)) { + error_setg(errp, "pgsize 0x%"PRIx64" too small", pgsizes); + return false; + } + + proxy->dma_pgsizes = pgsizes; + return true; +} + +static bool check_max_dma(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QNum *qn = qobject_to(QNum, qobj); + uint64_t max_dma; + + if (qn == NULL || !qnum_get_try_uint(qn, &max_dma)) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAP_MAX); + return false; + } + + /* can only lower it */ + if (max_dma > VFIO_USER_DEF_MAP_MAX) { + error_setg(errp, "%s too large", VFIO_USER_CAP_MAP_MAX); + return false; + } + + proxy->max_dma = max_dma; + return true; +} + +static bool check_migr(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict = qobject_to(QDict, qobj); + + if (qdict == NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP_MAX_FDS); + return true; + } + return caps_parse(proxy, qdict, caps_migr, errp); +} + +static struct cap_entry caps_cap[] = { + { VFIO_USER_CAP_MAX_FDS, check_max_fds }, + { VFIO_USER_CAP_MAX_XFER, check_max_xfer }, + { VFIO_USER_CAP_PGSIZES, check_pgsizes }, + { VFIO_USER_CAP_MAP_MAX, check_max_dma }, + { VFIO_USER_CAP_MIGR, check_migr }, + { NULL } +}; + +static bool check_cap(VFIOUserProxy *proxy, QObject *qobj, Error **errp) +{ + QDict *qdict = qobject_to(QDict, qobj); + + if (qdict == NULL) { + error_setg(errp, "malformed %s", VFIO_USER_CAP); + return false; + } + return caps_parse(proxy, qdict, caps_cap, errp); +} + +static struct cap_entry ver_0_0[] = { + { VFIO_USER_CAP, check_cap }, + { NULL } +}; + +static bool caps_check(VFIOUserProxy *proxy, int minor, const char *caps, + Error **errp) +{ + QObject *qobj; + QDict *qdict; + bool ret; + + qobj = qobject_from_json(caps, NULL); + if (qobj == NULL) { + error_setg(errp, "malformed capabilities %s", caps); + return false; + } + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + error_setg(errp, "capabilities %s not an object", caps); + qobject_unref(qobj); + return false; + } + ret = caps_parse(proxy, qdict, ver_0_0, errp); + + qobject_unref(qobj); + return ret; +} + +static GString *caps_json(void) +{ + QDict *dict = qdict_new(); + QDict *capdict = qdict_new(); + QDict *migdict = qdict_new(); + GString *str; + + qdict_put_int(migdict, VFIO_USER_CAP_PGSIZE, VFIO_USER_DEF_PGSIZE); + qdict_put_int(migdict, VFIO_USER_CAP_MAX_BITMAP, VFIO_USER_DEF_MAX_BITMAP); + qdict_put_obj(capdict, VFIO_USER_CAP_MIGR, QOBJECT(migdict)); + + qdict_put_int(capdict, VFIO_USER_CAP_MAX_FDS, VFIO_USER_MAX_MAX_FDS); + qdict_put_int(capdict, VFIO_USER_CAP_MAX_XFER, VFIO_USER_DEF_MAX_XFER); + qdict_put_int(capdict, VFIO_USER_CAP_PGSIZES, VFIO_USER_DEF_PGSIZE); + qdict_put_int(capdict, VFIO_USER_CAP_MAP_MAX, VFIO_USER_DEF_MAP_MAX); + + qdict_put_obj(dict, VFIO_USER_CAP, QOBJECT(capdict)); + + str = qobject_to_json(QOBJECT(dict)); + qobject_unref(dict); + return str; +} + +bool vfio_user_validate_version(VFIOUserProxy *proxy, Error **errp) +{ + g_autofree VFIOUserVersion *msgp = NULL; + GString *caps; + char *reply; + int size, caplen; + + caps = caps_json(); + caplen = caps->len + 1; + size = sizeof(*msgp) + caplen; + msgp = g_malloc0(size); + + vfio_user_request_msg(&msgp->hdr, VFIO_USER_VERSION, size, 0); + msgp->major = VFIO_USER_MAJOR_VER; + msgp->minor = VFIO_USER_MINOR_VER; + memcpy(&msgp->capabilities, caps->str, caplen); + g_string_free(caps, true); + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + + vfio_user_send_wait(proxy, &msgp->hdr, NULL, 0); + if (msgp->hdr.flags & VFIO_USER_ERROR) { + error_setg_errno(errp, msgp->hdr.error_reply, "version reply"); + return false; + } + + if (msgp->major != VFIO_USER_MAJOR_VER || + msgp->minor > VFIO_USER_MINOR_VER) { + error_setg(errp, "incompatible server version"); + return false; + } + + reply = msgp->capabilities; + if (reply[msgp->hdr.size - sizeof(*msgp) - 1] != '\0') { + error_setg(errp, "corrupt version reply"); + return false; + } + + if (!caps_check(proxy, msgp->minor, reply, errp)) { + return false; + } + + trace_vfio_user_version(msgp->major, msgp->minor, msgp->capabilities); + return true; +} diff --git a/hw/vfio-user/trace-events b/hw/vfio-user/trace-events index 89d6c11c4c..7a3645024f 100644 --- a/hw/vfio-user/trace-events +++ b/hw/vfio-user/trace-events @@ -4,3 +4,5 @@ vfio_user_recv_hdr(const char *name, uint16_t id, uint16_t cmd, uint32_t size, uint32_t flags) " (%s) id 0x%x cmd 0x%x size 0x%x flags 0x%x" vfio_user_recv_read(uint16_t id, int read) " id 0x%x read 0x%x" vfio_user_recv_request(uint16_t cmd) " command 0x%x" +vfio_user_send_write(uint16_t id, int wrote) " id 0x%x wrote 0x%x" +vfio_user_version(uint16_t major, uint16_t minor, const char *caps) " major %d minor %d caps: %s" -- 2.43.0