From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC0B4D525 for ; Fri, 10 Nov 2023 14:15:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=auristor.com header.i=jaltman@auristor.com header.b="abBXXAD1" Received: from sequoia-grove.ad.secure-endpoints.com (sequoia-grove.ad.secure-endpoints.com [IPv6:2001:470:1f07:f77:70f5:c082:a96a:5685]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFF00A7514 for ; Fri, 10 Nov 2023 06:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed; d=auristor.com; s=MDaemon; r=y; t=1699625739; x=1700230539; i=jaltman@auristor.com; q=dns/txt; h=Message-ID:Date: MIME-Version:User-Agent:Subject:Content-Language:To:Cc: References:From:Organization:In-Reply-To:Content-Type; bh=+zKZtf 8LBn9JS0V6QRsnAT35G9JDJNyQZEHRKQ5XcjU=; b=abBXXAD1gcW2K+Hsi6yaDx FmsZFLK7cgy+3E4YvBPce4mwirhk5zZ2JhIGMB1EE69M/XF4MgO1LEvG4GbI+cpf FHM5be2+RTrdQNax+KbH7I4Adv8NqucvDL4yftTm6kN2LBmpCYp++23HJsdppdEh RCSed/0+ek5X2reMx+UIg= X-MDAV-Result: clean X-MDAV-Processed: sequoia-grove.ad.secure-endpoints.com, Fri, 10 Nov 2023 09:15:39 -0500 Received: from [IPV6:2603:7000:73d:b00:d023:ff5f:54c2:9ec4] by auristor.com (IPv6:2001:470:1f07:f77:28d9:68fb:855d:c2a5) (MDaemon PRO v23.5.1c) with ESMTPSA id md5001003742719.msg; Fri, 10 Nov 2023 09:15:38 -0500 X-Spam-Processed: sequoia-grove.ad.secure-endpoints.com, Fri, 10 Nov 2023 09:15:38 -0500 (not processed: message from trusted or authenticated source) X-MDRemoteIP: 2603:7000:73d:b00:d023:ff5f:54c2:9ec4 X-MDHelo: [IPV6:2603:7000:73d:b00:d023:ff5f:54c2:9ec4] X-MDArrival-Date: Fri, 10 Nov 2023 09:15:38 -0500 X-MDOrigin-Country: US, NA X-Authenticated-Sender: jaltman@auristor.com X-Return-Path: prvs=1678d0c34b=jaltman@auristor.com X-Envelope-From: jaltman@auristor.com X-MDaemon-Deliver-To: linux-fsdevel@vger.kernel.org Message-ID: Date: Fri, 10 Nov 2023 09:15:31 -0500 Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 01/41] rxrpc: Fix RTT determination to use PING ACKs as a source Content-Language: en-US To: David Howells Cc: Marc Dionne , linux-afs@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org References: <6fadc6aa-4078-4f12-a4c7-235267d6e0b1@auristor.com> <20231109154004.3317227-1-dhowells@redhat.com> <20231109154004.3317227-2-dhowells@redhat.com> <3327953.1699567608@warthog.procyon.org.uk> From: Jeffrey E Altman Organization: AuriStor, Inc. In-Reply-To: <3327953.1699567608@warthog.procyon.org.uk> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="------------ms000100040607090605030007" X-MDCFSigsAdded: auristor.com This is a cryptographically signed message in MIME format. --------------ms000100040607090605030007 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 11/9/2023 5:06 PM, David Howells wrote: > I do not believe the ack_reason matters within rxrpc_input_ack(). As > long as >> the acked_serial is non-zero, >> rxrpc_complete_rtt_probe() can be called to attempt to compute an RTT. If >> there is an exact match for the >> acked_serial then an RTT can be computed and if acked_serial is later than the >> pending rtt probe, the probe >> can be abandoned with the following caveats. >> >> 1. Receiving an acked_serial that is later than the serial of the >> transmitted probe indicates that a packet >> transmitted after the probe was received first. Or that reordering >> of the transmitted packets occurred. >> Or that the probe was never received by the peer; or that the peer's >> response to the probe was lost in >> transit. >> 2. The serial number namespace is unsigned 32-bit shared across all of >> the call channels of the associated >> rx connection. As the serial numbers will wrap the use of after() >> within rxrpc_complete_rtt_probe to >> compare their values is questionable. If serial numbers will be >> compared in this manner then they >> need to be locally tracked and compared as unsigned 64-bit values >> where only the low 32-bits are >> transmitted on the wire and any wire serial number equal to zero is >> ignored. > I do ignore ack.serial == 0 for this purpose. Zero has the special meaning - this ACK is not explicitly in response to a received packet. However, as mentioned, the serial number counter wraps frequently and most RxRPC implementations do not transition from serial 0xffffffff -> 0x00000001 when wrapping. > I'm not sure how expanding it internally to 64-bits actually helps since the > upper 32 bits is not visible to the peer. rxrpc_complete_rtt_probe contains the following logic that relies on after() being able to detect a serial number wrap.         /* If a later serial is being acked, then mark this slot as          * being available.          */         if (after(acked_serial, orig_serial)) {             trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_obsolete, i,                        orig_serial, acked_serial, 0, 0);             clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);             smp_wmb();             set_bit(i, &call->rtt_avail);         } Otherwise, acked_serial = 0x01 will be considered smaller than orig_serial = 0xfffffffe and the slot will not be marked available. I will note that there is a similar problem with rxrpc_seq_t values which are u32 on the wire but which will wrap for calls that transmit more than approximately 5.5TB of data. Calls of this size are unlikely for a cache manager but are common for any service transmitting volume dumps. Jeffrey Altman --------------ms000100040607090605030007 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC DHEwggXSMIIEuqADAgECAhBAAYJpmi/rPn/F0fJyDlzMMA0GCSqGSIb3DQEBCwUAMDoxCzAJ BgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEz MB4XDTIyMDgwNDE2MDQ0OFoXDTI1MTAzMTE2MDM0OFowcDEvMC0GCgmSJomT8ixkAQETH0Ew MTQxMEQwMDAwMDE4MjY5OUEyRkQyMDAwMjMzQ0QxGTAXBgNVBAMTEEplZmZyZXkgRSBBbHRt YW4xFTATBgNVBAoTDEF1cmlTdG9yIEluYzELMAkGA1UEBhMCVVMwggEiMA0GCSqGSIb3DQEB AQUAA4IBDwAwggEKAoIBAQCkC7PKBBZnQqDKPtZPMLAy77zo2DPvwtGnd1hNjPvbXrpGxUb3 xHZRtv179LHKAOcsY2jIctzieMxf82OMyhpBziMPsFAG/ukihBMFj3/xEeZVso3K27pSAyyN fO/wJ0rX7G+ges22Dd7goZul8rPaTJBIxbZDuaykJMGpNq4PQ8VPcnYZx+6b+nJwJJoJ46kI EEfNh3UKvB/vM0qtxS690iAdgmQIhTl+qfXq4IxWB6b+3NeQxgR6KLU4P7v88/tvJTpxIKkg 9xj89ruzeThyRFd2DSe3vfdnq9+g4qJSHRXyTft6W3Lkp7UWTM4kMqOcc4VSRdufVKBQNXjG IcnhAgMBAAGjggKcMIICmDAOBgNVHQ8BAf8EBAMCBPAwgYQGCCsGAQUFBwEBBHgwdjAwBggr BgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVudHJ1c3QuY29tMEIGCCsGAQUF BzAChjZodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2NlcnRzL3RydXN0aWRjYWEx My5wN2MwHwYDVR0jBBgwFoAULbfeG1l+KpguzeHUG+PFEBJe6RQwCQYDVR0TBAIwADCCASsG A1UdIASCASIwggEeMIIBGgYLYIZIAYb5LwAGAgEwggEJMEoGCCsGAQUFBwIBFj5odHRwczov L3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmljYXRlcy9wb2xpY3kvdHMvaW5kZXguaHRt bDCBugYIKwYBBQUHAgIwga0MgapUaGlzIFRydXN0SUQgQ2VydGlmaWNhdGUgaGFzIGJlZW4g aXNzdWVkIGluIGFjY29yZGFuY2Ugd2l0aCBJZGVuVHJ1c3QncyBUcnVzdElEIENlcnRpZmlj YXRlIFBvbGljeSBmb3VuZCBhdCBodHRwczovL3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRp ZmljYXRlcy9wb2xpY3kvdHMvaW5kZXguaHRtbDBFBgNVHR8EPjA8MDqgOKA2hjRodHRwOi8v dmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2NybC90cnVzdGlkY2FhMTMuY3JsMB8GA1UdEQQY MBaBFGphbHRtYW5AYXVyaXN0b3IuY29tMB0GA1UdDgQWBBQB+nzqgljLocLTsiUn2yWqEc2s gjAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwDQYJKoZIhvcNAQELBQADggEBAJwV eycprp8Ox1npiTyfwc5QaVaqtoe8Dcg2JXZc0h4DmYGW2rRLHp8YL43snEV93rPJVk6B2v4c WLeQfaMrnyNeEuvHx/2CT44cdLtaEk5zyqo3GYJYlLcRVz6EcSGHv1qPXgDT0xB/25etwGYq utYF4Chkxu4KzIpq90eDMw5ajkexw+8ARQz4N5+d6NRbmMCovd7wTGi8th/BZvz8hgKUiUJo Qle4wDxrdXdnIhCP7g87InXKefWgZBF4VX21t2+hkc04qrhIJlHrocPG9mRSnnk2WpsY0MXt a8ivbVKtfpY7uSNDZSKTDi1izEFH5oeQdYRkgIGb319a7FjslV8wggaXMIIEf6ADAgECAhBA AXA7OrqBjMk8rp4OuNQSMA0GCSqGSIb3DQEBCwUAMEoxCzAJBgNVBAYTAlVTMRIwEAYDVQQK EwlJZGVuVHJ1c3QxJzAlBgNVBAMTHklkZW5UcnVzdCBDb21tZXJjaWFsIFJvb3QgQ0EgMTAe Fw0yMDAyMTIyMTA3NDlaFw0zMDAyMTIyMTA3NDlaMDoxCzAJBgNVBAYTAlVTMRIwEAYDVQQK EwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEzMIIBIjANBgkqhkiG9w0BAQEF AAOCAQ8AMIIBCgKCAQEAu6sUO01SDD99PM+QdZkNxKxJNt0NgQE+Zt6ixaNP0JKSjTd+SG5L wqxBWjnOgI/3dlwgtSNeN77AgSs+rA4bK4GJ75cUZZANUXRKw/et8pf9Qn6iqgB63OdHxBN/ 15KbM3HR+PyiHXQoUVIevCKW8nnlWnnZabT1FejOhRRKVUg5HACGOTfnCOONrlxlg+m1Vjgn o1uNqNuLM/jkD1z6phNZ/G9IfZGI0ppHX5AA/bViWceX248VmefNhSR14ADZJtlAAWOi2un0 3bqrBPHA9nDyXxI8rgWLfUP5rDy8jx2hEItg95+ORF5wfkGUq787HBjspE86CcaduLka/Bk2 VwIDAQABo4IChzCCAoMwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwgYkG CCsGAQUFBwEBBH0wezAwBggrBgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVu dHJ1c3QuY29tMEcGCCsGAQUFBzAChjtodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29t L3Jvb3RzL2NvbW1lcmNpYWxyb290Y2ExLnA3YzAfBgNVHSMEGDAWgBTtRBnA0/AGi+6ke75C 5yZUyI42djCCASQGA1UdIASCARswggEXMIIBEwYEVR0gADCCAQkwSgYIKwYBBQUHAgEWPmh0 dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRl eC5odG1sMIG6BggrBgEFBQcCAjCBrQyBqlRoaXMgVHJ1c3RJRCBDZXJ0aWZpY2F0ZSBoYXMg YmVlbiBpc3N1ZWQgaW4gYWNjb3JkYW5jZSB3aXRoIElkZW5UcnVzdCdzIFRydXN0SUQgQ2Vy dGlmaWNhdGUgUG9saWN5IGZvdW5kIGF0IGh0dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20v Y2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRleC5odG1sMEoGA1UdHwRDMEEwP6A9oDuGOWh0 dHA6Ly92YWxpZGF0aW9uLmlkZW50cnVzdC5jb20vY3JsL2NvbW1lcmNpYWxyb290Y2ExLmNy bDAdBgNVHQ4EFgQULbfeG1l+KpguzeHUG+PFEBJe6RQwHQYDVR0lBBYwFAYIKwYBBQUHAwIG CCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQB/7BKcygLX6Nl4a03cDHt7TLdPxCzFvDF2 bkVYCFTRX47UfeomF1gBPFDee3H/IPlLRmuTPoNt0qjdpfQzmDWN95jUXLdLPRToNxyaoB5s 0hOhcV6H08u3FHACBif55i0DTDzVSaBv0AZ9h1XeuGx4Fih1Vm3Xxz24GBqqVudvPRLyMJ7u 6hvBqTIKJ53uCs3dyQLZT9DXnp+kJv8y7ZSAY+QVrI/dysT8avtn8d7k7azNBkfnbRq+0e88 QoBnel6u+fpwbd5NLRHywXeH+phbzULCa+bLPRMqJaW2lbhvSWrMHRDy3/d8HvgnLCBFK2s4 Spns4YCN4xVcbqlGWzgolHCKUH39vpcsDo1ymZFrJ8QR6ihIn8FmJ5oKwAnnd/G6ADXFC9bu db9+532phSAXOZrrecIQn+vtP366PC+aClAPsIIDJDsotS5z4X2JUFsNIuEgXGqhiKE7SuZb rFG9sdcLprSlJN7TsRDc0W2b9nqwD+rj/5MN0C+eKwha+8ydv0+qzTyxPP90KRgaegGowC4d UsZyTk2n4Z3MuAHX5nAZL/Vh/SyDj/ajorV44yqZBzQ3ChKhXbfUSwe2xMmygA2Z5DRwMRJn p/BscizYdNk2WXJMTnH+wVLN8sLEwEtQR4eTLoFmQvrK2AMBS9kW5sBkMzINt/ZbbcZ3F+eA MDGCAxQwggMQAgEBME4wOjELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEXMBUG A1UEAxMOVHJ1c3RJRCBDQSBBMTMCEEABgmmaL+s+f8XR8nIOXMwwDQYJYIZIAWUDBAIBBQCg ggGXMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTIzMTExMDE0 MTUzMVowLwYJKoZIhvcNAQkEMSIEIDL9LJDtjjx467a6oRd2dbHe0yxpM30RIiDj2kpnYtXB MF0GCSsGAQQBgjcQBDFQME4wOjELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEX MBUGA1UEAxMOVHJ1c3RJRCBDQSBBMTMCEEABgmmaL+s+f8XR8nIOXMwwXwYLKoZIhvcNAQkQ AgsxUKBOMDoxCzAJBgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRy dXN0SUQgQ0EgQTEzAhBAAYJpmi/rPn/F0fJyDlzMMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZI AWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEAM53u jGA8KCVqvtcB/lO3Qt3X0DxJehyV9FV8vKMDGuFMPURKhn/AYXHt5tItWcRYJbI9vKB3pmnv E6UmB18075bjGHGI1rf2tRlimI4CbFL4SCO34yLmHnsS2WE/+pqlngYwWQ0lstaKpvVXvJDT hOP/Md3kchqqWonC+35MoztU/xymxA2tpgtcv/qnlZ+V3OXdmjXTmFobvj+j5M5MX3HXgRPL 8IBLs0b+CDNRFZJcHWgEKj0aeTkSG539nual04sUX9EwgvE1knsO/HINYY/JT+YPGEnp2eL3 UYf13CkLPb9ipraBMshAPia+pX6zpqm42AdtExPeUe1j1mbwCwAAAAAAAA== --------------ms000100040607090605030007--