From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailgw-01.dd24.net ([193.46.215.41]:47882 "EHLO mailgw01.dd24.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751593AbbCLEsQ (ORCPT ); Thu, 12 Mar 2015 00:48:16 -0400 Received: from mailpolicy-01.live.igb.homer.key-systems.net (mailpolicy-02.live.igb.homer.key-systems.net [192.168.1.27]) by mailgw01.dd24.net (Postfix) with ESMTP id DE0135FB5D for ; Thu, 12 Mar 2015 04:48:14 +0000 (UTC) Received: from mailgw01.dd24.net ([192.168.1.35]) by mailpolicy-01.live.igb.homer.key-systems.net (mailpolicy-02.live.igb.homer.key-systems.net [192.168.1.25]) (amavisd-new, port 10235) with ESMTP id iFBd5JxzvsTI for ; Thu, 12 Mar 2015 04:48:12 +0000 (UTC) Received: from heisenberg.fritz.box (ppp-93-104-121-105.dynamic.mnet-online.de [93.104.121.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mailgw01.dd24.net (Postfix) with ESMTPSA for ; Thu, 12 Mar 2015 04:48:12 +0000 (UTC) Message-ID: <1426135691.2574.46.camel@scientia.net> Subject: Re: is cryptographically secure integrity checking possible with btrfs? From: Christoph Anton Mitterer To: linux-btrfs@vger.kernel.org Date: Thu, 12 Mar 2015 05:48:11 +0100 In-Reply-To: <20150312040706.GA26641@localhost.localdomain> References: <1426131319.2574.18.camel@scientia.net> <20150312040706.GA26641@localhost.localdomain> Content-Type: multipart/signed; micalg="sha-512"; protocol="application/x-pkcs7-signature"; boundary="=-PlrC2eTu9/pWKA88kFbp" Mime-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org List-ID: --=-PlrC2eTu9/pWKA88kFbp Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, 2015-03-12 at 12:07 +0800, Liu Bo wrote:=20 > checksum is updated along with the corresponding datablock's change, but > updating the super block depends on when btrfs starts committing > transaction. Well I guess it was clear that you don't update the whole chain on every single write =3D) (other wise btrfs would write to it all the time)... but AFAIU it would be enough if everything is flushed out to disk at the end (i.e. when the user unmounts and the checksum is printed). Of course one general problem remains (which I didn't think of before): What happens in case the system crashes... then the user would have no way to now the last valid super-checksum. I'm not sure whether this can be easily solved.. if at all. One idea would be to make a snapshot of the whole fs in the beginning, so in case of a crash, the user could go at least back to that validated state. But since snapshots are on subvols and not the whole fs, this wouldn't really work. And even if there was some way implemented to keep the pre-mounted state of the fs (until unmount) this could cost a lot of space. Hmm,... I think this is actually a bigger problem on the whole idea, maybe it makes it unreasonable to use such functionality on "unattended" filesystems and even when I sit next to my harddisk while the system crashes (where I can be sure that no one forged any data after the crash) one would need to trust the storage device. > > On mount one could then specify the expected checksum for the > > superblock. If it differs already, then the mount should obviously fail= s > > right away (or try backup superblocks and that like, but again only use > > them if they match the sum). >=20 > That's exactly what we have now in btrfs, but the superblock checksum is = only to > verify superblock itself, nothing more. Sure but that's obviously different and not enough for cryptographic integrity validation of *all* the data in the fs. =20 > > Obviously one would also need an operation mode in which btrfs dies wit= h > > bells and red signs as soon as something (data or metadata) is read > > afterwards, which doesn't match the expected checksums. > > again, RAID copies, etc. could of course be tried - but if no valid cop= y > > is found, then it should assume compromise and the read operation shoul= d > > error out and not deliver any data at all (it may be compromised and > > nothing should use it). >=20 > Okay, if you kept using btrfs for a while, you can figure out the above > is basically true Hehe... well I've wrote about that some time ago, IMHO (and no one here should take this offensive) the documentation is in a suboptimal state, especially also since there are still many things going on. Especially what one could really expect from the system, things like - Over which data are checksums calculated, and what exactly happens on errors (both data and metadata). Endusers are do not necessarily know that btrfs will always verify checksums and with RAID look for a valid block and give back only such (unlike MD or hardware RAID do usually)- - What happens if I abort defrag (e.g. Ctrl-C)? Will it abort cleanly? Will all data be lost? What if the system crashes during defrag or balance? Are these procedures so safe that they cannot loose data in these cases? Will the procedure continue automatically after reboot? - Why does btrfs still need a log and what is it used for. Anyway,... different topic ;) > but there are some differences, > a) if we find a metadata checksum mismatch, we do go to > get another copy(we usually have two copies for metadata) for good copy > if it is, and if no, we dont make btrfs die but throw a warning(actually > it can refuse to mount if you got such errors during mount and couldnt > find good copies). And what if bad meta-data is found during normal operations (e.g. read/write) with no good copy being found? Does it just give a warning and try to do it's best to give back the read file/it's attrs/permissions/etc. (which could already be a problem)? Or will at also fail to read any files/dirs/etc. for which that meta-data would have been needed (e.g. their checksums). > > Of course people might still want to read such "compromised" blocks > > (e.g. when they are sure that they've only suffered from accidental dat= a > > corruption and try to rescue as much as possible), but that should then > > require a special mount option. >=20 > So you're asking for a strict mode, but for datablock corruption, is it > OK for you to just flip btrfs into readonly mode instead of making it > die? Not sure what you mean. If btrfs would be operating in "normal" (i.e. not a kind of data recovery mode explicitly chosen by the user at mount time) than it should fail to give back any data/metadata which it cannot verify. I'm not sure whether it would need to go into readonly mode... perhaps it would be a good idea, cause one could theoretically think about blocking attacks against such integrity protected filesystem, e.g. something like: | +--trusted-keys.d | \-- ... +--revoked-keys.d \-- ... If an attacker would manage to just corrupt the metadata for some of the revoked-keys and the fs would just not give them back, he could use that for tricky attacks. So indeed, it might be better to go into read-only mode when a unrecoverable validation error occurs... or perhaps even allow the user to make a combination of remount-ro + kernel panic (some software might still continue to work even when the fs is ro and for some users it might be better to completely die than to use bad/insecure data). Cheers, Chris. --=-PlrC2eTu9/pWKA88kFbp Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCCEZIw ggW/MIIDp6ADAgECAgMCOakwDQYJKoZIhvcNAQENBQAwVDEUMBIGA1UEChMLQ0FjZXJ0IEluYy4x HjAcBgNVBAsTFWh0dHA6Ly93d3cuQ0FjZXJ0Lm9yZzEcMBoGA1UEAxMTQ0FjZXJ0IENsYXNzIDMg Um9vdDAeFw0xNDA2MTIxNjM2MThaFw0xNjA2MTExNjM2MThaMHwxITAfBgNVBAMTGENocmlzdG9w aCBBbnRvbiBNaXR0ZXJlcjEkMCIGCSqGSIb3DQEJARYVY2FsZXN0eW9Ac2NpZW50aWEubmV0MTEw LwYJKoZIhvcNAQkBFiJtYWlsQGNocmlzdG9waC5hbnRvbi5taXR0ZXJlci5uYW1lMIIBIjANBgkq hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4phP/j9vT9dZT+k3ffHxvRWMOuzBnu5O3Fl4y2+WL7pL rfLiEhWzGXhHvjSqpt4vCNSdqy43453nnu8+hMb+uEtqSIL1AHU5eLhuDNVN9S4bt9E7nA2WKYBU LCUi/xCD/GL7ToyJNwhrhzcCZ7pXSc3xVqFoC4f6weU9ExhoEZQNRpTM0BFCOi4fRxvKFNnUYgjK hqy0Ta5H0Xx86mAp0Q4dxoD7mhI5iTF6TRkUheELxF24JCuAf04M89Cwft6DRH1FpJ3yvgW2B5U5 aFSL4ZnF4N/wyCB7Dkm1rQ7RCAvw5btkf0VdPnU7ccDCx8HEc2nxK/lbCjrznvh3sa1CCwIDAQAB o4IBcDCCAWwwDAYDVR0TAQH/BAIwADBWBglghkgBhvhCAQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNl cnRpZmljYXRlIGZvciBGUkVFIGhlYWQgb3ZlciB0byBodHRwOi8vd3d3LkNBY2VydC5vcmcwDgYD VR0PAQH/BAQDAgOoMEAGA1UdJQQ5MDcGCCsGAQUFBwMEBggrBgEFBQcDAgYKKwYBBAGCNwoDBAYK KwYBBAGCNwoDAwYJYIZIAYb4QgQBMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcwAYYWaHR0cDov L29jc3AuY2FjZXJ0Lm9yZzA4BgNVHR8EMTAvMC2gK6AphidodHRwOi8vY3JsLmNhY2VydC5vcmcv Y2xhc3MzLXJldm9rZS5jcmwwRAYDVR0RBD0wO4EVY2FsZXN0eW9Ac2NpZW50aWEubmV0gSJtYWls QGNocmlzdG9waC5hbnRvbi5taXR0ZXJlci5uYW1lMA0GCSqGSIb3DQEBDQUAA4ICAQBefctiLgGl e5baspuozyA4k7Up7SVhGHbif6pQfoFc/9Thx9GXnYpX+U64PMyWBfWwHZIy52Vg0RVkvPi1t6mi GyBfoSpC6ooR0bKWtUIogw/ymqKWlTLVR8kbLqRmRk4juMtCXG2K3yMygX/rjkuUSuFj2Bjpkmzg CtMojbUMYbszePmhQ7DJ62YEdtKpcjN94QAsI5GWlIAbs3KJazAcaNCRJeXCLcUMchyKHJA+NXH5 az/ekBxBMBzJP2An20PP88UI4JW18z31KiG9UVGa2uO4l4aWgVe2GnhNEdCD/o48msJEWKAt5vl2 yMqr7ihmNPocU2+/FW0xPe/vftdOTD9pgXdSGf4prdD+23q2YvpalOCzr2p8yCJZNVBPMxAP4mL0 3OEktXza4wohqAmceXKfGUNwRGBaPvtIGnPrpLhCQ+2YJDg8g1UEsk23bKyZlJWeKJyVqOBsDJmj aBsN/qKhQFnav+zQdqGhMeaSisF/53mD3gyVYg2JRl18apgGbg32kyLmomqa0JbhnY3Dc3FVtZfe +P+s2Cyep3pVKvFer2llRoGm8TwraG5Yhyx8Oq/1qETpstjbURJOVBLDCV4AjOEUj0ZnE/tEo/DK yexgGaViNvjp+IZdFdJhYmsVjw4Q3vG7O0pfsLiYEyQjeDgjNEWDfa5/MufPywIfxzCCBb8wggOn oAMCAQICAwI5qTANBgkqhkiG9w0BAQ0FADBUMRQwEgYDVQQKEwtDQWNlcnQgSW5jLjEeMBwGA1UE CxMVaHR0cDovL3d3dy5DQWNlcnQub3JnMRwwGgYDVQQDExNDQWNlcnQgQ2xhc3MgMyBSb290MB4X DTE0MDYxMjE2MzYxOFoXDTE2MDYxMTE2MzYxOFowfDEhMB8GA1UEAxMYQ2hyaXN0b3BoIEFudG9u IE1pdHRlcmVyMSQwIgYJKoZIhvcNAQkBFhVjYWxlc3R5b0BzY2llbnRpYS5uZXQxMTAvBgkqhkiG 9w0BCQEWIm1haWxAY2hyaXN0b3BoLmFudG9uLm1pdHRlcmVyLm5hbWUwggEiMA0GCSqGSIb3DQEB AQUAA4IBDwAwggEKAoIBAQDimE/+P29P11lP6Td98fG9FYw67MGe7k7cWXjLb5Yvukut8uISFbMZ eEe+NKqm3i8I1J2rLjfjneee7z6Exv64S2pIgvUAdTl4uG4M1U31Lhu30TucDZYpgFQsJSL/EIP8 YvtOjIk3CGuHNwJnuldJzfFWoWgLh/rB5T0TGGgRlA1GlMzQEUI6Lh9HG8oU2dRiCMqGrLRNrkfR fHzqYCnRDh3GgPuaEjmJMXpNGRSF4QvEXbgkK4B/Tgzz0LB+3oNEfUWknfK+BbYHlTloVIvhmcXg 3/DIIHsOSbWtDtEIC/Dlu2R/RV0+dTtxwMLHwcRzafEr+VsKOvOe+HexrUILAgMBAAGjggFwMIIB bDAMBgNVHRMBAf8EAjAAMFYGCWCGSAGG+EIBDQRJFkdUbyBnZXQgeW91ciBvd24gY2VydGlmaWNh dGUgZm9yIEZSRUUgaGVhZCBvdmVyIHRvIGh0dHA6Ly93d3cuQ0FjZXJ0Lm9yZzAOBgNVHQ8BAf8E BAMCA6gwQAYDVR0lBDkwNwYIKwYBBQUHAwQGCCsGAQUFBwMCBgorBgEEAYI3CgMEBgorBgEEAYI3 CgMDBglghkgBhvhCBAEwMgYIKwYBBQUHAQEEJjAkMCIGCCsGAQUFBzABhhZodHRwOi8vb2NzcC5j YWNlcnQub3JnMDgGA1UdHwQxMC8wLaAroCmGJ2h0dHA6Ly9jcmwuY2FjZXJ0Lm9yZy9jbGFzczMt cmV2b2tlLmNybDBEBgNVHREEPTA7gRVjYWxlc3R5b0BzY2llbnRpYS5uZXSBIm1haWxAY2hyaXN0 b3BoLmFudG9uLm1pdHRlcmVyLm5hbWUwDQYJKoZIhvcNAQENBQADggIBAF59y2IuAaV7ltqym6jP IDiTtSntJWEYduJ/qlB+gVz/1OHH0Zedilf5Trg8zJYF9bAdkjLnZWDRFWS8+LW3qaIbIF+hKkLq ihHRspa1QiiDD/KaopaVMtVHyRsupGZGTiO4y0JcbYrfIzKBf+uOS5RK4WPYGOmSbOAK0yiNtQxh uzN4+aFDsMnrZgR20qlyM33hACwjkZaUgBuzcolrMBxo0JEl5cItxQxyHIockD41cflrP96QHEEw HMk/YCfbQ8/zxQjglbXzPfUqIb1RUZra47iXhpaBV7YaeE0R0IP+jjyawkRYoC3m+XbIyqvuKGY0 +hxTb78VbTE97+9+105MP2mBd1IZ/imt0P7berZi+lqU4LOvanzIIlk1UE8zEA/iYvTc4SS1fNrj CiGoCZx5cp8ZQ3BEYFo++0gac+ukuEJD7ZgkODyDVQSyTbdsrJmUlZ4onJWo4GwMmaNoGw3+oqFA Wdq/7NB2oaEx5pKKwX/neYPeDJViDYlGXXxqmAZuDfaTIuaiaprQluGdjcNzcVW1l974/6zYLJ6n elUq8V6vaWVGgabxPCtobliHLHw6r/WoROmy2NtREk5UEsMJXgCM4RSPRmcT+0Sj8MrJ7GAZpWI2 +On4hl0V0mFiaxWPDhDe8bs7Sl+wuJgTJCN4OCM0RYN9rn8y58/LAh/HMIIGCDCCA/CgAwIBAgIB ATANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290IENBMR4wHAYDVQQLExVodHRwOi8vd3d3 LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNBIENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG 9w0BCQEWEnN1cHBvcnRAY2FjZXJ0Lm9yZzAeFw0wNTEwMTQwNzM2NTVaFw0zMzAzMjgwNzM2NTVa MFQxFDASBgNVBAoTC0NBY2VydCBJbmMuMR4wHAYDVQQLExVodHRwOi8vd3d3LkNBY2VydC5vcmcx HDAaBgNVBAMTE0NBY2VydCBDbGFzcyAzIFJvb3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQCrSTURSHzSJn5TlM9Dqd0o10Iqi/OHeBlYfA+e2ol94fvrcpANdKGWZKufoCSZc9riVXbH F3v1BKxGuMO+f2SNEGwk82GcwPKQ+lHm9WkBY8MPVuJKQs/iRIwlKKjFeQl9RrmK8+nzNCkIReQc n8uUBByBqBSzmGXEQ+xOgo0J0b2qW42S0OzekMV/CsLj6+YxWl50PpczWejDAz1gM7/30W9HxM3u YoNSbi4ImqTZFRiRpoWSR7CuSOtttyHshRpocjWr//AQXcD0lKdq1TuSfkyQBX6TwSyLpI5idBVx bgtxA+qvFTia1NIFcm+M+SvrWnIl+TlG43IbPgTDZCciECqKT1inA62+tC4T7V2qSNfVfdQqe1z6 RgRQ5MwOQluM7dvyz/yWk+DbETZUYjQ4jwxgmzuXVjit89Jbi6Bb6k6WuHzX1aCGcEDTkSm3ojyt 9Yy7zxqSiuQ0e8DYbF/pCsLDpyCaWt8sXVJcukfVm+8kKHA4IC/VfynAskEDaJLM4JzMl0tF7zoQ CqtwOpiVcK01seqFK6QcgCExqa5geoAmSAC4AcCTY1UikTxW56/bOiXzjzFU6iaLgVn5odFTEcV7 nQP2dBHgbbEsPyyGkZlxmqZ3izRg0RS0LKydr4wQ05/EavhvE/xzWfdmQnQeiuP43NJvmJzLR5iV QAX76QIDAQABo4G/MIG8MA8GA1UdEwEB/wQFMAMBAf8wXQYIKwYBBQUHAQEEUTBPMCMGCCsGAQUF BzABhhdodHRwOi8vb2NzcC5DQWNlcnQub3JnLzAoBggrBgEFBQcwAoYcaHR0cDovL3d3dy5DQWNl cnQub3JnL2NhLmNydDBKBgNVHSAEQzBBMD8GCCsGAQQBgZBKMDMwMQYIKwYBBQUHAgEWJWh0dHA6 Ly93d3cuQ0FjZXJ0Lm9yZy9pbmRleC5waHA/aWQ9MTAwDQYJKoZIhvcNAQEEBQADggIBAH8IiKHa GlBJ2on7oQhy84r3HsQ6tHlbIDCxRd7CXdNlafHCXVRUPIVfuXtCkcKZ/RtRm6tGpaEQU55tiKxz biwzpvD0nuB1wT6IRanhZkP+VlrRekF490DaSjrxC1uluxYG5sLnk7mFTZdPsR44Q4Dvmw2M77in YACHV30eRBzLI++bPJmdr7UpHEV5FpZNJ23xHGzDwlVks7wU4vOkHx4y/CcVBc/dLq4+gmF78CEQ GPZE6lM5+dzQmiDgxrvgu1pPxJnIB721vaLbLmINQjRBvP+LivVRIqqIMADisNS8vmW61QNXeZvo 3MhN+FDtkaVSKKKs+zZYPumUK5FQhxvWXtaMzPcPEAxSTtAWYeXlCmy/F8dyRlecmPVsYGN6b165 Ti/Iubm7aoW8mA3t+T6XhDSUrgCvoeXnkm5OvfPi2RSLXNLrAWygF6UtEOucekq9ve7O/e0iQKtw OIj1CodqwqsFYMlIBdpTwd5Ed2qz8zw87YC8pjhKKSRf/lk7myV6VmMAZLldpGJ9VzZPrYPvH5JT oI53V93lYRE9IwCQTDz6o2CTBKOvNfYOao9PSmCnhQVsRqGP9Md246FZV/dxssRuFFxtbUFm3xuT sdQAw+7Lzzw9IYCpX2Nl/N3gX6T0K/CFcUHUZyX7GrGXrtaZghNB0m6lG5kngOcLqagAMYIC7TCC AukCAQEwWzBUMRQwEgYDVQQKEwtDQWNlcnQgSW5jLjEeMBwGA1UECxMVaHR0cDovL3d3dy5DQWNl cnQub3JnMRwwGgYDVQQDExNDQWNlcnQgQ2xhc3MgMyBSb290AgMCOakwDQYJYIZIAWUDBAIDBQCg ggFjMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE1MDMxMjA0NDgx MVowTwYJKoZIhvcNAQkEMUIEQE8cRQ3RYkYPgH/x2Kk6Kvv46ars5vuE4C1kobPCCzmevltM31cH zm+dLQIf9JtKvkNWV/NBIYo5GB7Im2Cq6xcwagYJKwYBBAGCNxAEMV0wWzBUMRQwEgYDVQQKEwtD QWNlcnQgSW5jLjEeMBwGA1UECxMVaHR0cDovL3d3dy5DQWNlcnQub3JnMRwwGgYDVQQDExNDQWNl cnQgQ2xhc3MgMyBSb290AgMCOakwbAYLKoZIhvcNAQkQAgsxXaBbMFQxFDASBgNVBAoTC0NBY2Vy dCBJbmMuMR4wHAYDVQQLExVodHRwOi8vd3d3LkNBY2VydC5vcmcxHDAaBgNVBAMTE0NBY2VydCBD bGFzcyAzIFJvb3QCAwI5qTANBgkqhkiG9w0BAQEFAASCAQBdEdh5UTNiTdb02120P/XpuLnTG7R2 7e5EImr6jPSwde0LmJbqxX2Pa6DmcLdfvcH0TuEPB2k7cbr+BG3jcltdXZAPtq9C5u2A1JqplY0X CuQu9cAOSIbND3oXSxvfW0Q2HmsGcd9sw0dACqO1iST5xSXqrvTwMqE2bxZ/RDYgXg3sQH5/U3yf mHeE4cJ1yJbGY7UTs+7n/nnAAvdxaKxfi45+B0LxcrUoFH0Y/yrQfDhTzi1AKwoiKQMcuGcGjqso 7KedkO9mLH0ND4SHyc342OjyFyVjmpwscB5LN3l4Wev+vyupI74T1FkEJu8bCX7z+zcIFCX7zUDq gNzjIXIUAAAAAAAA --=-PlrC2eTu9/pWKA88kFbp--