* [RFC] apfs: thoughts on upstreaming an out-of-tree module
@ 2025-02-28 1:53 Ethan Carter Edwards
2025-02-28 12:55 ` Theodore Ts'o
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Ethan Carter Edwards @ 2025-02-28 1:53 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-fsdevel, linux-staging, asahi
Lately, I have been thinking a lot about the lack of APFS support on
Linux. I was wondering what I could do about that. APFS support is not
in-tree, but there is a proprietary module sold by Paragon software [0].
Obviously, this could not be used in-tree. However, there is also an
open source driver that, from what I can tell, was once planned to be
upstreamed [1] with associated filesystem progs [2]. I think I would
base most of my work off of the existing FOSS tree.
The biggest barrier I see currently is the driver's use of bufferheads.
I realize that there has been a lot of work to move existing filesystem
implementations to iomap/folios, and adding a filesystem that uses
bufferheads would be antithetical to the purpose of that effort.
Additionally, there is a lot of ifndefs/C preprocessor magic littered
throughout the codebase that fixes functionality with various different
versions of Linux.
The first step would be to move away from bufferheads and the
versioning. I plan to start my work in the next few weeks, and hope to
have a working driver to submit to staging by the end of June. From
there, I will work to have it meet more kernel standards and hopefully
move into fs/ by the end of the year.
Before I started, I was wondering if anyone had any thoughts. I am open
to feedback. If you think this is a bad idea, let me know. I am very
passionate about the Asahi Linux project. I think this would be a good
way to indirectly give back and contribute to the project. While I
recognize that it is not one of Asahi's project goals (those being
mostly hardware support), I am confident many users would find it
helpful. I sure would.
Thanks,
Ethan Carter Edwards <ethan@ethancedwards.com>
[0]: https://www.paragon-software.com/us/home/apfs-linux/
[1]: https://github.com/linux-apfs/linux-apfs-rw
[2]: https://github.com/linux-apfs/apfsprogs
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards @ 2025-02-28 12:55 ` Theodore Ts'o 2025-03-01 16:26 ` Ethan Carter Edwards 2025-02-28 23:04 ` Sven Peter 2025-03-02 10:55 ` Dan Carpenter 2 siblings, 1 reply; 10+ messages in thread From: Theodore Ts'o @ 2025-02-28 12:55 UTC (permalink / raw) To: Ethan Carter Edwards; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi On Thu, Feb 27, 2025 at 08:53:56PM -0500, Ethan Carter Edwards wrote: > Lately, I have been thinking a lot about the lack of APFS support on > Linux. I was wondering what I could do about that. APFS support is not > in-tree, but there is a proprietary module sold by Paragon software [0]. > Obviously, this could not be used in-tree. However, there is also an > open source driver that, from what I can tell, was once planned to be > upstreamed [1] with associated filesystem progs [2]. I think I would > base most of my work off of the existing FOSS tree. > > The biggest barrier I see currently is the driver's use of bufferheads. > I realize that there has been a lot of work to move existing filesystem > implementations to iomap/folios, and adding a filesystem that uses > bufferheads would be antithetical to the purpose of that effort. > Additionally, there is a lot of ifndefs/C preprocessor magic littered > throughout the codebase that fixes functionality with various different > versions of Linux. I don't see the use of bufferheads as a fundamental barrier to the mainline kernel; certainly not for staging. First of all, there are a huge number of file systems which still use buffer heads, including: adfs affs befs bfs ecryptfs efs exfat ext2 ext4 fat freevxfs gfs2 hfs hfsplus hpfs isofs jfs minix nilfs2 ntfs3 ocfs2 omfs pstore qnx4 qnx6 romfs sysv udf ufs There are many reasons to move to folios, including better performance, and making it easier to make sure things are done correctly if you can take advantage of iomap. For example, with ext4 we plan to work towards moving to use folios and iomap for the data plane operations for buffered write (we already use iomap for Direct I/O, FIEMAP support, etc.) and while we might want to move away from buffer heads for metadata blocks, we would need to change the jbd2 layer to use some simplified layer that looks an awful lot like buffer heads before we could do that. We might try to fork buffer heads, and strip out everything we don't need, and then merge that with jbd2's journal_head structure, for example. But that's a mid-term to long-term project, because using bufferheads doesn't actually hurt anyone. (That being said, if anyone wants to help out with the project of allowing us to move jbd2 away from buffer heads, let me know --- patches are welcome.) In any case, cleaning up preprocessor magic and other thigns that were needed because the code was designed for out of tree use would be something that I'd encourage you to focus on first, and then try a proposal to submit it to staging. Cheers, - Ted P.S. Something that you might want to consider using is fstests (AKA xfstests), which is the gold standard for file system testing in Linux. I have a test appliance VM of xfstests, which you can find here[1]. I test x86 and arm64 kernels using Google Cloud, and on local systems, using qemu/kvm. For qemu/kvm testing, this is being used on Debian, Fedora, OpenSuSE, and MacOS. [1] https://github.com/tytso/xfstests-bld For kernel development on my Macbook Air M2, I can build arm64 kernels using Debian running in a Parallels VM, and then to avoid the double virtualization overhead, I run qemu on MacOS using the hvf accelerator. It shouldn't be hard to make this work on your Ashai Linux development system; see more on that below. For more details about this test infrastructure, including its use on Google Cloud see the presentation here[2]. I am using gce-xfstests to perform continuous integration testing by watching a git branch, and when it gets updated, the test manager (running in an e2-micro VM) automatically starts a 4 CPU VM to build the kernel, and then launches multiple 2 CPU VM's to test multiple file system configurations in parallel --- for example, I am currently running over two dozen fs kernels testing ext4, xfs, btrfs, and f2fs on a Linux-next branch every day). Running a smoke test costs pennies. A full-up test of a dozen ext4 configuration (a dozen VM's, running for 2 hours of wall clock time), costs under $2 at US retail prices. For APFS, if you start with a single configuration, with many of the tests disable because APFS won't many of the advanced file systems of ext4 and xfs, I'm guessing it will cost you less than 25 cents per test run. [2] https://thunk.org/gce-xfstests Or of course you can use qemu-xfstests or kvm-xfstests using local compute. I do love though being able to fire off a set of tests, then suspend my laptop, knowing that I will receive e-mail with the test results when they are ready. If you are interested in trying to use this on Asahi linux, I'd certainly be happy help you with it. I suspect modulo some instructures about which packages are needed, it shouldn't be that hard to run a test appliance. Building new versions of the appliance does require a Debian build chroot, which might be tricker to set up on Asahi, but that's not necessary while you are getting started. In any case, I strongly encourage file system developers to use xfstests earlier rather than later. See the last slide of [2] for my opinion of "File system development without testing". :-) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-02-28 12:55 ` Theodore Ts'o @ 2025-03-01 16:26 ` Ethan Carter Edwards 0 siblings, 0 replies; 10+ messages in thread From: Ethan Carter Edwards @ 2025-03-01 16:26 UTC (permalink / raw) To: Theodore Ts'o; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi On 25/02/28 07:55AM, Theodore Ts'o wrote: > On Thu, Feb 27, 2025 at 08:53:56PM -0500, Ethan Carter Edwards wrote: > > Lately, I have been thinking a lot about the lack of APFS support on > > Linux. I was wondering what I could do about that. APFS support is not > > in-tree, but there is a proprietary module sold by Paragon software [0]. > > Obviously, this could not be used in-tree. However, there is also an > > open source driver that, from what I can tell, was once planned to be > > upstreamed [1] with associated filesystem progs [2]. I think I would > > base most of my work off of the existing FOSS tree. > > > > The biggest barrier I see currently is the driver's use of bufferheads. > > I realize that there has been a lot of work to move existing filesystem > > implementations to iomap/folios, and adding a filesystem that uses > > bufferheads would be antithetical to the purpose of that effort. > > Additionally, there is a lot of ifndefs/C preprocessor magic littered > > throughout the codebase that fixes functionality with various different > > versions of Linux. > > I don't see the use of bufferheads as a fundamental barrier to the > mainline kernel; certainly not for staging. First of all, there are a > huge number of file systems which still use buffer heads, including: > > adfs affs befs bfs ecryptfs efs exfat ext2 ext4 fat > freevxfs gfs2 hfs hfsplus hpfs isofs jfs minix nilfs2 > ntfs3 ocfs2 omfs pstore qnx4 qnx6 romfs sysv udf ufs > Good to know. I did not realized so many fs's stil used them. I will not let that stop me from submitting a the driver to staging. I definitely plan on moving them over as time permits. > There are many reasons to move to folios, including better > performance, and making it easier to make sure things are done > correctly if you can take advantage of iomap. > > For example, with ext4 we plan to work towards moving to use folios > and iomap for the data plane operations for buffered write (we already > use iomap for Direct I/O, FIEMAP support, etc.) and while we might > want to move away from buffer heads for metadata blocks, we would need > to change the jbd2 layer to use some simplified layer that looks an > awful lot like buffer heads before we could do that. We might try to > fork buffer heads, and strip out everything we don't need, and then > merge that with jbd2's journal_head structure, for example. But > that's a mid-term to long-term project, because using bufferheads > doesn't actually hurt anyone. (That being said, if anyone wants to > help out with the project of allowing us to move jbd2 away from buffer > heads, let me know --- patches are welcome.) > > In any case, cleaning up preprocessor magic and other thigns that were > needed because the code was designed for out of tree use would be > something that I'd encourage you to focus on first, and then try a > proposal to submit it to staging. Will do. > > Cheers, > > - Ted > > P.S. Something that you might want to consider using is fstests (AKA > xfstests), which is the gold standard for file system testing in > Linux. I have a test appliance VM of xfstests, which you can find > here[1]. I test x86 and arm64 kernels using Google Cloud, and on > local systems, using qemu/kvm. For qemu/kvm testing, this is being > used on Debian, Fedora, OpenSuSE, and MacOS. > > [1] https://github.com/tytso/xfstests-bld > > For kernel development on my Macbook Air M2, I can build arm64 kernels > using Debian running in a Parallels VM, and then to avoid the double > virtualization overhead, I run qemu on MacOS using the hvf > accelerator. It shouldn't be hard to make this work on your Ashai > Linux development system; see more on that below. > > For more details about this test infrastructure, including its use on > Google Cloud see the presentation here[2]. I am using gce-xfstests to > perform continuous integration testing by watching a git branch, and > when it gets updated, the test manager (running in an e2-micro VM) > automatically starts a 4 CPU VM to build the kernel, and then launches > multiple 2 CPU VM's to test multiple file system configurations in > parallel --- for example, I am currently running over two dozen fs > kernels testing ext4, xfs, btrfs, and f2fs on a Linux-next branch > every day). Running a smoke test costs pennies. A full-up test of a > dozen ext4 configuration (a dozen VM's, running for 2 hours of wall > clock time), costs under $2 at US retail prices. For APFS, if you > start with a single configuration, with many of the tests disable > because APFS won't many of the advanced file systems of ext4 and xfs, > I'm guessing it will cost you less than 25 cents per test run. > > [2] https://thunk.org/gce-xfstests > > Or of course you can use qemu-xfstests or kvm-xfstests using local > compute. I do love though being able to fire off a set of tests, then > suspend my laptop, knowing that I will receive e-mail with the test > results when they are ready. > > If you are interested in trying to use this on Asahi linux, I'd > certainly be happy help you with it. I suspect modulo some > instructures about which packages are needed, it shouldn't be that > hard to run a test appliance. Building new versions of the appliance > does require a Debian build chroot, which might be tricker to set up > on Asahi, but that's not necessary while you are getting started. > > In any case, I strongly encourage file system developers to use > xfstests earlier rather than later. See the last slide of [2] for my > opinion of "File system development without testing". :-) Good to know. I certainly agree with that last slide ;). Filesystems are far too important to develop without a test suite. Tests have certainly saved my butt in other projects, I have no doubt that it is the same for fs devel. Thank you for your detailed response. I was nervous that this idea would not be well-recieved since there is an existing out-of-tree driver, but since I am a believer in "upstream-first", I wanted to give this a try. Thanks, Ethan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards 2025-02-28 12:55 ` Theodore Ts'o @ 2025-02-28 23:04 ` Sven Peter 2025-03-01 16:39 ` Ethan Carter Edwards 2025-03-02 10:55 ` Dan Carpenter 2 siblings, 1 reply; 10+ messages in thread From: Sven Peter @ 2025-02-28 23:04 UTC (permalink / raw) To: Ethan Carter Edwards, linux-kernel; +Cc: linux-fsdevel, linux-staging, asahi Hi, On Fri, Feb 28, 2025, at 02:53, Ethan Carter Edwards wrote: > Lately, I have been thinking a lot about the lack of APFS support on > Linux. I was wondering what I could do about that. APFS support is not > in-tree, but there is a proprietary module sold by Paragon software [0]. > Obviously, this could not be used in-tree. However, there is also an > open source driver that, from what I can tell, was once planned to be > upstreamed [1] with associated filesystem progs [2]. I think I would > base most of my work off of the existing FOSS tree. > > The biggest barrier I see currently is the driver's use of bufferheads. > I realize that there has been a lot of work to move existing filesystem > implementations to iomap/folios, and adding a filesystem that uses > bufferheads would be antithetical to the purpose of that effort. > Additionally, there is a lot of ifndefs/C preprocessor magic littered > throughout the codebase that fixes functionality with various different > versions of Linux. > > The first step would be to move away from bufferheads and the > versioning. I plan to start my work in the next few weeks, and hope to > have a working driver to submit to staging by the end of June. From > there, I will work to have it meet more kernel standards and hopefully > move into fs/ by the end of the year. > > Before I started, I was wondering if anyone had any thoughts. I am open > to feedback. If you think this is a bad idea, let me know. I am very > passionate about the Asahi Linux project. I think this would be a good > way to indirectly give back and contribute to the project. While I > recognize that it is not one of Asahi's project goals (those being > mostly hardware support), I am confident many users would find it > helpful. I sure would. Agreed, I think it would be helpful for many people (especially those who still regularly dual boot between macOS and Linux) to have a working APFS driver upstream. This may also be useful once we fully bring up the Secure Enclave and need to read/write to at least a single file on the xART partition which has to be APFS because it's shared between all operating systems running on a single machine. It looks like there's still recent activity on that linux-apfs github repository. Have you reached out to the people working on it to see what their plans for upstreaming and/or future work are? Best, Sven ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-02-28 23:04 ` Sven Peter @ 2025-03-01 16:39 ` Ethan Carter Edwards 2025-03-05 7:23 ` Aditya Garg 0 siblings, 1 reply; 10+ messages in thread From: Ethan Carter Edwards @ 2025-03-01 16:39 UTC (permalink / raw) To: Sven Peter, Theodore Ts'o Cc: linux-kernel, linux-fsdevel, linux-staging, asahi On 25/03/01 12:04AM, Sven Peter wrote: > Hi, > > > On Fri, Feb 28, 2025, at 02:53, Ethan Carter Edwards wrote: > > Lately, I have been thinking a lot about the lack of APFS support on > > Linux. I was wondering what I could do about that. APFS support is not > > in-tree, but there is a proprietary module sold by Paragon software [0]. > > Obviously, this could not be used in-tree. However, there is also an > > open source driver that, from what I can tell, was once planned to be > > upstreamed [1] with associated filesystem progs [2]. I think I would > > base most of my work off of the existing FOSS tree. > > > > The biggest barrier I see currently is the driver's use of bufferheads. > > I realize that there has been a lot of work to move existing filesystem > > implementations to iomap/folios, and adding a filesystem that uses > > bufferheads would be antithetical to the purpose of that effort. > > Additionally, there is a lot of ifndefs/C preprocessor magic littered > > throughout the codebase that fixes functionality with various different > > versions of Linux. > > > > The first step would be to move away from bufferheads and the > > versioning. I plan to start my work in the next few weeks, and hope to > > have a working driver to submit to staging by the end of June. From > > there, I will work to have it meet more kernel standards and hopefully > > move into fs/ by the end of the year. > > > > Before I started, I was wondering if anyone had any thoughts. I am open > > to feedback. If you think this is a bad idea, let me know. I am very > > passionate about the Asahi Linux project. I think this would be a good > > way to indirectly give back and contribute to the project. While I > > recognize that it is not one of Asahi's project goals (those being > > mostly hardware support), I am confident many users would find it > > helpful. I sure would. > > Agreed, I think it would be helpful for many people (especially those > who still regularly dual boot between macOS and Linux) to have a working > APFS driver upstream. > This may also be useful once we fully bring up the Secure Enclave and need > to read/write to at least a single file on the xART partition which has > to be APFS because it's shared between all operating systems running on > a single machine. > > > It looks like there's still recent activity on that linux-apfs github > repository. Have you reached out to the people working on it to see > what their plans for upstreaming and/or future work are? I did ask the upstream maintainer and he said he did not see it happening. He specifically mentioned the use of bufferheads as a barrier to mainline merging, but I get the sense that he does not have the time/desire to commit to upstreaming it. [0] I did have a question/concern over the inclusion of the LZFSE/LZVN [1] compression library included in the module. It is BSD3 licensed, which, as far as I know is GPL-compatible, but what is the kernel's policy on external algorithms being included? It was originally developed by Apple and as far as I can tell is pretty necessary to read (and write) compressed files on APFS. Also, the library does produce an objtool warning. Ted, looping you in here, your thoughts? > > > > Best, > > > Sven > Thanks, Ethan [0]: https://github.com/linux-apfs/linux-apfs-rw/issues/68 [1]: https://github.com/linux-apfs/linux-apfs-rw/tree/master/lzfse ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-03-01 16:39 ` Ethan Carter Edwards @ 2025-03-05 7:23 ` Aditya Garg 2025-03-06 18:04 ` Theodore Ts'o 0 siblings, 1 reply; 10+ messages in thread From: Aditya Garg @ 2025-03-05 7:23 UTC (permalink / raw) To: Ethan Carter Edwards Cc: Sven Peter, Theodore Ts'o, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev, asahi@lists.linux.dev, ernesto@corellium.com Hi Ccing Ernesto >>> Lately, I have been thinking a lot about the lack of APFS support on >>> Linux. I was wondering what I could do about that. APFS support is not >>> in-tree, but there is a proprietary module sold by Paragon software [0]. >>> Obviously, this could not be used in-tree. However, there is also an >>> open source driver that, from what I can tell, was once planned to be >>> upstreamed [1] with associated filesystem progs [2]. I think I would >>> base most of my work off of the existing FOSS tree. >>> >>> The biggest barrier I see currently is the driver's use of bufferheads. >>> I realize that there has been a lot of work to move existing filesystem >>> implementations to iomap/folios, and adding a filesystem that uses >>> bufferheads would be antithetical to the purpose of that effort. >>> Additionally, there is a lot of ifndefs/C preprocessor magic littered >>> throughout the codebase that fixes functionality with various different >>> versions of Linux. >>> >>> The first step would be to move away from bufferheads and the >>> versioning. I plan to start my work in the next few weeks, and hope to >>> have a working driver to submit to staging by the end of June. From >>> there, I will work to have it meet more kernel standards and hopefully >>> move into fs/ by the end of the year. >>> >>> Before I started, I was wondering if anyone had any thoughts. I am open >>> to feedback. If you think this is a bad idea, let me know. I am very >>> passionate about the Asahi Linux project. I think this would be a good >>> way to indirectly give back and contribute to the project. While I >>> recognize that it is not one of Asahi's project goals (those being >>> mostly hardware support), I am confident many users would find it >>> helpful. I sure would. This driver tbh will not ‘really’ be helpful as far as T2 Macs are concerned. On these Macs, the T2 Security Chip encrypts all the APFS partitions on the internal SSD, and the key is in the T2 Chip. Even proprietary APFS drivers cannot read these partitions. I dunno how it works in Apple Silicon Macs. One practical use that we actually do have with this driver is that the recovery partition of macOS is still unencrypted. Now brcmfmac driver on Linux needs wifi firmware to work which on these is obtained from macOS. We mount the macOS Recovery partition and use a helper script [1] to get the firmware. [1] https://wiki.t2linux.org/guides/wifi-bluetooth/#setting-up >> >> Agreed, I think it would be helpful for many people (especially those >> who still regularly dual boot between macOS and Linux) to have a working >> APFS driver upstream. >> This may also be useful once we fully bring up the Secure Enclave and need >> to read/write to at least a single file on the xART partition which has >> to be APFS because it's shared between all operating systems running on >> a single machine. >> >> >> It looks like there's still recent activity on that linux-apfs github >> repository. Have you reached out to the people working on it to see >> what their plans for upstreaming and/or future work are? > > I did ask the upstream maintainer and he said he did not see it > happening. He specifically mentioned the use of bufferheads as a barrier > to mainline merging, but I get the sense that he does not have the > time/desire to commit to upstreaming it. [0] > > I did have a question/concern over the inclusion of the LZFSE/LZVN [1] > compression library included in the module. It is BSD3 licensed, which, > as far as I know is GPL-compatible, but what is the kernel's policy on > external algorithms being included? It was originally developed by Apple > and as far as I can tell is pretty necessary to read (and write) > compressed files on APFS. Also, the library does produce an objtool > warning. The original author was clear about this. Its is compatible with GPL: https://github.com/linux-apfs/linux-apfs-rw/issues/68#issuecomment-2156731178 > > Ted, looping you in here, your thoughts? ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-03-05 7:23 ` Aditya Garg @ 2025-03-06 18:04 ` Theodore Ts'o 2025-03-06 19:39 ` Sven Peter 0 siblings, 1 reply; 10+ messages in thread From: Theodore Ts'o @ 2025-03-06 18:04 UTC (permalink / raw) To: Aditya Garg Cc: Ethan Carter Edwards, Sven Peter, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev, asahi@lists.linux.dev, ernesto@corellium.com On Wed, Mar 05, 2025 at 07:23:55AM +0000, Aditya Garg wrote: > > This driver tbh will not ‘really’ be helpful as far as T2 Macs are > concerned. > > On these Macs, the T2 Security Chip encrypts all the APFS partitions > on the internal SSD, and the key is in the T2 Chip. Even proprietary > APFS drivers cannot read these partitions. I dunno how it works in > Apple Silicon Macs. How this workings on Apple Silicon Macs is described in this article: https://eclecticlight.co/2022/04/23/explainer-filevault/ It appears such a driver will also be useful if there are external SSD's using APFS. (Although I suspect many external SSD's would end up using some other file system that might be more portable like VFS.) In terms of making it work with the internal SSD, it sounds like Linux would need to talk to the secure enclave on the T2 Security Chip and convince it to upload the encryption key into the hardware in-line encryption engine. I don't know if presenting the user's password is sufficient, or if there is a requirement that the OS prove that it is "approved" software that was loaded via a certified boot chain, which various secure enclaves (such as TPM) are wont to do. - Ted ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-03-06 18:04 ` Theodore Ts'o @ 2025-03-06 19:39 ` Sven Peter 2025-03-07 16:50 ` Ernesto A. Fernández 0 siblings, 1 reply; 10+ messages in thread From: Sven Peter @ 2025-03-06 19:39 UTC (permalink / raw) To: Theodore Ts'o, Aditya Garg Cc: Ethan Carter Edwards, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev, asahi@lists.linux.dev, ernesto@corellium.com Hi, On Thu, Mar 6, 2025, at 19:04, Theodore Ts'o wrote: > On Wed, Mar 05, 2025 at 07:23:55AM +0000, Aditya Garg wrote: >> >> This driver tbh will not ‘really’ be helpful as far as T2 Macs are >> concerned. >> >> On these Macs, the T2 Security Chip encrypts all the APFS partitions >> on the internal SSD, and the key is in the T2 Chip. Even proprietary >> APFS drivers cannot read these partitions. I dunno how it works in >> Apple Silicon Macs. > > How this workings on Apple Silicon Macs is described in this article: > > https://eclecticlight.co/2022/04/23/explainer-filevault/ > > It appears such a driver will also be useful if there are external > SSD's using APFS. (Although I suspect many external SSD's would end > up using some other file system that might be more portable like VFS.) > > In terms of making it work with the internal SSD, it sounds like Linux > would need to talk to the secure enclave on the T2 Security Chip and > convince it to upload the encryption key into the hardware in-line > encryption engine. I don't know if presenting the user's password is > sufficient, or if there is a requirement that the OS prove that it is > "approved" software that was loaded via a certified boot chain, which > various secure enclaves (such as TPM) are wont to do. At least on Apple Silicon all you need is the user password (and a working Secure Enclave driver and a way to forward entangled keys from the Secure Enclave to the NVMe co-processor). It's still possible to unlock the encryption keys inside the Secure Enclave when booting into a secondary macOS installation with all security features disabled (and with a modified kernel). I'd assume the same applies to T2/x86 machines since the T2 is an ancestor of the M-series Apple Silicon SoCs. The only limitation that I'm aware of is that access to DRM keys (HDCP, FairPlay for video streaming, etc.) is only allowed via a certified boot chain. Sven ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-03-06 19:39 ` Sven Peter @ 2025-03-07 16:50 ` Ernesto A. Fernández 0 siblings, 0 replies; 10+ messages in thread From: Ernesto A. Fernández @ 2025-03-07 16:50 UTC (permalink / raw) To: Sven Peter Cc: Theodore Ts'o, Aditya Garg, Ethan Carter Edwards, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev, asahi@lists.linux.dev, ernesto@corellium.com Hi everyone, I don't mind putting in the work to prepare my driver for upstream. I just can't make a case for it myself, since it sounds like a lot of work for the reviewers and I suspect it won't be all that useful in practice. I think the driver is reliable enough under linux-only use; the subset of xfstests that I managed to get to run stopped finding intermittent bugs last year. I'm less confident about our compatibility with the official driver, since I recently fixed a terrible corruption bug for all shared containers above 1.32 TiB in size. There is an official reference for the layout, but it's incomplete and has a few errors. > > (Although I suspect many external SSD's would end > > up using some other file system that might be more portable like VFS.) That's what I would expect too. The driver does get cloned a lot, and it's been packaged for debian for years, so I guess some people must be using it, but I don't really know for sure. > > In terms of making it work with the internal SSD, it sounds like Linux > > would need to talk to the secure enclave on the T2 Security Chip and > > convince it to upload the encryption key into the hardware in-line > > encryption engine. I don't know much about the hardware side, but I think my driver will also need some changes to get this to work. Right now we don't support any form of encryption. It's the biggest missing feature I believe. Ernesto ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module 2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards 2025-02-28 12:55 ` Theodore Ts'o 2025-02-28 23:04 ` Sven Peter @ 2025-03-02 10:55 ` Dan Carpenter 2 siblings, 0 replies; 10+ messages in thread From: Dan Carpenter @ 2025-03-02 10:55 UTC (permalink / raw) To: Ethan Carter Edwards; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi If you're going to do a major re-work it might be easier to do that first before you upstream it. Once it's in the kernel then you have to follow all the rules like breaking your commits up into reviewable patches and not breaking git bisect. The rules are good for quality control but they can be burdensome for people who want to move fast and break things. regards, dan carpenter ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2025-03-07 16:51 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards 2025-02-28 12:55 ` Theodore Ts'o 2025-03-01 16:26 ` Ethan Carter Edwards 2025-02-28 23:04 ` Sven Peter 2025-03-01 16:39 ` Ethan Carter Edwards 2025-03-05 7:23 ` Aditya Garg 2025-03-06 18:04 ` Theodore Ts'o 2025-03-06 19:39 ` Sven Peter 2025-03-07 16:50 ` Ernesto A. Fernández 2025-03-02 10:55 ` Dan Carpenter
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox