* [RFC] apfs: thoughts on upstreaming an out-of-tree module
@ 2025-02-28 1:53 Ethan Carter Edwards
2025-02-28 12:55 ` Theodore Ts'o
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Ethan Carter Edwards @ 2025-02-28 1:53 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-fsdevel, linux-staging, asahi
Lately, I have been thinking a lot about the lack of APFS support on
Linux. I was wondering what I could do about that. APFS support is not
in-tree, but there is a proprietary module sold by Paragon software [0].
Obviously, this could not be used in-tree. However, there is also an
open source driver that, from what I can tell, was once planned to be
upstreamed [1] with associated filesystem progs [2]. I think I would
base most of my work off of the existing FOSS tree.
The biggest barrier I see currently is the driver's use of bufferheads.
I realize that there has been a lot of work to move existing filesystem
implementations to iomap/folios, and adding a filesystem that uses
bufferheads would be antithetical to the purpose of that effort.
Additionally, there is a lot of ifndefs/C preprocessor magic littered
throughout the codebase that fixes functionality with various different
versions of Linux.
The first step would be to move away from bufferheads and the
versioning. I plan to start my work in the next few weeks, and hope to
have a working driver to submit to staging by the end of June. From
there, I will work to have it meet more kernel standards and hopefully
move into fs/ by the end of the year.
Before I started, I was wondering if anyone had any thoughts. I am open
to feedback. If you think this is a bad idea, let me know. I am very
passionate about the Asahi Linux project. I think this would be a good
way to indirectly give back and contribute to the project. While I
recognize that it is not one of Asahi's project goals (those being
mostly hardware support), I am confident many users would find it
helpful. I sure would.
Thanks,
Ethan Carter Edwards <ethan@ethancedwards.com>
[0]: https://www.paragon-software.com/us/home/apfs-linux/
[1]: https://github.com/linux-apfs/linux-apfs-rw
[2]: https://github.com/linux-apfs/apfsprogs
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards
@ 2025-02-28 12:55 ` Theodore Ts'o
2025-03-01 16:26 ` Ethan Carter Edwards
2025-02-28 23:04 ` Sven Peter
2025-03-02 10:55 ` Dan Carpenter
2 siblings, 1 reply; 10+ messages in thread
From: Theodore Ts'o @ 2025-02-28 12:55 UTC (permalink / raw)
To: Ethan Carter Edwards; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi
On Thu, Feb 27, 2025 at 08:53:56PM -0500, Ethan Carter Edwards wrote:
> Lately, I have been thinking a lot about the lack of APFS support on
> Linux. I was wondering what I could do about that. APFS support is not
> in-tree, but there is a proprietary module sold by Paragon software [0].
> Obviously, this could not be used in-tree. However, there is also an
> open source driver that, from what I can tell, was once planned to be
> upstreamed [1] with associated filesystem progs [2]. I think I would
> base most of my work off of the existing FOSS tree.
>
> The biggest barrier I see currently is the driver's use of bufferheads.
> I realize that there has been a lot of work to move existing filesystem
> implementations to iomap/folios, and adding a filesystem that uses
> bufferheads would be antithetical to the purpose of that effort.
> Additionally, there is a lot of ifndefs/C preprocessor magic littered
> throughout the codebase that fixes functionality with various different
> versions of Linux.
I don't see the use of bufferheads as a fundamental barrier to the
mainline kernel; certainly not for staging. First of all, there are a
huge number of file systems which still use buffer heads, including:
adfs affs befs bfs ecryptfs efs exfat ext2 ext4 fat
freevxfs gfs2 hfs hfsplus hpfs isofs jfs minix nilfs2
ntfs3 ocfs2 omfs pstore qnx4 qnx6 romfs sysv udf ufs
There are many reasons to move to folios, including better
performance, and making it easier to make sure things are done
correctly if you can take advantage of iomap.
For example, with ext4 we plan to work towards moving to use folios
and iomap for the data plane operations for buffered write (we already
use iomap for Direct I/O, FIEMAP support, etc.) and while we might
want to move away from buffer heads for metadata blocks, we would need
to change the jbd2 layer to use some simplified layer that looks an
awful lot like buffer heads before we could do that. We might try to
fork buffer heads, and strip out everything we don't need, and then
merge that with jbd2's journal_head structure, for example. But
that's a mid-term to long-term project, because using bufferheads
doesn't actually hurt anyone. (That being said, if anyone wants to
help out with the project of allowing us to move jbd2 away from buffer
heads, let me know --- patches are welcome.)
In any case, cleaning up preprocessor magic and other thigns that were
needed because the code was designed for out of tree use would be
something that I'd encourage you to focus on first, and then try a
proposal to submit it to staging.
Cheers,
- Ted
P.S. Something that you might want to consider using is fstests (AKA
xfstests), which is the gold standard for file system testing in
Linux. I have a test appliance VM of xfstests, which you can find
here[1]. I test x86 and arm64 kernels using Google Cloud, and on
local systems, using qemu/kvm. For qemu/kvm testing, this is being
used on Debian, Fedora, OpenSuSE, and MacOS.
[1] https://github.com/tytso/xfstests-bld
For kernel development on my Macbook Air M2, I can build arm64 kernels
using Debian running in a Parallels VM, and then to avoid the double
virtualization overhead, I run qemu on MacOS using the hvf
accelerator. It shouldn't be hard to make this work on your Ashai
Linux development system; see more on that below.
For more details about this test infrastructure, including its use on
Google Cloud see the presentation here[2]. I am using gce-xfstests to
perform continuous integration testing by watching a git branch, and
when it gets updated, the test manager (running in an e2-micro VM)
automatically starts a 4 CPU VM to build the kernel, and then launches
multiple 2 CPU VM's to test multiple file system configurations in
parallel --- for example, I am currently running over two dozen fs
kernels testing ext4, xfs, btrfs, and f2fs on a Linux-next branch
every day). Running a smoke test costs pennies. A full-up test of a
dozen ext4 configuration (a dozen VM's, running for 2 hours of wall
clock time), costs under $2 at US retail prices. For APFS, if you
start with a single configuration, with many of the tests disable
because APFS won't many of the advanced file systems of ext4 and xfs,
I'm guessing it will cost you less than 25 cents per test run.
[2] https://thunk.org/gce-xfstests
Or of course you can use qemu-xfstests or kvm-xfstests using local
compute. I do love though being able to fire off a set of tests, then
suspend my laptop, knowing that I will receive e-mail with the test
results when they are ready.
If you are interested in trying to use this on Asahi linux, I'd
certainly be happy help you with it. I suspect modulo some
instructures about which packages are needed, it shouldn't be that
hard to run a test appliance. Building new versions of the appliance
does require a Debian build chroot, which might be tricker to set up
on Asahi, but that's not necessary while you are getting started.
In any case, I strongly encourage file system developers to use
xfstests earlier rather than later. See the last slide of [2] for my
opinion of "File system development without testing". :-)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards
2025-02-28 12:55 ` Theodore Ts'o
@ 2025-02-28 23:04 ` Sven Peter
2025-03-01 16:39 ` Ethan Carter Edwards
2025-03-02 10:55 ` Dan Carpenter
2 siblings, 1 reply; 10+ messages in thread
From: Sven Peter @ 2025-02-28 23:04 UTC (permalink / raw)
To: Ethan Carter Edwards, linux-kernel; +Cc: linux-fsdevel, linux-staging, asahi
Hi,
On Fri, Feb 28, 2025, at 02:53, Ethan Carter Edwards wrote:
> Lately, I have been thinking a lot about the lack of APFS support on
> Linux. I was wondering what I could do about that. APFS support is not
> in-tree, but there is a proprietary module sold by Paragon software [0].
> Obviously, this could not be used in-tree. However, there is also an
> open source driver that, from what I can tell, was once planned to be
> upstreamed [1] with associated filesystem progs [2]. I think I would
> base most of my work off of the existing FOSS tree.
>
> The biggest barrier I see currently is the driver's use of bufferheads.
> I realize that there has been a lot of work to move existing filesystem
> implementations to iomap/folios, and adding a filesystem that uses
> bufferheads would be antithetical to the purpose of that effort.
> Additionally, there is a lot of ifndefs/C preprocessor magic littered
> throughout the codebase that fixes functionality with various different
> versions of Linux.
>
> The first step would be to move away from bufferheads and the
> versioning. I plan to start my work in the next few weeks, and hope to
> have a working driver to submit to staging by the end of June. From
> there, I will work to have it meet more kernel standards and hopefully
> move into fs/ by the end of the year.
>
> Before I started, I was wondering if anyone had any thoughts. I am open
> to feedback. If you think this is a bad idea, let me know. I am very
> passionate about the Asahi Linux project. I think this would be a good
> way to indirectly give back and contribute to the project. While I
> recognize that it is not one of Asahi's project goals (those being
> mostly hardware support), I am confident many users would find it
> helpful. I sure would.
Agreed, I think it would be helpful for many people (especially those
who still regularly dual boot between macOS and Linux) to have a working
APFS driver upstream.
This may also be useful once we fully bring up the Secure Enclave and need
to read/write to at least a single file on the xART partition which has
to be APFS because it's shared between all operating systems running on
a single machine.
It looks like there's still recent activity on that linux-apfs github
repository. Have you reached out to the people working on it to see
what their plans for upstreaming and/or future work are?
Best,
Sven
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-02-28 12:55 ` Theodore Ts'o
@ 2025-03-01 16:26 ` Ethan Carter Edwards
0 siblings, 0 replies; 10+ messages in thread
From: Ethan Carter Edwards @ 2025-03-01 16:26 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi
On 25/02/28 07:55AM, Theodore Ts'o wrote:
> On Thu, Feb 27, 2025 at 08:53:56PM -0500, Ethan Carter Edwards wrote:
> > Lately, I have been thinking a lot about the lack of APFS support on
> > Linux. I was wondering what I could do about that. APFS support is not
> > in-tree, but there is a proprietary module sold by Paragon software [0].
> > Obviously, this could not be used in-tree. However, there is also an
> > open source driver that, from what I can tell, was once planned to be
> > upstreamed [1] with associated filesystem progs [2]. I think I would
> > base most of my work off of the existing FOSS tree.
> >
> > The biggest barrier I see currently is the driver's use of bufferheads.
> > I realize that there has been a lot of work to move existing filesystem
> > implementations to iomap/folios, and adding a filesystem that uses
> > bufferheads would be antithetical to the purpose of that effort.
> > Additionally, there is a lot of ifndefs/C preprocessor magic littered
> > throughout the codebase that fixes functionality with various different
> > versions of Linux.
>
> I don't see the use of bufferheads as a fundamental barrier to the
> mainline kernel; certainly not for staging. First of all, there are a
> huge number of file systems which still use buffer heads, including:
>
> adfs affs befs bfs ecryptfs efs exfat ext2 ext4 fat
> freevxfs gfs2 hfs hfsplus hpfs isofs jfs minix nilfs2
> ntfs3 ocfs2 omfs pstore qnx4 qnx6 romfs sysv udf ufs
>
Good to know. I did not realized so many fs's stil used them. I will
not let that stop me from submitting a the driver to staging. I
definitely plan on moving them over as time permits.
> There are many reasons to move to folios, including better
> performance, and making it easier to make sure things are done
> correctly if you can take advantage of iomap.
>
> For example, with ext4 we plan to work towards moving to use folios
> and iomap for the data plane operations for buffered write (we already
> use iomap for Direct I/O, FIEMAP support, etc.) and while we might
> want to move away from buffer heads for metadata blocks, we would need
> to change the jbd2 layer to use some simplified layer that looks an
> awful lot like buffer heads before we could do that. We might try to
> fork buffer heads, and strip out everything we don't need, and then
> merge that with jbd2's journal_head structure, for example. But
> that's a mid-term to long-term project, because using bufferheads
> doesn't actually hurt anyone. (That being said, if anyone wants to
> help out with the project of allowing us to move jbd2 away from buffer
> heads, let me know --- patches are welcome.)
>
> In any case, cleaning up preprocessor magic and other thigns that were
> needed because the code was designed for out of tree use would be
> something that I'd encourage you to focus on first, and then try a
> proposal to submit it to staging.
Will do.
>
> Cheers,
>
> - Ted
>
> P.S. Something that you might want to consider using is fstests (AKA
> xfstests), which is the gold standard for file system testing in
> Linux. I have a test appliance VM of xfstests, which you can find
> here[1]. I test x86 and arm64 kernels using Google Cloud, and on
> local systems, using qemu/kvm. For qemu/kvm testing, this is being
> used on Debian, Fedora, OpenSuSE, and MacOS.
>
> [1] https://github.com/tytso/xfstests-bld
>
> For kernel development on my Macbook Air M2, I can build arm64 kernels
> using Debian running in a Parallels VM, and then to avoid the double
> virtualization overhead, I run qemu on MacOS using the hvf
> accelerator. It shouldn't be hard to make this work on your Ashai
> Linux development system; see more on that below.
>
> For more details about this test infrastructure, including its use on
> Google Cloud see the presentation here[2]. I am using gce-xfstests to
> perform continuous integration testing by watching a git branch, and
> when it gets updated, the test manager (running in an e2-micro VM)
> automatically starts a 4 CPU VM to build the kernel, and then launches
> multiple 2 CPU VM's to test multiple file system configurations in
> parallel --- for example, I am currently running over two dozen fs
> kernels testing ext4, xfs, btrfs, and f2fs on a Linux-next branch
> every day). Running a smoke test costs pennies. A full-up test of a
> dozen ext4 configuration (a dozen VM's, running for 2 hours of wall
> clock time), costs under $2 at US retail prices. For APFS, if you
> start with a single configuration, with many of the tests disable
> because APFS won't many of the advanced file systems of ext4 and xfs,
> I'm guessing it will cost you less than 25 cents per test run.
>
> [2] https://thunk.org/gce-xfstests
>
> Or of course you can use qemu-xfstests or kvm-xfstests using local
> compute. I do love though being able to fire off a set of tests, then
> suspend my laptop, knowing that I will receive e-mail with the test
> results when they are ready.
>
> If you are interested in trying to use this on Asahi linux, I'd
> certainly be happy help you with it. I suspect modulo some
> instructures about which packages are needed, it shouldn't be that
> hard to run a test appliance. Building new versions of the appliance
> does require a Debian build chroot, which might be tricker to set up
> on Asahi, but that's not necessary while you are getting started.
>
> In any case, I strongly encourage file system developers to use
> xfstests earlier rather than later. See the last slide of [2] for my
> opinion of "File system development without testing". :-)
Good to know. I certainly agree with that last slide ;). Filesystems are
far too important to develop without a test suite. Tests have certainly
saved my butt in other projects, I have no doubt that it is the same for
fs devel.
Thank you for your detailed response. I was nervous that this idea would
not be well-recieved since there is an existing out-of-tree driver, but
since I am a believer in "upstream-first", I wanted to give this a try.
Thanks,
Ethan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-02-28 23:04 ` Sven Peter
@ 2025-03-01 16:39 ` Ethan Carter Edwards
2025-03-05 7:23 ` Aditya Garg
0 siblings, 1 reply; 10+ messages in thread
From: Ethan Carter Edwards @ 2025-03-01 16:39 UTC (permalink / raw)
To: Sven Peter, Theodore Ts'o
Cc: linux-kernel, linux-fsdevel, linux-staging, asahi
On 25/03/01 12:04AM, Sven Peter wrote:
> Hi,
>
>
> On Fri, Feb 28, 2025, at 02:53, Ethan Carter Edwards wrote:
> > Lately, I have been thinking a lot about the lack of APFS support on
> > Linux. I was wondering what I could do about that. APFS support is not
> > in-tree, but there is a proprietary module sold by Paragon software [0].
> > Obviously, this could not be used in-tree. However, there is also an
> > open source driver that, from what I can tell, was once planned to be
> > upstreamed [1] with associated filesystem progs [2]. I think I would
> > base most of my work off of the existing FOSS tree.
> >
> > The biggest barrier I see currently is the driver's use of bufferheads.
> > I realize that there has been a lot of work to move existing filesystem
> > implementations to iomap/folios, and adding a filesystem that uses
> > bufferheads would be antithetical to the purpose of that effort.
> > Additionally, there is a lot of ifndefs/C preprocessor magic littered
> > throughout the codebase that fixes functionality with various different
> > versions of Linux.
> >
> > The first step would be to move away from bufferheads and the
> > versioning. I plan to start my work in the next few weeks, and hope to
> > have a working driver to submit to staging by the end of June. From
> > there, I will work to have it meet more kernel standards and hopefully
> > move into fs/ by the end of the year.
> >
> > Before I started, I was wondering if anyone had any thoughts. I am open
> > to feedback. If you think this is a bad idea, let me know. I am very
> > passionate about the Asahi Linux project. I think this would be a good
> > way to indirectly give back and contribute to the project. While I
> > recognize that it is not one of Asahi's project goals (those being
> > mostly hardware support), I am confident many users would find it
> > helpful. I sure would.
>
> Agreed, I think it would be helpful for many people (especially those
> who still regularly dual boot between macOS and Linux) to have a working
> APFS driver upstream.
> This may also be useful once we fully bring up the Secure Enclave and need
> to read/write to at least a single file on the xART partition which has
> to be APFS because it's shared between all operating systems running on
> a single machine.
>
>
> It looks like there's still recent activity on that linux-apfs github
> repository. Have you reached out to the people working on it to see
> what their plans for upstreaming and/or future work are?
I did ask the upstream maintainer and he said he did not see it
happening. He specifically mentioned the use of bufferheads as a barrier
to mainline merging, but I get the sense that he does not have the
time/desire to commit to upstreaming it. [0]
I did have a question/concern over the inclusion of the LZFSE/LZVN [1]
compression library included in the module. It is BSD3 licensed, which,
as far as I know is GPL-compatible, but what is the kernel's policy on
external algorithms being included? It was originally developed by Apple
and as far as I can tell is pretty necessary to read (and write)
compressed files on APFS. Also, the library does produce an objtool
warning.
Ted, looping you in here, your thoughts?
>
>
>
> Best,
>
>
> Sven
>
Thanks,
Ethan
[0]: https://github.com/linux-apfs/linux-apfs-rw/issues/68
[1]: https://github.com/linux-apfs/linux-apfs-rw/tree/master/lzfse
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards
2025-02-28 12:55 ` Theodore Ts'o
2025-02-28 23:04 ` Sven Peter
@ 2025-03-02 10:55 ` Dan Carpenter
2 siblings, 0 replies; 10+ messages in thread
From: Dan Carpenter @ 2025-03-02 10:55 UTC (permalink / raw)
To: Ethan Carter Edwards; +Cc: linux-kernel, linux-fsdevel, linux-staging, asahi
If you're going to do a major re-work it might be easier to do that first
before you upstream it. Once it's in the kernel then you have to follow
all the rules like breaking your commits up into reviewable patches and
not breaking git bisect. The rules are good for quality control but they
can be burdensome for people who want to move fast and break things.
regards,
dan carpenter
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-03-01 16:39 ` Ethan Carter Edwards
@ 2025-03-05 7:23 ` Aditya Garg
2025-03-06 18:04 ` Theodore Ts'o
0 siblings, 1 reply; 10+ messages in thread
From: Aditya Garg @ 2025-03-05 7:23 UTC (permalink / raw)
To: Ethan Carter Edwards
Cc: Sven Peter, Theodore Ts'o, linux-kernel@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev,
asahi@lists.linux.dev, ernesto@corellium.com
Hi
Ccing Ernesto
>>> Lately, I have been thinking a lot about the lack of APFS support on
>>> Linux. I was wondering what I could do about that. APFS support is not
>>> in-tree, but there is a proprietary module sold by Paragon software [0].
>>> Obviously, this could not be used in-tree. However, there is also an
>>> open source driver that, from what I can tell, was once planned to be
>>> upstreamed [1] with associated filesystem progs [2]. I think I would
>>> base most of my work off of the existing FOSS tree.
>>>
>>> The biggest barrier I see currently is the driver's use of bufferheads.
>>> I realize that there has been a lot of work to move existing filesystem
>>> implementations to iomap/folios, and adding a filesystem that uses
>>> bufferheads would be antithetical to the purpose of that effort.
>>> Additionally, there is a lot of ifndefs/C preprocessor magic littered
>>> throughout the codebase that fixes functionality with various different
>>> versions of Linux.
>>>
>>> The first step would be to move away from bufferheads and the
>>> versioning. I plan to start my work in the next few weeks, and hope to
>>> have a working driver to submit to staging by the end of June. From
>>> there, I will work to have it meet more kernel standards and hopefully
>>> move into fs/ by the end of the year.
>>>
>>> Before I started, I was wondering if anyone had any thoughts. I am open
>>> to feedback. If you think this is a bad idea, let me know. I am very
>>> passionate about the Asahi Linux project. I think this would be a good
>>> way to indirectly give back and contribute to the project. While I
>>> recognize that it is not one of Asahi's project goals (those being
>>> mostly hardware support), I am confident many users would find it
>>> helpful. I sure would.
This driver tbh will not ‘really’ be helpful as far as T2 Macs are concerned.
On these Macs, the T2 Security Chip encrypts all the APFS partitions on the internal SSD,
and the key is in the T2 Chip. Even proprietary APFS drivers cannot read these partitions.
I dunno how it works in Apple Silicon Macs.
One practical use that we actually do have with this driver is that the recovery partition of
macOS is still unencrypted. Now brcmfmac driver on Linux needs wifi firmware to work which
on these is obtained from macOS. We mount the macOS Recovery partition and use a helper
script [1] to get the firmware.
[1] https://wiki.t2linux.org/guides/wifi-bluetooth/#setting-up
>>
>> Agreed, I think it would be helpful for many people (especially those
>> who still regularly dual boot between macOS and Linux) to have a working
>> APFS driver upstream.
>> This may also be useful once we fully bring up the Secure Enclave and need
>> to read/write to at least a single file on the xART partition which has
>> to be APFS because it's shared between all operating systems running on
>> a single machine.
>>
>>
>> It looks like there's still recent activity on that linux-apfs github
>> repository. Have you reached out to the people working on it to see
>> what their plans for upstreaming and/or future work are?
>
> I did ask the upstream maintainer and he said he did not see it
> happening. He specifically mentioned the use of bufferheads as a barrier
> to mainline merging, but I get the sense that he does not have the
> time/desire to commit to upstreaming it. [0]
>
> I did have a question/concern over the inclusion of the LZFSE/LZVN [1]
> compression library included in the module. It is BSD3 licensed, which,
> as far as I know is GPL-compatible, but what is the kernel's policy on
> external algorithms being included? It was originally developed by Apple
> and as far as I can tell is pretty necessary to read (and write)
> compressed files on APFS. Also, the library does produce an objtool
> warning.
The original author was clear about this. Its is compatible with GPL:
https://github.com/linux-apfs/linux-apfs-rw/issues/68#issuecomment-2156731178
>
> Ted, looping you in here, your thoughts?
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-03-05 7:23 ` Aditya Garg
@ 2025-03-06 18:04 ` Theodore Ts'o
2025-03-06 19:39 ` Sven Peter
0 siblings, 1 reply; 10+ messages in thread
From: Theodore Ts'o @ 2025-03-06 18:04 UTC (permalink / raw)
To: Aditya Garg
Cc: Ethan Carter Edwards, Sven Peter, linux-kernel@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev,
asahi@lists.linux.dev, ernesto@corellium.com
On Wed, Mar 05, 2025 at 07:23:55AM +0000, Aditya Garg wrote:
>
> This driver tbh will not ‘really’ be helpful as far as T2 Macs are
> concerned.
>
> On these Macs, the T2 Security Chip encrypts all the APFS partitions
> on the internal SSD, and the key is in the T2 Chip. Even proprietary
> APFS drivers cannot read these partitions. I dunno how it works in
> Apple Silicon Macs.
How this workings on Apple Silicon Macs is described in this article:
https://eclecticlight.co/2022/04/23/explainer-filevault/
It appears such a driver will also be useful if there are external
SSD's using APFS. (Although I suspect many external SSD's would end
up using some other file system that might be more portable like VFS.)
In terms of making it work with the internal SSD, it sounds like Linux
would need to talk to the secure enclave on the T2 Security Chip and
convince it to upload the encryption key into the hardware in-line
encryption engine. I don't know if presenting the user's password is
sufficient, or if there is a requirement that the OS prove that it is
"approved" software that was loaded via a certified boot chain, which
various secure enclaves (such as TPM) are wont to do.
- Ted
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-03-06 18:04 ` Theodore Ts'o
@ 2025-03-06 19:39 ` Sven Peter
2025-03-07 16:50 ` Ernesto A. Fernández
0 siblings, 1 reply; 10+ messages in thread
From: Sven Peter @ 2025-03-06 19:39 UTC (permalink / raw)
To: Theodore Ts'o, Aditya Garg
Cc: Ethan Carter Edwards, linux-kernel@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-staging@lists.linux.dev,
asahi@lists.linux.dev, ernesto@corellium.com
Hi,
On Thu, Mar 6, 2025, at 19:04, Theodore Ts'o wrote:
> On Wed, Mar 05, 2025 at 07:23:55AM +0000, Aditya Garg wrote:
>>
>> This driver tbh will not ‘really’ be helpful as far as T2 Macs are
>> concerned.
>>
>> On these Macs, the T2 Security Chip encrypts all the APFS partitions
>> on the internal SSD, and the key is in the T2 Chip. Even proprietary
>> APFS drivers cannot read these partitions. I dunno how it works in
>> Apple Silicon Macs.
>
> How this workings on Apple Silicon Macs is described in this article:
>
> https://eclecticlight.co/2022/04/23/explainer-filevault/
>
> It appears such a driver will also be useful if there are external
> SSD's using APFS. (Although I suspect many external SSD's would end
> up using some other file system that might be more portable like VFS.)
>
> In terms of making it work with the internal SSD, it sounds like Linux
> would need to talk to the secure enclave on the T2 Security Chip and
> convince it to upload the encryption key into the hardware in-line
> encryption engine. I don't know if presenting the user's password is
> sufficient, or if there is a requirement that the OS prove that it is
> "approved" software that was loaded via a certified boot chain, which
> various secure enclaves (such as TPM) are wont to do.
At least on Apple Silicon all you need is the user password (and a working
Secure Enclave driver and a way to forward entangled keys from the Secure
Enclave to the NVMe co-processor). It's still possible to unlock the
encryption keys inside the Secure Enclave when booting into a secondary
macOS installation with all security features disabled (and with a
modified kernel). I'd assume the same applies to T2/x86 machines since
the T2 is an ancestor of the M-series Apple Silicon SoCs.
The only limitation that I'm aware of is that access to DRM keys
(HDCP, FairPlay for video streaming, etc.) is only allowed via a
certified boot chain.
Sven
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC] apfs: thoughts on upstreaming an out-of-tree module
2025-03-06 19:39 ` Sven Peter
@ 2025-03-07 16:50 ` Ernesto A. Fernández
0 siblings, 0 replies; 10+ messages in thread
From: Ernesto A. Fernández @ 2025-03-07 16:50 UTC (permalink / raw)
To: Sven Peter
Cc: Theodore Ts'o, Aditya Garg, Ethan Carter Edwards,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-staging@lists.linux.dev, asahi@lists.linux.dev,
ernesto@corellium.com
Hi everyone,
I don't mind putting in the work to prepare my driver for upstream. I just
can't make a case for it myself, since it sounds like a lot of work for the
reviewers and I suspect it won't be all that useful in practice.
I think the driver is reliable enough under linux-only use; the subset of
xfstests that I managed to get to run stopped finding intermittent bugs last
year. I'm less confident about our compatibility with the official driver,
since I recently fixed a terrible corruption bug for all shared containers
above 1.32 TiB in size. There is an official reference for the layout, but
it's incomplete and has a few errors.
> > (Although I suspect many external SSD's would end
> > up using some other file system that might be more portable like VFS.)
That's what I would expect too. The driver does get cloned a lot, and it's
been packaged for debian for years, so I guess some people must be using it,
but I don't really know for sure.
> > In terms of making it work with the internal SSD, it sounds like Linux
> > would need to talk to the secure enclave on the T2 Security Chip and
> > convince it to upload the encryption key into the hardware in-line
> > encryption engine.
I don't know much about the hardware side, but I think my driver will also
need some changes to get this to work. Right now we don't support any form
of encryption. It's the biggest missing feature I believe.
Ernesto
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2025-03-07 16:51 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-28 1:53 [RFC] apfs: thoughts on upstreaming an out-of-tree module Ethan Carter Edwards
2025-02-28 12:55 ` Theodore Ts'o
2025-03-01 16:26 ` Ethan Carter Edwards
2025-02-28 23:04 ` Sven Peter
2025-03-01 16:39 ` Ethan Carter Edwards
2025-03-05 7:23 ` Aditya Garg
2025-03-06 18:04 ` Theodore Ts'o
2025-03-06 19:39 ` Sven Peter
2025-03-07 16:50 ` Ernesto A. Fernández
2025-03-02 10:55 ` Dan Carpenter
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox