* Verifier running out of memory on ieee1275/powerpc64 @ 2020-03-17 17:15 Stefan Berger 2020-03-18 15:59 ` Simon Hardy 0 siblings, 1 reply; 7+ messages in thread From: Stefan Berger @ 2020-03-17 17:15 UTC (permalink / raw) To: grub-devel Hello! I trying to add (v)TPM support for the ieee1275/powerpc64 platform to grub. The issue I have been running into is that the verifier runs out of memory. At that point it has loaded the (~ 32MB) Linux kernel and now the verifier is invoked to load the file. Unfortunately it cannot load the file since it doesn't have enough memory to grub_malloc. I have played with increasing heap size(es) but it still doesn't work. The kernel and initramfs files on ppc64 can be rather big, thus we do not a lot of memory. The rescue initramfs here is for example 78MB, a regular initramfs from Fedora 31 is ~34MB. The kernel sizes on my system are 32MB, though a colleague was using an unstripped kernel of 127MB, so lots of (unfragmented) memory needs to be available to run verifiers. This is the code that's trying to allocate the memory to load the file into: http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/commands/verifiers.c#n128 I had increased the GRUB_IEEE_STATIC_HEAP_LEN to 0x4000000, but it didn't resolve the issue: http://git.savannah.gnu.org/cgit/grub.git/tree/include/grub/ieee1275/ieee1275.h#n93 I also tried to increase the HEAP_MAX_SIZE to 128 * 1024 * 1024 along with HEAP_MAX_ADDR, but that didn't help, either: http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/kern/ieee1275/init.c Does anybody have a suggestion? Regards, Stefan ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-17 17:15 Verifier running out of memory on ieee1275/powerpc64 Stefan Berger @ 2020-03-18 15:59 ` Simon Hardy 2020-03-18 19:32 ` Stefan Berger 0 siblings, 1 reply; 7+ messages in thread From: Simon Hardy @ 2020-03-18 15:59 UTC (permalink / raw) To: The development of GNU GRUB The 2020/03/17 13:15, Stefan Berger wrote: > I trying to add (v)TPM support for the ieee1275/powerpc64 platform to grub. > The issue I have been running into is that the verifier runs out of memory. > At that point it has loaded the (~ 32MB) Linux kernel and now the verifier > is invoked to load the file. Unfortunately it cannot load the file since it > doesn't have enough memory to grub_malloc. I have played with increasing > heap size(es) but it still doesn't work. The kernel and initramfs files on > ppc64 can be rather big, thus we do not a lot of memory. The rescue > initramfs here is for example 78MB, a regular initramfs from Fedora 31 is > ~34MB. The kernel sizes on my system are 32MB, though a colleague was using > an unstripped kernel of 127MB, so lots of (unfragmented) memory needs to be > available to run verifiers. The verifiers framework has a flag, GRUB_VERIFY_FLAGS_SINGLE_CHUNK, that is used by the platform-independent TPM module. This could be deferred to the platform-specific TPM file (see point 3 below). With this flag unset for your platform, you could verify the files in small chunks. This requires three further elements: 1. You will need to implement the chunk-by-chunk behaviour in verifiers.c, it doesn't exist yet. 2. You will need to add functionality to calculate a hash from chunks, or require that the crypto module is built into the core. 3. The firmware interface needs to support HashLogExtend with a user supplied hash instead of a memory buffer. For example the PC Conventional BIOS API has this, but the UEFI API does not. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-18 15:59 ` Simon Hardy @ 2020-03-18 19:32 ` Stefan Berger 2020-03-18 20:27 ` Stefan Berger 0 siblings, 1 reply; 7+ messages in thread From: Stefan Berger @ 2020-03-18 19:32 UTC (permalink / raw) To: The development of GNU GRUB, Simon Hardy On 3/18/20 11:59 AM, Simon Hardy wrote: > The 2020/03/17 13:15, Stefan Berger wrote: >> I trying to add (v)TPM support for the ieee1275/powerpc64 platform to grub. >> The issue I have been running into is that the verifier runs out of memory. >> At that point it has loaded the (~ 32MB) Linux kernel and now the verifier >> is invoked to load the file. Unfortunately it cannot load the file since it >> doesn't have enough memory to grub_malloc. I have played with increasing >> heap size(es) but it still doesn't work. The kernel and initramfs files on >> ppc64 can be rather big, thus we do not a lot of memory. The rescue >> initramfs here is for example 78MB, a regular initramfs from Fedora 31 is >> ~34MB. The kernel sizes on my system are 32MB, though a colleague was using >> an unstripped kernel of 127MB, so lots of (unfragmented) memory needs to be >> available to run verifiers. > The verifiers framework has a flag, GRUB_VERIFY_FLAGS_SINGLE_CHUNK, that is > used by the platform-independent TPM module. This could be deferred to the > platform-specific TPM file (see point 3 below). With this flag unset for your > platform, you could verify the files in small chunks. This requires three > further elements: > > 1. You will need to implement the chunk-by-chunk behaviour in > verifiers.c, it doesn't exist yet. > > 2. You will need to add functionality to calculate a hash from chunks, or > require that the crypto module is built into the core. > > 3. The firmware interface needs to support HashLogExtend with a user supplied > hash instead of a memory buffer. For example the PC Conventional BIOS API has > this, but the UEFI API does not. Simon, thanks a lot for your reply. TPM 2's logging behavior is different than that of a TPM 1.2 and it's not clear whether hashing in grub will produce the right hash or hashes (for different PCR banks) as needed by the firmware. I would rather leave the hashing entirely up to the firmware because it knows which PCR banks are activate and what hashes it wants to use for logging and PCR extending. I was wondering whether it would not be possible to load the raw file into memory, pass it to the firmware for hashing (and logging) via the verifier, and if we do not trust that the firmware treated the file data as a read-only array, load the file again into the same array right after. This way we wouldn't need more memory. [*] However, I am not sure how it fits into the architecture with the verifiers or whether the TPM verifier would have to take on a special role (possibly with a flag) then. You didn't pick up on the idea of a bigger heap. Is there a problem with the heap size somehow? My machine has GBs of memory, so it really shouldn't be a problem to get memory. [*] Obviously we can load the files. We can also load them while the TPM verifier is active but only when preventing the loading of those large files (actually allowing the grub_malloc() on the large size to occurr seems to cause issues booting). The memory allocations that are occurring due to the TPM verifier do not seem to cause memory fragmentation that would prevent the loading of those large files. So, makes me think, re-load the files into the same memory. Stefan > > > _______________________________________________ > Grub-devel mailing list > Grub-devel@gnu.org > https://lists.gnu.org/mailman/listinfo/grub-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-18 19:32 ` Stefan Berger @ 2020-03-18 20:27 ` Stefan Berger 2020-03-18 22:17 ` Simon Hardy 0 siblings, 1 reply; 7+ messages in thread From: Stefan Berger @ 2020-03-18 20:27 UTC (permalink / raw) To: The development of GNU GRUB, Simon Hardy On 3/18/20 3:32 PM, Stefan Berger wrote: > On 3/18/20 11:59 AM, Simon Hardy wrote: >> The 2020/03/17 13:15, Stefan Berger wrote: >>> I trying to add (v)TPM support for the ieee1275/powerpc64 platform >>> to grub. >>> The issue I have been running into is that the verifier runs out of >>> memory. >>> At that point it has loaded the (~ 32MB) Linux kernel and now the >>> verifier >>> is invoked to load the file. Unfortunately it cannot load the file >>> since it >>> doesn't have enough memory to grub_malloc. I have played with >>> increasing >>> heap size(es) but it still doesn't work. The kernel and initramfs >>> files on >>> ppc64 can be rather big, thus we do not a lot of memory. The rescue >>> initramfs here is for example 78MB, a regular initramfs from Fedora >>> 31 is >>> ~34MB. The kernel sizes on my system are 32MB, though a colleague >>> was using >>> an unstripped kernel of 127MB, so lots of (unfragmented) memory >>> needs to be >>> available to run verifiers. >> The verifiers framework has a flag, GRUB_VERIFY_FLAGS_SINGLE_CHUNK, >> that is >> used by the platform-independent TPM module. This could be deferred >> to the >> platform-specific TPM file (see point 3 below). With this flag unset >> for your >> platform, you could verify the files in small chunks. This requires >> three >> further elements: >> >> 1. You will need to implement the chunk-by-chunk behaviour in >> verifiers.c, it doesn't exist yet. >> >> 2. You will need to add functionality to calculate a hash from >> chunks, or >> require that the crypto module is built into the core. >> >> 3. The firmware interface needs to support HashLogExtend with a user >> supplied >> hash instead of a memory buffer. For example the PC Conventional BIOS >> API has >> this, but the UEFI API does not. > > Simon, thanks a lot for your reply. > > TPM 2's logging behavior is different than that of a TPM 1.2 and it's > not clear whether hashing in grub will produce the right hash or > hashes (for different PCR banks) as needed by the firmware. I would > rather leave the hashing entirely up to the firmware because it knows > which PCR banks are activate and what hashes it wants to use for > logging and PCR extending. > > I was wondering whether it would not be possible to load the raw file > into memory, pass it to the firmware for hashing (and logging) via the > verifier, and if we do not trust that the firmware treated the file > data as a read-only array, load the file again into the same array > right after. This way we wouldn't need more memory. [*] However, I am > not sure how it fits into the architecture with the verifiers or > whether the TPM verifier would have to take on a special role > (possibly with a flag) then. > > You didn't pick up on the idea of a bigger heap. Is there a problem > with the heap size somehow? My machine has GBs of memory, so it really > shouldn't be a problem to get memory. I think that's the problem to solve, at least for this platform, since none of the verifiers will work due to the memory exhaustion issue. Stefan ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-18 20:27 ` Stefan Berger @ 2020-03-18 22:17 ` Simon Hardy 2020-03-19 16:37 ` Stefan Berger 0 siblings, 1 reply; 7+ messages in thread From: Simon Hardy @ 2020-03-18 22:17 UTC (permalink / raw) To: The development of GNU GRUB > > I was wondering whether it would not be possible to load the raw file > > into memory, pass it to the firmware for hashing (and logging) via the > > verifier, and if we do not trust that the firmware treated the file data > > as a read-only array, load the file again into the same array right > > after. This way we wouldn't need more memory. [*] However, I am not sure > > how it fits into the architecture with the verifiers or whether the TPM > > verifier would have to take on a special role (possibly with a flag) > > then. > > > > You didn't pick up on the idea of a bigger heap. Is there a problem with > > the heap size somehow? My machine has GBs of memory, so it really > > shouldn't be a problem to get memory. > > I think that's the problem to solve, at least for this platform, since none > of the verifiers will work due to the memory exhaustion issue. Increasing the heap size may be the easiest way for you to solve this problem. I am not so familiar with the issues around doing this, but I have seen recent discussion on this list. The reason there is extra memory usage is that the verifier framework is built on top of the file filter framework. The process is: 1. The file is opened using grub_file_open. 2. The verifier framework opens the underlying file and reads all of it into a new heap-allocated memory buffer. 3. The verifier framework provides the buffer to the verifiers for verification. 4. The verifier framework returns a file handle. 5. Read operations on this file handle are served by copying from the verification buffer. 6. The file is closed. The verification framework frees its buffer. There are good reasons for doing this. It is more secure to ensure that verification is applied to the same data that is actually used. (An attacker could give you different data on a second read from the disk.) Some verifiers, like the TPM, prefer to receive the data in a single chunk. Existing code depends on the open/read/seek/close file access model. The problem you have run into is that where you have a use case that involves loading a large file into memory then you need enough free memory to hold two copies of that file. Ideally there would be an alternative grub_file_read_all function that loads the contents of a named file to a user-provided memory buffer. This doesn't look like it would be simple or easy, for example I don't know how this would work for compressed files. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-18 22:17 ` Simon Hardy @ 2020-03-19 16:37 ` Stefan Berger 2020-03-20 17:14 ` Eric Snowberg 0 siblings, 1 reply; 7+ messages in thread From: Stefan Berger @ 2020-03-19 16:37 UTC (permalink / raw) To: The development of GNU GRUB, Simon Hardy On 3/18/20 6:17 PM, Simon Hardy wrote: >>> I was wondering whether it would not be possible to load the raw file >>> into memory, pass it to the firmware for hashing (and logging) via the >>> verifier, and if we do not trust that the firmware treated the file data >>> as a read-only array, load the file again into the same array right >>> after. This way we wouldn't need more memory. [*] However, I am not sure >>> how it fits into the architecture with the verifiers or whether the TPM >>> verifier would have to take on a special role (possibly with a flag) >>> then. >>> >>> You didn't pick up on the idea of a bigger heap. Is there a problem with >>> the heap size somehow? My machine has GBs of memory, so it really >>> shouldn't be a problem to get memory. >> I think that's the problem to solve, at least for this platform, since none >> of the verifiers will work due to the memory exhaustion issue. > Increasing the heap size may be the easiest way for you to solve this > problem. I am not so familiar with the issues around doing this, but I > have seen recent discussion on this list. I found the lsmmap command and it shows me some impressive memory ranges to be known: base_addr = 0x4000, length = 0x17c000, available RAM base_addr = 0x1fffff, length = 0x1, available RAM base_addr = 0x21ffff, length = 0x1, available RAM base_addr = 0x2000000, length = 0x7bbf0000, available RAM base_addr = 0x7feffffff, length = 0x1, available RAM base_addr = 0x800903dc, length = 0x7ff6fc24, available RAM There's more than needed memory here. It doesn't seem to be available for grub_malloc, though. > > The reason there is extra memory usage is that the verifier framework is > built on top of the file filter framework. The process is: > 1. The file is opened using grub_file_open. > 2. The verifier framework opens the underlying file and reads all of it > into a new heap-allocated memory buffer. > 3. The verifier framework provides the buffer to the verifiers for > verification. > 4. The verifier framework returns a file handle. > 5. Read operations on this file handle are served by copying from the > verification buffer. Yes, saw this, verify_fs. > 6. The file is closed. The verification framework frees its buffer. > > There are good reasons for doing this. It is more secure to ensure that > verification is applied to the same data that is actually used. (An > attacker could give you different data on a second read from the disk.) Also true. > Some verifiers, like the TPM, prefer to receive the data in a single > chunk. Existing code depends on the open/read/seek/close file access > model. > > The problem you have run into is that where you have a use case that > involves loading a large file into memory then you need enough free > memory to hold two copies of that file. > > Ideally there would be an alternative grub_file_read_all function that > loads the contents of a named file to a user-provided memory buffer. > This doesn't look like it would be simple or easy, for example I don't > know how this would work for compressed files. Stefan > > _______________________________________________ > Grub-devel mailing list > Grub-devel@gnu.org > https://lists.gnu.org/mailman/listinfo/grub-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Verifier running out of memory on ieee1275/powerpc64 2020-03-19 16:37 ` Stefan Berger @ 2020-03-20 17:14 ` Eric Snowberg 0 siblings, 0 replies; 7+ messages in thread From: Eric Snowberg @ 2020-03-20 17:14 UTC (permalink / raw) To: The development of GNU GRUB; +Cc: Simon Hardy > On Mar 19, 2020, at 10:37 AM, Stefan Berger <stefanb@linux.ibm.com> wrote: > > On 3/18/20 6:17 PM, Simon Hardy wrote: >>>> I was wondering whether it would not be possible to load the raw file >>>> into memory, pass it to the firmware for hashing (and logging) via the >>>> verifier, and if we do not trust that the firmware treated the file data >>>> as a read-only array, load the file again into the same array right >>>> after. This way we wouldn't need more memory. [*] However, I am not sure >>>> how it fits into the architecture with the verifiers or whether the TPM >>>> verifier would have to take on a special role (possibly with a flag) >>>> then. >>>> >>>> You didn't pick up on the idea of a bigger heap. Is there a problem with >>>> the heap size somehow? My machine has GBs of memory, so it really >>>> shouldn't be a problem to get memory. >>> I think that's the problem to solve, at least for this platform, since none >>> of the verifiers will work due to the memory exhaustion issue. >> Increasing the heap size may be the easiest way for you to solve this >> problem. I am not so familiar with the issues around doing this, but I >> have seen recent discussion on this list. > > I found the lsmmap command and it shows me some impressive memory ranges to be known: > > base_addr = 0x4000, length = 0x17c000, available RAM > > base_addr = 0x1fffff, length = 0x1, available RAM > > base_addr = 0x21ffff, length = 0x1, available RAM > > base_addr = 0x2000000, length = 0x7bbf0000, available RAM > > base_addr = 0x7feffffff, length = 0x1, available RAM > > base_addr = 0x800903dc, length = 0x7ff6fc24, available RAM > > There's more than needed memory here. It doesn't seem to be available for grub_malloc, though. On SPARC, I thought the code was never written to claim memory in different regions. Does that same problem exist on PPC? I ran into a lot of memory corruption problems with ofdisk. I don’t know if this will solve your problem, but you could try this patch: https://lists.gnu.org/archive/html/grub-devel/2016-06/msg00035.html I couldn’t get any new ieee1275 code to work without it. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-03-20 17:14 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-03-17 17:15 Verifier running out of memory on ieee1275/powerpc64 Stefan Berger 2020-03-18 15:59 ` Simon Hardy 2020-03-18 19:32 ` Stefan Berger 2020-03-18 20:27 ` Stefan Berger 2020-03-18 22:17 ` Simon Hardy 2020-03-19 16:37 ` Stefan Berger 2020-03-20 17:14 ` Eric Snowberg
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.