RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Richard Chan]

I am interested in getting UEFI/PXE working using the EFI IP/TFTP
stack as hacked versions of grub-legacy (e,g., Fedora 17). Is there
any wisdom that the list would like to shed?

1. ON EFI platforms grub-legacy directly uses the  EFI TFTP stack and
not its own i386-pc/TFTP stack.
Network tftp-able files are referred to using the device (nd)

It has a (nd) "network device" and pulls, for it's configuration file,
(nd)/<UUID>
(nd)/<mac address)
(nd)/<IP address> reducing by the LS-Byte each time until finally
(nd)/efidefault

Subsequently, network tftp-able files, e.g. kernel, initrd are
referred to using the (nd) device.

2. GRUB refers to TFTP-able files using the (pxe:<server>) device;
these use the TFTP protocol which is implemented to the i386-pc TFTP
stack.

3. Some questions: would it be better to have a totally new device
name and protocol e.g. "(nd)" and "efitftp" so as not to clash with
the i386-pc infrastructure?

4. File system behaviour:  grub-legacy (with #ifdef for EFI) creates a
separate filesystem structure for EFI.
* mounting the device always succeeds if the PXE stack is started
* dir_open the files check using TFTP for the file size
* read, for the first read (offset 0), the whole file is downloaded
into to memory buffer
subsequent reads read from memory
Would these file  system semantics work for GRUB?

5. I have looked at the existing GRUB net+efinet stack. During my
simple testing I have to be unaable to trigger the code path where
grub populates the grub_net_structure from the UEFI PXE code. This
occurs in grub_efi_net_config_real()  in
grub-core/net/drivers/efi/efinet.c.

When I do
grub> insmod efinet
grub> net_ls_cards
grub> net_ls_addr
I cannot see any information from the PXE boot (i.e. I have booted
grub using UEFI PXE and I expected the efinet structures - at least
one of the cards - to be populated from PXE).

Nevertheless, since the UEFI/PXE stack works independently of Grub,
would it be ok to bypass the whole grub/net/drivers infrastructure and
use UEFI directly, as what grub-legacy currently does?

Thanks in advance for any comments and advice.

Richard

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]

On 25.04.2012 02:16, Richard Chan wrote:
> 5. I have looked at the existing GRUB net+efinet stack. During my
> simple testing I have to be unaable to trigger the code path where
> grub populates the grub_net_structure from the UEFI PXE code. This
> occurs in grub_efi_net_config_real()  in
> grub-core/net/drivers/efi/efinet.c.
The information from EFI network is used only if efinet is embed in
grub.efi. Otherwise you have to call net_bootp manually
> When I do
> grub> insmod efinet
> grub> net_ls_cards
That's not what is expected. Could you try debug where
grub_efinet_findcards in efinet.c stopped?
> grub> net_ls_addr
> I cannot see any information from the PXE boot (i.e. I have booted
> grub using UEFI PXE and I expected the efinet structures - at least
> one of the cards - to be populated from PXE).
>
> Nevertheless, since the UEFI/PXE stack works independently of Grub,
> would it be ok to bypass the whole grub/net/drivers infrastructure and
> use UEFI directly, as what grub-legacy currently does?
No. You basically say "code A has a bug, let's throw it away and write
code B, which will have other bugs". Such approach is unmaintainable.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

On Wed, Apr 25, 2012 at 8:16 AM, Richard Chan <rspchan@starhub.net.sg> wrote:
> I am interested in getting UEFI/PXE working using the EFI IP/TFTP
> stack as hacked versions of grub-legacy (e,g., Fedora 17). Is there
> any wisdom that the list would like to shed?
>
> 1. ON EFI platforms grub-legacy directly uses the  EFI TFTP stack and
> not its own i386-pc/TFTP stack.
> Network tftp-able files are referred to using the device (nd)
>
> It has a (nd) "network device" and pulls, for it's configuration file,
> (nd)/<UUID>
> (nd)/<mac address)
> (nd)/<IP address> reducing by the LS-Byte each time until finally
> (nd)/efidefault
>
> Subsequently, network tftp-able files, e.g. kernel, initrd are
> referred to using the (nd) device.
>
> 2. GRUB refers to TFTP-able files using the (pxe:<server>) device;
> these use the TFTP protocol which is implemented to the i386-pc TFTP
> stack.
>
> 3. Some questions: would it be better to have a totally new device
> name and protocol e.g. "(nd)" and "efitftp" so as not to clash with
> the i386-pc infrastructure?
>
> 4. File system behaviour:  grub-legacy (with #ifdef for EFI) creates a
> separate filesystem structure for EFI.
> * mounting the device always succeeds if the PXE stack is started
> * dir_open the files check using TFTP for the file size
> * read, for the first read (offset 0), the whole file is downloaded
> into to memory buffer
> subsequent reads read from memory
> Would these file  system semantics work for GRUB?
>
> 5. I have looked at the existing GRUB net+efinet stack. During my
> simple testing I have to be unaable to trigger the code path where
> grub populates the grub_net_structure from the UEFI PXE code. This
> occurs in grub_efi_net_config_real()  in
> grub-core/net/drivers/efi/efinet.c.
>
> When I do
> grub> insmod efinet
> grub> net_ls_cards
> grub> net_ls_addr
> I cannot see any information from the PXE boot (i.e. I have booted
> grub using UEFI PXE and I expected the efinet structures - at least
> one of the cards - to be populated from PXE).
>
> Nevertheless, since the UEFI/PXE stack works independently of Grub,
> would it be ok to bypass the whole grub/net/drivers infrastructure and
> use UEFI directly, as what grub-legacy currently does?
>
> Thanks in advance for any comments and advice.

Hi,

You should know that most UEFI implementation does not ship with
network drivers. Only some server machine and apple's efi have network
support. Although apple's firmware has snp driver, it doesn't have
tftp driver. So grub-legacy's method of relying on existing network
stack doesn't work very well in reality.

However, if you have an intel card, then you're in luck since intel
provides uefi driver download from their website, no other vendor
provides it as far as I know.

First, download intel driver from here:
http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=19186%20

Then, you need to start efi shell first, and use load command like this:

load E4404X3.efi (The latest version may have a different name)
load snp

Then load grub2 from efi shell, net_ls_cards should work now.

If you want the whole tftp stack, you need to load bc from efi shell as well.

load bc

snp and bc are compiled from edk source code, I can send you binary
copy if you don't know how to do it.

-- 
Best wishes
Bean

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Seth Goldberg]

Hi,

On Apr 25, 2012, at 8:59 AM, Bean wrote:

> On Wed, Apr 25, 2012 at 8:16 AM, Richard Chan <rspchan@starhub.net.sg> wrote:
>> I am interested in getting UEFI/PXE working using the EFI IP/TFTP
>> stack as hacked versions of grub-legacy (e,g., Fedora 17). Is there
>> any wisdom that the list would like to shed?
>> 
>> 1. ON EFI platforms grub-legacy directly uses the  EFI TFTP stack and
>> not its own i386-pc/TFTP stack.
>> Network tftp-able files are referred to using the device (nd)
>> 
>> It has a (nd) "network device" and pulls, for it's configuration file,
>> (nd)/<UUID>
>> (nd)/<mac address)
>> (nd)/<IP address> reducing by the LS-Byte each time until finally
>> (nd)/efidefault
>> 
>> Subsequently, network tftp-able files, e.g. kernel, initrd are
>> referred to using the (nd) device.
>> 
>> 2. GRUB refers to TFTP-able files using the (pxe:<server>) device;
>> these use the TFTP protocol which is implemented to the i386-pc TFTP
>> stack.

 Just to chime in here with some data -- I've found numerous UEFI systems' network functionality to be buggy (what a shock, right).  Specifically, using the TFTP APIs allow files to be retrieved, but using GRUB 2's TFTP stack, those same files fail to download, with the failure lying somewhere within the network driver.  In other words, there is close coupling between the network driver and the TFTP implementation in some vendors' UEFI implementations such that when you try to just use SNP, you end up with random timeouts that kill performance or dropped packets.  So, supporting use of UEFI's TFTP APIs seems like a good thing to do to deal with those types of systems.

 --S

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Richard Chan]

Hi Vladimir and list

"The information from EFI network is used only if efinet is embed in
grub.efi. Otherwise you have to call net_bootp manually"

1. How do you "embed" efinet.mod into the image so that
grub_efi_net_config is called?
I presume you mean somthing different from preload of module?

./grub-core/kern/efi/init.c:  if (!*device && grub_efi_net_config)

./grub-core/net/drivers/efi/efinet.c:  grub_efi_net_config =
grub_efi_net_config_real;

The latter function populates the NIC structure from PXE, but in my case, it is
not being called.

2. For net_bootp, I can see the BOOTP request/reply but GRUB will
timeout setting the IP address. It is as if it doesn't see the BOOTP
reply.

3. On a Dell UEFI I have successfully used GRUB net/tftp with manual
IP addressing but can't see a simple way to set the netmask
insmod tftp
insmod efinet
net_bootp efinet0
## timeout , even though I can see BOOTP request/reply on the wire

insmod tftp
insmod efinet
net_add_addr em1 efinet0 172.31.0.167
## sets /16 netmask, any way to make it /nn?
net_del_route em1:local
net_add_route em1:local 172.31.0.0/24 em1
## now I can actualy ping the interface from another machine
##
linux (pxe:1.2.3.4)/vmlinuz root=/dev/sda3 ro
initrd (pxe:1.2.3.4)/initramfs.img
## Dell EFI network driver + GRUB tftp works  :-) :-)

Richard

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On 25.04.2012 22:20, Richard Chan wrote:
> Hi Vladimir and list
>
> "The information from EFI network is used only if efinet is embed in
> grub.efi. Otherwise you have to call net_bootp manually"
>
> 1. How do you "embed" efinet.mod into the image so that
> grub_efi_net_config is called?
> I presume you mean somthing different from preload of module?
No, just preload
> ./grub-core/kern/efi/init.c:  if (!*device && grub_efi_net_config)
>
> ./grub-core/net/drivers/efi/efinet.c:  grub_efi_net_config =
> grub_efi_net_config_real;
>
> The latter function populates the NIC structure from PXE, but in my case, it is
> not being called.
Network config is started only if root is on network. Otherwise network
config is considered unnecessary and skipped.
>
> 2. For net_bootp, I can see the BOOTP request/reply but GRUB will
> timeout setting the IP address. It is as if it doesn't see the BOOTP
> reply.
Do you have full dumps? It's possible GRUB rejects them for some field
mismatch (not all bootp servers are good).


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Richard Chan]

I found what seems to be an incompatiblity when GRUB/UEFI bootp is
used with PXE active: you cannot reuse the same IP address. I.e. you
cannot DHCP twice - once by UEFI/PXE and the 2nd time by GRUB/bootp
The 2nd attempt will fail.

Eg., UEFI/PXE has received address 172.31.0.202.
If I try to reuse the same IP address (either manually by net_add_addr
or via net_bootp) I get the error message "timeout cannot resolve
hardware address".
If I use a different IP address it works.

Hence it seems that with the native PXE stack active you cannot reuse
the same IP for GRUB IP stack and UEFI/PXE.

Is there a command that allows me to shutdown the PXE stack from the
grub shell? Then I might be able to try net_bootp again.






2012/4/26 Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com>:
> On 25.04.2012 22:20, Richard Chan wrote:
>> Hi Vladimir and list
>>
>> "The information from EFI network is used only if efinet is embed in
>> grub.efi. Otherwise you have to call net_bootp manually"
>>
>> 1. How do you "embed" efinet.mod into the image so that
>> grub_efi_net_config is called?
>> I presume you mean somthing different from preload of module?
> No, just preload
>> ./grub-core/kern/efi/init.c:  if (!*device && grub_efi_net_config)
>>
>> ./grub-core/net/drivers/efi/efinet.c:  grub_efi_net_config =
>> grub_efi_net_config_real;
>>
>> The latter function populates the NIC structure from PXE, but in my case, it is
>> not being called.
> Network config is started only if root is on network. Otherwise network
> config is considered unnecessary and skipped.
>>
>> 2. For net_bootp, I can see the BOOTP request/reply but GRUB will
>> timeout setting the IP address. It is as if it doesn't see the BOOTP
>> reply.
> Do you have full dumps? It's possible GRUB rejects them for some field
> mismatch (not all bootp servers are good).
>
>
> --
> Regards
> Vladimir 'φ-coder/phcoder' Serbinenko
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

On Thu, Apr 26, 2012 at 1:57 AM, Seth Goldberg <seth.goldberg@oracle.com> wrote:
>  Just to chime in here with some data -- I've found numerous UEFI systems' network functionality to be buggy (what a shock, right).  Specifically, using the TFTP APIs allow files to be retrieved, but using GRUB 2's TFTP stack, those same files fail to download, with the failure lying somewhere within the network driver.  In other words, there is close coupling between the network driver and the TFTP implementation in some vendors' UEFI implementations such that when you try to just use SNP, you end up with random timeouts that kill performance or dropped packets.  So, supporting use of UEFI's TFTP APIs seems like a good thing to do to deal with those types of systems.

Hi,

Actually I believe the problem is not in snp, but in timeout handling
mechanism. I once implemented a tftp service using udp, and found its
performance very bad compared to the native driver. After some
debugging, I found out that it set an event which is signaled by snp
while udp set the timeout to 0 so that it always returned whether or
not there is available packet. When I use similar technique, my own
tftp run as fast as the native service.

-- 
Best wishes
Bean

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

On 25.04.2012 22:21, Bean wrote:
> On Thu, Apr 26, 2012 at 1:57 AM, Seth Goldberg <seth.goldberg@oracle.com> wrote:
>>  Just to chime in here with some data -- I've found numerous UEFI systems' network functionality to be buggy (what a shock, right).  Specifically, using the TFTP APIs allow files to be retrieved, but using GRUB 2's TFTP stack, those same files fail to download, with the failure lying somewhere within the network driver.  In other words, there is close coupling between the network driver and the TFTP implementation in some vendors' UEFI implementations such that when you try to just use SNP, you end up with random timeouts that kill performance or dropped packets.  So, supporting use of UEFI's TFTP APIs seems like a good thing to do to deal with those types of systems.
> Hi,
>
> Actually I believe the problem is not in snp, but in timeout handling
> mechanism. I once implemented a tftp service using udp, and found its
> performance very bad compared to the native driver. After some
> debugging, I found out that it set an event which is signaled by snp
> while udp set the timeout to 0 so that it always returned whether or
> not there is available packet. When I use similar technique, my own
> tftp run as fast as the native service.
It's very good that you found the real reason for this brain damage. I'm
happy that someone did. Do you have this in code/as patch?
>


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

2012/4/26 Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com>:
> On 25.04.2012 22:21, Bean wrote:
>> On Thu, Apr 26, 2012 at 1:57 AM, Seth Goldberg <seth.goldberg@oracle.com> wrote:
>>>  Just to chime in here with some data -- I've found numerous UEFI systems' network functionality to be buggy (what a shock, right).  Specifically, using the TFTP APIs allow files to be retrieved, but using GRUB 2's TFTP stack, those same files fail to download, with the failure lying somewhere within the network driver.  In other words, there is close coupling between the network driver and the TFTP implementation in some vendors' UEFI implementations such that when you try to just use SNP, you end up with random timeouts that kill performance or dropped packets.  So, supporting use of UEFI's TFTP APIs seems like a good thing to do to deal with those types of systems.
>> Hi,
>>
>> Actually I believe the problem is not in snp, but in timeout handling
>> mechanism. I once implemented a tftp service using udp, and found its
>> performance very bad compared to the native driver. After some
>> debugging, I found out that it set an event which is signaled by snp
>> while udp set the timeout to 0 so that it always returned whether or
>> not there is available packet. When I use similar technique, my own
>> tftp run as fast as the native service.
> It's very good that you found the real reason for this brain damage. I'm
> happy that someone did. Do you have this in code/as patch?

Hi,

This requires significant modification to the driver interface, change
the definition of get_card_packet from:

static struct grub_net_buff *
get_card_packet (const struct grub_net_card *dev)

to something like this:
static struct grub_net_buff *
get_card_packet (const struct grub_net_card *dev, int timeout)

The former uses async mode and return as soon as possible, while the
later uses sync mode and wait for the packet for timeout before
returning. Perhaps you can contact the network stack author to see if
such transformation is possible.

PS: the reason why async mode doesn't work very well is because it
spent too much time in upper layers, which increase the chance of
packet loss. Each lost packet would need to be re-transmitted by
server which is a major performance killer.

-- 
Best wishes
Bean

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Seth Goldberg]

On Apr 25, 2012, at 7:23 PM, Bean wrote:

> 2012/4/26 Vladimir 'φ-coder/phcoder' Serbinenko <phcoder@gmail.com>:
>> On 25.04.2012 22:21, Bean wrote:
>>> On Thu, Apr 26, 2012 at 1:57 AM, Seth Goldberg <seth.goldberg@oracle.com> wrote:
>>>>  Just to chime in here with some data -- I've found numerous UEFI systems' network functionality to be buggy (what a shock, right).  Specifically, using the TFTP APIs allow files to be retrieved, but using GRUB 2's TFTP stack, those same files fail to download, with the failure lying somewhere within the network driver.  In other words, there is close coupling between the network driver and the TFTP implementation in some vendors' UEFI implementations such that when you try to just use SNP, you end up with random timeouts that kill performance or dropped packets.  So, supporting use of UEFI's TFTP APIs seems like a good thing to do to deal with those types of systems.
>>> Hi,
>>> 
>>> Actually I believe the problem is not in snp, but in timeout handling
>>> mechanism. I once implemented a tftp service using udp, and found its
>>> performance very bad compared to the native driver. After some
>>> debugging, I found out that it set an event which is signaled by snp
>>> while udp set the timeout to 0 so that it always returned whether or
>>> not there is available packet. When I use similar technique, my own
>>> tftp run as fast as the native service.
>> It's very good that you found the real reason for this brain damage. I'm
>> happy that someone did. Do you have this in code/as patch?
> 
> Hi,
> 
> This requires significant modification to the driver interface, change
> the definition of get_card_packet from:
> 
> static struct grub_net_buff *
> get_card_packet (const struct grub_net_card *dev)
> 
> to something like this:
> static struct grub_net_buff *
> get_card_packet (const struct grub_net_card *dev, int timeout)
> 
> The former uses async mode and return as soon as possible, while the
> later uses sync mode and wait for the packet for timeout before
> returning. Perhaps you can contact the network stack author to see if
> such transformation is possible.
> 
> PS: the reason why async mode doesn't work very well is because it
> spent too much time in upper layers, which increase the chance of
> packet loss. Each lost packet would need to be re-transmitted by
> server which is a major performance killer.

  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.

 --S

> 
> -- 
> Best wishes
> Bean
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

On Thu, Apr 26, 2012 at 12:10 PM, Seth Goldberg
<seth.goldberg@oracle.com> wrote:
>  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.

Hi,

Normal OS has interrupt handler that removes the packet from nic
buffer as soon as possible, but grub2 is basically single thread and
use pull mode. So we should make the pulling loop as short as
possible. In async mode, if a packet is not found, it has to returned
to upper layer and retry, while in sync mode, the loop is inside the
driver which make it more efficient. It's something like reading 100
bytes from disk is much faster than 100 x 1 byte.


-- 
Best wishes
Bean

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Seth Goldberg]

On Apr 25, 2012, at 11:22 PM, Bean <bean123ch@gmail.com> wrote:

> On Thu, Apr 26, 2012 at 12:10 PM, Seth Goldberg
> <seth.goldberg@oracle.com> wrote:
>>  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.
> 
> Hi,
> 
> Normal OS has interrupt handler that removes the packet from nic
> buffer as soon as possible, but grub2 is basically single thread and
> use pull mode. So we should make the pulling loop as short as
> possible. In async mode, if a packet is not found, it has to returned
> to upper layer and retry, while in sync mode, the loop is inside the
> driver which make it more efficient. It's something like reading 100
> bytes from disk is much faster than 100 x 1 byte.
> 

  Marginally.  We still need to handle that packet inside grub once it is received.  Besides, modern nic hardware has receive rings and overrun is rare especially with non-pipelined tftp (unless there is a ton of broadcast or unicast packets sent to the client while in grub which is unlikely in practice, so I'm not seeing where the real benefit is here.  I can definitely understand your finding inefficiencies in the uefi udp-snp (or in systems I've seen it's more like UDP-MNP-SNP), so no argument there, but for grub 2, it's already using the lowest level interface it can (without calling undi directly).

 --S

> 
> -- 
> Best wishes
> Bean
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

On Thu, Apr 26, 2012 at 12:59 PM, Seth Goldberg
<seth.goldberg@oracle.com> wrote:
>
>
> On Apr 25, 2012, at 11:22 PM, Bean <bean123ch@gmail.com> wrote:
>
>> On Thu, Apr 26, 2012 at 12:10 PM, Seth Goldberg
>> <seth.goldberg@oracle.com> wrote:
>>>  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.
>>
>> Hi,
>>
>> Normal OS has interrupt handler that removes the packet from nic
>> buffer as soon as possible, but grub2 is basically single thread and
>> use pull mode. So we should make the pulling loop as short as
>> possible. In async mode, if a packet is not found, it has to returned
>> to upper layer and retry, while in sync mode, the loop is inside the
>> driver which make it more efficient. It's something like reading 100
>> bytes from disk is much faster than 100 x 1 byte.
>>
>
>  Marginally.  We still need to handle that packet inside grub once it is received.  Besides, modern nic hardware has receive rings and overrun is rare especially with non-pipelined tftp (unless there is a ton of broadcast or unicast packets sent to the client while in grub which is unlikely in practice, so I'm not seeing where the real benefit is here.  I can definitely understand your finding inefficiencies in the uefi udp-snp (or in systems I've seen it's more like UDP-MNP-SNP), so no argument there, but for grub 2, it's already using the lowest level interface it can (without calling undi directly).

Hi,

Well, it's possible that different uefi implementation has different
issues, but at least this method works quite well for the machines I
tested. I can send you a test program if you want to give it a try.

-- 
Best wishes
Bean

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Seth Goldberg]

On Apr 26, 2012, at 6:46 AM, Bean <bean123ch@gmail.com> wrote:

> On Thu, Apr 26, 2012 at 12:59 PM, Seth Goldberg
> <seth.goldberg@oracle.com> wrote:
>> 
>> 
>> On Apr 25, 2012, at 11:22 PM, Bean <bean123ch@gmail.com> wrote:
>> 
>>> On Thu, Apr 26, 2012 at 12:10 PM, Seth Goldberg
>>> <seth.goldberg@oracle.com> wrote:
>>>>  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.
>>> 
>>> Hi,
>>> 
>>> Normal OS has interrupt handler that removes the packet from nic
>>> buffer as soon as possible, but grub2 is basically single thread and
>>> use pull mode. So we should make the pulling loop as short as
>>> possible. In async mode, if a packet is not found, it has to returned
>>> to upper layer and retry, while in sync mode, the loop is inside the
>>> driver which make it more efficient. It's something like reading 100
>>> bytes from disk is much faster than 100 x 1 byte.
>>> 
>> 
>>  Marginally.  We still need to handle that packet inside grub once it is received.  Besides, modern nic hardware has receive rings and overrun is rare especially with non-pipelined tftp (unless there is a ton of broadcast or unicast packets sent to the client while in grub which is unlikely in practice, so I'm not seeing where the real benefit is here.  I can definitely understand your finding inefficiencies in the uefi udp-snp (or in systems I've seen it's more like UDP-MNP-SNP), so no argument there, but for grub 2, it's already using the lowest level interface it can (without calling undi directly).
> 
> Hi,
> 
> Well, it's possible that different uefi implementation has different
> issues, but at least this method works quite well for the machines I
> tested. I can send you a test program if you want to give it a try.

  Sure!  Thanks!

  --S


> 
> -- 
> Best wishes
> Bean
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: RFC: UEFI/PXE and emulating grub-legacy-uefi-hacked behaviour

[Bean]

On Fri, Apr 27, 2012 at 1:10 AM, Seth Goldberg <seth.goldberg@oracle.com> wrote:
>
>
> On Apr 26, 2012, at 6:46 AM, Bean <bean123ch@gmail.com> wrote:
>
>> On Thu, Apr 26, 2012 at 12:59 PM, Seth Goldberg
>> <seth.goldberg@oracle.com> wrote:
>>>
>>>
>>> On Apr 25, 2012, at 11:22 PM, Bean <bean123ch@gmail.com> wrote:
>>>
>>>> On Thu, Apr 26, 2012 at 12:10 PM, Seth Goldberg
>>>> <seth.goldberg@oracle.com> wrote:
>>>>>  How does this work around the issue?  I'm not seeing it -- we call SNP directly.  We don't go through UDP or any other upper layers in efinet.  When I did the investigation, I removed ALL other consumers of SNP manually via the efi shell before loading GRUB 2 and still saw packet loss.
>>>>
>>>> Hi,
>>>>
>>>> Normal OS has interrupt handler that removes the packet from nic
>>>> buffer as soon as possible, but grub2 is basically single thread and
>>>> use pull mode. So we should make the pulling loop as short as
>>>> possible. In async mode, if a packet is not found, it has to returned
>>>> to upper layer and retry, while in sync mode, the loop is inside the
>>>> driver which make it more efficient. It's something like reading 100
>>>> bytes from disk is much faster than 100 x 1 byte.
>>>>
>>>
>>>  Marginally.  We still need to handle that packet inside grub once it is received.  Besides, modern nic hardware has receive rings and overrun is rare especially with non-pipelined tftp (unless there is a ton of broadcast or unicast packets sent to the client while in grub which is unlikely in practice, so I'm not seeing where the real benefit is here.  I can definitely understand your finding inefficiencies in the uefi udp-snp (or in systems I've seen it's more like UDP-MNP-SNP), so no argument there, but for grub 2, it's already using the lowest level interface it can (without calling undi directly).
>>
>> Hi,
>>
>> Well, it's possible that different uefi implementation has different
>> issues, but at least this method works quite well for the machines I
>> tested. I can send you a test program if you want to give it a try.
>
>  Sure!  Thanks!

Hi,

I sent it to your email yesterday but it seems the mail server blocks
binary attachment, anyway, i upload it here:

http://download.gna.org/grubutil/efitest.zip

The main program is efiload.efi, I also include edk driver snp, bc and
pxedhcp4 just in case the firmware doesn't contain them.

Loading imagefile using native tftp function:
efiload /imagename 0 1

Loading imagefile using custom tftp function
efiload /imagename 16

The third parameter here is the block size (in KB), you can set it as
large as 60, while native tftp normally support up to 8K.

-- 
Best wishes
Bean