[RFC 0/2] UEFI-based HTTP Boot

[RFC 0/2] UEFI-based HTTP Boot

[Ken Lin]

This RFC patchset is stacked on the previous HTTP boot patchset:
https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
It re-uses some code from it, e.g. the DCHPACK
with vendor_class_identifier=HTTPClient.

Instead of implementing HTTP and HTTPS boot totally from software,
UEFI firmware already defines APIs for HTTP(s).
Please check UEFI spec. 2.5 and plus for the detail:

28.6 EFI HTTP Protocols

Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
the HTTP(s) APIs are not available. In the case,
Grub can fall back to the software-based implementation.
In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
is the code to query if the HTTP Protocol is supported by the UEFI firmware.

This patchset was tested on QEMU+OVMF and it works flawlessly.

The main goals of this RFC is to ask for opinions and suggestion to make
the first patch modularized as much as possible.
In the second patch, there is some codes related TCP re-transmission
that need to pass by for the HTTP Boot to work.

More details are described in the logs of each patch. 


Ken Lin (2):
  net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
  net: workaround to bypass corruption of the efihttp function pointer

 grub-core/Makefile.core.def         |   1 +
 grub-core/net/bootp.c               |   6 +
 grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
 grub-core/net/drivers/efi/efinet.c  |   1 +
 grub-core/net/net.c                 |  39 +++-
 include/grub/efi/api.h              |  17 ++
 include/grub/efi/http.h             | 221 +++++++++++++++++++++
 include/grub/err.h                  |   3 +-
 include/grub/net.h                  |   1 +
 9 files changed, 672 insertions(+), 3 deletions(-)
 create mode 100755 grub-core/net/drivers/efi/efihttp.c
 create mode 100755 include/grub/efi/http.h

-- 
2.7.4

[RFC 1/2] net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol

[Ken Lin]

This patch implements HTTP Boot by using UEFI HTTP Protocols
defined in the UEFI spec (v2.5+):

 28.6 EFI HTTP Protocols

It would be better to make efihttp as a single module similar to http.mod.
But I cannot fit my implementation to the interfaces (grub_net_app_protocol).
Some operations conflict: there is packets_pulled(), but no read().
So I placed efihttp in the net.mod.

It will be much appreciated to have suggestions of the best way
to get this patch integrated in grub.

Signed-off-by: Ken Lin <ken.lin@hpe.com>
Signed-off-by: Clay Chang <clayc@hpe.com>
Reviewed-by: Keng-Yu Lin <kengyu@hpe.com>
---
 grub-core/Makefile.core.def         |   1 +
 grub-core/net/bootp.c               |   6 +
 grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
 grub-core/net/drivers/efi/efinet.c  |   1 +
 grub-core/net/net.c                 |  36 +++-
 include/grub/efi/api.h              |  17 ++
 include/grub/efi/http.h             | 221 +++++++++++++++++++++
 include/grub/err.h                  |   3 +-
 include/grub/net.h                  |   1 +
 9 files changed, 669 insertions(+), 3 deletions(-)
 create mode 100755 grub-core/net/drivers/efi/efihttp.c
 create mode 100755 include/grub/efi/http.h

diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 2dfa22a..6a35951 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -2143,6 +2143,7 @@ module = {
   common = net/ethernet.c;
   common = net/arp.c;
   common = net/netbuff.c;
+  efi = net/drivers/efi/efihttp.c;
 };
 
 module = {
diff --git a/grub-core/net/bootp.c b/grub-core/net/bootp.c
index de9239c..17ac54c 100644
--- a/grub-core/net/bootp.c
+++ b/grub-core/net/bootp.c
@@ -26,6 +26,9 @@
 #include <grub/datetime.h>
 #include <grub/time.h>
 #include <grub/list.h>
+#ifdef GRUB_MACHINE_EFI
+#include <grub/efi/http.h>
+#endif
 
 static int
 dissect_url (const char *url, char **proto, char **host, char **path)
@@ -339,6 +342,9 @@ grub_net_configure_by_dhcp_ack (const char *name,
 	    }
 	  else
 	    grub_errno = GRUB_ERR_NONE;
+#ifdef GRUB_MACHINE_EFI
+          grub_efihttp_configure (card, bp);
+#endif
 
 	  grub_free (proto);
 	  grub_free (ip);
diff --git a/grub-core/net/drivers/efi/efihttp.c b/grub-core/net/drivers/efi/efihttp.c
new file mode 100755
index 0000000..7e63f5c
--- /dev/null
+++ b/grub-core/net/drivers/efi/efihttp.c
@@ -0,0 +1,386 @@
+/* efihttp.c - EFI HTTP. */
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2009  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <grub/misc.h>
+#include <grub/efi/efi.h>
+#include <grub/efi/api.h>
+#include <grub/efi/http.h>
+#include <grub/charset.h>
+
+/* EFI-HTTP(S) Boot variables */
+grub_efi_http_t *grub_efihttp = NULL;
+static grub_efi_boolean_t grub_efihttp_request_callback_done;
+static grub_efi_boolean_t grub_efihttp_response_callback_done;
+static grub_uint8_t *efihttp_rx_buf;
+
+static void
+grub_efihttp_request_callback (grub_efi_event_t event, void *context)
+{
+  grub_dprintf ("efihttp", "grub_efihttp_request_callback(), event:%p, context:%p\n", event, context);
+  grub_efihttp_request_callback_done = 1;
+}
+
+static void
+grub_efihttp_response_callback (grub_efi_event_t event, void *context)
+{
+  grub_dprintf ("efihttp", "grub_efihttp_request_callback(), event:%p, context:%p\n", event, context);
+  grub_efihttp_response_callback_done = 1;
+}
+
+grub_err_t
+grub_efihttp_configure (struct grub_net_card *card, const struct grub_net_bootp_packet *bp)
+{ 
+  grub_efi_guid_t grub_efihttp_sb_guid = GRUB_EFI_HTTP_SERVICE_BINDING_PROTOCOL_GUID;
+  grub_efi_guid_t grub_efihttp_guid = GRUB_EFI_HTTP_PROTOCOL_GUID;
+  grub_efi_service_binding_t *grub_efihttp_sb = NULL;
+  grub_efi_handle_t grub_efihttp_handle = NULL;
+  grub_efi_http_config_data_t grub_efihttp_config_data;
+  grub_efi_httpv4_access_point_t grub_efihttp_ipv4_node;
+  grub_efi_boot_services_t *b = grub_efi_system_table->boot_services;
+  grub_efi_status_t status;
+
+  grub_dprintf ("efihttp", "Enter grub_efihttp_configure()\n");
+  
+  grub_efihttp_sb = grub_efi_open_protocol (card->efi_handle, &grub_efihttp_sb_guid, GRUB_EFI_OPEN_PROTOCOL_GET_PROTOCOL);
+  if (!grub_efihttp_sb)
+  {
+      grub_dprintf ("efihttp", "Fail to open the Service Binding protocol!\n");
+      return GRUB_ERR_EFI;
+  }
+
+  grub_dprintf ("efihttp", "sb->create_child()\n");
+  status = efi_call_2 (grub_efihttp_sb->create_child, grub_efihttp_sb, &grub_efihttp_handle);
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Fail to create child! status=%d\n", (int)status);
+      return GRUB_ERR_EFI;
+  }
+
+  grub_dprintf ("efihttp", "b->handle_protocol()\n");
+  status = efi_call_3(b->handle_protocol, grub_efihttp_handle, &grub_efihttp_guid, (void**)&grub_efihttp);
+  if (!grub_efihttp || GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Error! Fail to get HTTP protocol! status=%d\n", (int)status);
+      return GRUB_ERR_EFI;
+  }
+
+  grub_memset (&grub_efihttp_config_data, 0, sizeof(grub_efihttp_config_data));
+  grub_efihttp_config_data.http_version = GRUB_EFI_HTTPVERSION11;
+  grub_efihttp_config_data.timeout_millisec = 5000;
+  grub_efihttp_config_data.local_address_is_ipv6 = 0;
+  grub_memset (&grub_efihttp_ipv4_node, 0, sizeof(grub_efihttp_ipv4_node));
+  grub_efihttp_ipv4_node.use_default_address = 0;
+  grub_memcpy ((void*)grub_efihttp_ipv4_node.local_address, &bp->your_ip, sizeof (bp->your_ip));
+  grub_memcpy ((void*)grub_efihttp_ipv4_node.local_subnet, &bp->subnet_mask, sizeof (bp->subnet_mask));
+  grub_efihttp_ipv4_node.local_port = 0;
+  grub_efihttp_config_data.access_point.ipv4_node = &grub_efihttp_ipv4_node;
+
+  grub_dprintf ("efihttp", "grub_efihttp->configure()\n");
+  status = efi_call_2 (grub_efihttp->configure, grub_efihttp, &grub_efihttp_config_data);
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Fail to do configuration! status=%d\n", (int)status);
+      return GRUB_ERR_EFI;
+  }
+
+  grub_dprintf ("efihttp", "Leave grub_efihttp_configure()\n");
+
+  return GRUB_ERR_NONE;
+}
+  
+grub_err_t
+grub_efihttp_open (grub_file_t file, const char *filename)
+{
+  grub_efi_http_request_data_t request_data;
+  grub_efi_http_header_t request_headers[3];
+  grub_efi_http_message_t *request_message;
+  grub_efi_http_token_t *request_token;
+  grub_efi_http_response_data_t response_data;
+  grub_efi_http_message_t *response_message;
+  grub_efi_http_token_t *response_token;
+  grub_efi_boot_services_t *b = grub_efi_system_table->boot_services;
+  grub_efi_status_t status;
+  grub_efi_http_status_code_t http_status;
+  const char *http = "http://";
+  char *url;
+  grub_efi_char16_t *usc2_url;
+  grub_uint32_t url_len, usc2_url_len;
+  grub_uint32_t offset, length, i;
+
+  grub_dprintf ("efihttp", "Enter grub_efihttp_open(), file->name:%s\n", file->name);
+  grub_dprintf ("efihttp", "grub_efihttp:%p, grub_efihttp->request:%p\n", grub_efihttp, grub_efihttp->request);
+
+  /* init */
+  grub_memset (&request_data, 0, sizeof (grub_efi_http_request_data_t));
+  request_message = grub_zalloc (sizeof (grub_efi_http_message_t));
+  request_token = grub_zalloc (sizeof (grub_efi_http_token_t));
+  grub_memset (&response_data, 0, sizeof (grub_efi_http_response_data_t));
+  response_message = grub_zalloc (sizeof (grub_efi_http_message_t));
+  response_token = grub_zalloc (sizeof (grub_efi_http_token_t));
+
+  request_data.method = GRUB_EFI_HTTPMETHODGET;
+
+  /* url */
+  url_len = grub_strlen (http) + grub_strlen (file->device->net->server) + grub_strlen (file->device->net->name);
+  url = grub_malloc ((url_len + 1) * sizeof (url[0]));
+  grub_memset (url, 0, url_len);
+  grub_strncpy (url, http, grub_strlen(http));
+  offset = grub_strlen (http);
+  grub_strncpy (url + offset, file->device->net->server, grub_strlen (file->device->net->server));
+  offset += grub_strlen (file->device->net->server);
+  grub_strncpy (url + offset, file->device->net->name, grub_strlen (file->device->net->name));
+  url[url_len] = 0;
+  grub_dprintf ("efihttp", "url:%s\n", url);
+  usc2_url_len = url_len * GRUB_MAX_UTF16_PER_UTF8;
+  usc2_url = grub_malloc ((usc2_url_len + 1) * sizeof (usc2_url[0]));
+  usc2_url_len = grub_utf8_to_utf16 (usc2_url, usc2_url_len, (grub_uint8_t *)url, url_len, NULL); /* convert string format from ascii to usc2 */
+  usc2_url[usc2_url_len] = 0;
+  request_data.url = usc2_url;
+
+  /* headers */
+  request_headers[0].field_name = (grub_efi_char8_t*)"Host";
+  request_headers[0].field_value = (grub_efi_char8_t*)file->device->net->server;
+  request_headers[1].field_name = (grub_efi_char8_t*)"Accept";
+  request_headers[1].field_value = (grub_efi_char8_t*)"*/*";
+  request_headers[2].field_name = (grub_efi_char8_t*)"User-Agent";
+  request_headers[2].field_value = (grub_efi_char8_t*)"UefiHttpBoot/1.0";
+
+  request_message->data.request = &request_data;
+  request_message->header_count = 3;
+  request_message->headers = request_headers;
+  request_message->body_length = 0;
+  request_message->body = NULL;
+
+  /* request token */
+  request_token->event = NULL;
+  request_token->status = GRUB_EFI_NOT_READY;
+  request_token->message = request_message;
+  grub_efihttp_request_callback_done = 0;
+  grub_dprintf ("efihttp", "b->create_event()\n");
+  status = efi_call_5 (b->create_event,
+                       GRUB_EFI_EVT_NOTIFY_SIGNAL,
+                       GRUB_EFI_TPL_CALLBACK,
+                       grub_efihttp_request_callback,
+                       NULL,
+                       &request_token->event);
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Fail to create an event! status=%d\n", (int)status);
+      return status;
+  }
+
+  grub_dprintf ("efihttp", "grub_efihttp:%p, grub_efihttp->request:%p\n", grub_efihttp, grub_efihttp->request);
+
+  /* make a HTTP request */
+  grub_dprintf ("efihttp", "Before grub_efihttp->request(), url:%s\n", url);
+  status = efi_call_2 (grub_efihttp->request, grub_efihttp, request_token);
+  grub_dprintf ("efihttp", "After grub_efihttp->request()\n");
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Fail to send a request! status=%d\n", (int)status);
+      return GRUB_ERR_EFI;
+  }
+  /* allow the network stack 10 seconds to send the request successfully */
+  while (!grub_efihttp_request_callback_done)
+  {
+      efi_call_1(grub_efihttp->poll, grub_efihttp); // give the http driver more motivation
+  }
+
+  response_data.status_code = GRUB_EFI_HTTP_STATUS_UNSUPPORTED_STATUS;
+  response_message->data.response = &response_data;
+  response_message->header_count = 0; // herader_count will be updated by the HTTP driver on response
+  response_message->headers = NULL; // headers will be populated by the driver on response
+  /* use zero BodyLength to only receive the response headers */
+  response_message->body_length = 0;
+  response_message->body = NULL;
+  response_token->event = NULL;
+  efi_call_5 (b->create_event,
+              GRUB_EFI_EVT_NOTIFY_SIGNAL,
+              GRUB_EFI_TPL_CALLBACK,
+              grub_efihttp_response_callback,
+              NULL,
+              &response_token->event);
+  response_token->status = GRUB_EFI_SUCCESS;
+  response_token->message = response_message;
+
+  /* wait for HTTP response */
+  grub_efihttp_response_callback_done = 0;
+  grub_dprintf ("efihttp", "Before grub_efihttp->response()\n");
+  status = efi_call_2 (grub_efihttp->response, grub_efihttp, response_token);
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Fail to receive a response! status=%d\n", (int)status);
+      return status;
+  }
+  while (!grub_efihttp_response_callback_done)
+  {
+      efi_call_1 (grub_efihttp->poll, grub_efihttp);
+  }
+  grub_dprintf ("efihttp", "After grub_efihttp->response(), response_message->body_length:%d\n", response_message->body_length);
+
+  /* check the HTTP status code */
+  http_status = response_token->message->data.response->status_code;
+  grub_dprintf ("efihttp", "http_status=%d\n", (int)http_status);
+
+  /* parse the length of the file from the ContentLength header */
+  grub_dprintf ("efihttp", "response_message->header_count:%d\n", response_message->header_count);
+  for (length = 0, i = 0; i < response_message->header_count; ++i)
+  {
+      if (!grub_strcmp((const char*)response_message->headers[i].field_name, "Content-Length"))
+      {
+          length = grub_strtoul((const char*)response_message->headers[i].field_value, 0, 10);
+          break;
+      }
+  }
+  file->size = (grub_off_t)length;
+  
+  file->not_easily_seekable = 0;
+  file->data = (void*)filename;
+  file->device->net->offset = 0;
+  efihttp_rx_buf = grub_malloc (EFIHTTP_RX_BUF_LEN);
+  
+  /* release */
+  grub_free (request_message);
+  grub_free (request_token);
+  grub_free (response_message);
+  grub_free (response_token);
+  
+  grub_dprintf ("efihttp", "Leave grub_efihttp_open(), file->size:%d, file->offset:%d\n", (int)file->size, (int)file->offset);
+
+  return GRUB_ERR_NONE;
+}
+
+grub_err_t
+grub_efihttp_close (grub_file_t file)
+{
+  grub_efi_status_t status;
+
+  grub_dprintf ("efihttp", "Enter grub_efihttp_close(), file->device->net->name:%s, file->offset:%d\n", file->device->net->name, (int)file->offset);  
+  status = efi_call_2 (grub_efihttp->cancel, grub_efihttp, NULL);
+  if (GRUB_EFI_SUCCESS != status)
+  {
+      grub_dprintf ("efihttp", "Error! status=%d\n", (int)status);
+      return GRUB_ERR_EFI;
+  }
+  grub_free (efihttp_rx_buf);
+  file->offset = 0;
+  file->device->net->offset = 0;
+  grub_dprintf ("efihttp", "Leave grub_efihttp_close(), file->device->net->name:%s, file->offset:%d\n", file->device->net->name, (int)file->offset);
+
+  return GRUB_ERR_NONE;
+}
+
+grub_ssize_t
+grub_efihttp_read (grub_file_t file, char *buf, grub_size_t len)
+{
+  grub_efi_http_response_data_t response_data;
+  grub_efi_http_message_t *response_message;
+  grub_efi_http_token_t *response_token;
+  grub_efi_boot_services_t *b = grub_efi_system_table->boot_services; 
+  grub_efi_status_t status;
+  grub_efi_http_status_code_t http_status;
+  char *ptr = buf;
+  grub_size_t amount, total = 0;
+
+  grub_dprintf ("efihttp", "Enter grub_efihttp_read(), len:%d\n", (int)len);
+
+  /* zero init */
+  grub_memset (&response_data, 0, sizeof (grub_efi_http_response_data_t));
+  response_message = grub_zalloc (sizeof (grub_efi_http_message_t));
+  response_token = grub_zalloc (sizeof (grub_efi_http_token_t));
+
+  /* receive the data */
+  response_message->data.response = &response_data;
+  response_message->header_count = 0; // herader_count will be updated by the HTTP driver on response
+  response_message->headers = NULL; // headers will be populated by the driver on response
+  response_message->body_length = EFIHTTP_RX_BUF_LEN;
+  response_message->body = efihttp_rx_buf;
+  response_token->event = NULL;
+  response_token->status = GRUB_EFI_NOT_READY;
+  response_token->message = response_message;
+  grub_efihttp_response_callback_done = 0;
+  efi_call_5 (b->create_event,
+              GRUB_EFI_EVT_NOTIFY_SIGNAL,
+              GRUB_EFI_TPL_CALLBACK,
+              grub_efihttp_response_callback,
+              NULL,
+              &response_token->event);
+
+  while (len > 0)
+  {
+      grub_dprintf ("efihttp", "file->device->net->offset:%d, file->size:%d\n", (int)file->device->net->offset, (int)file->size);
+      amount = EFIHTTP_RX_BUF_LEN;
+      if (amount > len)
+      {
+          amount = len;
+      }
+
+      response_message->data.response = NULL;
+      if (!response_message->headers)
+      {
+          grub_free(response_message->headers);
+      }
+      response_message->header_count = 0;
+      response_message->headers = NULL;
+      response_message->body_length = amount;
+      grub_memset(efihttp_rx_buf, 0, amount);
+
+      /* accept another response */
+      response_token->status = GRUB_EFI_NOT_READY;
+      grub_efihttp_response_callback_done = 0; //false;
+      grub_dprintf ("efihttp", "Before grub_efihttp->response(), response_message->body_length:%d\n", response_message->body_length);
+      status = efi_call_2 (grub_efihttp->response, grub_efihttp, response_token);
+      if (GRUB_EFI_SUCCESS != status)
+      {
+          grub_dprintf ("efihttp", "Error! status=%d\n", (int)status);
+          return 0;
+      }
+
+      while (!grub_efihttp_response_callback_done)
+      {
+          efi_call_1(grub_efihttp->poll, grub_efihttp);
+      }
+
+      grub_dprintf ("efihttp", "After grub_efihttp->response(), response_message->body_length:%d, response_token.status:%d\n",
+                               response_message->body_length, (int)response_token->status);
+
+      /* check the HTTP status code */
+      http_status = response_token->message->data.response->status_code;
+      grub_dprintf ("efihttp", "http_status=%d\n", (int)http_status);
+
+      len -= response_message->body_length;
+      total += response_message->body_length;
+      file->device->net->offset += response_message->body_length;
+      if (buf)
+      {
+        grub_memcpy (ptr, efihttp_rx_buf, response_message->body_length);
+        ptr += response_message->body_length;
+      }
+      grub_dprintf ("efihttp", "len:%d, total:%d, file->device->net->offset:%d\n",
+                    (int)len, (int)total, (int)file->device->net->offset);
+  }
+  
+  /* release */
+  grub_free (response_message);
+  grub_free (response_token);  
+
+  grub_dprintf ("efihttp", "Leave grub_efihttp_read(), file->offset:%d\n", (int)file->offset);
+
+  return total;
+}
diff --git a/grub-core/net/drivers/efi/efinet.c b/grub-core/net/drivers/efi/efinet.c
index 82a28fb..82b0fdd 100644
--- a/grub-core/net/drivers/efi/efinet.c
+++ b/grub-core/net/drivers/efi/efinet.c
@@ -518,6 +518,7 @@ grub_efinet_create_dhcp_ack_from_device_path (grub_efi_device_path_t *dp, int *u
 	}
       grub_memcpy (bp->boot_file, uri_dp->uri, uri_len);
       grub_memcpy (&bp->your_ip, ipv4->local_ip_address, sizeof (bp->your_ip));
+      grub_memcpy (&bp->subnet_mask, ipv4->subnet_mask, sizeof (bp->subnet_mask));
       grub_memcpy (&bp->server_ip, ipv4->remote_ip_address, sizeof (bp->server_ip));
 
       bp->vendor[0] = GRUB_NET_BOOTP_RFC1048_MAGIC_0;
diff --git a/grub-core/net/net.c b/grub-core/net/net.c
index 2b329ee..90c5b48 100644
--- a/grub-core/net/net.c
+++ b/grub-core/net/net.c
@@ -32,6 +32,9 @@
 #include <grub/loader.h>
 #include <grub/bufio.h>
 #include <grub/kernel.h>
+#ifdef GRUB_MACHINE_EFI
+#include <grub/efi/http.h>
+#endif
 
 GRUB_MOD_LICENSE ("GPLv3+");
 
@@ -1504,7 +1507,12 @@ grub_net_fs_open (struct grub_file *file_out, const char *name)
       return grub_errno;
     }
 
-  err = file->device->net->protocol->open (file, name);
+#ifdef GRUB_MACHINE_EFI
+  if (grub_efihttp)
+      err =  grub_efihttp_open (file, name);
+  else
+#endif
+      err = file->device->net->protocol->open (file, name);
   if (err)
     {
       while (file->device->net->packs.first)
@@ -1543,7 +1551,12 @@ grub_net_fs_close (grub_file_t file)
       grub_netbuff_free (file->device->net->packs.first->nb);
       grub_net_remove_packet (file->device->net->packs.first);
     }
-  file->device->net->protocol->close (file);
+#ifdef GRUB_MACHINE_EFI
+  if (grub_efihttp)
+      grub_efihttp_close (file);
+  else
+#endif
+      file->device->net->protocol->close (file);
   grub_free (file->device->net->name);
   return GRUB_ERR_NONE;
 }
@@ -1774,6 +1787,25 @@ grub_net_seek_real (struct grub_file *file, grub_off_t offset)
 static grub_ssize_t
 grub_net_fs_read (grub_file_t file, char *buf, grub_size_t len)
 {
+#ifdef GRUB_MACHINE_EFI
+  if (grub_efihttp)
+  {
+      /* adjust the offset */
+      if (file->offset > file->device->net->offset)
+      {
+          grub_efihttp_read (file, NULL, (file->offset - file->device->net->offset));
+      }
+      else if (file->offset < file->device->net->offset)
+      {
+          grub_efihttp_close (file);
+          grub_efihttp_open (file, file->device->net->name);
+          if (file->offset)
+              grub_efihttp_read (file, NULL, file->offset);
+      }
+
+      return grub_efihttp_read (file, buf, len);
+  }
+#endif
   if (file->offset != file->device->net->offset)
     {
       grub_err_t err;
diff --git a/include/grub/efi/api.h b/include/grub/efi/api.h
index 99ba068..6a37eb9 100644
--- a/include/grub/efi/api.h
+++ b/include/grub/efi/api.h
@@ -982,6 +982,23 @@ struct grub_efi_bios_device_path
 } GRUB_PACKED;
 typedef struct grub_efi_bios_device_path grub_efi_bios_device_path_t;
 
+/* Service Binding definitions */
+struct grub_efi_service_binding;
+
+typedef grub_efi_status_t
+(*grub_efi_service_binding_create_child) (struct grub_efi_service_binding *this,
+                                          grub_efi_handle_t *child_handle);
+
+typedef grub_efi_status_t
+(*grub_efi_service_binding_destroy_child) (struct grub_efi_service_binding *this,
+                                           grub_efi_handle_t *child_handle);
+
+typedef struct grub_efi_service_binding
+{
+  grub_efi_service_binding_create_child create_child;
+  grub_efi_service_binding_destroy_child destroy_child;
+} grub_efi_service_binding_t;
+
 struct grub_efi_open_protocol_information_entry
 {
   grub_efi_handle_t agent_handle;
diff --git a/include/grub/efi/http.h b/include/grub/efi/http.h
new file mode 100755
index 0000000..3a2725b
--- /dev/null
+++ b/include/grub/efi/http.h
@@ -0,0 +1,221 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2006,2007,2008  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef GRUB_EFI_HTTP_HEADER
+#define GRUB_EFI_HTTP_HEADER	1
+
+#include <grub/symbol.h>
+#include <grub/net.h>
+#include <grub/efi/api.h>
+
+#define GRUB_EFI_HTTP_SERVICE_BINDING_PROTOCOL_GUID \
+  { 0xbdc8e6af, 0xd9bc, 0x4379, \
+      { 0xa7, 0x2a, 0xe0, 0xc4, 0xe7, 0x5d, 0xae, 0x1c } \
+  }
+
+#define GRUB_EFI_HTTP_PROTOCOL_GUID \
+  { 0x7A59B29B, 0x910B, 0x4171, \
+      { 0x82, 0x42, 0xA8, 0x5A, 0x0D, 0xF2, 0x5B, 0x5B } \
+  }
+
+#define EFIHTTP_WAIT_TIME 10000 // 10000ms = 10s
+#define EFIHTTP_RX_BUF_LEN 10240
+
+//******************************************
+// Protocol Interface Structure
+//******************************************
+struct grub_efi_http;
+
+//******************************************
+// EFI_HTTP_VERSION
+//******************************************
+typedef enum {
+  GRUB_EFI_HTTPVERSION10,
+  GRUB_EFI_HTTPVERSION11,
+  GRUB_EFI_HTTPVERSIONUNSUPPORTED
+} grub_efi_http_version_t;
+
+//******************************************
+// EFI_HTTPv4_ACCESS_POINT
+//******************************************
+typedef struct {
+  grub_efi_boolean_t use_default_address;
+  grub_efi_ipv4_address_t local_address;
+  grub_efi_ipv4_address_t local_subnet;
+  grub_efi_uint16_t local_port;
+} grub_efi_httpv4_access_point_t;
+
+//******************************************
+// EFI_HTTPv6_ACCESS_POINT
+//******************************************
+typedef struct {
+  grub_efi_ipv6_address_t local_address;
+  grub_efi_uint16_t local_port;
+} grub_efi_httpv6_access_point_t;
+
+//******************************************
+// EFI_HTTP_CONFIG_DATA
+//******************************************
+typedef struct {
+  grub_efi_http_version_t http_version;
+  grub_efi_uint32_t timeout_millisec;
+  grub_efi_boolean_t local_address_is_ipv6; 
+  union {
+    grub_efi_httpv4_access_point_t *ipv4_node;
+    grub_efi_httpv6_access_point_t *ipv6_node;
+  } access_point;
+} grub_efi_http_config_data_t;
+
+//******************************************
+// EFI_HTTP_METHOD
+//******************************************
+typedef enum {
+  GRUB_EFI_HTTPMETHODGET,
+  GRUB_EFI_HTTPMETHODPOST,
+  GRUB_EFI_HTTPMETHODPATCH,
+  GRUB_EFI_HTTPMETHODOPTIONS,
+  GRUB_EFI_HTTPMETHODCONNECT,
+  GRUB_EFI_HTTPMETHODHEAD,
+  GRUB_EFI_HTTPMETHODPUT,
+  GRUB_EFI_HTTPMETHODDELETE,
+  GRUB_EFI_HTTPMETHODTRACE,
+} grub_efi_http_method_t;
+
+//******************************************
+// EFI_HTTP_REQUEST_DATA
+//******************************************
+typedef struct {
+  grub_efi_http_method_t method;
+  grub_efi_char16_t *url;
+} grub_efi_http_request_data_t;
+
+typedef enum {
+  GRUB_EFI_HTTP_STATUS_UNSUPPORTED_STATUS = 0,
+  GRUB_EFI_HTTP_STATUS_100_CONTINUE,
+  GRUB_EFI_HTTP_STATUS_101_SWITCHING_PROTOCOLS,
+  GRUB_EFI_HTTP_STATUS_200_OK,
+  GRUB_EFI_HTTP_STATUS_201_CREATED,
+  GRUB_EFI_HTTP_STATUS_202_ACCEPTED,
+  GRUB_EFI_HTTP_STATUS_203_NON_AUTHORITATIVE_INFORMATION,
+  GRUB_EFI_HTTP_STATUS_204_NO_CONTENT,
+  GRUB_EFI_HTTP_STATUS_205_RESET_CONTENT,
+  GRUB_EFI_HTTP_STATUS_206_PARTIAL_CONTENT,
+  GRUB_EFI_HTTP_STATUS_300_MULTIPLE_CHIOCES,
+  GRUB_EFI_HTTP_STATUS_301_MOVED_PERMANENTLY,
+  GRUB_EFI_HTTP_STATUS_302_FOUND,
+  GRUB_EFI_HTTP_STATUS_303_SEE_OTHER,
+  GRUB_EFI_HTTP_STATUS_304_NOT_MODIFIED,
+  GRUB_EFI_HTTP_STATUS_305_USE_PROXY,
+  GRUB_EFI_HTTP_STATUS_307_TEMPORARY_REDIRECT,
+  GRUB_EFI_HTTP_STATUS_400_BAD_REQUEST,
+  GRUB_EFI_HTTP_STATUS_401_UNAUTHORIZED,
+  GRUB_EFI_HTTP_STATUS_402_PAYMENT_REQUIRED,
+  GRUB_EFI_HTTP_STATUS_403_FORBIDDEN,
+  GRUB_EFI_HTTP_STATUS_404_NOT_FOUND,
+  GRUB_EFI_HTTP_STATUS_405_METHOD_NOT_ALLOWED,
+  GRUB_EFI_HTTP_STATUS_406_NOT_ACCEPTABLE,
+  GRUB_EFI_HTTP_STATUS_407_PROXY_AUTHENTICATION_REQUIRED,
+  GRUB_EFI_HTTP_STATUS_408_REQUEST_TIME_OUT,
+  GRUB_EFI_HTTP_STATUS_409_CONFLICT,
+  GRUB_EFI_HTTP_STATUS_410_GONE,
+  GRUB_EFI_HTTP_STATUS_411_LENGTH_REQUIRED,
+  GRUB_EFI_HTTP_STATUS_412_PRECONDITION_FAILED,
+  GRUB_EFI_HTTP_STATUS_413_REQUEST_ENTITY_TOO_LARGE,
+  GRUB_EFI_HTTP_STATUS_414_REQUEST_URI_TOO_LARGE,
+  GRUB_EFI_HTTP_STATUS_415_UNSUPPORTED_MEDIA_TYPE,
+  GRUB_EFI_HTTP_STATUS_416_REQUESTED_RANGE_NOT_SATISFIED,
+  GRUB_EFI_HTTP_STATUS_417_EXPECTATION_FAILED,
+  GRUB_EFI_HTTP_STATUS_500_INTERNAL_SERVER_ERROR,
+  GRUB_EFI_HTTP_STATUS_501_NOT_IMPLEMENTED,
+  GRUB_EFI_HTTP_STATUS_502_BAD_GATEWAY,
+  GRUB_EFI_HTTP_STATUS_503_SERVICE_UNAVAILABLE,
+  GRUB_EFI_HTTP_STATUS_504_GATEWAY_TIME_OUT,
+  GRUB_EFI_HTTP_STATUS_505_HTTP_VERSION_NOT_SUPPORTED
+} grub_efi_http_status_code_t;
+
+//******************************************
+// EFI_HTTP_RESPONSE_DATA
+//******************************************
+typedef struct {
+  grub_efi_http_status_code_t status_code;
+} grub_efi_http_response_data_t;
+
+//******************************************
+// EFI_HTTP_HEADER
+//******************************************
+typedef struct {
+  grub_efi_char8_t *field_name;
+  grub_efi_char8_t *field_value;
+} grub_efi_http_header_t;
+
+//******************************************
+// EFI_HTTP_MESSAGE
+//******************************************
+typedef struct {
+  union {
+    grub_efi_http_request_data_t *request;
+    grub_efi_http_response_data_t *response;
+  } data;
+  grub_efi_uint32_t header_count;
+  grub_efi_http_header_t *headers;
+  grub_efi_uint32_t body_length;  
+  void *body;
+} grub_efi_http_message_t;
+
+//******************************************
+// EFI_HTTP_TOKEN
+//******************************************
+typedef struct {
+  grub_efi_event_t event;
+  grub_efi_status_t status;
+  grub_efi_http_message_t *message;
+} grub_efi_http_token_t;
+
+struct grub_efi_http {
+  grub_efi_status_t
+  (*get_mode_data) (struct grub_efi_http *this,
+                    grub_efi_http_config_data_t *http_config_data);
+  
+  grub_efi_status_t
+  (*configure) (struct grub_efi_http *this,
+                grub_efi_http_config_data_t *http_config_data);
+  
+  grub_efi_status_t
+  (*request) (struct grub_efi_http *this,
+              grub_efi_http_token_t *token);
+  
+  grub_efi_status_t
+  (*cancel) (struct grub_efi_http *this,
+             grub_efi_http_token_t *token);
+ 
+  grub_efi_status_t
+  (*response) (struct grub_efi_http *this,
+               grub_efi_http_token_t *token);
+
+  grub_efi_status_t
+  (*poll) (struct grub_efi_http *this);
+};
+typedef struct grub_efi_http grub_efi_http_t;
+
+extern grub_efi_http_t *grub_efihttp;
+grub_err_t grub_efihttp_configure (struct grub_net_card *card, const struct grub_net_bootp_packet *bp);
+grub_err_t grub_efihttp_open (grub_file_t file, const char *filename);
+grub_err_t grub_efihttp_close (grub_file_t file);
+grub_ssize_t grub_efihttp_read (grub_file_t file, char *buf, grub_size_t len);
+
+#endif /* !GRUB_EFI_HTTP_HEADER */
diff --git a/include/grub/err.h b/include/grub/err.h
index 1590c68..eb0fc0e 100644
--- a/include/grub/err.h
+++ b/include/grub/err.h
@@ -71,7 +71,8 @@ typedef enum
     GRUB_ERR_NET_PACKET_TOO_BIG,
     GRUB_ERR_NET_NO_DOMAIN,
     GRUB_ERR_EOF,
-    GRUB_ERR_BAD_SIGNATURE
+    GRUB_ERR_BAD_SIGNATURE,
+    GRUB_ERR_EFI
   }
 grub_err_t;
 
diff --git a/include/grub/net.h b/include/grub/net.h
index 67c801e..d175b8f 100644
--- a/include/grub/net.h
+++ b/include/grub/net.h
@@ -435,6 +435,7 @@ struct grub_net_bootp_packet
   grub_uint16_t flags;
   grub_uint32_t	client_ip;
   grub_uint32_t your_ip;
+  grub_uint32_t subnet_mask;
   grub_uint32_t	server_ip;
   grub_uint32_t	gateway_ip;
   grub_net_bootp_mac_addr_t mac_addr;
-- 
2.7.4

[RFC 2/2] net: workaround to bypass corruption of the efihttp function pointer

[Ken Lin]

It's a workaround to bypass the corruption of the Request function pointer.
The function pointer crashes after the grub menu is shown. Here is the log:

net/drivers/efi/efihttp.c:175: b->create_event()
net/drivers/efi/efihttp.c:188: grub_efihttp:0x3edc4020,
grub_efihttp->request:0xafafafafafafafaf
net/drivers/efi/efihttp.c:191: Before grub_efihttp->request(),
url:http://192.168.111.1/boot/grub/x86_64-efi/part_gpt.mod
!!!! X64 Exception Type - 0D(#GP - General Protection)  CPU Apic ID - 00000000 !!!!
RIP  - 000000003E74C59B, CS  - 0000000000000038, RFLAGS - 0000000000210202
ExceptionData - 0000000000000000
RAX  - 000000003BD24280, RCX - 000000003EDC4020, RDX - 000000003BC5B500
RBX  - 000000003BC5B600, RSP - 000000003FAFA6D0, RBP - 000000003BC5B4A0
RSI  - 000000003EDC4020, RDI - AFAFAFAFAFAFAFAF
R8   - 000000003BC5B340, R9  - 000000003BC5B1A0, R10 - 0000000000000013
R11  - 0000000000000058, R12 - 000000003E74C594, R13 - 000000003E754D13
R14  - 000000003BC5B460, R15 - 0000000000000000
DS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030
GS   - 0000000000000030, SS  - 0000000000000030
CR0  - 0000000080000033, CR2 - 0000000000000000, CR3 - 000000003FA99000
CR4  - 0000000000000668, CR8 - 0000000000000000
DR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3  - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000003FA87698 0000000000000047, LDTR - 0000000000000000
IDTR - 000000003F4FA018 0000000000000FFF,   TR - 0000000000000000
FXSAVE_STATE - 000000003FAFA330
!!!! Find PE image (No PDB)  (ImageBase=000000003E74C000, EntryPoint=000000003E74C400) !!!!

HTTP Boot can’t work without this workaround.
The root cause is unknown. It is much appreciated for any kind of suggestions
for debugging this.

Signed-off-by: Ken Lin <ken.lin@hpe.com>
Signed-off-by: Clay Chang <clayc@hpe.com>
Reviewed-by: Keng-Yu Lin <kengyu@hpe.com>
---
 grub-core/net/net.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/grub-core/net/net.c b/grub-core/net/net.c
index 90c5b48..e258109 100644
--- a/grub-core/net/net.c
+++ b/grub-core/net/net.c
@@ -1661,6 +1661,9 @@ static void
 grub_net_poll_cards_idle_real (void)
 {
   struct grub_net_card *card;
+#ifdef GRUB_MACHINE_EFI
+  if (grub_efihttp) return;
+#endif
   FOR_NET_CARDS (card)
   {
     grub_uint64_t ctime = grub_get_time_ms ();
-- 
2.7.4

Re: [RFC 0/2] UEFI-based HTTP Boot

[Andrei Borzenkov]

On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
> This RFC patchset is stacked on the previous HTTP boot patchset:
> https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
> It re-uses some code from it, e.g. the DCHPACK
> with vendor_class_identifier=HTTPClient.
>
> Instead of implementing HTTP and HTTPS boot totally from software,
> UEFI firmware already defines APIs for HTTP(s).
> Please check UEFI spec. 2.5 and plus for the detail:
>
> 28.6 EFI HTTP Protocols
>

Without reviewing patches themselves - we usually prefer to rely on
firmware as little as possible. We already have HTTP support, so what
is missing in grub that requires what amounts to full
re-implementation? Cannot we rather fix our HTTP support instead? This
will automatically benefit all supported platforms, of which EFI is
just one.

> Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
> the HTTP(s) APIs are not available. In the case,
> Grub can fall back to the software-based implementation.
> In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
> is the code to query if the HTTP Protocol is supported by the UEFI firmware.
>
> This patchset was tested on QEMU+OVMF and it works flawlessly.
>
> The main goals of this RFC is to ask for opinions and suggestion to make
> the first patch modularized as much as possible.
> In the second patch, there is some codes related TCP re-transmission
> that need to pass by for the HTTP Boot to work.
>
> More details are described in the logs of each patch.
>
>
> Ken Lin (2):
>   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
>   net: workaround to bypass corruption of the efihttp function pointer
>
>  grub-core/Makefile.core.def         |   1 +
>  grub-core/net/bootp.c               |   6 +
>  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
>  grub-core/net/drivers/efi/efinet.c  |   1 +
>  grub-core/net/net.c                 |  39 +++-
>  include/grub/efi/api.h              |  17 ++
>  include/grub/efi/http.h             | 221 +++++++++++++++++++++
>  include/grub/err.h                  |   3 +-
>  include/grub/net.h                  |   1 +
>  9 files changed, 672 insertions(+), 3 deletions(-)
>  create mode 100755 grub-core/net/drivers/efi/efihttp.c
>  create mode 100755 include/grub/efi/http.h
>
> --
> 2.7.4
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Chang, Clay (HPS OE-Linux TDC)]

Hi Andrei,

We didn’t “re-implement” the existing http feature in grub2. What we do is putting a piece of code to check the availability of the http protocol in the firmware. If available, we use the http protocol offered by the firmware. If not, we follow the original http path.

Does this make sense to you?

Regards,
Clay

On 1/20/17, 10:50 PM, "Andrei Borzenkov" <arvidjaar@gmail.com> wrote:

    On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
    > This RFC patchset is stacked on the previous HTTP boot patchset:
    > https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
    > It re-uses some code from it, e.g. the DCHPACK
    > with vendor_class_identifier=HTTPClient.
    >
    > Instead of implementing HTTP and HTTPS boot totally from software,
    > UEFI firmware already defines APIs for HTTP(s).
    > Please check UEFI spec. 2.5 and plus for the detail:
    >
    > 28.6 EFI HTTP Protocols
    >
    
    Without reviewing patches themselves - we usually prefer to rely on
    firmware as little as possible. We already have HTTP support, so what
    is missing in grub that requires what amounts to full
    re-implementation? Cannot we rather fix our HTTP support instead? This
    will automatically benefit all supported platforms, of which EFI is
    just one.
    
    > Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
    > the HTTP(s) APIs are not available. In the case,
    > Grub can fall back to the software-based implementation.
    > In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
    > is the code to query if the HTTP Protocol is supported by the UEFI firmware.
    >
    > This patchset was tested on QEMU+OVMF and it works flawlessly.
    >
    > The main goals of this RFC is to ask for opinions and suggestion to make
    > the first patch modularized as much as possible.
    > In the second patch, there is some codes related TCP re-transmission
    > that need to pass by for the HTTP Boot to work.
    >
    > More details are described in the logs of each patch.
    >
    >
    > Ken Lin (2):
    >   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
    >   net: workaround to bypass corruption of the efihttp function pointer
    >
    >  grub-core/Makefile.core.def         |   1 +
    >  grub-core/net/bootp.c               |   6 +
    >  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
    >  grub-core/net/drivers/efi/efinet.c  |   1 +
    >  grub-core/net/net.c                 |  39 +++-
    >  include/grub/efi/api.h              |  17 ++
    >  include/grub/efi/http.h             | 221 +++++++++++++++++++++
    >  include/grub/err.h                  |   3 +-
    >  include/grub/net.h                  |   1 +
    >  9 files changed, 672 insertions(+), 3 deletions(-)
    >  create mode 100755 grub-core/net/drivers/efi/efihttp.c
    >  create mode 100755 include/grub/efi/http.h
    >
    > --
    > 2.7.4
    >
    >
    > _______________________________________________
    > Grub-devel mailing list
    > Grub-devel@gnu.org
    > https://lists.gnu.org/mailman/listinfo/grub-devel
    

Re: [RFC 0/2] UEFI-based HTTP Boot

[Lin, Keng-Yu]

The main reason is HTTPS. We've been looking at it for a while. Recently a few patches of HTTPS Boot were already merge in the EDK2 main tree (before this, it was in the EDK2-staging tree).

Please see: https://github.com/tianocore/edk2/search?q=tls&type=Commits

For the software-based implementation, it will be a lot of works to support this, e.g. interfaces to import or remove certificates, grub has to depend on GnuTLS or OpenSSL libraries. This means that grub has to release a new version once GnuTLS or OpenSSL bump their ABIs, etc.

The ideal picture from us is as below:
 * For the UEFI firmware v2.4 and older: use the software-based HTTP boot. The limitation is this does not include HTTPS.

 * For the UEFI firmware v2.5 and newer: use the UEFI-based HTTP(S) boot (this RFC patchset). Both HTTP and HTTPS are supported.

I hope this help you and the other maintainers understand the background of the HTTP(s) Boot patches from us.
________________________________________
From: Andrei Borzenkov <arvidjaar@gmail.com>
Sent: Friday, January 20, 2017 10:50:56 PM
To: The development of GNU GRUB
Cc: Lin, Ken (HPS OE-Linux TDC); Chang, Clay (HPS OE-Linux TDC); Lin, Keng-Yu; Knippers, Linda; Ruan, Michael (HPS OE-Linux/VMware TDC)
Subject: Re: [RFC 0/2] UEFI-based HTTP Boot

On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
> This RFC patchset is stacked on the previous HTTP boot patchset:
> https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
> It re-uses some code from it, e.g. the DCHPACK
> with vendor_class_identifier=HTTPClient.
>
> Instead of implementing HTTP and HTTPS boot totally from software,
> UEFI firmware already defines APIs for HTTP(s).
> Please check UEFI spec. 2.5 and plus for the detail:
>
> 28.6 EFI HTTP Protocols
>

Without reviewing patches themselves - we usually prefer to rely on
firmware as little as possible. We already have HTTP support, so what
is missing in grub that requires what amounts to full
re-implementation? Cannot we rather fix our HTTP support instead? This
will automatically benefit all supported platforms, of which EFI is
just one.

> Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
> the HTTP(s) APIs are not available. In the case,
> Grub can fall back to the software-based implementation.
> In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
> is the code to query if the HTTP Protocol is supported by the UEFI firmware.
>
> This patchset was tested on QEMU+OVMF and it works flawlessly.
>
> The main goals of this RFC is to ask for opinions and suggestion to make
> the first patch modularized as much as possible.
> In the second patch, there is some codes related TCP re-transmission
> that need to pass by for the HTTP Boot to work.
>
> More details are described in the logs of each patch.
>
>
> Ken Lin (2):
>   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
>   net: workaround to bypass corruption of the efihttp function pointer
>
>  grub-core/Makefile.core.def         |   1 +
>  grub-core/net/bootp.c               |   6 +
>  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
>  grub-core/net/drivers/efi/efinet.c  |   1 +
>  grub-core/net/net.c                 |  39 +++-
>  include/grub/efi/api.h              |  17 ++
>  include/grub/efi/http.h             | 221 +++++++++++++++++++++
>  include/grub/err.h                  |   3 +-
>  include/grub/net.h                  |   1 +
>  9 files changed, 672 insertions(+), 3 deletions(-)
>  create mode 100755 grub-core/net/drivers/efi/efihttp.c
>  create mode 100755 include/grub/efi/http.h
>
> --
> 2.7.4
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Andrei Borzenkov]

21.01.2017 06:37, Lin, Keng-Yu пишет:
> 

> The main reason is HTTPS. We've been looking at it for a while.
Recently a few patches of HTTPS Boot were already merge in the EDK2 main
tree (before this, it was in the EDK2-staging tree).
> 
> Please see:
> https://github.com/tianocore/edk2/search?q=tls&type=Commits
> 
> For the software-based implementation, it will be a lot of works to
support this, e.g. interfaces to import or remove certificates, grub has

I do not see how your patch solves this either. UEFI provides no
guarantee that necessary certificates are available in non-volatile
store, so either your patch relies on some third-party software to do it
on every boot or on implementation specific behavior that stores
information permanently. Having hooks in grub to manage this information
would free it from these assumptions and then it does not matter whether
grub will manage it via EFI interface or using own internal implementation.

to depend on GnuTLS or OpenSSL libraries. This means that grub has to
release a new version once GnuTLS or OpenSSL bump their ABIs, etc.

GRUB already reuses libgcrypto. It may be possible to reuse gnutls in
the same way. Sure, it is a lot of work, but then this will also be
available on every platform.

> 
> The ideal picture from us is as below: * For the UEFI firmware v2.4
> and older: use the software-based HTTP
boot. The limitation is this does not include HTTPS.
> 
> * For the UEFI firmware v2.5 and newer: use the UEFI-based HTTP(S)
boot (this RFC patchset). Both HTTP and HTTPS are supported.

This hardly can be used in this form. First, grub is using SNP and
relies on being the only consumer of network device. We already had
quite serious problems when firmware raced with grub for card access.
Your patch spawns off additional protocol(s) that compete with grub.
This cannot work reliably unless we switch grub to MNP.

Second your patch completely bypasses grub event loop and busy waits for
transfer completion. It should hook into general grub net polling
framework.

> 
> I hope this help you and the other maintainers understand the
background of the HTTP(s) Boot patches from us.
> ________________________________________
> From: Andrei Borzenkov <arvidjaar@gmail.com>
> Sent: Friday, January 20, 2017 10:50:56 PM
> To: The development of GNU GRUB
> Cc: Lin, Ken (HPS OE-Linux TDC); Chang, Clay (HPS OE-Linux TDC); Lin, Keng-Yu; Knippers, Linda; Ruan, Michael (HPS OE-Linux/VMware TDC)
> Subject: Re: [RFC 0/2] UEFI-based HTTP Boot
> 
> On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
>> This RFC patchset is stacked on the previous HTTP boot patchset:
>> https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
>> It re-uses some code from it, e.g. the DCHPACK
>> with vendor_class_identifier=HTTPClient.
>>
>> Instead of implementing HTTP and HTTPS boot totally from software,
>> UEFI firmware already defines APIs for HTTP(s).
>> Please check UEFI spec. 2.5 and plus for the detail:
>>
>> 28.6 EFI HTTP Protocols
>>
> 
> Without reviewing patches themselves - we usually prefer to rely on
> firmware as little as possible. We already have HTTP support, so what
> is missing in grub that requires what amounts to full
> re-implementation? Cannot we rather fix our HTTP support instead? This
> will automatically benefit all supported platforms, of which EFI is
> just one.
> 
>> Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
>> the HTTP(s) APIs are not available. In the case,
>> Grub can fall back to the software-based implementation.
>> In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
>> is the code to query if the HTTP Protocol is supported by the UEFI firmware.
>>
>> This patchset was tested on QEMU+OVMF and it works flawlessly.
>>
>> The main goals of this RFC is to ask for opinions and suggestion to make
>> the first patch modularized as much as possible.
>> In the second patch, there is some codes related TCP re-transmission
>> that need to pass by for the HTTP Boot to work.
>>
>> More details are described in the logs of each patch.
>>
>>
>> Ken Lin (2):
>>   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
>>   net: workaround to bypass corruption of the efihttp function pointer
>>
>>  grub-core/Makefile.core.def         |   1 +
>>  grub-core/net/bootp.c               |   6 +
>>  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
>>  grub-core/net/drivers/efi/efinet.c  |   1 +
>>  grub-core/net/net.c                 |  39 +++-
>>  include/grub/efi/api.h              |  17 ++
>>  include/grub/efi/http.h             | 221 +++++++++++++++++++++
>>  include/grub/err.h                  |   3 +-
>>  include/grub/net.h                  |   1 +
>>  9 files changed, 672 insertions(+), 3 deletions(-)
>>  create mode 100755 grub-core/net/drivers/efi/efihttp.c
>>  create mode 100755 include/grub/efi/http.h
>>
>> --
>> 2.7.4
>>
>>
>> _______________________________________________
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Michael Chang]

On Fri, Jan 20, 2017 at 05:50:56PM +0300, Andrei Borzenkov wrote:
> On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
> > This RFC patchset is stacked on the previous HTTP boot patchset:
> > https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
> > It re-uses some code from it, e.g. the DCHPACK
> > with vendor_class_identifier=HTTPClient.
> >
> > Instead of implementing HTTP and HTTPS boot totally from software,
> > UEFI firmware already defines APIs for HTTP(s).
> > Please check UEFI spec. 2.5 and plus for the detail:
> >
> > 28.6 EFI HTTP Protocols
> >
> 
> Without reviewing patches themselves - we usually prefer to rely on
> firmware as little as possible. We already have HTTP support, so what
> is missing in grub that requires what amounts to full
> re-implementation? Cannot we rather fix our HTTP support instead? This
> will automatically benefit all supported platforms, of which EFI is
> just one.

Nothing wrong in providing firmware based approach in addition to grub's native
stack of getting the similar things done. And there's no prioity for what has
to be implemented first imho. Occasionally people would prefer firmware based
stack because they need new features it provides that haven't been worked out
in grub, such as the https or fibre networks, or simply to avoid bug from grub,
like the SNP woes among some UEFI box.

Thanks,
Michael

> 
> > Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
> > the HTTP(s) APIs are not available. In the case,
> > Grub can fall back to the software-based implementation.
> > In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
> > is the code to query if the HTTP Protocol is supported by the UEFI firmware.
> >
> > This patchset was tested on QEMU+OVMF and it works flawlessly.
> >
> > The main goals of this RFC is to ask for opinions and suggestion to make
> > the first patch modularized as much as possible.
> > In the second patch, there is some codes related TCP re-transmission
> > that need to pass by for the HTTP Boot to work.
> >
> > More details are described in the logs of each patch.
> >
> >
> > Ken Lin (2):
> >   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
> >   net: workaround to bypass corruption of the efihttp function pointer
> >
> >  grub-core/Makefile.core.def         |   1 +
> >  grub-core/net/bootp.c               |   6 +
> >  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
> >  grub-core/net/drivers/efi/efinet.c  |   1 +
> >  grub-core/net/net.c                 |  39 +++-
> >  include/grub/efi/api.h              |  17 ++
> >  include/grub/efi/http.h             | 221 +++++++++++++++++++++
> >  include/grub/err.h                  |   3 +-
> >  include/grub/net.h                  |   1 +
> >  9 files changed, 672 insertions(+), 3 deletions(-)
> >  create mode 100755 grub-core/net/drivers/efi/efihttp.c
> >  create mode 100755 include/grub/efi/http.h
> >
> > --
> > 2.7.4
> >
> >
> > _______________________________________________
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Andrei Borzenkov]

On Wed, Jan 25, 2017 at 11:09 AM, Michael Chang <mchang@suse.com> wrote:
> On Fri, Jan 20, 2017 at 05:50:56PM +0300, Andrei Borzenkov wrote:
>> On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
>> > This RFC patchset is stacked on the previous HTTP boot patchset:
>> > https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
>> > It re-uses some code from it, e.g. the DCHPACK
>> > with vendor_class_identifier=HTTPClient.
>> >
>> > Instead of implementing HTTP and HTTPS boot totally from software,
>> > UEFI firmware already defines APIs for HTTP(s).
>> > Please check UEFI spec. 2.5 and plus for the detail:
>> >
>> > 28.6 EFI HTTP Protocols
>> >
>>
>> Without reviewing patches themselves - we usually prefer to rely on
>> firmware as little as possible. We already have HTTP support, so what
>> is missing in grub that requires what amounts to full
>> re-implementation? Cannot we rather fix our HTTP support instead? This
>> will automatically benefit all supported platforms, of which EFI is
>> just one.
>
> Nothing wrong in providing firmware based approach in addition to grub's native
> stack of getting the similar things done.

You cannot both shut off all layered protocols on physical adapter and
makes use of these layered protocols. This will need to implement
alternative networking stack first.

> And there's no prioity for what has
> to be implemented first imho. Occasionally people would prefer firmware based
> stack because they need new features it provides that haven't been worked out
> in grub, such as the https or fibre networks, or simply to avoid bug from grub,
> like the SNP woes among some UEFI box.
>
> Thanks,
> Michael
>
>>
>> > Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
>> > the HTTP(s) APIs are not available. In the case,
>> > Grub can fall back to the software-based implementation.
>> > In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
>> > is the code to query if the HTTP Protocol is supported by the UEFI firmware.
>> >
>> > This patchset was tested on QEMU+OVMF and it works flawlessly.
>> >
>> > The main goals of this RFC is to ask for opinions and suggestion to make
>> > the first patch modularized as much as possible.
>> > In the second patch, there is some codes related TCP re-transmission
>> > that need to pass by for the HTTP Boot to work.
>> >
>> > More details are described in the logs of each patch.
>> >
>> >
>> > Ken Lin (2):
>> >   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
>> >   net: workaround to bypass corruption of the efihttp function pointer
>> >
>> >  grub-core/Makefile.core.def         |   1 +
>> >  grub-core/net/bootp.c               |   6 +
>> >  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
>> >  grub-core/net/drivers/efi/efinet.c  |   1 +
>> >  grub-core/net/net.c                 |  39 +++-
>> >  include/grub/efi/api.h              |  17 ++
>> >  include/grub/efi/http.h             | 221 +++++++++++++++++++++
>> >  include/grub/err.h                  |   3 +-
>> >  include/grub/net.h                  |   1 +
>> >  9 files changed, 672 insertions(+), 3 deletions(-)
>> >  create mode 100755 grub-core/net/drivers/efi/efihttp.c
>> >  create mode 100755 include/grub/efi/http.h
>> >
>> > --
>> > 2.7.4
>> >
>> >
>> > _______________________________________________
>> > Grub-devel mailing list
>> > Grub-devel@gnu.org
>> > https://lists.gnu.org/mailman/listinfo/grub-devel
>>
>> _______________________________________________
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> https://lists.gnu.org/mailman/listinfo/grub-devel
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Michael Chang]

On Wed, Jan 25, 2017 at 11:19:19AM +0300, Andrei Borzenkov wrote:
> On Wed, Jan 25, 2017 at 11:09 AM, Michael Chang <mchang@suse.com> wrote:
> > On Fri, Jan 20, 2017 at 05:50:56PM +0300, Andrei Borzenkov wrote:
> >> On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
> >> > This RFC patchset is stacked on the previous HTTP boot patchset:
> >> > https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
> >> > It re-uses some code from it, e.g. the DCHPACK
> >> > with vendor_class_identifier=HTTPClient.
> >> >
> >> > Instead of implementing HTTP and HTTPS boot totally from software,
> >> > UEFI firmware already defines APIs for HTTP(s).
> >> > Please check UEFI spec. 2.5 and plus for the detail:
> >> >
> >> > 28.6 EFI HTTP Protocols
> >> >
> >>
> >> Without reviewing patches themselves - we usually prefer to rely on
> >> firmware as little as possible. We already have HTTP support, so what
> >> is missing in grub that requires what amounts to full
> >> re-implementation? Cannot we rather fix our HTTP support instead? This
> >> will automatically benefit all supported platforms, of which EFI is
> >> just one.
> >
> > Nothing wrong in providing firmware based approach in addition to grub's native
> > stack of getting the similar things done.
> 
> You cannot both shut off all layered protocols on physical adapter and
> makes use of these layered protocols. This will need to implement
> alternative networking stack first.

They don't have to be runtime switch to operate at the same time. Make them
distinct modules, and provid a swith for controlling the image being built. We
already have --native switch in grub2-install to incorporate native disk modules
rather than firmware's.

Thanks,
Michael

> 
> > And there's no prioity for what has
> > to be implemented first imho. Occasionally people would prefer firmware based
> > stack because they need new features it provides that haven't been worked out
> > in grub, such as the https or fibre networks, or simply to avoid bug from grub,
> > like the SNP woes among some UEFI box.
> >
> > Thanks,
> > Michael
> >
> >>
> >> > Then why two implementations? For older UEFI firmwares (UEFI 2.4 and older),
> >> > the HTTP(s) APIs are not available. In the case,
> >> > Grub can fall back to the software-based implementation.
> >> > In the first patch of this patchset, grub-core/net/drivers/efi/efihttp.c:76 to 81
> >> > is the code to query if the HTTP Protocol is supported by the UEFI firmware.
> >> >
> >> > This patchset was tested on QEMU+OVMF and it works flawlessly.
> >> >
> >> > The main goals of this RFC is to ask for opinions and suggestion to make
> >> > the first patch modularized as much as possible.
> >> > In the second patch, there is some codes related TCP re-transmission
> >> > that need to pass by for the HTTP Boot to work.
> >> >
> >> > More details are described in the logs of each patch.
> >> >
> >> >
> >> > Ken Lin (2):
> >> >   net: add efihttp to do HTTP(S) Boot by UEFI HTTP Protocol
> >> >   net: workaround to bypass corruption of the efihttp function pointer
> >> >
> >> >  grub-core/Makefile.core.def         |   1 +
> >> >  grub-core/net/bootp.c               |   6 +
> >> >  grub-core/net/drivers/efi/efihttp.c | 386 ++++++++++++++++++++++++++++++++++++
> >> >  grub-core/net/drivers/efi/efinet.c  |   1 +
> >> >  grub-core/net/net.c                 |  39 +++-
> >> >  include/grub/efi/api.h              |  17 ++
> >> >  include/grub/efi/http.h             | 221 +++++++++++++++++++++
> >> >  include/grub/err.h                  |   3 +-
> >> >  include/grub/net.h                  |   1 +
> >> >  9 files changed, 672 insertions(+), 3 deletions(-)
> >> >  create mode 100755 grub-core/net/drivers/efi/efihttp.c
> >> >  create mode 100755 include/grub/efi/http.h
> >> >
> >> > --
> >> > 2.7.4
> >> >
> >> >
> >> > _______________________________________________
> >> > Grub-devel mailing list
> >> > Grub-devel@gnu.org
> >> > https://lists.gnu.org/mailman/listinfo/grub-devel
> >>
> >> _______________________________________________
> >> Grub-devel mailing list
> >> Grub-devel@gnu.org
> >> https://lists.gnu.org/mailman/listinfo/grub-devel
> >
> > _______________________________________________
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC 0/2] UEFI-based HTTP Boot

[Andrei Borzenkov]

On Wed, Jan 25, 2017 at 11:49 AM, Michael Chang <mchang@suse.com> wrote:
> On Wed, Jan 25, 2017 at 11:19:19AM +0300, Andrei Borzenkov wrote:
>> On Wed, Jan 25, 2017 at 11:09 AM, Michael Chang <mchang@suse.com> wrote:
>> > On Fri, Jan 20, 2017 at 05:50:56PM +0300, Andrei Borzenkov wrote:
>> >> On Fri, Jan 20, 2017 at 4:13 AM, Ken Lin <ken.lin@hpe.com> wrote:
>> >> > This RFC patchset is stacked on the previous HTTP boot patchset:
>> >> > https://lists.gnu.org/archive/html/grub-devel/2016-12/msg00088.html
>> >> > It re-uses some code from it, e.g. the DCHPACK
>> >> > with vendor_class_identifier=HTTPClient.
>> >> >
>> >> > Instead of implementing HTTP and HTTPS boot totally from software,
>> >> > UEFI firmware already defines APIs for HTTP(s).
>> >> > Please check UEFI spec. 2.5 and plus for the detail:
>> >> >
>> >> > 28.6 EFI HTTP Protocols
>> >> >
>> >>
>> >> Without reviewing patches themselves - we usually prefer to rely on
>> >> firmware as little as possible. We already have HTTP support, so what
>> >> is missing in grub that requires what amounts to full
>> >> re-implementation? Cannot we rather fix our HTTP support instead? This
>> >> will automatically benefit all supported platforms, of which EFI is
>> >> just one.
>> >
>> > Nothing wrong in providing firmware based approach in addition to grub's native
>> > stack of getting the similar things done.
>>
>> You cannot both shut off all layered protocols on physical adapter and
>> makes use of these layered protocols. This will need to implement
>> alternative networking stack first.
>
> They don't have to be runtime switch to operate at the same time. Make them
> distinct modules, and provid a swith for controlling the image being built. We
> already have --native switch in grub2-install to incorporate native disk modules
> rather than firmware's.
>

These patches hook into network stack of grub. So they either must
comply with this stack (which means - provide alternative network
driver for reasons I mentioned) or they must pretend "efihttp" is
disk.