[Qemu-devel] [SeaBIOS] [PATCH V3 0/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Qemu-devel] [SeaBIOS] [PATCH V3 0/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Marcel Apfelbaum]

v2 -> v3:
 - Addressed Michael S. Tsirkin's comments:
   - I/O and Prefetchable Memory are optional. Do not allocate ranges
     if they are not implemented (2/2).
 - Note that 2/2 patch can be seen as a separate fix. However, it
   is related to ranges reservation.

v1 -> v2:
 - Thanks Gerd Hoffmann for the review.
 - Addressed Michael S. Tsirkin's comments:
   - Limit capabilities query to 256 iterations, to make sure we
     don't get into an infinite loop with a broken device.


If a pci-2-pci bridge supports hot-plug functionality but there are no devices
connected to it, reserve IO/mem in order to be able to attach devices
later. Do not waste space, use minimum allowed.

Marcel Apfelbaum (2):
  hw/pci: reserve IO and mem for pci-2-pci bridges with no devices
    attached
  hw/pci: check if pci2pci bridges implement optional limit registers

 src/fw/pciinit.c | 12 +++++-----
 src/hw/pci.c     | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/hw/pci.h     | 10 +++++++++
 3 files changed, 82 insertions(+), 7 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH V3 1/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Marcel Apfelbaum]

If a pci-2-pci bridge supports hot-plug functionality but there are no devices
connected to it, reserve IO/mem in order to be able to attach devices
later. Do not waste space, use minimum allowed.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
---
 src/fw/pciinit.c |  3 +++
 src/hw/pci.c     | 19 +++++++++++++++++++
 src/hw/pci.h     |  1 +
 3 files changed, 23 insertions(+)

diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
index 64f1d41..9b5d7ad 100644
--- a/src/fw/pciinit.c
+++ b/src/fw/pciinit.c
@@ -677,12 +677,15 @@ static int pci_bios_check_devices(struct pci_bus *busses)
             continue;
         struct pci_bus *parent = &busses[pci_bdf_to_bus(s->bus_dev->bdf)];
         int type;
+        u8 shpc_cap = pci_find_capability(s->bus_dev, PCI_CAP_ID_SHPC);
         for (type = 0; type < PCI_REGION_TYPE_COUNT; type++) {
             u64 align = (type == PCI_REGION_TYPE_IO) ?
                 PCI_BRIDGE_IO_MIN : PCI_BRIDGE_MEM_MIN;
             if (pci_region_align(&s->r[type]) > align)
                  align = pci_region_align(&s->r[type]);
             u64 sum = pci_region_sum(&s->r[type]);
+            if (!sum && shpc_cap)
+                sum = align; /* reserve min size for hot-plug */
             u64 size = ALIGN(sum, align);
             int is64 = pci_bios_bridge_region_is64(&s->r[type],
                                             s->bus_dev, type);
diff --git a/src/hw/pci.c b/src/hw/pci.c
index caf9265..055353d 100644
--- a/src/hw/pci.c
+++ b/src/hw/pci.c
@@ -225,6 +225,25 @@ pci_find_init_device(const struct pci_device_id *ids, void *arg)
     return NULL;
 }
 
+u8 pci_find_capability(struct pci_device *pci, u8 cap_id)
+{
+    int i;
+    u8 cap;
+    u16 status = pci_config_readw(pci->bdf, PCI_STATUS);
+
+    if (!(status & PCI_STATUS_CAP_LIST))
+        return 0;
+
+    for (i = 0, cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
+         (i <= 0xff) && cap;
+         i++, cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
+        if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
+            return cap;
+
+    return 0;
+}
+
+
 void
 pci_reboot(void)
 {
diff --git a/src/hw/pci.h b/src/hw/pci.h
index 167a027..e828225 100644
--- a/src/hw/pci.h
+++ b/src/hw/pci.h
@@ -116,6 +116,7 @@ int pci_init_device(const struct pci_device_id *ids
                     , struct pci_device *pci, void *arg);
 struct pci_device *pci_find_init_device(const struct pci_device_id *ids
                                         , void *arg);
+u8 pci_find_capability(struct pci_device *pci, u8 cap_id);
 void pci_reboot(void);
 
 #endif
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH V3 1/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Michael S. Tsirkin]

On Thu, Apr 10, 2014 at 04:29:40PM +0300, Marcel Apfelbaum wrote:
> If a pci-2-pci bridge supports hot-plug functionality but there are no devices
> connected to it, reserve IO/mem in order to be able to attach devices
> later. Do not waste space, use minimum allowed.
> 
> Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>

Looks good to me
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>

but let's apply with patch 2 (after it's fixed): otherwise
there's no way for hypervisors to avoid wasting resources
unnecessaily on bridges.

> ---
>  src/fw/pciinit.c |  3 +++
>  src/hw/pci.c     | 19 +++++++++++++++++++
>  src/hw/pci.h     |  1 +
>  3 files changed, 23 insertions(+)
> 
> diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
> index 64f1d41..9b5d7ad 100644
> --- a/src/fw/pciinit.c
> +++ b/src/fw/pciinit.c
> @@ -677,12 +677,15 @@ static int pci_bios_check_devices(struct pci_bus *busses)
>              continue;
>          struct pci_bus *parent = &busses[pci_bdf_to_bus(s->bus_dev->bdf)];
>          int type;
> +        u8 shpc_cap = pci_find_capability(s->bus_dev, PCI_CAP_ID_SHPC);
>          for (type = 0; type < PCI_REGION_TYPE_COUNT; type++) {
>              u64 align = (type == PCI_REGION_TYPE_IO) ?
>                  PCI_BRIDGE_IO_MIN : PCI_BRIDGE_MEM_MIN;
>              if (pci_region_align(&s->r[type]) > align)
>                   align = pci_region_align(&s->r[type]);
>              u64 sum = pci_region_sum(&s->r[type]);
> +            if (!sum && shpc_cap)
> +                sum = align; /* reserve min size for hot-plug */
>              u64 size = ALIGN(sum, align);
>              int is64 = pci_bios_bridge_region_is64(&s->r[type],
>                                              s->bus_dev, type);
> diff --git a/src/hw/pci.c b/src/hw/pci.c
> index caf9265..055353d 100644
> --- a/src/hw/pci.c
> +++ b/src/hw/pci.c
> @@ -225,6 +225,25 @@ pci_find_init_device(const struct pci_device_id *ids, void *arg)
>      return NULL;
>  }
>  
> +u8 pci_find_capability(struct pci_device *pci, u8 cap_id)
> +{
> +    int i;
> +    u8 cap;
> +    u16 status = pci_config_readw(pci->bdf, PCI_STATUS);
> +
> +    if (!(status & PCI_STATUS_CAP_LIST))
> +        return 0;
> +
> +    for (i = 0, cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
> +         (i <= 0xff) && cap;
> +         i++, cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
> +        if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
> +            return cap;
> +
> +    return 0;
> +}
> +
> +
>  void
>  pci_reboot(void)
>  {
> diff --git a/src/hw/pci.h b/src/hw/pci.h
> index 167a027..e828225 100644
> --- a/src/hw/pci.h
> +++ b/src/hw/pci.h
> @@ -116,6 +116,7 @@ int pci_init_device(const struct pci_device_id *ids
>                      , struct pci_device *pci, void *arg);
>  struct pci_device *pci_find_init_device(const struct pci_device_id *ids
>                                          , void *arg);
> +u8 pci_find_capability(struct pci_device *pci, u8 cap_id);
>  void pci_reboot(void);
>  
>  #endif
> -- 
> 1.8.3.1

Re: [Qemu-devel] [PATCH V3 1/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Kevin O'Connor]

On Thu, Apr 10, 2014 at 04:29:40PM +0300, Marcel Apfelbaum wrote:
[...]
> +    for (i = 0, cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
> +         (i <= 0xff) && cap;
> +         i++, cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
> +        if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
> +            return cap;

I hate to nitpick, but I find that for-loop hard to read.  What about
something like:

    int i;
    u8 cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
    for (i = 0, cap && i <= 0xff; i++) {
        if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
            return cap;
        cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
    }

Otherwise, your patches look fine to me.

-Kevin

Re: [Qemu-devel] [PATCH V3 1/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Marcel Apfelbaum]

On Thu, 2014-04-10 at 12:45 -0400, Kevin O'Connor wrote:
> On Thu, Apr 10, 2014 at 04:29:40PM +0300, Marcel Apfelbaum wrote:
> [...]
> > +    for (i = 0, cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
> > +         (i <= 0xff) && cap;
> > +         i++, cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
> > +        if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
> > +            return cap;
> 
> I hate to nitpick, but I find that for-loop hard to read.  What about
> something like:
> 
>     int i;
>     u8 cap = pci_config_readb(pci->bdf, PCI_CAPABILITY_LIST);
>     for (i = 0, cap && i <= 0xff; i++) {
>         if (pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_ID) == cap_id)
>             return cap;
>         cap = pci_config_readb(pci->bdf, cap + PCI_CAP_LIST_NEXT))
>     }
Sure!

Thanks for the review,
Marcel
> 
> Otherwise, your patches look fine to me.
> 
> -Kevin

[Qemu-devel] [PATCH V3 2/2] hw/pci: check if pci2pci bridges implement optional limit registers

[Marcel Apfelbaum]

<I/O Base Register, I/O Limit Register> pair and
<Prefetchable Memory Base Register, Prefetchable Memory Limit Register> pair
are both optional.
Do not reserve ranges if the above registers are not implemented.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
---
 src/fw/pciinit.c |  9 ++-------
 src/hw/pci.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 src/hw/pci.h     |  9 +++++++++
 3 files changed, 59 insertions(+), 7 deletions(-)

diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
index 9b5d7ad..bbaecd6 100644
--- a/src/fw/pciinit.c
+++ b/src/fw/pciinit.c
@@ -26,13 +26,6 @@
 #define PCI_BRIDGE_MEM_MIN    (1<<21)  // 2M == hugepage size
 #define PCI_BRIDGE_IO_MIN      0x1000  // mandated by pci bridge spec
 
-enum pci_region_type {
-    PCI_REGION_TYPE_IO,
-    PCI_REGION_TYPE_MEM,
-    PCI_REGION_TYPE_PREFMEM,
-    PCI_REGION_TYPE_COUNT,
-};
-
 static const char *region_type_name[] = {
     [ PCI_REGION_TYPE_IO ]      = "io",
     [ PCI_REGION_TYPE_MEM ]     = "mem",
@@ -681,6 +674,8 @@ static int pci_bios_check_devices(struct pci_bus *busses)
         for (type = 0; type < PCI_REGION_TYPE_COUNT; type++) {
             u64 align = (type == PCI_REGION_TYPE_IO) ?
                 PCI_BRIDGE_IO_MIN : PCI_BRIDGE_MEM_MIN;
+            if (!pci_bridge_has_region(s->bus_dev, type))
+                continue;
             if (pci_region_align(&s->r[type]) > align)
                  align = pci_region_align(&s->r[type]);
             u64 sum = pci_region_sum(&s->r[type]);
diff --git a/src/hw/pci.c b/src/hw/pci.c
index 055353d..27e7b1c 100644
--- a/src/hw/pci.c
+++ b/src/hw/pci.c
@@ -243,6 +243,54 @@ u8 pci_find_capability(struct pci_device *pci, u8 cap_id)
     return 0;
 }
 
+static int pci_config_writableb(struct pci_device *pci, u32 addr, u8 test_val)
+{
+    u8 val;
+
+    val = pci_config_readb(pci->bdf, addr);
+    pci_config_writeb(pci->bdf, addr, test_val);
+
+    if (!(pci_config_readb(pci->bdf, addr)))
+        return 0;
+
+    pci_config_writeb(pci->bdf, addr, val);
+    return 1;
+}
+
+static int pci_config_writablew(struct pci_device *pci, u32 addr, u16 test_val)
+{
+    u16 val;
+
+    val = pci_config_readw(pci->bdf, addr);
+    pci_config_writew(pci->bdf, addr, test_val);
+
+    if (!(pci_config_readw(pci->bdf, addr)))
+        return 0;
+
+    pci_config_writew(pci->bdf, addr, val);
+    return 1;
+}
+
+int pci_bridge_has_region(struct pci_device *pci,
+                          enum pci_region_type region_type)
+{
+    if (pci->class != PCI_CLASS_BRIDGE_PCI)
+        return 0;
+
+    switch (region_type) {
+    case PCI_REGION_TYPE_IO:
+        return pci_config_writableb(pci, PCI_IO_BASE, 0xF0) &&
+               pci_config_writableb(pci, PCI_IO_LIMIT, 0xF0);
+    case PCI_REGION_TYPE_PREFMEM:
+        return pci_config_writablew(pci, PCI_PREF_MEMORY_BASE, 0xFFF0) &&
+               pci_config_writablew(pci, PCI_PREF_MEMORY_LIMIT, 0xFFF0);
+    case PCI_REGION_TYPE_MEM:    /* fall through */
+    default:
+        return 1;
+    }
+
+    return 1;
+}
 
 void
 pci_reboot(void)
diff --git a/src/hw/pci.h b/src/hw/pci.h
index e828225..0aaa84c 100644
--- a/src/hw/pci.h
+++ b/src/hw/pci.h
@@ -12,6 +12,13 @@
 #define PCI_NUM_REGIONS 7
 #define PCI_BRIDGE_NUM_REGIONS 2
 
+enum pci_region_type {
+    PCI_REGION_TYPE_IO,
+    PCI_REGION_TYPE_MEM,
+    PCI_REGION_TYPE_PREFMEM,
+    PCI_REGION_TYPE_COUNT,
+};
+
 static inline u8 pci_bdf_to_bus(u16 bdf) {
     return bdf >> 8;
 }
@@ -117,6 +124,8 @@ int pci_init_device(const struct pci_device_id *ids
 struct pci_device *pci_find_init_device(const struct pci_device_id *ids
                                         , void *arg);
 u8 pci_find_capability(struct pci_device *pci, u8 cap_id);
+int pci_bridge_has_region(struct pci_device *pci,
+                          enum pci_region_type region_type);
 void pci_reboot(void);
 
 #endif
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH V3 2/2] hw/pci: check if pci2pci bridges implement optional limit registers

[Michael S. Tsirkin]

On Thu, Apr 10, 2014 at 04:29:41PM +0300, Marcel Apfelbaum wrote:
> <I/O Base Register, I/O Limit Register> pair and
> <Prefetchable Memory Base Register, Prefetchable Memory Limit Register> pair
> are both optional.
> Do not reserve ranges if the above registers are not implemented.
> 
> Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>


Reviewed-by: Michael S. Tsirkin <mst@redhat.com>




> ---
>  src/fw/pciinit.c |  9 ++-------
>  src/hw/pci.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
>  src/hw/pci.h     |  9 +++++++++
>  3 files changed, 59 insertions(+), 7 deletions(-)
> 
> diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
> index 9b5d7ad..bbaecd6 100644
> --- a/src/fw/pciinit.c
> +++ b/src/fw/pciinit.c
> @@ -26,13 +26,6 @@
>  #define PCI_BRIDGE_MEM_MIN    (1<<21)  // 2M == hugepage size
>  #define PCI_BRIDGE_IO_MIN      0x1000  // mandated by pci bridge spec
>  
> -enum pci_region_type {
> -    PCI_REGION_TYPE_IO,
> -    PCI_REGION_TYPE_MEM,
> -    PCI_REGION_TYPE_PREFMEM,
> -    PCI_REGION_TYPE_COUNT,
> -};
> -
>  static const char *region_type_name[] = {
>      [ PCI_REGION_TYPE_IO ]      = "io",
>      [ PCI_REGION_TYPE_MEM ]     = "mem",
> @@ -681,6 +674,8 @@ static int pci_bios_check_devices(struct pci_bus *busses)
>          for (type = 0; type < PCI_REGION_TYPE_COUNT; type++) {
>              u64 align = (type == PCI_REGION_TYPE_IO) ?
>                  PCI_BRIDGE_IO_MIN : PCI_BRIDGE_MEM_MIN;
> +            if (!pci_bridge_has_region(s->bus_dev, type))
> +                continue;
>              if (pci_region_align(&s->r[type]) > align)
>                   align = pci_region_align(&s->r[type]);
>              u64 sum = pci_region_sum(&s->r[type]);
> diff --git a/src/hw/pci.c b/src/hw/pci.c
> index 055353d..27e7b1c 100644
> --- a/src/hw/pci.c
> +++ b/src/hw/pci.c
> @@ -243,6 +243,54 @@ u8 pci_find_capability(struct pci_device *pci, u8 cap_id)
>      return 0;
>  }
>  
> +static int pci_config_writableb(struct pci_device *pci, u32 addr, u8 test_val)

Don't like this name.
pci_config_writable_byte or better see below.

> +{
> +    u8 val;
> +
> +    val = pci_config_readb(pci->bdf, addr);
> +    pci_config_writeb(pci->bdf, addr, test_val);
> +
> +    if (!(pci_config_readb(pci->bdf, addr)))
> +        return 0;

This only works for readonly fields which return 0
on read. Maybe name should reflect this:
how about pci_config_is_reserved() ?

There's no need to pass in test value either,
it makes the interface fragile:
passing e.g. 0x0 as value won't work.

> +
> +    pci_config_writeb(pci->bdf, addr, val);

Practically that's overkill.

Just set base > limit and there won't be
need to save and restore: range is disabled.



> +    return 1;
> +}
> +
> +static int pci_config_writablew(struct pci_device *pci, u32 addr, u16 test_val)
> +{
> +    u16 val;
> +
> +    val = pci_config_readw(pci->bdf, addr);
> +    pci_config_writew(pci->bdf, addr, test_val);
> +
> +    if (!(pci_config_readw(pci->bdf, addr)))
> +        return 0;
> +
> +    pci_config_writew(pci->bdf, addr, val);
> +    return 1;
> +}


This one isn't needed at all.

> +
> +int pci_bridge_has_region(struct pci_device *pci,
> +                          enum pci_region_type region_type)
> +{
> +    if (pci->class != PCI_CLASS_BRIDGE_PCI)
> +        return 0;

Do we really need this test here?

> +
> +    switch (region_type) {
> +    case PCI_REGION_TYPE_IO:
> +        return pci_config_writableb(pci, PCI_IO_BASE, 0xF0) &&
> +               pci_config_writableb(pci, PCI_IO_LIMIT, 0xF0);

I think we should only test base.
Testing limit like this might cause issues as bridge now
claims some resources temporarily.
Also why 0xF0 and not 0xFF? Seems like a random value.


> +    case PCI_REGION_TYPE_PREFMEM:
> +        return pci_config_writablew(pci, PCI_PREF_MEMORY_BASE, 0xFFF0) &&
> +               pci_config_writablew(pci, PCI_PREF_MEMORY_LIMIT, 0xFFF0);
> +    case PCI_REGION_TYPE_MEM:    /* fall through */
> +    default:
> +        return 1;
> +    }
> +
> +    return 1;
> +}
>  
>  void
>  pci_reboot(void)


Same comments apply to prefetch and writeableb,
but also - there is no need to write all bytes.
So really

Basically:

int pci_bridge_has_region(struct pci_device *pci,
> +                          enum pci_region_type region_type)
> +{
> +    if (pci->class != PCI_CLASS_BRIDGE_PCI)
> +        return 0;

Do we really need this test here?

> +
> +    switch (region_type) {
> +    case PCI_REGION_TYPE_IO:
> +        return pci_config_writableb(pci, PCI_IO_BASE, 0xF0) &&
> +               pci_config_writableb(pci, PCI_IO_LIMIT, 0xF0);

I think we should only test base.
Testing limit like this might cause issues as bridge now
claims some resources temporarily.
Also why 0xF0 and not 0xFF? Seems like a random value.


> +    case PCI_REGION_TYPE_PREFMEM:
> +        return pci_config_writablew(pci, PCI_PREF_MEMORY_BASE, 0xFFF0) &&
> +               pci_config_writablew(pci, PCI_PREF_MEMORY_LIMIT, 0xFFF0);
> +    case PCI_REGION_TYPE_MEM:    /* fall through */
> +    default:
> +        return 1;
> +    }
> +
> +    return 1;

	unreacheable code.

> +}




Here's a simpler implementation.

int pci_bridge_has_region(struct pci_device *pci,
                          enum pci_region_type region_type)
{
    u8 base;
  
    switch (region_type) {
    case PCI_REGION_TYPE_IO:
	base = PCI_IO_BASE;
	break;
    case PCI_REGION_TYPE_PREFMEM:
	base = PCI_PREF_MEMORY_BASE;
	break;
    default:
	/* Regular memory support is mandatory */
        return 1;
    }

    /* Check that base is writeable. Safe as this increases base so it
     * won't cause bridge to claim new resources.
     */
    pci_config_writeb(pci->bdf, base, 0xFF);
    return !!pci_config_readb(pci->bdf, base);
}




> diff --git a/src/hw/pci.h b/src/hw/pci.h
> index e828225..0aaa84c 100644
> --- a/src/hw/pci.h
> +++ b/src/hw/pci.h
> @@ -12,6 +12,13 @@
>  #define PCI_NUM_REGIONS 7
>  #define PCI_BRIDGE_NUM_REGIONS 2
>  
> +enum pci_region_type {
> +    PCI_REGION_TYPE_IO,
> +    PCI_REGION_TYPE_MEM,
> +    PCI_REGION_TYPE_PREFMEM,
> +    PCI_REGION_TYPE_COUNT,
> +};
> +
>  static inline u8 pci_bdf_to_bus(u16 bdf) {
>      return bdf >> 8;
>  }
> @@ -117,6 +124,8 @@ int pci_init_device(const struct pci_device_id *ids
>  struct pci_device *pci_find_init_device(const struct pci_device_id *ids
>                                          , void *arg);
>  u8 pci_find_capability(struct pci_device *pci, u8 cap_id);
> +int pci_bridge_has_region(struct pci_device *pci,
> +                          enum pci_region_type region_type);
>  void pci_reboot(void);
>  
>  #endif
> -- 
> 1.8.3.1

Re: [Qemu-devel] [PATCH V3 2/2] hw/pci: check if pci2pci bridges implement optional limit registers

[Marcel Apfelbaum]

On Thu, 2014-04-10 at 18:46 +0300, Michael S. Tsirkin wrote:
> On Thu, Apr 10, 2014 at 04:29:41PM +0300, Marcel Apfelbaum wrote:
> > <I/O Base Register, I/O Limit Register> pair and
> > <Prefetchable Memory Base Register, Prefetchable Memory Limit Register> pair
> > are both optional.
> > Do not reserve ranges if the above registers are not implemented.
> > 
> > Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
> 
> 
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> 
> 
> 
> 
> > ---
> >  src/fw/pciinit.c |  9 ++-------
> >  src/hw/pci.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  src/hw/pci.h     |  9 +++++++++
> >  3 files changed, 59 insertions(+), 7 deletions(-)
> > 
> > diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
> > index 9b5d7ad..bbaecd6 100644
> > --- a/src/fw/pciinit.c
> > +++ b/src/fw/pciinit.c
> > @@ -26,13 +26,6 @@
> >  #define PCI_BRIDGE_MEM_MIN    (1<<21)  // 2M == hugepage size
> >  #define PCI_BRIDGE_IO_MIN      0x1000  // mandated by pci bridge spec
> >  
> > -enum pci_region_type {
> > -    PCI_REGION_TYPE_IO,
> > -    PCI_REGION_TYPE_MEM,
> > -    PCI_REGION_TYPE_PREFMEM,
> > -    PCI_REGION_TYPE_COUNT,
> > -};
> > -
> >  static const char *region_type_name[] = {
> >      [ PCI_REGION_TYPE_IO ]      = "io",
> >      [ PCI_REGION_TYPE_MEM ]     = "mem",
> > @@ -681,6 +674,8 @@ static int pci_bios_check_devices(struct pci_bus *busses)
> >          for (type = 0; type < PCI_REGION_TYPE_COUNT; type++) {
> >              u64 align = (type == PCI_REGION_TYPE_IO) ?
> >                  PCI_BRIDGE_IO_MIN : PCI_BRIDGE_MEM_MIN;
> > +            if (!pci_bridge_has_region(s->bus_dev, type))
> > +                continue;
> >              if (pci_region_align(&s->r[type]) > align)
> >                   align = pci_region_align(&s->r[type]);
> >              u64 sum = pci_region_sum(&s->r[type]);
> > diff --git a/src/hw/pci.c b/src/hw/pci.c
> > index 055353d..27e7b1c 100644
> > --- a/src/hw/pci.c
> > +++ b/src/hw/pci.c
> > @@ -243,6 +243,54 @@ u8 pci_find_capability(struct pci_device *pci, u8 cap_id)
> >      return 0;
> >  }
> >  
> > +static int pci_config_writableb(struct pci_device *pci, u32 addr, u8 test_val)
> 
> Don't like this name.
> pci_config_writable_byte or better see below.
> 
> > +{
> > +    u8 val;
> > +
> > +    val = pci_config_readb(pci->bdf, addr);
> > +    pci_config_writeb(pci->bdf, addr, test_val);
> > +
> > +    if (!(pci_config_readb(pci->bdf, addr)))
> > +        return 0;
> 
> This only works for readonly fields which return 0
> on read. Maybe name should reflect this:
> how about pci_config_is_reserved() ?
No problem with that

> 
> There's no need to pass in test value either,
> it makes the interface fragile:
> passing e.g. 0x0 as value won't work.
Yes, the value would be the caller responsibility.
But I think that a value like 0xF0 (explained below)
will cover all we need.

> 
> > +
> > +    pci_config_writeb(pci->bdf, addr, val);
> 
> Practically that's overkill.
I have to restore prev settings. I do not see this as
an overkill: a query should preserve the prev value.
And is also not expensive, and it is made only by bridges,
not a real issue from my point of view.

> 
> Just set base > limit and there won't be
> need to save and restore: range is disabled.
That would be possible, yes. But again, I don't want
to change values in a query.

> 
> 
> 
> > +    return 1;
> > +}
> > +
> > +static int pci_config_writablew(struct pci_device *pci, u32 addr, u16 test_val)
> > +{
> > +    u16 val;
> > +
> > +    val = pci_config_readw(pci->bdf, addr);
> > +    pci_config_writew(pci->bdf, addr, test_val);
> > +
> > +    if (!(pci_config_readw(pci->bdf, addr)))
> > +        return 0;
> > +
> > +    pci_config_writew(pci->bdf, addr, val);
> > +    return 1;
> > +}
> 
> 
> This one isn't needed at all.
I can use the byte function, yes, thanks, I'll drop it.

> 
> > +
> > +int pci_bridge_has_region(struct pci_device *pci,
> > +                          enum pci_region_type region_type)
> > +{
> > +    if (pci->class != PCI_CLASS_BRIDGE_PCI)
> > +        return 0;
> 
> Do we really need this test here?
Better safe than sorry. If is not a bridge this will mess up
pci device's configuration and the bug will be hard to find.

> 
> > +
> > +    switch (region_type) {
> > +    case PCI_REGION_TYPE_IO:
> > +        return pci_config_writableb(pci, PCI_IO_BASE, 0xF0) &&
> > +               pci_config_writableb(pci, PCI_IO_LIMIT, 0xF0);
> 
> I think we should only test base.
> Testing limit like this might cause issues as bridge now
> claims some resources temporarily.
I can test only one, yes. The spec said that both should be 0,
but going for one of them should be enough.

> Also why 0xF0 and not 0xFF? Seems like a random value.
0xF0 is not random, the last 4 bits are *always* read only, this
is why I don't test them. Any value starting from 0x10 will do.
0xFF it will still work, as the last 4 bits will be silently
dropped, but why not follow the spec. 
 
> 
> 
> > +    case PCI_REGION_TYPE_PREFMEM:
> > +        return pci_config_writablew(pci, PCI_PREF_MEMORY_BASE, 0xFFF0) &&
> > +               pci_config_writablew(pci, PCI_PREF_MEMORY_LIMIT, 0xFFF0);
> > +    case PCI_REGION_TYPE_MEM:    /* fall through */
> > +    default:
> > +        return 1;
> > +    }
> > +
> > +    return 1;
> > +}
> >  
> >  void
> >  pci_reboot(void)
> 
[...]

> > +    }
> > +
> > +    return 1;
> 
> 	unreacheable code.
Right. Some compilation flags and automatic check tools might
have a problem with that, this why I left it there. I can remove
it, of course.
 
> 
> > +}
> 
> 
> 
> 
> Here's a simpler implementation.
> 
> int pci_bridge_has_region(struct pci_device *pci,
>                           enum pci_region_type region_type)
> {
>     u8 base;
>   
>     switch (region_type) {
>     case PCI_REGION_TYPE_IO:
> 	base = PCI_IO_BASE;
> 	break;
>     case PCI_REGION_TYPE_PREFMEM:
> 	base = PCI_PREF_MEMORY_BASE;
> 	break;
>     default:
> 	/* Regular memory support is mandatory */
>         return 1;
>     }
> 
>     /* Check that base is writeable. Safe as this increases base so it
>      * won't cause bridge to claim new resources.
Yes, but it can claim less. Why mess with the orig value for
a query? I think a query should not modify a value, even if does
not affect the system. It is hard to track this kind of changes,
and restoring the value is not a big price to pay IMHO.

Thanks for the review,
Marcel 
 
>      */
>     pci_config_writeb(pci->bdf, base, 0xFF);
>     return !!pci_config_readb(pci->bdf, base);
> }
> 
> 
> 
> 
> > diff --git a/src/hw/pci.h b/src/hw/pci.h
> > index e828225..0aaa84c 100644
> > --- a/src/hw/pci.h
> > +++ b/src/hw/pci.h
> > @@ -12,6 +12,13 @@
> >  #define PCI_NUM_REGIONS 7
> >  #define PCI_BRIDGE_NUM_REGIONS 2
> >  
> > +enum pci_region_type {
> > +    PCI_REGION_TYPE_IO,
> > +    PCI_REGION_TYPE_MEM,
> > +    PCI_REGION_TYPE_PREFMEM,
> > +    PCI_REGION_TYPE_COUNT,
> > +};
> > +
> >  static inline u8 pci_bdf_to_bus(u16 bdf) {
> >      return bdf >> 8;
> >  }
> > @@ -117,6 +124,8 @@ int pci_init_device(const struct pci_device_id *ids
> >  struct pci_device *pci_find_init_device(const struct pci_device_id *ids
> >                                          , void *arg);
> >  u8 pci_find_capability(struct pci_device *pci, u8 cap_id);
> > +int pci_bridge_has_region(struct pci_device *pci,
> > +                          enum pci_region_type region_type);
> >  void pci_reboot(void);
> >  
> >  #endif
> > -- 
> > 1.8.3.1

Re: [Qemu-devel] [SeaBIOS] [PATCH V3 0/2] hw/pci: reserve IO and mem for pci-2-pci bridges with no devices attached

[Michael S. Tsirkin]

On Thu, Apr 10, 2014 at 04:29:39PM +0300, Marcel Apfelbaum wrote:
> v2 -> v3:
>  - Addressed Michael S. Tsirkin's comments:
>    - I/O and Prefetchable Memory are optional. Do not allocate ranges
>      if they are not implemented (2/2).
>  - Note that 2/2 patch can be seen as a separate fix. However, it
>    is related to ranges reservation.
> 
> v1 -> v2:
>  - Thanks Gerd Hoffmann for the review.
>  - Addressed Michael S. Tsirkin's comments:
>    - Limit capabilities query to 256 iterations, to make sure we
>      don't get into an infinite loop with a broken device.
> 


An additional possible enhancement would be to avoid allocating
for devices that are behind a non IO forwarding bridge.

Can be a patch on top.


> If a pci-2-pci bridge supports hot-plug functionality but there are no devices
> connected to it, reserve IO/mem in order to be able to attach devices
> later. Do not waste space, use minimum allowed.
> 
> Marcel Apfelbaum (2):
>   hw/pci: reserve IO and mem for pci-2-pci bridges with no devices
>     attached
>   hw/pci: check if pci2pci bridges implement optional limit registers
> 
>  src/fw/pciinit.c | 12 +++++-----
>  src/hw/pci.c     | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  src/hw/pci.h     | 10 +++++++++
>  3 files changed, 82 insertions(+), 7 deletions(-)
> 
> -- 
> 1.8.3.1