Linux USB
 help / color / mirror / Atom feed
From: Mika Westerberg <mika.westerberg@linux.intel.com>
To: Dag B <dag@bakke.com>
Cc: "Bjorn Helgaas" <helgaas@kernel.org>,
	linux-pci@vger.kernel.org, linux-usb@vger.kernel.org,
	"Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>,
	"Lukas Wunner" <lukas@wunner.de>
Subject: Re: Connecting multiple TB3 eGPUs via USB4 hub?'
Date: Thu, 11 Jun 2026 06:41:14 +0200	[thread overview]
Message-ID: <20260611044114.GU2990@black.igk.intel.com> (raw)
In-Reply-To: <9507bfaa-c14e-49fc-8016-cf65bed9a76d@bakke.com>

Hi,

On Thu, Jun 11, 2026 at 12:48:40AM +0200, Dag B wrote:
> 
> On 6/9/26 14:08, Mika Westerberg wrote:
> > Hi,
> > 
> > On Mon, Jun 08, 2026 at 10:26:40AM -0500, Bjorn Helgaas wrote:
> > > [+cc linux-usb (Thunderbolt maintainers), Ilpo, Lukas]
> > > 
> > > On Sun, May 24, 2026 at 04:56:56PM +0200, Dag B wrote:
> > > > I am attempting to connect two TB3 enclosures to the same USB4 port via a
> > > > USB4 hub.
> > > > 
> > > > I cannot figure out of this is in violation of the TB3, TB4 or USB4 spec.
> > > I'm not aware of a spec issue here.
> Thank you very much Bjorn, slightly more interesting trying to get it to
> work then.
> > It should be fine but typically PCIe resources for one tunneled PCIe root
> > port may not be enough for multiple GPUs. Therefore I suggest to connect
> > them directly to the host USB4 ports without a hub.
> 
> That makes perfect sense. Until you run out of physical ports. The entire
> point of this exercise started with trying to connect a 5th 3090 to my fw13
> motherboard. :-D
> 
> Hence the USB4 hub. But even with just the hub and two connected GPUs, it
> fails.

Yes most likely because a single GPU takes lots of resources and there is
quite small window reserved per root port for example:

  Memory behind bridge: 60000000-6c1fffff [size=194M] [32-bit]
  Prefetchable memory behind bridge: 6040000000-605bffffff [size=448M] [32-bit]

Looking at your logs you have all 4 PCIe root ports for tunneling listed
there so I would expect you have 4 USB4/TB ports available as well
(typically the root ports not available for tunneling will not be visible
to the OS).

> For now, I have limited my testing to just the USB4 hub and either one or
> two GPUs.
> 
> 
> > Second thing is that you have bunch of PCIe related command line parameters
> > that may affect. I suggest removing all of them, and retry. There is hardly
> > any reason to add these - the kernel should be able to handle this by
> > default.
> 
> Mhm. I have stripped the command line parameters for some of my tests. It
> absolutely works with nothing extra for a single GPU. And it absolutely did
> not work without command-line params and with 4 GPUs  when I started on this
> adventure. But that was around 6.16-6.17.
> 
> 
> > 
> > Then provide full dmesg along with output of 'sudo lspci -vv' so we can
> > look a the resource allocation.
> 
> This very quickly becomes a lot of data as I test various things. I put a
> repo with various data up on github.
> 
> https://github.com/dagbdagb/usb4_to_tb3_troubleshooting/

There seems to be no dmesg + lspci without the command line parameters? Can
you do this:

  1. Remove all the non-default command line parameters.
  2. Boot the system up, nothing connected.
  3. Plug in the USB4 dock, enable PCIe tunnel (this probably is automatic)
  4. Plug in the first eGPU to the dock, enable PCIe tunnel
  5. Plug in the second eGPU to the dock, enable PCIe tunnel

From this take full dmesg and output of 'sudo lspci -vv'. At least we can
see how the "default" kernel resource allocation fails.

> If you truly want all this on the mailing list, please let me know.
> 
> > 
> > > Below it looks like you hot-added the hub.  Does it make any
> > > difference if everything is connected before boot?
> 
> Yes. With both GPUs connected at boot, none of them will work. With just
> one, it works.

Okay that tells the BIOS does not even try to assign the resources for
both of them.

  reply	other threads:[~2026-06-11  4:41 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <f600c4de-955f-4403-9f47-4de1088a9f56@bakke.com>
2026-06-08 15:26 ` Connecting multiple TB3 eGPUs via USB4 hub?' Bjorn Helgaas
2026-06-09 12:08   ` Mika Westerberg
2026-06-10 22:48     ` Dag B
2026-06-11  4:41       ` Mika Westerberg [this message]
2026-06-11  5:28         ` Mika Westerberg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260611044114.GU2990@black.igk.intel.com \
    --to=mika.westerberg@linux.intel.com \
    --cc=dag@bakke.com \
    --cc=helgaas@kernel.org \
    --cc=ilpo.jarvinen@linux.intel.com \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=lukas@wunner.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox