Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Greg KH <gregkh@linuxfoundation.org>]

On Tue, Apr 28, 2026 at 04:11:48PM +0200, Mika Westerberg wrote:
> On Tue, Apr 28, 2026 at 07:54:51AM -0600, Greg KH wrote:
> > On Tue, Apr 28, 2026 at 02:03:14PM +0200, Mika Westerberg wrote:
> > > On Tue, Apr 28, 2026 at 05:57:37AM -0600, Greg KH wrote:
> > > > On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > > > > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > > > > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > > > > through a character device without need to go through the network stack.
> > > > > 
> > > > > Any application that supports read(2) and write(2) in some form should
> > > > > be able to use the device without changes. The data is sent out to the
> > > > > other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> > > > > device is called /dev/tbstreamX where X is the minor number starting
> > > > > from 0.
> > > > > 
> > > > > All stream devices need to be configured first. This is done through
> > > > > ConfigFS interface. There can be multiple streams at the same time (this
> > > > > depends on number of DMA rings and available HopIDs) and a single stream
> > > > > supports traffic in both directions. For example there could be an
> > > > > application that uses one stream as control channel and another one as
> > > > > bi-directional data channel.
> > > > > 
> > > > > A real use-case for this is to take a backup as a part of recovery
> > > > > initramfs tooling (no need to setup networking or have ssh or similar
> > > > > tooling as part of the initramfs). Say we want to backup the disk of
> > > > > host1 to host2. First Thunderbolt/USB4 cable is connected between the
> > > > > hosts (there can be devices in the middle too) then the receiving side
> > > > > configures the stream:
> > > > > 
> > > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
> > > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
> > > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
> > > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> > > > > 
> > > > > We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> > > > > >From this point forward the /dev/tbstream0 can be used pretty much as
> > > > > regular file:
> > > > > 
> > > > >   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> > > > > 
> > > > > The host that is being backed up then configures the stream accordingly:
> > > > > 
> > > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
> > > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> > > > > 
> > > > > Here we take advantage of the fact that host2 also announces the active
> > > > > streams through XDomain properties so the name "backup" gives us the
> > > > > HopIDs. It is also possible to configure them manually in the same way
> > > > > we did for host2.
> > > > > 
> > > > > Then it is just a matter of copying the data over:
> > > > > 
> > > > >   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> > > > > 
> > > > > Similarly it is possible to transfer parts of the filesystem. For
> > > > > example copy contents of mydir over to the host2:
> > > > > 
> > > > >   host2 # gunzip < /dev/tbstream0 | tar xf -
> > > > >   host1 # tar cf - mydir | gzip > /dev/tbstream0
> > > > > 
> > > > > Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> > > > > to desktop (host2):
> > > > > 
> > > > >   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
> > > > >                          autovideosink
> > > > > 
> > > > >   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
> > > > >                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> > > > > 
> > > > > Once the streams are no longer needed they can be removed:
> > > > > 
> > > > >   host1 # cd /sys/kernel/config/thunderbolt/stream/
> > > > >   host1 # rmdir -p 0-503.0/backup
> > > > > 
> > > > >   host2 # cd /sys/kernel/config/thunderbolt/stream
> > > > >   host2 # rmdir -p 0-1.0/backup
> > > > 
> > > > Very cool, but shouldn't the above be in some documentation somewhere so
> > > > that people know how to use it?
> > > 
> > > Sure, I can add it part of the Documentation/admin-guide/thunderbolt.rs for
> > > example.
> > > 
> > > > And why do you need a whole major for this, why not just use a misc
> > > > device that it dynamically created for every new dev?
> > > 
> > > We do use this:
> > > 
> > >        ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
> > >                                   "tbstream");
> > > 
> > > that should be dynamically allocated, no?
> > 
> > Yes, but you are using up a whole major number for this, and in reality
> > there's only going to be 1-2, maybe 4, different devices needed at once,
> > right?  So just use the miscdev interface instead?
> 
> There could be 11 per host controller in Intel hardware (we have 12 DMA
> rings, one of which is reserved for control traffic), and we have 2 host
> conrollers in recent systems. Due to the dedicated flow control we use now
> that's not possible but we are planning to make it to use shared flow
> control instead which allows more.
> 
> Not sure if anybody ever will create that many, though.

Yeah, that's not many, and a bit of a waste of a full major number.

> Second thing is that we use cdev_device_add() to manage the char device and
> the stream device as they are part of the same structure. I don't think
> that can be done with miscdevice.

Not yet, but see the patches on the list for how to do that properly.
You will have issues with disconnect/open/close that you need to handle
very carefully, especially as you are a dynamic device.  See this
thread:
	https://lore.kernel.org/r/20260427134659.95181-1-tzungbi@kernel.org

thanks,

greg k-h