[PATCH 0/9] thunderbolt: Introduce USB4STREAM

[PATCH 0/9] thunderbolt: Introduce USB4STREAM

[Mika Westerberg]

Hi all,

This series adds support for a new protocol over USB4/Thunderbolt cable
called USB4STREAM. The protocol is super-simple and basically just
transfers raw packets from one host to another. It is documented as part of
the thunderbolt_stream driver.

The driver exposes /dev/tbstreamX devices on each side of the link that can
be used to transfer data using regular filesystem operations such as
read(2) and write(2):

  host1 # cat /dev/tbstream0
  host2 # echo hello > /dev/tbstream0

This can be useful in cases where network tooling is not available or just
for existing applications like 'dd' and 'cat' that do not support sockets.

thunderbolt_stream can be used at the same time with thunderbolt_net so
they don't rule each other our. 

thunderbolt_stream allows multiple streams to be created, for example one
stream for control traffic and another for data (there are some limitations
in the core USB4/Thunderbolt driver due to dedicated flow control scheme
but this is likely change in the future). Each stream is bi-directional
tunnel over the fabric.

There are a couple of additional usage examples in the last patch that adds
the driver itself.

This applies on top of my XDomain improvements series [1].

[1] https://lore.kernel.org/linux-usb/20260427081109.2337731-1-mika.westerberg@linux.intel.com/

Mika Westerberg (9):
  thunderbolt: Add tb_property_merge_dir()
  thunderbolt: Add KUnit test for tb_property_merge_dir()
  thunderbolt: Allow service drivers to specify their own properties
  thunderbolt / net: Move ring_frame_size() to thunderbolt.h
  thunderbolt / net: Let the service drivers configure interrupt throttling
  thunderbolt: Add helper to figure size of the ring
  thunderbolt: Add tb_ring_flush()
  thunderbolt: Add support for ConfigFS
  thunderbolt: Add support for USB4STREAM

 .../ABI/testing/configfs-thunderbolt_stream   |   77 +
 drivers/net/thunderbolt/main.c                |   23 +-
 drivers/thunderbolt/Kconfig                   |   15 +
 drivers/thunderbolt/Makefile                  |    4 +
 drivers/thunderbolt/configfs.c                |   61 +
 drivers/thunderbolt/dma_test.c                |    5 +
 drivers/thunderbolt/domain.c                  |    2 +
 drivers/thunderbolt/nhi.c                     |   86 +-
 drivers/thunderbolt/nhi_regs.h                |    3 +-
 drivers/thunderbolt/property.c                |  154 +-
 drivers/thunderbolt/stream.c                  | 1693 +++++++++++++++++
 drivers/thunderbolt/tb.h                      |    8 +
 drivers/thunderbolt/test.c                    |   82 +
 drivers/thunderbolt/xdomain.c                 |   95 +-
 include/linux/thunderbolt.h                   |   44 +-
 15 files changed, 2257 insertions(+), 95 deletions(-)
 create mode 100644 Documentation/ABI/testing/configfs-thunderbolt_stream
 create mode 100644 drivers/thunderbolt/configfs.c
 create mode 100644 drivers/thunderbolt/stream.c

-- 
2.50.1

[PATCH 1/9] thunderbolt: Add tb_property_merge_dir()

[Mika Westerberg]

This allows merging one XDomain property directory into another. We are
going to use this in the subsequent patch.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/thunderbolt/property.c | 154 ++++++++++++++++++++++++---------
 include/linux/thunderbolt.h    |   3 +
 2 files changed, 114 insertions(+), 43 deletions(-)

diff --git a/drivers/thunderbolt/property.c b/drivers/thunderbolt/property.c
index 50cbfc92fe65..6b9666b61181 100644
--- a/drivers/thunderbolt/property.c
+++ b/drivers/thunderbolt/property.c
@@ -38,6 +38,7 @@ struct tb_property_dir_entry {
 static struct tb_property_dir *__tb_property_parse_dir(const u32 *block,
 	size_t block_len, unsigned int dir_offset, size_t dir_len,
 	bool is_root);
+static struct tb_property *tb_property_copy(const struct tb_property *property);
 
 static inline void parse_dwdata(void *dst, const void *src, size_t dwords)
 {
@@ -507,17 +508,9 @@ ssize_t tb_property_format_dir(const struct tb_property_dir *dir, u32 *block,
 	return ret < 0 ? ret : 0;
 }
 
-/**
- * tb_property_copy_dir() - Take a deep copy of directory
- * @dir: Directory to copy
- *
- * The resulting directory needs to be released by calling tb_property_free_dir().
- *
- * Return: Pointer to &struct tb_property_dir, %NULL in case of failure.
- */
-struct tb_property_dir *tb_property_copy_dir(const struct tb_property_dir *dir)
+static struct tb_property_dir *copy_dir(const struct tb_property_dir *dir)
 {
-	struct tb_property *property, *p = NULL;
+	struct tb_property *property, *p;
 	struct tb_property_dir *d;
 
 	if (!dir)
@@ -528,56 +521,131 @@ struct tb_property_dir *tb_property_copy_dir(const struct tb_property_dir *dir)
 		return NULL;
 
 	list_for_each_entry(property, &dir->properties, list) {
-		struct tb_property *p;
-
-		p = tb_property_alloc(property->key, property->type);
+		p = tb_property_copy(property);
 		if (!p)
 			goto err_free;
+		list_add_tail(&p->list, &d->properties);
+	}
 
-		p->length = property->length;
+	return d;
 
-		switch (property->type) {
-		case TB_PROPERTY_TYPE_DIRECTORY:
-			p->value.dir = tb_property_copy_dir(property->value.dir);
-			if (!p->value.dir)
-				goto err_free;
-			break;
+err_free:
+	tb_property_free_dir(d);
+	return NULL;
+}
 
-		case TB_PROPERTY_TYPE_DATA:
-			p->value.data = kmemdup(property->value.data,
-						property->length * 4,
-						GFP_KERNEL);
-			if (!p->value.data)
-				goto err_free;
-			break;
+static struct tb_property *tb_property_copy(const struct tb_property *property)
+{
+	struct tb_property *p;
 
-		case TB_PROPERTY_TYPE_TEXT:
-			p->value.text = kzalloc(p->length * 4, GFP_KERNEL);
-			if (!p->value.text)
-				goto err_free;
-			strcpy(p->value.text, property->value.text);
-			break;
+	p = tb_property_alloc(property->key, property->type);
+	if (!p)
+		return NULL;
 
-		case TB_PROPERTY_TYPE_VALUE:
-			p->value.immediate = property->value.immediate;
-			break;
+	p->length = property->length;
+	switch (property->type) {
+	case TB_PROPERTY_TYPE_DIRECTORY:
+		p->value.dir = copy_dir(property->value.dir);
+		if (!p->value.dir)
+			goto err_free;
+		break;
 
-		default:
-			break;
-		}
+	case TB_PROPERTY_TYPE_DATA:
+		p->value.data = kmemdup(property->value.data,
+					property->length * 4,
+					GFP_KERNEL);
+		if (!p->value.data)
+			goto err_free;
+		break;
 
-		list_add_tail(&p->list, &d->properties);
+	case TB_PROPERTY_TYPE_TEXT:
+		p->value.text = kzalloc(p->length * 4, GFP_KERNEL);
+		if (!p->value.text)
+			goto err_free;
+		strcpy(p->value.text, property->value.text);
+		break;
+
+	case TB_PROPERTY_TYPE_VALUE:
+		p->value.immediate = property->value.immediate;
+		break;
+
+	default:
+		break;
 	}
 
-	return d;
+	return p;
 
 err_free:
 	kfree(p);
-	tb_property_free_dir(d);
-
 	return NULL;
 }
 
+/**
+ * tb_property_copy_dir() - Take a deep copy of directory
+ * @dir: Directory to copy
+ *
+ * The resulting directory needs to be released by calling tb_property_free_dir().
+ *
+ * Return: Pointer to &struct tb_property_dir, %NULL in case of failure.
+ */
+struct tb_property_dir *tb_property_copy_dir(const struct tb_property_dir *dir)
+{
+	return copy_dir(dir);
+}
+EXPORT_SYMBOL_GPL(tb_property_copy_dir);
+
+/**
+ * tb_property_merge_dir() - Merges directory into parent
+ * @parent: Directory to merge @dir
+ * @dir: Directory that is merged
+ * @replace: Replace existing entries
+ *
+ * This will merge @dir into @parent. Both must have same UUID. The
+ * properties in @dir will overwrite overlapping properties in @parent
+ * if @replace is %true. Contents of @dir is copied (so if it is not
+ * needed afterwards it needs to relesed by calling tb_property_free_dir()).
+ */
+int tb_property_merge_dir(struct tb_property_dir *parent,
+			  const struct tb_property_dir *dir,
+			  bool replace)
+{
+	const struct tb_property *property;
+
+	if (WARN_ON(parent == dir))
+		return -EINVAL;
+
+	if (!uuid_equal(parent->uuid, dir->uuid))
+		return -EINVAL;
+
+	list_for_each_entry(property, &dir->properties, list) {
+		struct tb_property *p, *tmp;
+
+		tmp = tb_property_copy(property);
+		if (!tmp)
+			return -ENOMEM;
+
+		p = tb_property_find(parent, property->key, property->type);
+		if (p) {
+			if (replace) {
+				/*
+				 * Found existing property in parent so
+				 * replace with the new one.
+				 */
+				list_replace(&p->list, &tmp->list);
+				tb_property_free(p);
+			} else {
+				tb_property_free(tmp);
+				continue;
+			}
+		} else {
+			list_add_tail(&tmp->list, &parent->properties);
+		}
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(tb_property_merge_dir);
+
 /**
  * tb_property_add_immediate() - Add immediate property to directory
  * @parent: Directory to add the property
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index bbdbbc84c999..e98d569779f9 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -153,6 +153,9 @@ struct tb_property_dir *tb_property_parse_dir(const u32 *block,
 ssize_t tb_property_format_dir(const struct tb_property_dir *dir, u32 *block,
 			       size_t block_len);
 struct tb_property_dir *tb_property_copy_dir(const struct tb_property_dir *dir);
+int tb_property_merge_dir(struct tb_property_dir *parent,
+			  const struct tb_property_dir *dir,
+			  bool replace);
 struct tb_property_dir *tb_property_create_dir(const uuid_t *uuid);
 void tb_property_free_dir(struct tb_property_dir *dir);
 int tb_property_add_immediate(struct tb_property_dir *parent, const char *key,
-- 
2.50.1

[PATCH 2/9] thunderbolt: Add KUnit test for tb_property_merge_dir()

[Mika Westerberg]

This makes sure it keeps working if we ever need to change it.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/thunderbolt/test.c | 82 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/drivers/thunderbolt/test.c b/drivers/thunderbolt/test.c
index 1f4318249c22..ce14ab9ef7dd 100644
--- a/drivers/thunderbolt/test.c
+++ b/drivers/thunderbolt/test.c
@@ -2852,6 +2852,87 @@ static void tb_test_property_copy(struct kunit *test)
 	tb_property_free_dir(src);
 }
 
+static void tb_test_property_merge(struct kunit *test)
+{
+	struct tb_property_dir *dir1, *dir2, *dir3;
+	struct tb_property *p;
+	uuid_t uuid;
+	int ret;
+
+	dir1 = tb_property_create_dir(&network_dir_uuid);
+	KUNIT_ASSERT_NOT_NULL(test, dir1);
+	ret = tb_property_add_immediate(dir1, "prtcid", 1);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	ret = tb_property_add_immediate(dir1, "prtcvers", 1);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	ret = tb_property_add_immediate(dir1, "prtcrevs", 0);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	ret = tb_property_add_immediate(dir1, "prtcstns", 0);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+
+	dir2 = tb_property_create_dir(&network_dir_uuid);
+	KUNIT_ASSERT_NOT_NULL(test, dir2);
+	ret = tb_property_add_text(dir2, "descr", "This is text");
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	/* This replaces the value in dir1 */
+	ret = tb_property_add_immediate(dir2, "prtcvers", 0x1234);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+
+	uuid_gen(&uuid);
+	dir3 = tb_property_create_dir(&uuid);
+	KUNIT_ASSERT_NOT_NULL(test, dir3);
+	ret = tb_property_add_immediate(dir3, "value0", 0);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	ret = tb_property_add_text(dir3, "value1", "Text value");
+	KUNIT_EXPECT_EQ(test, ret, 0);
+	ret = tb_property_add_dir(dir2, "my", dir3);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+
+	ret = tb_property_merge_dir(dir1, dir2, true);
+	KUNIT_EXPECT_EQ(test, ret, 0);
+
+	p = tb_property_get_next(dir1, NULL);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "prtcid");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_VALUE);
+	KUNIT_ASSERT_EQ(test, p->length, 1);
+	KUNIT_ASSERT_EQ(test, p->value.immediate, 1);
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "prtcvers");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_VALUE);
+	KUNIT_ASSERT_EQ(test, p->length, 1);
+	KUNIT_ASSERT_EQ(test, p->value.immediate, 0x1234);
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "prtcrevs");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_VALUE);
+	KUNIT_ASSERT_EQ(test, p->length, 1);
+	KUNIT_ASSERT_EQ(test, p->value.immediate, 0);
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "prtcstns");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_VALUE);
+	KUNIT_ASSERT_EQ(test, p->length, 1);
+	KUNIT_ASSERT_EQ(test, p->value.immediate, 0);
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "descr");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_TEXT);
+	KUNIT_ASSERT_EQ(test, p->length, 4);
+	KUNIT_ASSERT_STREQ(test, p->value.text, "This is text");
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NOT_NULL(test, p);
+	KUNIT_ASSERT_STREQ(test, &p->key[0], "my");
+	KUNIT_ASSERT_EQ(test, p->type, TB_PROPERTY_TYPE_DIRECTORY);
+	compare_dirs(test, p->value.dir, dir3);
+	p = tb_property_get_next(dir1, p);
+	KUNIT_ASSERT_NULL(test, p);
+
+	tb_property_free_dir(dir2);
+	tb_property_free_dir(dir1);
+}
+
 static struct kunit_case tb_test_cases[] = {
 	KUNIT_CASE(tb_test_path_basic),
 	KUNIT_CASE(tb_test_path_not_connected_walk),
@@ -2892,6 +2973,7 @@ static struct kunit_case tb_test_cases[] = {
 	KUNIT_CASE(tb_test_property_parse),
 	KUNIT_CASE(tb_test_property_format),
 	KUNIT_CASE(tb_test_property_copy),
+	KUNIT_CASE(tb_test_property_merge),
 	{ }
 };
 
-- 
2.50.1

[PATCH 3/9] thunderbolt: Allow service drivers to specify their own properties

[Mika Westerberg]

The XDomain properties can be useful for service drivers, for example to
implement a registry for the services they expose. So far there has been
no need for service drivers to specify these but with the USB4STREAM
driver that we are going to use them.

This adds remote and local side properties that the service drivers have
access to. Remote side is read-only but the local side can be changed by
a service driver. Also provide a mechanism to notify the remote side
that there are changes.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/thunderbolt/xdomain.c | 95 ++++++++++++++++++++++++++++++-----
 include/linux/thunderbolt.h   | 12 +++++
 2 files changed, 94 insertions(+), 13 deletions(-)

diff --git a/drivers/thunderbolt/xdomain.c b/drivers/thunderbolt/xdomain.c
index 6e83f93eee83..781d88d06b93 100644
--- a/drivers/thunderbolt/xdomain.c
+++ b/drivers/thunderbolt/xdomain.c
@@ -640,6 +640,32 @@ void tb_unregister_protocol_handler(struct tb_protocol_handler *handler)
 }
 EXPORT_SYMBOL_GPL(tb_unregister_protocol_handler);
 
+static int update_service_properties(struct device *dev, void *data)
+{
+	struct tb_property_dir *root = data;
+	struct tb_service *svc;
+	struct tb_property *p;
+
+	svc = tb_to_service(dev);
+	if (!svc)
+		return 0;
+
+	guard(mutex)(&svc->lock);
+
+	/*
+	 * Replace the static service properties with the dynamic one.
+	 * Typically this is the same but service drivers can add their
+	 * own dynamic properties here too.
+	 */
+	p = tb_property_find(root, svc->key, TB_PROPERTY_TYPE_DIRECTORY);
+	if (!p)
+		return 0;
+	if (svc->local_properties)
+		return tb_property_merge_dir(p->value.dir,
+					     svc->local_properties, false);
+	return 0;
+}
+
 static void update_property_block(struct tb_xdomain *xd)
 {
 	mutex_lock(&xdomain_lock);
@@ -664,6 +690,9 @@ static void update_property_block(struct tb_xdomain *xd)
 		tb_property_add_text(dir, "deviceid", utsname()->nodename);
 		tb_property_add_immediate(dir, "maxhopid", xd->local_max_hopid);
 
+		/* Add service specific dynamic properties */
+		device_for_each_child(&xd->dev, dir, update_service_properties);
+
 		ret = tb_property_format_dir(dir, NULL, 0);
 		if (ret < 0) {
 			dev_warn(&xd->dev, "local property block creation failed\n");
@@ -936,6 +965,40 @@ void tb_unregister_service_driver(struct tb_service_driver *drv)
 }
 EXPORT_SYMBOL_GPL(tb_unregister_service_driver);
 
+static int update_xdomain(struct device *dev, void *data)
+{
+	struct tb_xdomain *xd;
+
+	xd = tb_to_xdomain(dev);
+	if (xd) {
+		queue_delayed_work(xd->tb->wq, &xd->properties_changed_work,
+				   msecs_to_jiffies(50));
+	}
+
+	return 0;
+}
+
+/**
+ * tb_service_properties_changed() - Notify the other host about changes
+ * @svc: Service whose properties changed
+ *
+ * Notifies the other host that service properties may have been
+ * changed. This should be called whenever @svc->local_properties is
+ * updated.
+ */
+void tb_service_properties_changed(struct tb_service *svc)
+{
+	struct tb_xdomain *xd = tb_service_parent(svc);
+
+	if (xd->is_unplugged)
+		return;
+
+	scoped_guard(mutex, &xdomain_lock)
+		xdomain_property_block_gen++;
+	update_xdomain(&xd->dev, NULL);
+}
+EXPORT_SYMBOL_GPL(tb_service_properties_changed);
+
 static ssize_t key_show(struct device *dev, struct device_attribute *attr,
 			char *buf)
 {
@@ -1035,6 +1098,7 @@ static void tb_service_release(struct device *dev)
 	struct tb_service *svc = container_of(dev, struct tb_service, dev);
 	struct tb_xdomain *xd = tb_service_parent(svc);
 
+	tb_property_free_dir(svc->remote_properties);
 	ida_free(&xd->service_ids, svc->id);
 	kfree(svc->key);
 	kfree(svc);
@@ -1049,6 +1113,16 @@ const struct device_type tb_service_type = {
 };
 EXPORT_SYMBOL_GPL(tb_service_type);
 
+static void update_service(struct tb_service *svc, struct tb_property *property)
+{
+	struct tb_property_dir *dir = property->value.dir;
+
+	guard(mutex)(&svc->lock);
+	tb_property_free_dir(svc->remote_properties);
+	svc->remote_properties = tb_property_copy_dir(dir);
+	kobject_uevent(&svc->dev.kobj, KOBJ_CHANGE);
+}
+
 static void __unregister_service(struct device *dev)
 {
 	struct tb_service *svc = tb_to_service(dev);
@@ -1109,6 +1183,12 @@ static int populate_service(struct tb_service *svc,
 	if (!svc->key)
 		return -ENOMEM;
 
+	svc->remote_properties = tb_property_copy_dir(dir);
+	if (!svc->remote_properties) {
+		kfree(svc->key);
+		return -ENOMEM;
+	}
+
 	return 0;
 }
 
@@ -1133,6 +1213,7 @@ static void enumerate_services(struct tb_xdomain *xd)
 		/* If the service exists already we are fine */
 		dev = device_find_child(&xd->dev, p, find_service);
 		if (dev) {
+			update_service(tb_to_service(dev), p);
 			put_device(dev);
 			continue;
 		}
@@ -1156,6 +1237,7 @@ static void enumerate_services(struct tb_xdomain *xd)
 		svc->dev.bus = &tb_bus_type;
 		svc->dev.type = &tb_service_type;
 		svc->dev.parent = get_device(&xd->dev);
+		mutex_init(&svc->lock);
 		dev_set_name(&svc->dev, "%s.%d", dev_name(&xd->dev), svc->id);
 
 		tb_service_debugfs_init(svc);
@@ -2549,19 +2631,6 @@ bool tb_xdomain_handle_request(struct tb *tb, enum tb_cfg_pkg_type type,
 	return ret > 0;
 }
 
-static int update_xdomain(struct device *dev, void *data)
-{
-	struct tb_xdomain *xd;
-
-	xd = tb_to_xdomain(dev);
-	if (xd) {
-		queue_delayed_work(xd->tb->wq, &xd->properties_changed_work,
-				   msecs_to_jiffies(50));
-	}
-
-	return 0;
-}
-
 static void update_all_xdomains(void)
 {
 	bus_for_each_dev(&tb_bus_type, NULL, NULL, update_xdomain);
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index e98d569779f9..f60e3a1aecae 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -397,6 +397,10 @@ void tb_unregister_protocol_handler(struct tb_protocol_handler *handler);
  * @prtcvers: Protocol version from the properties directory
  * @prtcrevs: Protocol software revision from the properties directory
  * @prtcstns: Protocol settings mask from the properties directory
+ * @lock: Protects this structure
+ * @local_properties: Properties owned by the service driver
+ * @remote_properties: Properties read from the remote service. These
+ *		       are read-only.
  * @debugfs_dir: Pointer to the service debugfs directory. Always created
  *		 when debugfs is enabled. Can be used by service drivers to
  *		 add their own entries under the service.
@@ -404,6 +408,9 @@ void tb_unregister_protocol_handler(struct tb_protocol_handler *handler);
  * Each domain exposes set of services it supports as collection of
  * properties. For each service there will be one corresponding
  * &struct tb_service. Service drivers are bound to these.
+ *
+ * Service drivers can add their own dynamic properties to
+ * @local_properties but whenever they do so @lock must be held.
  */
 struct tb_service {
 	struct device dev;
@@ -413,6 +420,9 @@ struct tb_service {
 	u32 prtcvers;
 	u32 prtcrevs;
 	u32 prtcstns;
+	struct mutex lock;
+	struct tb_property_dir *local_properties;
+	struct tb_property_dir *remote_properties;
 	struct dentry *debugfs_dir;
 };
 
@@ -481,6 +491,8 @@ static inline struct tb_xdomain *tb_service_parent(struct tb_service *svc)
 	return tb_to_xdomain(svc->dev.parent);
 }
 
+void tb_service_properties_changed(struct tb_service *svc);
+
 /**
  * struct tb_nhi - thunderbolt native host interface
  * @lock: Must be held during ring creation/destruction. Is acquired by
-- 
2.50.1

[PATCH 4/9] thunderbolt / net: Move ring_frame_size() to thunderbolt.h

[Mika Westerberg]

This function can be used outside of thunderbolt networking driver so
move it to the common header.

No functional changes.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/net/thunderbolt/main.c | 16 ++++++----------
 include/linux/thunderbolt.h    | 10 +++++++++-
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index d8fcf18fc55c..49673f7e0055 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -38,7 +38,7 @@
 #define TBNET_MATCH_FRAGS_ID	BIT(1)
 #define TBNET_64K_FRAMES	BIT(2)
 #define TBNET_MAX_MTU		SZ_64K
-#define TBNET_FRAME_SIZE	SZ_4K
+#define TBNET_FRAME_SIZE	TB_MAX_FRAME_SIZE
 #define TBNET_MAX_PAYLOAD_SIZE	\
 	(TBNET_FRAME_SIZE - sizeof(struct thunderbolt_ip_frame_header))
 /* Rx packets need to hold space for skb_shared_info */
@@ -327,11 +327,6 @@ static void stop_login(struct tbnet *net)
 	netdev_dbg(net->dev, "login stopped\n");
 }
 
-static inline unsigned int tbnet_frame_size(const struct tbnet_frame *tf)
-{
-	return tf->frame.size ? : TBNET_FRAME_SIZE;
-}
-
 static void tbnet_free_buffers(struct tbnet_ring *ring)
 {
 	unsigned int i;
@@ -561,7 +556,7 @@ static struct tbnet_frame *tbnet_get_tx_buffer(struct tbnet *net)
 	tf->frame.size = 0;
 
 	dma_sync_single_for_cpu(dma_dev, tf->frame.buffer_phy,
-				tbnet_frame_size(tf), DMA_TO_DEVICE);
+				tb_ring_frame_size(&tf->frame), DMA_TO_DEVICE);
 
 	return tf;
 }
@@ -743,7 +738,7 @@ static bool tbnet_check_frame(struct tbnet *net, const struct tbnet_frame *tf,
 	}
 
 	/* Should be greater than just header i.e. contains data */
-	size = tbnet_frame_size(tf);
+	size = tb_ring_frame_size(&tf->frame);
 	if (size <= sizeof(*hdr)) {
 		net->stats.rx_length_errors++;
 		return false;
@@ -1010,7 +1005,8 @@ static bool tbnet_xmit_csum_and_map(struct tbnet *net, struct sk_buff *skb,
 						hdr->frame_index, hdr->frame_count);
 			dma_sync_single_for_device(dma_dev,
 				frames[i]->frame.buffer_phy,
-				tbnet_frame_size(frames[i]), DMA_TO_DEVICE);
+				tb_ring_frame_size(&frames[i]->frame),
+						   DMA_TO_DEVICE);
 		}
 
 		return true;
@@ -1084,7 +1080,7 @@ static bool tbnet_xmit_csum_and_map(struct tbnet *net, struct sk_buff *skb,
 	 */
 	for (i = 0; i < frame_count; i++) {
 		dma_sync_single_for_device(dma_dev, frames[i]->frame.buffer_phy,
-			tbnet_frame_size(frames[i]), DMA_TO_DEVICE);
+			tb_ring_frame_size(&frames[i]->frame), DMA_TO_DEVICE);
 	}
 
 	return true;
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index f60e3a1aecae..1d1bd458b5af 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -628,7 +628,15 @@ struct ring_frame {
 };
 
 /* Minimum size for ring_rx */
-#define TB_FRAME_SIZE		0x100
+#define TB_FRAME_SIZE		256
+#define TB_MAX_FRAME_SIZE	4096
+
+static inline size_t tb_ring_frame_size(const struct ring_frame *frame)
+{
+	if (frame->size)
+		return frame->size;
+	return TB_MAX_FRAME_SIZE;
+}
 
 struct tb_ring *tb_ring_alloc_tx(struct tb_nhi *nhi, int hop, int size,
 				 unsigned int flags);
-- 
2.50.1

[PATCH 5/9] thunderbolt / net: Let the service drivers configure interrupt throttling

[Mika Westerberg]

Instead of the core driver programming fixed value for throttling let
the service drivers to specify the interval if they need this. We also
allow user to tune this through a module parameter if the default is not
good fit.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/net/thunderbolt/main.c |  7 ++++
 drivers/thunderbolt/dma_test.c |  5 +++
 drivers/thunderbolt/nhi.c      | 58 ++++++++++++++++++----------------
 drivers/thunderbolt/nhi_regs.h |  3 +-
 include/linux/thunderbolt.h    |  5 +++
 5 files changed, 50 insertions(+), 28 deletions(-)

diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index 49673f7e0055..8771ca807933 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -218,6 +218,10 @@ static bool tbnet_e2e = true;
 module_param_named(e2e, tbnet_e2e, bool, 0444);
 MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)");
 
+static unsigned int tbnet_throttling = 128000;
+module_param_named(throttling, tbnet_throttling, uint, 0444);
+MODULE_PARM_DESC(throttling, "Interrupt throttling rate in ns (default: 128000)");
+
 static void tbnet_fill_header(struct thunderbolt_ip_header *hdr, u64 route,
 	u8 sequence, const uuid_t *initiator_uuid, const uuid_t *target_uuid,
 	enum thunderbolt_ip_type type, size_t size, u32 command_id)
@@ -955,6 +959,9 @@ static int tbnet_open(struct net_device *dev)
 	}
 	net->rx_ring.ring = ring;
 
+	tb_ring_throttling(net->tx_ring.ring, tbnet_throttling);
+	tb_ring_throttling(net->rx_ring.ring, tbnet_throttling);
+
 	napi_enable(&net->napi);
 	start_login(net);
 
diff --git a/drivers/thunderbolt/dma_test.c b/drivers/thunderbolt/dma_test.c
index af1e6bc9c7cd..7877319b1b03 100644
--- a/drivers/thunderbolt/dma_test.c
+++ b/drivers/thunderbolt/dma_test.c
@@ -155,6 +155,8 @@ static int dma_test_start_rings(struct dma_test *dt)
 		dt->tx_ring = ring;
 		e2e_tx_hop = ring->hop;
 
+		tb_ring_throttling(ring, 128000);
+
 		ret = tb_xdomain_alloc_out_hopid(xd, -1);
 		if (ret < 0) {
 			dma_test_free_rings(dt);
@@ -162,6 +164,7 @@ static int dma_test_start_rings(struct dma_test *dt)
 		}
 
 		dt->tx_hopid = ret;
+
 	}
 
 	if (dt->packets_to_receive) {
@@ -180,6 +183,8 @@ static int dma_test_start_rings(struct dma_test *dt)
 
 		dt->rx_ring = ring;
 
+		tb_ring_throttling(ring, 128000);
+
 		ret = tb_xdomain_alloc_in_hopid(xd, -1);
 		if (ret < 0) {
 			dma_test_free_rings(dt);
diff --git a/drivers/thunderbolt/nhi.c b/drivers/thunderbolt/nhi.c
index 1a2051673067..13009246e617 100644
--- a/drivers/thunderbolt/nhi.c
+++ b/drivers/thunderbolt/nhi.c
@@ -93,7 +93,7 @@ static void ring_interrupt_active(struct tb_ring *ring, bool active)
 	u32 old, new;
 
 	if (ring->irq > 0) {
-		u32 step, shift, ivr, misc;
+		u32 step, shift, ivr, misc, itr;
 		void __iomem *ivr_base;
 		int auto_clear_bit;
 		int index;
@@ -131,6 +131,12 @@ static void ring_interrupt_active(struct tb_ring *ring, bool active)
 		if (active)
 			ivr |= ring->vector << shift;
 		iowrite32(ivr, ivr_base + step);
+
+		/* Throttling is specified in 256ns increments */
+		itr = DIV_ROUND_UP(ring->interval_nsec, 256);
+		itr &= REG_INT_THROTTLING_RATE_INTERVAL_MASK;
+		iowrite32(itr, ring->nhi->iobase + REG_INT_THROTTLING_RATE +
+			  ring->vector * 4);
 	}
 
 	old = ioread32(ring->nhi->iobase + reg);
@@ -854,6 +860,26 @@ void tb_ring_free(struct tb_ring *ring)
 }
 EXPORT_SYMBOL_GPL(tb_ring_free);
 
+/**
+ * tb_ring_throttling() - Configure throttling for ring interrupt
+ * @ring: Ring to configure
+ * @interval_nsec: Interval counter for moderation (in ns), %0 disables
+ *
+ * Enables or disables ring interrupt throttling. The ring must be
+ * stopped for this to be called. Granularity is 256 ns.
+ *
+ * Return: %0 on success, negative errno otherwise.
+ */
+int tb_ring_throttling(struct tb_ring *ring, unsigned int interval_nsec)
+{
+	guard(spinlock_irqsave)(&ring->lock);
+	if (WARN_ON_ONCE(ring->running))
+		return -EBUSY;
+	ring->interval_nsec = interval_nsec;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(tb_ring_throttling);
+
 /**
  * nhi_mailbox_cmd() - Send a command through NHI mailbox
  * @nhi: Pointer to the NHI structure
@@ -1035,22 +1061,6 @@ static int nhi_poweroff_noirq(struct device *dev)
 	return __nhi_suspend_noirq(dev, wakeup);
 }
 
-static void nhi_enable_int_throttling(struct tb_nhi *nhi)
-{
-	/* Throttling is specified in 256ns increments */
-	u32 throttle = DIV_ROUND_UP(128 * NSEC_PER_USEC, 256);
-	unsigned int i;
-
-	/*
-	 * Configure interrupt throttling for all vectors even if we
-	 * only use few.
-	 */
-	for (i = 0; i < MSIX_MAX_VECS; i++) {
-		u32 reg = REG_INT_THROTTLING_RATE + i * 4;
-		iowrite32(throttle, nhi->iobase + reg);
-	}
-}
-
 static int nhi_resume_noirq(struct device *dev)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
@@ -1065,13 +1075,10 @@ static int nhi_resume_noirq(struct device *dev)
 	 */
 	if (!pci_device_is_present(pdev)) {
 		nhi->going_away = true;
-	} else {
-		if (nhi->ops && nhi->ops->resume_noirq) {
-			ret = nhi->ops->resume_noirq(nhi);
-			if (ret)
-				return ret;
-		}
-		nhi_enable_int_throttling(tb->nhi);
+	} else if (nhi->ops && nhi->ops->resume_noirq) {
+		ret = nhi->ops->resume_noirq(nhi);
+		if (ret)
+			return ret;
 	}
 
 	return tb_domain_resume_noirq(tb);
@@ -1133,7 +1140,6 @@ static int nhi_runtime_resume(struct device *dev)
 			return ret;
 	}
 
-	nhi_enable_int_throttling(nhi);
 	return tb_domain_runtime_resume(tb);
 }
 
@@ -1271,8 +1277,6 @@ static int nhi_init_msi(struct tb_nhi *nhi)
 	/* In case someone left them on. */
 	nhi_disable_interrupts(nhi);
 
-	nhi_enable_int_throttling(nhi);
-
 	ida_init(&nhi->msix_ida);
 
 	/*
diff --git a/drivers/thunderbolt/nhi_regs.h b/drivers/thunderbolt/nhi_regs.h
index cf5222bee971..d6a197fabc74 100644
--- a/drivers/thunderbolt/nhi_regs.h
+++ b/drivers/thunderbolt/nhi_regs.h
@@ -101,7 +101,8 @@ struct ring_desc {
 
 #define REG_RING_INTERRUPT_MASK_CLEAR_BASE	0x38208
 
-#define REG_INT_THROTTLING_RATE	0x38c00
+#define REG_INT_THROTTLING_RATE			0x38c00
+#define REG_INT_THROTTLING_RATE_INTERVAL_MASK	GENMASK(15, 0)
 
 /* Interrupt Vector Allocation */
 #define REG_INT_VEC_ALLOC_BASE	0x38c40
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index 1d1bd458b5af..1160e0bf5c5b 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -554,6 +554,8 @@ struct tb_nhi {
  * @start_poll: Called when ring interrupt is triggered to start
  *		polling. Passing %NULL keeps the ring in interrupt mode.
  * @poll_data: Data passed to @start_poll
+ * @interval_nsec: Interval counter if interrupt throttling is to be
+ *		   used with this ring (in ns)
  */
 struct tb_ring {
 	spinlock_t lock;
@@ -577,6 +579,7 @@ struct tb_ring {
 	u16 eof_mask;
 	void (*start_poll)(void *data);
 	void *poll_data;
+	unsigned int interval_nsec;
 };
 
 /* Leave ring interrupt enabled on suspend */
@@ -697,6 +700,8 @@ static inline int tb_ring_tx(struct tb_ring *ring, struct ring_frame *frame)
 struct ring_frame *tb_ring_poll(struct tb_ring *ring);
 void tb_ring_poll_complete(struct tb_ring *ring);
 
+int tb_ring_throttling(struct tb_ring *ring, unsigned int interval_nsec);
+
 /**
  * tb_ring_dma_device() - Return device used for DMA mapping
  * @ring: Ring whose DMA device is retrieved
-- 
2.50.1

Re: [PATCH 5/9] thunderbolt / net: Let the service drivers configure interrupt throttling

[Andrew Lunn]

On Tue, Apr 28, 2026 at 09:22:05AM +0200, Mika Westerberg wrote:
> Instead of the core driver programming fixed value for throttling let
> the service drivers to specify the interval if they need this. We also
> allow user to tune this through a module parameter if the default is not
> good fit.
> 
> Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> ---
>  drivers/net/thunderbolt/main.c |  7 ++++
>  drivers/thunderbolt/dma_test.c |  5 +++
>  drivers/thunderbolt/nhi.c      | 58 ++++++++++++++++++----------------
>  drivers/thunderbolt/nhi_regs.h |  3 +-
>  include/linux/thunderbolt.h    |  5 +++
>  5 files changed, 50 insertions(+), 28 deletions(-)
> 
> diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
> index 49673f7e0055..8771ca807933 100644
> --- a/drivers/net/thunderbolt/main.c
> +++ b/drivers/net/thunderbolt/main.c
> @@ -218,6 +218,10 @@ static bool tbnet_e2e = true;
>  module_param_named(e2e, tbnet_e2e, bool, 0444);
>  MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)");
>  
> +static unsigned int tbnet_throttling = 128000;
> +module_param_named(throttling, tbnet_throttling, uint, 0444);
> +MODULE_PARM_DESC(throttling, "Interrupt throttling rate in ns (default: 128000)");

As i mentioned elsewhere, netdev does not allow module
parameters. They are hard to use, especially when you have lots of
instances of a device, or you need to set it on the kernel command
line because by the time the kernel has booted, it is too late, etc.
And they are undocumented, and every driver does it differently.

The correct way to do this for netdev it ethtool -C.

For something which is not a netdev, like your stream file, you have
more flexibility, but the same usability issues apply.

	Andrew

Re: [PATCH 5/9] thunderbolt / net: Let the service drivers configure interrupt throttling

[Mika Westerberg]

On Tue, Apr 28, 2026 at 04:59:58PM +0200, Andrew Lunn wrote:
> On Tue, Apr 28, 2026 at 09:22:05AM +0200, Mika Westerberg wrote:
> > Instead of the core driver programming fixed value for throttling let
> > the service drivers to specify the interval if they need this. We also
> > allow user to tune this through a module parameter if the default is not
> > good fit.
> > 
> > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> > ---
> >  drivers/net/thunderbolt/main.c |  7 ++++
> >  drivers/thunderbolt/dma_test.c |  5 +++
> >  drivers/thunderbolt/nhi.c      | 58 ++++++++++++++++++----------------
> >  drivers/thunderbolt/nhi_regs.h |  3 +-
> >  include/linux/thunderbolt.h    |  5 +++
> >  5 files changed, 50 insertions(+), 28 deletions(-)
> > 
> > diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
> > index 49673f7e0055..8771ca807933 100644
> > --- a/drivers/net/thunderbolt/main.c
> > +++ b/drivers/net/thunderbolt/main.c
> > @@ -218,6 +218,10 @@ static bool tbnet_e2e = true;
> >  module_param_named(e2e, tbnet_e2e, bool, 0444);
> >  MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)");
> >  
> > +static unsigned int tbnet_throttling = 128000;
> > +module_param_named(throttling, tbnet_throttling, uint, 0444);
> > +MODULE_PARM_DESC(throttling, "Interrupt throttling rate in ns (default: 128000)");
> 
> As i mentioned elsewhere, netdev does not allow module
> parameters. They are hard to use, especially when you have lots of
> instances of a device, or you need to set it on the kernel command
> line because by the time the kernel has booted, it is too late, etc.
> And they are undocumented, and every driver does it differently.

Right. This was meant to be a "chicken bit" or "escape hatch" not something
used in daily basis.

I can drop that part from the patch, no problem.

Re: [PATCH 5/9] thunderbolt / net: Let the service drivers configure interrupt throttling

[Greg KH]

On Tue, Apr 28, 2026 at 07:26:29PM +0200, Mika Westerberg wrote:
> On Tue, Apr 28, 2026 at 04:59:58PM +0200, Andrew Lunn wrote:
> > On Tue, Apr 28, 2026 at 09:22:05AM +0200, Mika Westerberg wrote:
> > > Instead of the core driver programming fixed value for throttling let
> > > the service drivers to specify the interval if they need this. We also
> > > allow user to tune this through a module parameter if the default is not
> > > good fit.
> > > 
> > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> > > ---
> > >  drivers/net/thunderbolt/main.c |  7 ++++
> > >  drivers/thunderbolt/dma_test.c |  5 +++
> > >  drivers/thunderbolt/nhi.c      | 58 ++++++++++++++++++----------------
> > >  drivers/thunderbolt/nhi_regs.h |  3 +-
> > >  include/linux/thunderbolt.h    |  5 +++
> > >  5 files changed, 50 insertions(+), 28 deletions(-)
> > > 
> > > diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
> > > index 49673f7e0055..8771ca807933 100644
> > > --- a/drivers/net/thunderbolt/main.c
> > > +++ b/drivers/net/thunderbolt/main.c
> > > @@ -218,6 +218,10 @@ static bool tbnet_e2e = true;
> > >  module_param_named(e2e, tbnet_e2e, bool, 0444);
> > >  MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)");
> > >  
> > > +static unsigned int tbnet_throttling = 128000;
> > > +module_param_named(throttling, tbnet_throttling, uint, 0444);
> > > +MODULE_PARM_DESC(throttling, "Interrupt throttling rate in ns (default: 128000)");
> > 
> > As i mentioned elsewhere, netdev does not allow module
> > parameters. They are hard to use, especially when you have lots of
> > instances of a device, or you need to set it on the kernel command
> > line because by the time the kernel has booted, it is too late, etc.
> > And they are undocumented, and every driver does it differently.
> 
> Right. This was meant to be a "chicken bit" or "escape hatch" not something
> used in daily basis.

Put it in configfs if you really want it.

You shouldn't need any new module parameters.

thanks,

greg k-h

Re: [PATCH 5/9] thunderbolt / net: Let the service drivers configure interrupt throttling

[Alan Stern]

On Tue, Apr 28, 2026 at 12:10:49PM -0600, Greg KH wrote:
> On Tue, Apr 28, 2026 at 07:26:29PM +0200, Mika Westerberg wrote:
> > On Tue, Apr 28, 2026 at 04:59:58PM +0200, Andrew Lunn wrote:
> > > On Tue, Apr 28, 2026 at 09:22:05AM +0200, Mika Westerberg wrote:
> > > > Instead of the core driver programming fixed value for throttling let
> > > > the service drivers to specify the interval if they need this. We also
> > > > allow user to tune this through a module parameter if the default is not
> > > > good fit.
> > > > 
> > > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> > > > ---
> > > >  drivers/net/thunderbolt/main.c |  7 ++++
> > > >  drivers/thunderbolt/dma_test.c |  5 +++
> > > >  drivers/thunderbolt/nhi.c      | 58 ++++++++++++++++++----------------
> > > >  drivers/thunderbolt/nhi_regs.h |  3 +-
> > > >  include/linux/thunderbolt.h    |  5 +++
> > > >  5 files changed, 50 insertions(+), 28 deletions(-)
> > > > 
> > > > diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
> > > > index 49673f7e0055..8771ca807933 100644
> > > > --- a/drivers/net/thunderbolt/main.c
> > > > +++ b/drivers/net/thunderbolt/main.c
> > > > @@ -218,6 +218,10 @@ static bool tbnet_e2e = true;
> > > >  module_param_named(e2e, tbnet_e2e, bool, 0444);
> > > >  MODULE_PARM_DESC(e2e, "USB4NET full end-to-end flow control (default: true)");
> > > >  
> > > > +static unsigned int tbnet_throttling = 128000;
> > > > +module_param_named(throttling, tbnet_throttling, uint, 0444);
> > > > +MODULE_PARM_DESC(throttling, "Interrupt throttling rate in ns (default: 128000)");
> > > 
> > > As i mentioned elsewhere, netdev does not allow module
> > > parameters. They are hard to use, especially when you have lots of
> > > instances of a device, or you need to set it on the kernel command
> > > line because by the time the kernel has booted, it is too late, etc.
> > > And they are undocumented, and every driver does it differently.
> > 
> > Right. This was meant to be a "chicken bit" or "escape hatch" not something
> > used in daily basis.
> 
> Put it in configfs if you really want it.

Or even debugfs, if that's appropriate for the use case.

Alan Stern

> You shouldn't need any new module parameters.
> 
> thanks,
> 
> greg k-h
> 

[PATCH 6/9] thunderbolt: Add helper to figure size of the ring

[Mika Westerberg]

Add to common header a function that returns size of the ring. This can
be used in the drivers instead of rolling own version.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 include/linux/thunderbolt.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index 1160e0bf5c5b..9df8a356396f 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -641,6 +641,11 @@ static inline size_t tb_ring_frame_size(const struct ring_frame *frame)
 	return TB_MAX_FRAME_SIZE;
 }
 
+static inline size_t tb_ring_size(const struct tb_ring *ring)
+{
+	return ring->size;
+}
+
 struct tb_ring *tb_ring_alloc_tx(struct tb_nhi *nhi, int hop, int size,
 				 unsigned int flags);
 struct tb_ring *tb_ring_alloc_rx(struct tb_nhi *nhi, int hop, int size,
-- 
2.50.1

[PATCH 7/9] thunderbolt: Add tb_ring_flush()

[Mika Westerberg]

This allows the caller to wait for the ring to be empty. We are going to
need this in the upcoming userspace tunneling support.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/thunderbolt/nhi.c   | 28 ++++++++++++++++++++++++++++
 include/linux/thunderbolt.h |  3 +++
 2 files changed, 31 insertions(+)

diff --git a/drivers/thunderbolt/nhi.c b/drivers/thunderbolt/nhi.c
index 13009246e617..a0a789bfb680 100644
--- a/drivers/thunderbolt/nhi.c
+++ b/drivers/thunderbolt/nhi.c
@@ -325,6 +325,8 @@ static void ring_work(struct work_struct *work)
 		if (frame->callback)
 			frame->callback(ring, frame, canceled);
 	}
+
+	wake_up(&ring->wait);
 }
 
 int __tb_ring_enqueue(struct tb_ring *ring, struct ring_frame *frame)
@@ -601,6 +603,7 @@ static struct tb_ring *tb_ring_alloc(struct tb_nhi *nhi, u32 hop, int size,
 	INIT_LIST_HEAD(&ring->queue);
 	INIT_LIST_HEAD(&ring->in_flight);
 	INIT_WORK(&ring->work, ring_work);
+	init_waitqueue_head(&ring->wait);
 
 	ring->nhi = nhi;
 	ring->hop = hop;
@@ -760,6 +763,31 @@ void tb_ring_start(struct tb_ring *ring)
 }
 EXPORT_SYMBOL_GPL(tb_ring_start);
 
+static bool tb_ring_empty(struct tb_ring *ring)
+{
+	guard(spinlock_irqsave)(&ring->lock);
+	return list_empty(&ring->in_flight);
+}
+
+/**
+ * tb_ring_flush() - Waits for a ring to be empty
+ * @ring: Ring to wait
+ * @timeout_msec: Timeout in ms how long to wait.
+ *
+ * This can be called before stopping a ring to make sure all the frames
+ * submitted prior have been completed.
+ *
+ * Return: %true if the ring is empty now, %false otherwise.
+ */
+bool tb_ring_flush(struct tb_ring *ring, unsigned int timeout_msec)
+{
+	if (!wait_event_timeout(ring->wait, tb_ring_empty(ring),
+				msecs_to_jiffies(timeout_msec)))
+		return false;
+	return tb_ring_empty(ring);
+}
+EXPORT_SYMBOL_GPL(tb_ring_flush);
+
 /**
  * tb_ring_stop() - shutdown a ring
  * @ring: Ring to stop
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index 9df8a356396f..9c5cb5e4f23d 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -556,6 +556,7 @@ struct tb_nhi {
  * @poll_data: Data passed to @start_poll
  * @interval_nsec: Interval counter if interrupt throttling is to be
  *		   used with this ring (in ns)
+ * @wait: Used to signal that the ring may be empty now
  */
 struct tb_ring {
 	spinlock_t lock;
@@ -580,6 +581,7 @@ struct tb_ring {
 	void (*start_poll)(void *data);
 	void *poll_data;
 	unsigned int interval_nsec;
+	wait_queue_head_t wait;
 };
 
 /* Leave ring interrupt enabled on suspend */
@@ -653,6 +655,7 @@ struct tb_ring *tb_ring_alloc_rx(struct tb_nhi *nhi, int hop, int size,
 				 u16 sof_mask, u16 eof_mask,
 				 void (*start_poll)(void *), void *poll_data);
 void tb_ring_start(struct tb_ring *ring);
+bool tb_ring_flush(struct tb_ring *ring, unsigned int timeout_msec);
 void tb_ring_stop(struct tb_ring *ring);
 void tb_ring_free(struct tb_ring *ring);
 
-- 
2.50.1

[PATCH 8/9] thunderbolt: Add support for ConfigFS

[Mika Westerberg]

This adds ConfigFS support to USB4/Thunderbolt bus. By itself this just
creates the subsystem but it exposes functions that can be used to
register groups under it.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 drivers/thunderbolt/Kconfig    |  4 +++
 drivers/thunderbolt/Makefile   |  1 +
 drivers/thunderbolt/configfs.c | 61 ++++++++++++++++++++++++++++++++++
 drivers/thunderbolt/domain.c   |  2 ++
 drivers/thunderbolt/tb.h       |  8 +++++
 include/linux/thunderbolt.h    |  6 ++++
 6 files changed, 82 insertions(+)
 create mode 100644 drivers/thunderbolt/configfs.c

diff --git a/drivers/thunderbolt/Kconfig b/drivers/thunderbolt/Kconfig
index db3b0bef48f4..9b4aaa456e32 100644
--- a/drivers/thunderbolt/Kconfig
+++ b/drivers/thunderbolt/Kconfig
@@ -18,6 +18,10 @@ menuconfig USB4
 
 if USB4
 
+config USB4_CONFIGFS
+	def_tristate USB4
+	depends on CONFIGFS_FS && !(USB4=y && CONFIGFS_FS=m)
+
 config USB4_DEBUGFS_WRITE
 	bool "Enable write by debugfs to configuration spaces (DANGEROUS)"
 	help
diff --git a/drivers/thunderbolt/Makefile b/drivers/thunderbolt/Makefile
index b44b32dcb832..d5b367dfda1e 100644
--- a/drivers/thunderbolt/Makefile
+++ b/drivers/thunderbolt/Makefile
@@ -7,6 +7,7 @@ thunderbolt-objs += usb4_port.o nvm.o retimer.o quirks.o clx.o
 
 thunderbolt-${CONFIG_ACPI} += acpi.o
 thunderbolt-$(CONFIG_DEBUG_FS) += debugfs.o
+thunderbolt-$(CONFIG_USB4_CONFIGFS) += configfs.o
 thunderbolt-${CONFIG_USB4_KUNIT_TEST} += test.o
 CFLAGS_test.o += $(DISABLE_STRUCTLEAK_PLUGIN)
 
diff --git a/drivers/thunderbolt/configfs.c b/drivers/thunderbolt/configfs.c
new file mode 100644
index 000000000000..dc6bc363dfe8
--- /dev/null
+++ b/drivers/thunderbolt/configfs.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ConfigFS support
+ *
+ * Copyright (C) 2026, Intel Corporation
+ * Author: Mika Westerberg <mika.westerberg@linux.intel.com>
+ */
+
+#include <linux/configfs.h>
+#include <linux/export.h>
+
+#include "tb.h"
+
+static const struct config_item_type tb_root_group_type = {
+	.ct_owner = THIS_MODULE,
+};
+
+static struct configfs_subsystem tb_configfs = {
+	.su_group = {
+		.cg_item = {
+			.ci_namebuf = "thunderbolt",
+			.ci_type = &tb_root_group_type,
+		},
+	},
+};
+
+/**
+ * tb_configfs_register_group() - Register Thunderbolt ConfigFS group
+ * @group: Group to register.
+ *
+ * Registers the new @group under Thunderbolt subsystem ConfigFS.
+ *
+ * Return: 0% in case of success, negative errno otherwise.
+ */
+int tb_configfs_register_group(struct config_group *group)
+{
+	return configfs_register_group(&tb_configfs.su_group, group);
+}
+EXPORT_SYMBOL_GPL(tb_configfs_register_group);
+
+/**
+ * tb_configfs_unregister_group() - Unregister previously registered group
+ * @group: Group to unregister.
+ */
+void tb_configfs_unregister_group(struct config_group *group)
+{
+	configfs_unregister_group(group);
+}
+EXPORT_SYMBOL_GPL(tb_configfs_unregister_group);
+
+int tb_configfs_init(void)
+{
+	config_group_init(&tb_configfs.su_group);
+	mutex_init(&tb_configfs.su_mutex);
+	return configfs_register_subsystem(&tb_configfs);
+}
+
+void tb_configfs_exit(void)
+{
+	configfs_unregister_subsystem(&tb_configfs);
+}
diff --git a/drivers/thunderbolt/domain.c b/drivers/thunderbolt/domain.c
index d83719a37b4c..b381f184340e 100644
--- a/drivers/thunderbolt/domain.c
+++ b/drivers/thunderbolt/domain.c
@@ -887,6 +887,7 @@ int tb_domain_init(void)
 {
 	int ret;
 
+	tb_configfs_init();
 	tb_debugfs_init();
 	tb_acpi_init();
 
@@ -916,4 +917,5 @@ void tb_domain_exit(void)
 	tb_xdomain_exit();
 	tb_acpi_exit();
 	tb_debugfs_exit();
+	tb_configfs_exit();
 }
diff --git a/drivers/thunderbolt/tb.h b/drivers/thunderbolt/tb.h
index 229b9e7961fb..e60f1bc3764e 100644
--- a/drivers/thunderbolt/tb.h
+++ b/drivers/thunderbolt/tb.h
@@ -1558,4 +1558,12 @@ static inline void tb_retimer_debugfs_init(struct tb_retimer *rt) { }
 static inline void tb_retimer_debugfs_remove(struct tb_retimer *rt) { }
 #endif
 
+#if IS_REACHABLE(CONFIG_CONFIGFS_FS)
+int tb_configfs_init(void);
+void tb_configfs_exit(void);
+#else
+static inline int tb_configfs_init(void) { return 0; }
+static inline void tb_configfs_exit(void) { }
+#endif
+
 #endif
diff --git a/include/linux/thunderbolt.h b/include/linux/thunderbolt.h
index 9c5cb5e4f23d..0be9b298e692 100644
--- a/include/linux/thunderbolt.h
+++ b/include/linux/thunderbolt.h
@@ -13,6 +13,7 @@
 
 #include <linux/types.h>
 
+struct config_group;
 struct fwnode_handle;
 struct device;
 
@@ -727,6 +728,11 @@ static inline struct device *tb_ring_dma_device(struct tb_ring *ring)
 bool usb4_usb3_port_match(struct device *usb4_port_dev,
 			  const struct fwnode_handle *usb3_port_fwnode);
 
+#if IS_REACHABLE(CONFIG_CONFIGFS_FS)
+int tb_configfs_register_group(struct config_group *group);
+void tb_configfs_unregister_group(struct config_group *group);
+#endif
+
 #else /* CONFIG_USB4 */
 static inline bool usb4_usb3_port_match(struct device *usb4_port_dev,
 					const struct fwnode_handle *usb3_port_fwnode)
-- 
2.50.1

[PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Mika Westerberg]

Introduce USB4STREAM protocol and Linux implementation. This allows two
(or more) hosts to transfer data directly over Thunderbolt/USB4 cable
through a character device without need to go through the network stack.

Any application that supports read(2) and write(2) in some form should
be able to use the device without changes. The data is sent out to the
other side over a tunnel inside Thunderbolt/USB4 fabric. The character
device is called /dev/tbstreamX where X is the minor number starting
from 0.

All stream devices need to be configured first. This is done through
ConfigFS interface. There can be multiple streams at the same time (this
depends on number of DMA rings and available HopIDs) and a single stream
supports traffic in both directions. For example there could be an
application that uses one stream as control channel and another one as
bi-directional data channel.

A real use-case for this is to take a backup as a part of recovery
initramfs tooling (no need to setup networking or have ssh or similar
tooling as part of the initramfs). Say we want to backup the disk of
host1 to host2. First Thunderbolt/USB4 cable is connected between the
hosts (there can be devices in the middle too) then the receiving side
configures the stream:

  host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
  host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
  host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
  host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid

We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
From this point forward the /dev/tbstream0 can be used pretty much as
regular file:

  host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k

The host that is being backed up then configures the stream accordingly:

  host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
  host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup

Here we take advantage of the fact that host2 also announces the active
streams through XDomain properties so the name "backup" gives us the
HopIDs. It is also possible to configure them manually in the same way
we did for host2.

Then it is just a matter of copying the data over:

  host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k

Similarly it is possible to transfer parts of the filesystem. For
example copy contents of mydir over to the host2:

  host2 # gunzip < /dev/tbstream0 | tar xf -
  host1 # tar cf - mydir | gzip > /dev/tbstream0

Other end of the spectrum use-case is "borrowing" laptop (host1) camera
to desktop (host2):

  host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
                         autovideosink

  host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
                         jpegenc quality=90 ! filesink location=/dev/tbstream0

Once the streams are no longer needed they can be removed:

  host1 # cd /sys/kernel/config/thunderbolt/stream/
  host1 # rmdir -p 0-503.0/backup

  host2 # cd /sys/kernel/config/thunderbolt/stream
  host2 # rmdir -p 0-1.0/backup

Co-developed-by: Alan Borzeszkowski <alan.borzeszkowski@linux.intel.com>
Signed-off-by: Alan Borzeszkowski <alan.borzeszkowski@linux.intel.com>
Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
---
 .../ABI/testing/configfs-thunderbolt_stream   |   77 +
 drivers/thunderbolt/Kconfig                   |   11 +
 drivers/thunderbolt/Makefile                  |    3 +
 drivers/thunderbolt/stream.c                  | 1693 +++++++++++++++++
 4 files changed, 1784 insertions(+)
 create mode 100644 Documentation/ABI/testing/configfs-thunderbolt_stream
 create mode 100644 drivers/thunderbolt/stream.c

diff --git a/Documentation/ABI/testing/configfs-thunderbolt_stream b/Documentation/ABI/testing/configfs-thunderbolt_stream
new file mode 100644
index 000000000000..e403fda92765
--- /dev/null
+++ b/Documentation/ABI/testing/configfs-thunderbolt_stream
@@ -0,0 +1,77 @@
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		Configuration group for a stream Thunderbolt/USB4
+		service. It is possible to create groups even if there
+		is no connection yet to the other host. Once a
+		connection established and there is stream service on
+		the remote side that matches, this configuration is
+		applied to it.
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		Creates new stream with $name and fills it with the
+		default values. If there is an advertised remote stream
+		with the same name, uses its values as the default.
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/index
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		This matches the X in /dev/tbstreamX and allows userspace
+		to map the configfs directory to the corresponding character
+		device.
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/in_hopid
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		In HopID used with the read path of the tunnel. Available HopIDs
+		for tunneling start from 8. You can pass also -1 for automatic
+		allocation. The allocated value can be read here. Writing 0 will
+		de-allocate if the stream is not in use.
+
+		To figure out the maximum HopID you can run tbget from
+		tbtools [1] for the lane adapter. For example below we check
+		for lane adapter number 1 (first USB4 port):
+
+		  # tbget -r 0 -a 1 -D ADP_CS_5.Max\ Input\ HopID
+		  19
+
+		This allows to use anything between 8 and 19 inclusive.
+
+		[1] https://github.com/intel/tbtools
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/out_hopid
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		Out HopID used with the write path of the tunnel. Available HopIDs
+		for tunneling start from 8. You can pass also -1 for automatic
+		allocation. The allocated value can be read here. Writing 0 will
+		de-allocate if the stream is not in use. See @in_hopid
+		for how to figure out the maximum HopID.
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/ring_size
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		Size of the TX/RX rings. Can be adjusted between 32 and
+		4096. The default is 256.
+
+What:		/sys/kernel/config/thunderbolt/stream/<xdomain>.<service>/$name/throttling
+Date:		Sep 2026
+KernelVersion:	v7.2
+Contact:	Mika Westerberg <mika.westerberg@linux.intel.com>
+Description:
+		Interrupt throttling rate in ns. Lower values can give
+		better latency. The default is 8192 ns.
diff --git a/drivers/thunderbolt/Kconfig b/drivers/thunderbolt/Kconfig
index 9b4aaa456e32..294b3227a545 100644
--- a/drivers/thunderbolt/Kconfig
+++ b/drivers/thunderbolt/Kconfig
@@ -64,4 +64,15 @@ config USB4_DMA_TEST
 	  To compile this driver a module, choose M here. The module will be
 	  called thunderbolt_dma_test.
 
+config USB4_STREAM
+	tristate "Stream data over Thunderbolt/USB4 cable"
+	depends on USB4_CONFIGFS
+	help
+	  This adds support for USB4STREAM protocol that allows two
+	  hosts to stream data directly over Thunderbolt/USB4 cable
+	  through /dev/tbstreamX devices.
+
+	  To compile this driver a module, choose M here. The module will be
+	  called thunderbolt_stream.
+
 endif # USB4
diff --git a/drivers/thunderbolt/Makefile b/drivers/thunderbolt/Makefile
index d5b367dfda1e..c792b8084ffa 100644
--- a/drivers/thunderbolt/Makefile
+++ b/drivers/thunderbolt/Makefile
@@ -13,3 +13,6 @@ CFLAGS_test.o += $(DISABLE_STRUCTLEAK_PLUGIN)
 
 thunderbolt_dma_test-${CONFIG_USB4_DMA_TEST} += dma_test.o
 obj-$(CONFIG_USB4_DMA_TEST) += thunderbolt_dma_test.o
+
+thunderbolt_stream-${CONFIG_USB4_STREAM} += stream.o
+obj-$(CONFIG_USB4_STREAM) += thunderbolt_stream.o
diff --git a/drivers/thunderbolt/stream.c b/drivers/thunderbolt/stream.c
new file mode 100644
index 000000000000..684ab4e080c2
--- /dev/null
+++ b/drivers/thunderbolt/stream.c
@@ -0,0 +1,1693 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Stream data over Thunderbolt/USB4 cable
+ *
+ * Copyright (C) 2026, Intel Corporation
+ * Authors: Alan Borzeszkowski <alan.borzeszkowski@linux.intel.com>
+ *	    Mika Westerberg <mika.westerberg@linux.intel.com>
+ */
+
+#include <linux/cdev.h>
+#include <linux/configfs.h>
+#include <linux/device/class.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/idr.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/poll.h>
+#include <linux/sizes.h>
+#include <linux/thunderbolt.h>
+#include <linux/uaccess.h>
+#include <linux/uio.h>
+#include <linux/uuid.h>
+#include <linux/wait.h>
+
+/*
+ * USB4STREAM - Stream data directly over Thunderbolt/USB4 cable
+ *
+ * HopIDs are configured by the user. In Linux this is done through
+ * ConfigFS. Once that is done paths are be established the first time
+ * the stream is opened. Typically the read side is opened first to make
+ * sure all the data will be received.
+ *
+ * End-to-end flow control is mandatory on both sides.
+ *
+ * Data is sent to the other side as tunneled DATA packets. All the data
+ * is owned by the user and passed as-is from the writer to the reader.
+ *
+ * Once the stream device is closed, a CLOSE packet is sent to the peer
+ * so it can take the necessary action. On Linux this typically results
+ * in EOF being returned to the reader.
+ *
+ * Tunneled packet types:
+ *
+ * +-------+---------+------------------+
+ * |  PDF  |  Type   | Payload size     |
+ * +-------+---------+------------------+
+ * |   2   | DATA    | up to 4 KiB      |
+ * |   3   | CLOSE   | up to 256 bytes  |
+ * +-------+---------+------------------+
+ *
+ * Each stream can optionally publish configuration values under its own
+ * XDomain property directory. The name of the directory is the name of
+ * the stream in question and the UUID is up to the stream. For example
+ * if the stream exposes video output then the directory name could be
+ * "video".
+ *
+ * Below values are reserved and can be used by the stream:
+ *
+ * +----------+-----------+-------------------------+
+ * |   Key    |   Type    | Contents                |
+ * +----------+-----------+-------------------------+
+ * | inhopid  | IMMEDIATE | Configured input HopID  |
+ * | outhopid | IMMEDIATE | Configured output HopID |
+ * +----------+-----------+-------------------------+
+ *
+ * It is allowed to add more stream specific properties as well if the
+ * above are not enough.
+ */
+
+#define TBSTREAM_DEV_MINORS		(MINORMASK + 1)
+#define TBSTREAM_DEV_RING_SIZE		256
+#define TBSTREAM_DEV_MIN_RING_SIZE	32
+#define TBSTREAM_DEV_MAX_RING_SIZE	4096
+#define TBSTREAM_DEV_THROTTLING		8192
+#define TBSTREAM_DEV_MAX_THROTTLING	16776960
+
+/**
+ * enum tbstream_frame_pdf - PDF numbers for tunneled frames
+ * @TBSTREAM_FRAME_START: PDF of the start of the frame
+ * @TBSTREAM_DATA: PDF of the DATA frame
+ * @TBSTREAM_CLOSE: PDF of the CLOSE frame
+ */
+enum tbstream_frame_pdf {
+	TBSTREAM_FRAME_START = 1,
+	TBSTREAM_DATA,
+	TBSTREAM_CLOSE,
+};
+
+/**
+ * struct tbstream_frame - Frame submitted to/from the rings
+ * @sdev: Pointer to the stream device
+ * @page: Page holding the packet
+ * @offset: Offset inside @page if partial read is done
+ * @completed: %true if the RX frame is completed
+ * @frame: Underlying frame structure
+ */
+struct tbstream_frame {
+	struct tbstream_dev *sdev;
+	struct page *page;
+	unsigned int offset;
+	bool completed;
+	struct ring_frame frame;
+};
+
+/**
+ * struct tbstream_ring - Stream RX/TX ring structure
+ * @ring: Pointer to the API ring
+ * @prod: Current value of producer
+ * @cons: Current value of consumer
+ * @frames: Holds the ring frames
+ */
+struct tbstream_ring {
+	struct tb_ring *ring;
+	unsigned long prod;
+	unsigned long cons;
+	struct tbstream_frame *frames;
+};
+
+/**
+ * struct tbstream_dev - Stream character device
+ * @group: ConfigFS group for this device
+ * @stream: Pointer to the stream if it is attached (%NULL otherwise)
+ * @cdev: Character device used for tunneling
+ * @dev: Stream device
+ * @index: Unique identifier for the character device
+ * @in_hopid: In HopID
+ * @out_hopid: Out HopID
+ * @ring_size: Size of the rings
+ * @throttling: Interrupt throttling rate in ns
+ * @users: Number of times @cdev has been opened
+ * @closed: CLOSE packet was received
+ * @wait: Waitqueue for open, read and write
+ * @lock: Lock protecting this structure
+ * @tx_ring: Transmit ring
+ * @rx_ring: Receive ring
+ * @list: Stream devices are linked through this
+ */
+struct tbstream_dev {
+	struct config_group group;
+	struct tbstream *stream;
+	struct cdev cdev;
+	struct device dev;
+	int index;
+	int in_hopid;
+	int out_hopid;
+	unsigned int ring_size;
+	unsigned int throttling;
+	int users;
+	bool closed;
+	wait_queue_head_t wait;
+	struct mutex lock;
+	struct tbstream_ring tx_ring;
+	struct tbstream_ring rx_ring;
+	struct list_head list;
+};
+
+/**
+ * struct tbstream_group - Config group for stream
+ * @group: ConfigFS group for @stream
+ * @stream: Stream the ConfigFS group is attached to. %NULL if there is
+ *	    no stream attached.
+ * @lock: Lock protecting this structure
+ * @dev_list: List of stream devices
+ *
+ * This is the ConfigFS directory for one connection to another host.
+ * There can be several &struct stream_dev linked through @dev_list of
+ * this structure.
+ */
+struct tbstream_group {
+	struct config_group group;
+	struct tbstream *stream;
+	struct mutex lock;
+	struct list_head dev_list;
+};
+
+/**
+ * struct tbstream - Stream service private data
+ * @kref: Reference count
+ * @svc: Pointer to the service device
+ * @list: Streams are linked through this in @stream_list
+ *
+ * This represents the actual physical connection between two domains.
+ */
+struct tbstream {
+	struct kref kref;
+	struct tb_service *svc;
+	struct list_head list;
+};
+
+/* Major and minor numbers of the stream devices (/dev/tbstreamX) */
+static dev_t tbstream_devt;
+static DEFINE_IDA(tbstream_minors);
+
+/* Protects tbstream_list */
+static DEFINE_MUTEX(tbstream_lock);
+static LIST_HEAD(tbstream_list);
+
+/* Stream property directory UUID: 3a1cb984-c4d9-4469-a277-ce2fdfd11f0d */
+static const uuid_t tbstream_dir_uuid =
+	UUID_INIT(0x3a1cb984, 0xc4d9, 0x4469,
+		  0xa2, 0x77, 0xce, 0x2f, 0xdf, 0xd1, 0x1f, 0x0d);
+
+static struct tb_property_dir *tbstream_dir;
+
+static const struct class tbstream_class = {
+	.name = "thunderbolt_stream",
+};
+
+static void tbstream_release(struct kref *kref)
+{
+	struct tbstream *stream = container_of(kref, typeof(*stream), kref);
+
+	tb_service_put(stream->svc);
+	kfree(stream);
+}
+
+static void tbstream_put(struct tbstream *stream)
+{
+	if (stream)
+		kref_put(&stream->kref, tbstream_release);
+}
+
+static struct tbstream *tbstream_get(struct tbstream *stream)
+{
+	if (stream)
+		kref_get(&stream->kref);
+	return stream;
+}
+
+static inline bool tbstream_valid(const struct tbstream *stream)
+{
+	if (!stream)
+		return false;
+	return !tb_service_parent(stream->svc)->is_unplugged;
+}
+
+static void tbstream_ring_free(struct tbstream_ring *ring)
+{
+	struct device *dma_dev = tb_ring_dma_device(ring->ring);
+	enum dma_data_direction dir;
+	int i;
+
+	if (ring->ring->is_tx)
+		dir = DMA_TO_DEVICE;
+	else
+		dir = DMA_FROM_DEVICE;
+
+	for (i = 0; i < tb_ring_size(ring->ring); i++) {
+		struct tbstream_frame *sf = &ring->frames[i];
+
+		if (sf->frame.buffer_phy)
+			dma_unmap_page(dma_dev, sf->frame.buffer_phy,
+				       tb_ring_frame_size(&sf->frame), dir);
+		sf->frame.buffer_phy = 0;
+		if (sf->page)
+			__free_page(sf->page);
+		sf->page = NULL;
+	}
+
+	ring->prod = 0;
+	ring->cons = 0;
+	kfree(ring->frames);
+}
+
+static inline bool tbstream_ring_available(const struct tbstream_ring *ring)
+{
+	return ring->prod > ring->cons;
+}
+
+static inline struct tbstream_dev *tbstream_dev_get(struct tbstream_dev *sdev)
+{
+	get_device(&sdev->dev);
+	return sdev;
+}
+
+static inline void tbstream_dev_put(struct tbstream_dev *sdev)
+{
+	put_device(&sdev->dev);
+}
+
+static inline struct tbstream_dev *to_tbstream_dev(struct device *dev)
+{
+	return container_of(dev, struct tbstream_dev, dev);
+}
+
+static inline struct tb_xdomain *tbstream_dev_xdomain(struct tbstream_dev *sdev)
+{
+	if (sdev->stream)
+		return tb_service_parent(sdev->stream->svc);
+	return NULL;
+}
+
+static inline int tbstream_dev_valid(const struct tbstream_dev *sdev)
+{
+	const struct tbstream *stream = sdev->stream;
+
+	if (!tbstream_valid(stream))
+		return -ENXIO;
+	if (sdev->in_hopid <= 0 || sdev->out_hopid <= 0)
+		return -EINVAL;
+	return 0;
+}
+
+static inline bool tbstream_dev_closed(const struct tbstream_dev *sdev)
+{
+	return sdev->closed;
+}
+
+static void tbstream_dev_release(struct device *dev)
+{
+	struct tbstream_dev *sdev = to_tbstream_dev(dev);
+
+	if (sdev->stream) {
+		struct tb_xdomain *xd = tbstream_dev_xdomain(sdev);
+
+		if (sdev->out_hopid > 0)
+			tb_xdomain_release_out_hopid(xd, sdev->out_hopid);
+		if (sdev->in_hopid > 0)
+			tb_xdomain_release_in_hopid(xd, sdev->in_hopid);
+
+		tbstream_put(sdev->stream);
+	}
+	ida_free(&tbstream_minors, sdev->index);
+	kfree(sdev);
+}
+
+static void
+tbstream_dev_rx_callback(struct tb_ring *ring, struct ring_frame *frame,
+			 bool canceled)
+{
+	struct tbstream_frame *sf = container_of(frame, typeof(*sf), frame);
+	struct tbstream_dev *sdev = sf->sdev;
+
+	if (canceled)
+		return;
+
+	sf->completed = true;
+	sdev->rx_ring.prod++;
+
+	if (sf->frame.flags & RING_DESC_CRC_ERROR)
+		dev_warn(&sdev->dev, "RX CRC error\n");
+	else if (sf->frame.flags & RING_DESC_BUFFER_OVERRUN)
+		dev_warn(&sdev->dev, "RX buffer overrun\n");
+	else
+		wake_up_interruptible_poll(&sdev->wait, EPOLLIN | EPOLLRDNORM);
+}
+
+static struct tbstream_frame *
+tbstream_dev_completed_rx(struct tbstream_dev *sdev)
+{
+	struct device *dma_dev = tb_ring_dma_device(sdev->rx_ring.ring);
+	struct tbstream_frame *sf;
+	int index;
+
+	index = sdev->rx_ring.cons % tb_ring_size(sdev->rx_ring.ring);
+	sf = &sdev->rx_ring.frames[index];
+	if (!sf->completed)
+		return NULL;
+
+	dma_sync_single_for_cpu(dma_dev, sf->frame.buffer_phy,
+				tb_ring_frame_size(&sf->frame),
+				DMA_FROM_DEVICE);
+	return sf;
+}
+
+static int tbstream_dev_consume_rx(struct tbstream_dev *sdev)
+{
+	struct device *dma_dev = tb_ring_dma_device(sdev->rx_ring.ring);
+	struct tbstream_frame *sf;
+	int index;
+
+	index = sdev->rx_ring.cons % tb_ring_size(sdev->rx_ring.ring);
+	sdev->rx_ring.cons++;
+
+	sf = &sdev->rx_ring.frames[index];
+	sf->completed = false;
+	sf->offset = 0;
+	sf->frame.size = 0;
+
+	dma_sync_single_for_device(dma_dev, sf->frame.buffer_phy,
+				   tb_ring_frame_size(&sf->frame),
+				   DMA_FROM_DEVICE);
+
+	return tb_ring_rx(sdev->rx_ring.ring, &sf->frame);
+}
+
+static int tbstream_dev_alloc_rx_buffers(struct tbstream_dev *sdev)
+{
+	size_t ring_size = tb_ring_size(sdev->rx_ring.ring);
+	int i;
+
+	sdev->rx_ring.frames = kcalloc(ring_size, sizeof(struct tbstream_frame),
+				       GFP_KERNEL);
+	if (!sdev->rx_ring.frames)
+		return -ENOMEM;
+
+	for (i = 0; i < ring_size; i++) {
+		struct device *dma_dev = tb_ring_dma_device(sdev->rx_ring.ring);
+		struct tbstream_frame *sf = &sdev->rx_ring.frames[i];
+		dma_addr_t dma_addr;
+
+		sf->page = alloc_page(GFP_KERNEL);
+		if (!sf->page)
+			return -ENOMEM;
+
+		dma_addr = dma_map_page(dma_dev, sf->page, 0, TB_MAX_FRAME_SIZE,
+					DMA_FROM_DEVICE);
+		if (dma_mapping_error(dma_dev, dma_addr)) {
+			__free_page(sf->page);
+			sf->page = NULL;
+			return -ENOMEM;
+		}
+
+		sf->sdev = sdev;
+		sf->frame.callback = tbstream_dev_rx_callback;
+		sf->frame.buffer_phy = dma_addr;
+
+		tb_ring_rx(sdev->rx_ring.ring, &sf->frame);
+	}
+
+	sdev->rx_ring.cons = 0;
+	sdev->rx_ring.prod = 0;
+	return 0;
+}
+
+static void
+tbstream_dev_tx_callback(struct tb_ring *ring, struct ring_frame *frame,
+			 bool canceled)
+{
+	struct tbstream_frame *sf = container_of(frame, typeof(*sf), frame);
+	struct tbstream_dev *sdev = sf->sdev;
+
+	if (canceled)
+		return;
+
+	sdev->tx_ring.prod++;
+	if (sf->frame.eof == TBSTREAM_DATA)
+		wake_up_interruptible_poll(&sdev->wait, EPOLLOUT | EPOLLWRNORM);
+}
+
+static int tbstream_dev_alloc_tx_buffers(struct tbstream_dev *sdev)
+{
+	struct device *dma_dev = tb_ring_dma_device(sdev->tx_ring.ring);
+	size_t ring_size = tb_ring_size(sdev->tx_ring.ring);
+	int i;
+
+	sdev->tx_ring.frames = kcalloc(ring_size, sizeof(struct tbstream_frame),
+				       GFP_KERNEL);
+	if (!sdev->tx_ring.frames)
+		return -ENOMEM;
+
+	for (i = 0; i < ring_size; i++) {
+		struct tbstream_frame *sf = &sdev->tx_ring.frames[i];
+		dma_addr_t dma_addr;
+
+		sf->page = alloc_page(GFP_KERNEL);
+		if (!sf->page)
+			return -ENOMEM;
+
+		dma_addr = dma_map_page(dma_dev, sf->page, 0, TB_MAX_FRAME_SIZE,
+					DMA_TO_DEVICE);
+		if (dma_mapping_error(dma_dev, dma_addr)) {
+			__free_page(sf->page);
+			sf->page = NULL;
+			return -ENOMEM;
+		}
+
+		sf->sdev = sdev;
+		sf->frame.callback = tbstream_dev_tx_callback;
+		sf->frame.buffer_phy = dma_addr;
+		sf->frame.sof = TBSTREAM_FRAME_START;
+	}
+
+	sdev->tx_ring.cons = 0;
+	sdev->tx_ring.prod = ring_size - 1;
+	return 0;
+}
+
+static struct tbstream_frame *
+tbstream_dev_alloc_tx(struct tbstream_dev *sdev, enum tbstream_frame_pdf pdf,
+		      struct iov_iter *from, size_t size)
+{
+	struct device *dma_dev = tb_ring_dma_device(sdev->tx_ring.ring);
+	struct tbstream_frame *sf;
+	int index;
+
+	if (!tbstream_ring_available(&sdev->tx_ring))
+		return ERR_PTR(-ENOBUFS);
+
+	index = sdev->tx_ring.cons % tb_ring_size(sdev->tx_ring.ring);
+	sdev->tx_ring.cons++;
+
+	sf = &sdev->tx_ring.frames[index];
+	sf->frame.size = size < TB_MAX_FRAME_SIZE ? size : 0;
+	sf->frame.eof = pdf;
+
+	dma_sync_single_for_cpu(dma_dev, sf->frame.buffer_phy, size,
+				DMA_TO_DEVICE);
+	if (pdf == TBSTREAM_DATA) {
+		if (copy_page_from_iter(sf->page, 0, size, from) != size)
+			return ERR_PTR(-EFAULT);
+	} else {
+		memset(page_address(sf->page), 0, size);
+	}
+	dma_sync_single_for_device(dma_dev, sf->frame.buffer_phy, size,
+				   DMA_TO_DEVICE);
+	return sf;
+}
+
+static int
+tbstream_dev_send_data(struct tbstream_dev *sdev, struct iov_iter *from,
+		       size_t size)
+{
+	struct tbstream_frame *sf;
+
+	sf = tbstream_dev_alloc_tx(sdev, TBSTREAM_DATA, from, size);
+	if (IS_ERR(sf))
+		return PTR_ERR(sf);
+	return tb_ring_tx(sdev->tx_ring.ring, &sf->frame);
+}
+
+static int tbstream_dev_send_close(struct tbstream_dev *sdev)
+{
+	struct tbstream_frame *sf;
+
+	sf = tbstream_dev_alloc_tx(sdev, TBSTREAM_CLOSE, NULL, SZ_256);
+	if (IS_ERR(sf))
+		return PTR_ERR(sf);
+	return tb_ring_tx(sdev->tx_ring.ring, &sf->frame);
+}
+
+static int tbstream_dev_start(struct tbstream_dev *sdev)
+{
+	struct tb_xdomain *xd = tbstream_dev_xdomain(sdev);
+	u16 sof_mask, eof_mask;
+	struct tb_ring *ring;
+	int ret, e2e_tx_hop;
+
+	ring = tb_ring_alloc_tx(xd->tb->nhi, -1, sdev->ring_size,
+				RING_FLAG_FRAME | RING_FLAG_E2E);
+	if (!ring)
+		return -ENOMEM;
+	sdev->tx_ring.ring = ring;
+
+	ret = tbstream_dev_alloc_tx_buffers(sdev);
+	if (ret)
+		goto err_free_tx;
+
+	e2e_tx_hop = ring->hop;
+	sof_mask = BIT(TBSTREAM_FRAME_START);
+	eof_mask = BIT(TBSTREAM_DATA) | BIT(TBSTREAM_CLOSE);
+
+	ring = tb_ring_alloc_rx(xd->tb->nhi, -1, sdev->ring_size,
+				RING_FLAG_FRAME | RING_FLAG_E2E, e2e_tx_hop,
+				sof_mask, eof_mask, NULL, NULL);
+	if (!ring) {
+		ret = -ENOMEM;
+		goto err_free_tx_buffers;
+	}
+	sdev->rx_ring.ring = ring;
+
+	ret = tb_xdomain_enable_paths(xd, sdev->out_hopid,
+				     sdev->tx_ring.ring->hop,
+				     sdev->in_hopid,
+				     sdev->rx_ring.ring->hop);
+	if (ret)
+		goto err_free_rx;
+
+	tb_ring_throttling(sdev->tx_ring.ring, sdev->throttling);
+	tb_ring_throttling(sdev->rx_ring.ring, sdev->throttling);
+
+	tb_ring_start(sdev->tx_ring.ring);
+	tb_ring_start(sdev->rx_ring.ring);
+
+	ret = tbstream_dev_alloc_rx_buffers(sdev);
+	if (ret)
+		goto err_stop;
+	return 0;
+
+err_stop:
+	tb_ring_stop(sdev->rx_ring.ring);
+	tb_ring_stop(sdev->tx_ring.ring);
+err_free_rx:
+	tb_ring_free(sdev->rx_ring.ring);
+err_free_tx_buffers:
+	tbstream_ring_free(&sdev->tx_ring);
+err_free_tx:
+	tb_ring_free(sdev->tx_ring.ring);
+
+	return ret;
+}
+
+static void tbstream_dev_stop(struct tbstream_dev *sdev)
+{
+	struct tb_xdomain *xd;
+
+	/* Wait for the ring to complete any outstanding frames */
+	tb_ring_flush(sdev->tx_ring.ring, 500);
+	tb_ring_stop(sdev->tx_ring.ring);
+	tb_ring_flush(sdev->rx_ring.ring, 500);
+	tb_ring_stop(sdev->rx_ring.ring);
+
+	xd = tbstream_dev_xdomain(sdev);
+	if (xd) {
+		tb_xdomain_disable_paths(xd, sdev->out_hopid,
+					 sdev->tx_ring.ring->hop,
+					 sdev->in_hopid,
+					 sdev->rx_ring.ring->hop);
+	}
+
+	tbstream_ring_free(&sdev->rx_ring);
+	tb_ring_free(sdev->rx_ring.ring);
+	sdev->rx_ring.ring = NULL;
+	tbstream_ring_free(&sdev->tx_ring);
+	tb_ring_free(sdev->tx_ring.ring);
+	sdev->tx_ring.ring = NULL;
+}
+
+static ssize_t
+tbstream_dev_fops_read_iter(struct kiocb *kiocb, struct iov_iter *to)
+{
+	struct file *file = kiocb->ki_filp;
+	struct tbstream_dev *sdev = file->private_data;
+	size_t nbytes;
+	int ret;
+
+	ret = tbstream_dev_valid(sdev);
+	if (ret)
+		return ret;
+
+	if (mutex_lock_interruptible(&sdev->lock))
+		return -ERESTARTSYS;
+
+	while (!tbstream_ring_available(&sdev->rx_ring)) {
+		mutex_unlock(&sdev->lock);
+
+		if (file->f_flags & O_NONBLOCK)
+			return -EAGAIN;
+		ret = wait_event_interruptible(sdev->wait,
+				tbstream_ring_available(&sdev->rx_ring) ||
+				tbstream_dev_valid(sdev) != 0 ||
+				tbstream_dev_closed(sdev));
+		if (ret)
+			return ret;
+
+		ret = tbstream_dev_valid(sdev);
+		if (ret)
+			return ret;
+
+		if (mutex_lock_interruptible(&sdev->lock))
+			return -ERESTARTSYS;
+	}
+
+	nbytes = 0;
+	while (nbytes < iov_iter_count(to)) {
+		struct tbstream_frame *sf;
+		size_t size, sf_size;
+
+		sf = tbstream_dev_completed_rx(sdev);
+		if (!sf)
+			break;
+		/*
+		 * CLOSE tunneled packet. If userspace already read
+		 * something then we stop processing now and return
+		 * those bytes. Next time the first frame will be CLOSE
+		 * in which case we return EOF to the user.
+		 */
+		if (sf->frame.eof == TBSTREAM_CLOSE) {
+			if (!nbytes) {
+				tbstream_dev_consume_rx(sdev);
+				sdev->closed = true;
+			}
+			break;
+		}
+
+		sf_size = tb_ring_frame_size(&sf->frame);
+		size = min(iov_iter_count(to) - nbytes, sf_size);
+
+		if (copy_page_to_iter(sf->page, sf->offset, size, to) != size) {
+			ret = -EFAULT;
+			break;
+		}
+
+		/*
+		 * If not all data from the frame is read so leave it in
+		 * place and update the offset accordingly so next read
+		 * gets the rest.
+		 */
+		if (size < sf_size) {
+			sf->offset += size;
+			sf->frame.size = sf_size - size;
+		} else {
+			ret = tbstream_dev_consume_rx(sdev);
+			if (ret)
+				break;
+		}
+
+		nbytes += size;
+	}
+
+	mutex_unlock(&sdev->lock);
+	if (ret)
+		return ret;
+	return nbytes;
+}
+
+static ssize_t
+tbstream_dev_fops_write_iter(struct kiocb *kiocb, struct iov_iter *from)
+{
+	struct file *file = kiocb->ki_filp;
+	struct tbstream_dev *sdev = file->private_data;
+	size_t nbytes;
+	int ret;
+
+	ret = tbstream_dev_valid(sdev);
+	if (ret)
+		return ret;
+
+	if (mutex_lock_interruptible(&sdev->lock))
+		return -ERESTARTSYS;
+
+	while (!tbstream_ring_available(&sdev->tx_ring)) {
+		mutex_unlock(&sdev->lock);
+
+		if (file->f_flags & O_NONBLOCK)
+			return -EAGAIN;
+		ret = wait_event_interruptible(sdev->wait,
+				tbstream_ring_available(&sdev->tx_ring) ||
+				tbstream_dev_valid(sdev) != 0);
+		if (ret)
+			return ret;
+
+		ret = tbstream_dev_valid(sdev);
+		if (ret)
+			return ret;
+
+		if (tbstream_dev_closed(sdev))
+			return 0;
+
+		if (mutex_lock_interruptible(&sdev->lock))
+			return -ERESTARTSYS;
+	}
+
+	nbytes = 0;
+	while (nbytes < iov_iter_count(from)) {
+		size_t size;
+
+		size = min(iov_iter_count(from) - nbytes, TB_MAX_FRAME_SIZE);
+		ret = tbstream_dev_send_data(sdev, from, size);
+		if (ret) {
+			/*
+			 * If there are no more buffers we are done for
+			 * this write.
+			 */
+			if (ret == -ENOBUFS)
+				ret = 0;
+			break;
+		}
+
+		nbytes += size;
+	}
+
+	mutex_unlock(&sdev->lock);
+	if (ret)
+		return ret;
+	return nbytes;
+}
+
+static __poll_t
+tbstream_dev_fops_poll(struct file *file, struct poll_table_struct *wait)
+{
+	struct tbstream_dev *sdev = file->private_data;
+	__poll_t mask = 0;
+
+	poll_wait(file, &sdev->wait, wait);
+	guard(mutex)(&sdev->lock);
+	if (tbstream_dev_valid(sdev) != 0) {
+		mask |= EPOLLHUP | EPOLLERR;
+	} else {
+		if (tbstream_ring_available(&sdev->tx_ring))
+			mask |= EPOLLOUT | EPOLLWRNORM;
+		if (tbstream_ring_available(&sdev->rx_ring))
+			mask |= EPOLLIN | EPOLLRDNORM;
+	}
+	return mask;
+}
+
+static int tbstream_dev_fops_open(struct inode *inode, struct file *file)
+{
+	struct tbstream_dev *sdev;
+	struct device *dev;
+	int ret;
+
+	/*
+	 * The matching tbstream_dev_put() is done in tbstream_dev_fops_release()
+	 * to keep the reference as long as the device is open.
+	 */
+	dev = class_find_device_by_devt(&tbstream_class, inode->i_rdev);
+	if (!dev)
+		return -ENODEV;
+	sdev = to_tbstream_dev(dev);
+
+	if (mutex_lock_interruptible(&sdev->lock)) {
+		tbstream_dev_put(sdev);
+		return -ERESTARTSYS;
+	}
+
+	/*
+	 * If there is no stream attached yet, block until it appears
+	 * unless this is opened in non-blocking mode.
+	 */
+	while ((ret = tbstream_dev_valid(sdev))) {
+		mutex_unlock(&sdev->lock);
+
+		if (ret != -ENXIO || (file->f_flags & O_NONBLOCK))
+			goto err_put;
+
+		ret = wait_event_interruptible(sdev->wait,
+				tbstream_dev_valid(sdev) == 0);
+		if (ret)
+			goto err_put;
+
+		if (mutex_lock_interruptible(&sdev->lock)) {
+			ret = -ERESTARTSYS;
+			goto err_put;
+		}
+	}
+
+	/* Only on first open we allocate rings and enable paths */
+	if (!sdev->users++) {
+		ret = tbstream_dev_start(sdev);
+		if (ret) {
+			sdev->users--;
+			goto err_unlock;
+		}
+		sdev->closed = false;
+	}
+
+	file->private_data = sdev;
+	mutex_unlock(&sdev->lock);
+	return 0;
+
+err_unlock:
+	mutex_unlock(&sdev->lock);
+err_put:
+	tbstream_dev_put(sdev);
+
+	return ret;
+}
+
+static int tbstream_dev_fops_release(struct inode *inode, struct file *file)
+{
+	struct tbstream_dev *sdev = file->private_data;
+
+	mutex_lock(&sdev->lock);
+	if (--sdev->users == 0) {
+		/*
+		 * Send CLOSE tunneled packet to notify the other end
+		 * that we are closing the file. We do this twice if the
+		 * first one fails.
+		 */
+		tbstream_dev_send_close(sdev);
+		tbstream_dev_stop(sdev);
+	}
+	mutex_unlock(&sdev->lock);
+
+	file->private_data = NULL;
+	tbstream_dev_put(sdev);
+	return 0;
+}
+
+static const struct file_operations tbstream_dev_fops = {
+	.owner = THIS_MODULE,
+	.llseek = noop_llseek,
+	.read_iter = tbstream_dev_fops_read_iter,
+	.write_iter = tbstream_dev_fops_write_iter,
+	.poll = tbstream_dev_fops_poll,
+	.open = tbstream_dev_fops_open,
+	.release = tbstream_dev_fops_release,
+};
+
+static inline struct tbstream_dev *
+tbstream_dev_from_group(struct config_group *group)
+{
+	return container_of(group, struct tbstream_dev, group);
+}
+
+static ssize_t tbstream_dev_index_show(struct config_item *item, char *buf)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	return sysfs_emit(buf, "%d\n", sdev->index);
+}
+CONFIGFS_ATTR_RO(tbstream_dev_, index);
+
+static ssize_t tbstream_dev_in_hopid_show(struct config_item *item, char *buf)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	return sysfs_emit(buf, "%d\n", sdev->in_hopid);
+}
+
+/* svc->lock must be held */
+static void service_remove_properties(struct tb_service *svc, const char *name)
+{
+	struct tb_property *p;
+
+	if (!svc->local_properties)
+		return;
+
+	p = tb_property_find(svc->local_properties, name,
+			     TB_PROPERTY_TYPE_DIRECTORY);
+	if (p) {
+		tb_property_free_dir(p->value.dir);
+		tb_property_remove(p);
+
+		dev_dbg(&svc->dev, "removed local directory %s\n", name);
+
+		/*
+		 * Is the service directory empty already? If it is then
+		 * we can release it as well.
+		 */
+		tb_property_for_each(svc->local_properties, p) {
+			if (p->type == TB_PROPERTY_TYPE_DIRECTORY)
+				return;
+		}
+
+		tb_property_free_dir(svc->local_properties);
+		svc->local_properties = NULL;
+	}
+}
+
+static int service_update_properties(struct tb_service *svc, const char *name,
+				     int in_hopid, int out_hopid)
+{
+	struct tb_property_dir *dir;
+	struct tb_property *p;
+
+	guard(mutex)(&svc->lock);
+
+	if (in_hopid < 8 || out_hopid < 8) {
+		service_remove_properties(svc, name);
+		return 0;
+	}
+
+	if (!svc->local_properties) {
+		/*
+		 * Add the service directory first time we
+		 * populate the entries.
+		 */
+		svc->local_properties = tb_property_copy_dir(tbstream_dir);
+		if (!svc->local_properties)
+			return -ENOMEM;
+	}
+
+	p = tb_property_find(svc->local_properties, name,
+			     TB_PROPERTY_TYPE_DIRECTORY);
+	if (p) {
+		dir = p->value.dir;
+
+		p = tb_property_find(dir, "inhopid", TB_PROPERTY_TYPE_VALUE);
+		if (p && p->value.immediate != in_hopid)
+			p->value.immediate = in_hopid;
+		p = tb_property_find(dir, "outhopid", TB_PROPERTY_TYPE_VALUE);
+		if (p && p->value.immediate != out_hopid)
+			p->value.immediate = out_hopid;
+
+		dev_dbg(&svc->dev,
+			"updated local directory %s: in HopID %d, out HopID %d\n",
+			name, in_hopid, out_hopid);
+	} else {
+		uuid_t uuid;
+		int ret;
+
+		uuid_gen(&uuid);
+		dir = tb_property_create_dir(&uuid);
+		if (!dir)
+			return -ENOMEM;
+
+		tb_property_add_immediate(dir, "inhopid", in_hopid);
+		tb_property_add_immediate(dir, "outhopid", out_hopid);
+
+		ret = tb_property_add_dir(svc->local_properties, name, dir);
+		if (ret) {
+			tb_property_free_dir(dir);
+			return ret;
+		}
+
+		dev_dbg(&svc->dev,
+			"added local directory %s: in HopID %d, out HopID %d\n",
+			name, in_hopid, out_hopid);
+	}
+
+	return 0;
+}
+
+static int tbstream_dev_update_properties(struct tbstream_dev *sdev)
+{
+	struct tbstream *stream;
+	int ret;
+
+	stream = tbstream_get(sdev->stream);
+	if (!stream)
+		return 0;
+
+	ret = service_update_properties(stream->svc,
+					config_item_name(&sdev->group.cg_item),
+					sdev->in_hopid, sdev->out_hopid);
+	if (!ret)
+		tb_service_properties_changed(stream->svc);
+
+	tbstream_put(stream);
+	return ret;
+}
+
+static int tbstream_dev_alloc_in_hopid(struct tbstream_dev *sdev, int hopid)
+{
+	struct tb_xdomain *xd = tbstream_dev_xdomain(sdev);
+	int ret;
+
+	if (sdev->in_hopid > 0 && sdev->in_hopid != hopid)
+		tb_xdomain_release_in_hopid(xd, sdev->in_hopid);
+	if (!hopid) {
+		sdev->in_hopid = hopid;
+		return 0;
+	}
+	ret = tb_xdomain_alloc_in_hopid(xd, hopid);
+	if (ret < 0)
+		return ret;
+	/*
+	 * If specific HopID was asked by the user and we did not get
+	 * that one then release and return error instead.
+	 */
+	if (hopid > 0 && hopid != ret) {
+		tb_xdomain_release_in_hopid(xd, ret);
+		return -EBUSY;
+	}
+	sdev->in_hopid = ret;
+	return 0;
+}
+
+static int tbstream_dev_alloc_out_hopid(struct tbstream_dev *sdev, int hopid)
+{
+	struct tb_xdomain *xd = tbstream_dev_xdomain(sdev);
+	int ret;
+
+	if (sdev->out_hopid > 0 && sdev->out_hopid != hopid)
+		tb_xdomain_release_out_hopid(xd, sdev->out_hopid);
+	if (!hopid) {
+		sdev->out_hopid = hopid;
+		return 0;
+	}
+	ret = tb_xdomain_alloc_out_hopid(xd, hopid);
+	if (ret < 0)
+		return ret;
+	if (hopid > 0 && hopid != ret) {
+		tb_xdomain_release_out_hopid(xd, ret);
+		return -EBUSY;
+	}
+	sdev->out_hopid = ret;
+	return 0;
+}
+
+static ssize_t
+tbstream_dev_in_hopid_store(struct config_item *item, const char *buf,
+			    size_t count)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+	int ret, in_hopid;
+
+	ret = kstrtoint(buf, 0, &in_hopid);
+	if (ret)
+		return ret;
+
+	guard(mutex)(&sdev->lock);
+	if (sdev->users)
+		return -EBUSY;
+	if (sdev->stream) {
+		ret = tbstream_dev_alloc_in_hopid(sdev, in_hopid);
+		if (ret)
+			return ret;
+		ret = tbstream_dev_update_properties(sdev);
+	} else {
+		sdev->in_hopid = in_hopid;
+	}
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(tbstream_dev_, in_hopid);
+
+static ssize_t tbstream_dev_out_hopid_show(struct config_item *item, char *buf)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	return sysfs_emit(buf, "%d\n", sdev->out_hopid);
+}
+
+static ssize_t
+tbstream_dev_out_hopid_store(struct config_item *item, const char *buf,
+			     size_t count)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+	int ret, out_hopid;
+
+	ret = kstrtoint(buf, 0, &out_hopid);
+	if (ret)
+		return ret;
+
+	guard(mutex)(&sdev->lock);
+	if (sdev->users)
+		return -EBUSY;
+	if (sdev->stream) {
+		ret = tbstream_dev_alloc_out_hopid(sdev, out_hopid);
+		if (ret)
+			return ret;
+		ret = tbstream_dev_update_properties(sdev);
+	} else {
+		sdev->out_hopid = out_hopid;
+	}
+	return ret ? ret : count;
+}
+CONFIGFS_ATTR(tbstream_dev_, out_hopid);
+
+static ssize_t tbstream_dev_ring_size_show(struct config_item *item, char *buf)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	return sysfs_emit(buf, "%u\n", sdev->ring_size);
+}
+
+static ssize_t
+tbstream_dev_ring_size_store(struct config_item *item, const char *buf,
+			     size_t count)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+	unsigned int ring_size;
+	int ret;
+
+	ret = kstrtouint(buf, 0, &ring_size);
+	if (ret)
+		return ret;
+
+	if (ring_size < TBSTREAM_DEV_MIN_RING_SIZE ||
+	    ring_size > TBSTREAM_DEV_MAX_RING_SIZE)
+		return -EINVAL;
+
+	guard(mutex)(&sdev->lock);
+	if (sdev->users)
+		return -EBUSY;
+	sdev->ring_size = ring_size;
+	return count;
+}
+CONFIGFS_ATTR(tbstream_dev_, ring_size);
+
+static ssize_t tbstream_dev_throttling_show(struct config_item *item, char *buf)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	return sysfs_emit(buf, "%u\n", sdev->throttling);
+}
+
+static ssize_t
+tbstream_dev_throttling_store(struct config_item *item, const char *buf,
+			      size_t count)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+	unsigned int throttling;
+	int ret;
+
+	ret = kstrtouint(buf, 0, &throttling);
+	if (ret)
+		return ret;
+
+	if (throttling > TBSTREAM_DEV_MAX_THROTTLING)
+		return -EINVAL;
+
+	guard(mutex)(&sdev->lock);
+	if (sdev->users)
+		return -EBUSY;
+	sdev->throttling = throttling;
+	return count;
+}
+CONFIGFS_ATTR(tbstream_dev_, throttling);
+
+static struct configfs_attribute *tbstream_dev_attrs[] = {
+	&tbstream_dev_attr_index,
+	&tbstream_dev_attr_in_hopid,
+	&tbstream_dev_attr_out_hopid,
+	&tbstream_dev_attr_ring_size,
+	&tbstream_dev_attr_throttling,
+	NULL,
+};
+
+static void tbstream_dev_item_release(struct config_item *item)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(group);
+
+	/* Undo device_initialize() + cdev_device_add() */
+	cdev_device_del(&sdev->cdev, &sdev->dev);
+	tbstream_dev_put(sdev);
+}
+
+static struct configfs_item_operations tbstream_dev_item_ops = {
+	.release = tbstream_dev_item_release,
+};
+
+static const struct config_item_type tbstream_dev_type = {
+	.ct_owner = THIS_MODULE,
+	.ct_item_ops = &tbstream_dev_item_ops,
+	.ct_attrs = tbstream_dev_attrs,
+};
+
+static void service_get_hopids(struct tb_service *svc, const char *name,
+			       int *in_hopid, int *out_hopid)
+{
+	struct tb_property_dir *dir;
+	struct tb_property *p;
+
+	guard(mutex)(&svc->lock);
+
+	/* See if we have directory entry with the matching name */
+	p = tb_property_find(svc->remote_properties, name,
+			     TB_PROPERTY_TYPE_DIRECTORY);
+	if (!p)
+		return;
+
+	dir = p->value.dir;
+
+	/*
+	 * We need to reverse the HopIDs on our end so that in becomes
+	 * out and vice versa.
+	 */
+	p = tb_property_find(dir, "inhopid", TB_PROPERTY_TYPE_VALUE);
+	if (p && p->value.immediate >= 8)
+		*out_hopid = p->value.immediate;
+	p = tb_property_find(dir, "outhopid", TB_PROPERTY_TYPE_VALUE);
+	if (p && p->value.immediate >= 8)
+		*in_hopid = p->value.immediate;
+}
+
+static void
+tbstream_dev_attach_stream(struct tbstream_dev *sdev, struct tbstream_group *sg)
+{
+	const char *name = config_item_name(&sdev->group.cg_item);
+	struct tbstream *stream;
+
+	stream = tbstream_get(sg->stream);
+	if (!stream)
+		return;
+
+	scoped_guard(mutex, &sdev->lock) {
+		sdev->stream = stream;
+		/*
+		 * If there is no existing configuration (or automatic
+		 * configuration is being used) check if the other side
+		 * has configuration for this and use it.
+		 */
+		if (sdev->in_hopid <= 0 && sdev->out_hopid <= 0)
+			service_get_hopids(stream->svc, name, &sdev->in_hopid,
+					   &sdev->out_hopid);
+		if (sdev->in_hopid)
+			tbstream_dev_alloc_in_hopid(sdev, sdev->in_hopid);
+		if (sdev->out_hopid)
+			tbstream_dev_alloc_out_hopid(sdev, sdev->out_hopid);
+	}
+
+	service_update_properties(stream->svc, name, sdev->in_hopid,
+				  sdev->out_hopid);
+	tb_service_properties_changed(stream->svc);
+
+	/* Notify any openerers that the stream is now attached */
+	wake_up_interruptible(&sdev->wait);
+}
+
+static void tbstream_dev_detach_stream(struct tbstream_dev *sdev)
+{
+	const char *name = config_item_name(&sdev->group.cg_item);
+	struct tbstream *stream;
+	struct tb_xdomain *xd;
+
+	scoped_guard(mutex, &sdev->lock) {
+		stream = sdev->stream;
+		if (!stream)
+			return;
+		sdev->stream = NULL;
+
+		xd = tb_service_parent(stream->svc);
+		if (sdev->out_hopid > 0)
+			tb_xdomain_release_out_hopid(xd, sdev->out_hopid);
+		if (sdev->in_hopid > 0)
+			tb_xdomain_release_in_hopid(xd, sdev->in_hopid);
+	}
+
+	service_update_properties(stream->svc, name, 0, 0);
+	tb_service_properties_changed(stream->svc);
+
+	tbstream_put(stream);
+
+	/* Notify any task that the stream is not valid anymore */
+	wake_up_interruptible_poll(&sdev->wait, EPOLLHUP | EPOLLERR);
+}
+
+static inline struct tbstream_group *
+to_tbstream_group(struct config_group *group)
+{
+	return container_of(group, struct tbstream_group, group);
+}
+
+static struct config_group *
+tbstream_dev_make_group(struct config_group *group, const char *name)
+{
+	struct tbstream_group *sg = to_tbstream_group(group);
+	struct tbstream_dev *sdev;
+	int ret, index;
+
+	/*
+	 * We want the names to be suitable for passing as property
+	 * directory names.
+	 */
+	if (strlen(name) > TB_PROPERTY_KEY_SIZE)
+		return ERR_PTR(-ENAMETOOLONG);
+
+	sdev = kzalloc_obj(*sdev, GFP_KERNEL);
+	if (!sdev)
+		return ERR_PTR(-ENOMEM);
+
+	index = ida_alloc_max(&tbstream_minors, TBSTREAM_DEV_MINORS - 1,
+			      GFP_KERNEL);
+	if (index < 0) {
+		kfree(sdev);
+		return ERR_PTR(index);
+	}
+
+	sdev->index = index;
+	sdev->ring_size = TBSTREAM_DEV_RING_SIZE;
+	sdev->throttling = TBSTREAM_DEV_THROTTLING;
+	mutex_init(&sdev->lock);
+	init_waitqueue_head(&sdev->wait);
+	INIT_LIST_HEAD(&sdev->list);
+
+	sdev->dev.devt = MKDEV(MAJOR(tbstream_devt), index);
+	sdev->dev.class = &tbstream_class;
+	sdev->dev.release = tbstream_dev_release;
+	/* This point forward tbstream_dev_put() must be used to release sdev */
+	device_initialize(&sdev->dev);
+
+	ret = dev_set_name(&sdev->dev, "tbstream%d", index);
+	if (ret) {
+		tbstream_dev_put(sdev);
+		return ERR_PTR(ret);
+	}
+
+	config_group_init_type_name(&sdev->group, name, &tbstream_dev_type);
+
+	scoped_guard(mutex, &sg->lock)
+		list_add_tail(&sdev->list, &sg->dev_list);
+
+	tbstream_dev_attach_stream(sdev, sg);
+
+	cdev_init(&sdev->cdev, &tbstream_dev_fops);
+	ret = cdev_device_add(&sdev->cdev, &sdev->dev);
+	if (ret) {
+		tbstream_dev_detach_stream(sdev);
+		/* Calls tbstream_dev_put() */
+		config_group_put(&sdev->group);
+		return ERR_PTR(ret);
+	}
+
+	return &sdev->group;
+}
+
+static void
+tbstream_dev_drop_item(struct config_group *group, struct config_item *item)
+{
+	struct config_group *sdev_group = to_config_group(item);
+	struct tbstream_dev *sdev = tbstream_dev_from_group(sdev_group);
+	struct tbstream_group *sg = to_tbstream_group(group);
+
+	tbstream_dev_detach_stream(sdev);
+	scoped_guard(mutex, &sg->lock)
+		list_del(&sdev->list);
+	config_item_put(item);
+}
+
+static struct configfs_group_operations tbstream_dev_group_ops = {
+	.make_group = tbstream_dev_make_group,
+	.drop_item = tbstream_dev_drop_item,
+};
+
+static void tbstream_item_release(struct config_item *item)
+{
+	struct config_group *group = to_config_group(item);
+	struct tbstream_group *sg = to_tbstream_group(group);
+
+	tbstream_put(sg->stream);
+	kfree(sg);
+}
+
+static struct configfs_item_operations tbstream_item_ops = {
+	.release = tbstream_item_release,
+};
+
+static const struct config_item_type tbstream_dev_group_type = {
+	.ct_owner = THIS_MODULE,
+	.ct_group_ops = &tbstream_dev_group_ops,
+	.ct_item_ops = &tbstream_item_ops,
+};
+
+static struct config_group *
+tbstream_make_group(struct config_group *group, const char *name)
+{
+	struct tbstream_group *sg;
+	struct tbstream *stream;
+	int domain, index;
+	u64 route;
+
+	/* Make sure the format is correct */
+	if (sscanf(name, "%u-%llx.%u", &domain, &route, &index) != 3)
+		return ERR_PTR(-EINVAL);
+
+	sg = kzalloc_obj(*sg, GFP_KERNEL);
+	if (!sg)
+		return ERR_PTR(-ENOMEM);
+
+	mutex_init(&sg->lock);
+	INIT_LIST_HEAD(&sg->dev_list);
+
+	guard(mutex)(&tbstream_lock);
+	list_for_each_entry(stream, &tbstream_list, list) {
+		if (sysfs_streq(name, dev_name(&stream->svc->dev))) {
+			sg->stream = tbstream_get(stream);
+			break;
+		}
+	}
+
+	config_group_init_type_name(&sg->group, name, &tbstream_dev_group_type);
+	return &sg->group;
+}
+
+static struct configfs_group_operations tbstream_group_ops = {
+	.make_group = tbstream_make_group,
+};
+
+static const struct config_item_type tbstream_group_type = {
+	.ct_owner = THIS_MODULE,
+	.ct_group_ops = &tbstream_group_ops,
+};
+
+static struct config_group tbstream_group = {
+	.cg_item = {
+		.ci_namebuf = "stream",
+		.ci_type = &tbstream_group_type,
+	},
+};
+
+/* Returns reference count increased */
+static struct tbstream_group *tbstream_group_find(struct tbstream *stream)
+{
+	struct config_item *item;
+
+	guard(mutex)(&tbstream_group.cg_subsys->su_mutex);
+	item = config_group_find_item(&tbstream_group,
+				      dev_name(&stream->svc->dev));
+	if (item)
+		return to_tbstream_group(to_config_group(item));
+	return NULL;
+}
+
+static void tbstream_group_attach_stream(struct tbstream *stream)
+{
+	struct tbstream_group *sg;
+	struct tbstream_dev *sdev;
+
+	sg = tbstream_group_find(stream);
+	if (!sg)
+		return;
+
+	guard(mutex)(&sg->lock);
+	if (WARN_ON(sg->stream)) {
+		config_group_put(&sg->group);
+		return;
+	}
+	sg->stream = tbstream_get(stream);
+	/*
+	 * If there are existing stream devices, attach the stream to
+	 * them now.
+	 */
+	list_for_each_entry(sdev, &sg->dev_list, list)
+		tbstream_dev_attach_stream(sdev, sg);
+
+	config_group_put(&sg->group);
+}
+
+static void tbstream_group_detach_stream(struct tbstream *stream)
+{
+	struct tbstream_group *sg;
+	struct tbstream_dev *sdev;
+
+	sg = tbstream_group_find(stream);
+	if (!sg)
+		return;
+
+	guard(mutex)(&sg->lock);
+	if (sg->stream) {
+		/* Detach this stream from the stream devices */
+		list_for_each_entry_reverse(sdev, &sg->dev_list, list)
+			tbstream_dev_detach_stream(sdev);
+		tbstream_put(sg->stream);
+		sg->stream = NULL;
+	}
+
+	config_group_put(&sg->group);
+}
+
+static int tbstream_probe(struct tb_service *svc, const struct tb_service_id *id)
+{
+	struct tbstream *stream;
+
+	stream = kzalloc_obj(*stream, GFP_KERNEL);
+	if (!stream)
+		return -ENOMEM;
+
+	/* After this point, release stream by calling tbstream_put() */
+	kref_init(&stream->kref);
+	stream->svc = tb_service_get(svc);
+	INIT_LIST_HEAD(&stream->list);
+
+	scoped_guard(mutex, &tbstream_lock)
+		list_add_tail(&stream->list, &tbstream_list);
+
+	tbstream_group_attach_stream(stream);
+	tb_service_set_drvdata(svc, stream);
+	return 0;
+}
+
+static void tbstream_remove(struct tb_service *svc)
+{
+	struct tbstream *stream = tb_service_get_drvdata(svc);
+
+	tbstream_group_detach_stream(stream);
+	scoped_guard(mutex, &tbstream_lock)
+		list_del(&stream->list);
+	tbstream_put(stream);
+}
+
+static int __maybe_unused tbstream_suspend(struct device *dev)
+{
+	struct tb_service *svc = tb_to_service(dev);
+	struct tbstream *stream = tb_service_get_drvdata(svc);
+	struct tbstream_group *sg;
+	struct tbstream_dev *sdev;
+
+	sg = tbstream_group_find(stream);
+	if (!sg)
+		return 0;
+
+	list_for_each_entry_reverse(sdev, &sg->dev_list, list) {
+		/* Stop the stream (if it was open) */
+		if (sdev->users)
+			tbstream_dev_stop(sdev);
+	}
+
+	config_group_put(&sg->group);
+	return 0;
+}
+
+static int __maybe_unused tbstream_resume(struct device *dev)
+{
+	struct tb_service *svc = tb_to_service(dev);
+	struct tbstream *stream = tb_service_get_drvdata(svc);
+	struct tbstream_group *sg;
+	struct tbstream_dev *sdev;
+
+	sg = tbstream_group_find(stream);
+	if (!sg)
+		return 0;
+
+	list_for_each_entry(sdev, &sg->dev_list, list) {
+		int ret;
+
+		if (!sdev->users)
+			continue;
+		ret = tbstream_dev_start(sdev);
+		if (ret) {
+			config_group_put(&sg->group);
+			return ret;
+		}
+	}
+
+	config_group_put(&sg->group);
+	return 0;
+}
+
+static const struct dev_pm_ops tbstream_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(tbstream_suspend, tbstream_resume)
+};
+
+static const struct tb_service_id tbstream_ids[] = {
+	{ TB_SERVICE("stream", 1) },
+	{ },
+};
+MODULE_DEVICE_TABLE(tbsvc, tbstream_ids);
+
+static struct tb_service_driver tbstream_driver = {
+	.driver = {
+		.owner = THIS_MODULE,
+		.name = "thunderbolt_stream",
+		.pm = &tbstream_pm_ops,
+	},
+	.probe = tbstream_probe,
+	.remove = tbstream_remove,
+	.id_table = tbstream_ids,
+};
+
+static int __init tbstream_init(void)
+{
+	int ret;
+
+	ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
+				  "tbstream");
+	if (ret)
+		return ret;
+
+	ret = class_register(&tbstream_class);
+	if (ret)
+		goto err_unregister_chrdev;
+
+	tbstream_dir = tb_property_create_dir(&tbstream_dir_uuid);
+	if (!tbstream_dir) {
+		ret = -ENOMEM;
+		goto err_unregister_class;
+	}
+
+	tb_property_add_immediate(tbstream_dir, "prtcid", 1);
+	tb_property_add_immediate(tbstream_dir, "prtcvers", 1);
+	tb_property_add_immediate(tbstream_dir, "prtcrevs", 0);
+	tb_property_add_immediate(tbstream_dir, "prtcstns", 0);
+
+	ret = tb_register_property_dir("stream", tbstream_dir);
+	if (ret)
+		goto err_free_dir;
+
+	config_group_init(&tbstream_group);
+	ret = tb_configfs_register_group(&tbstream_group);
+	if (ret)
+		goto err_unregister_dir;
+
+	ret = tb_register_service_driver(&tbstream_driver);
+	if (ret)
+		goto err_unregister_group;
+	return 0;
+
+err_unregister_group:
+	tb_configfs_unregister_group(&tbstream_group);
+err_unregister_dir:
+	tb_unregister_property_dir("stream", tbstream_dir);
+err_free_dir:
+	tb_property_free_dir(tbstream_dir);
+err_unregister_class:
+	class_unregister(&tbstream_class);
+err_unregister_chrdev:
+	unregister_chrdev_region(tbstream_devt, TBSTREAM_DEV_MINORS);
+
+	return ret;
+}
+module_init(tbstream_init);
+
+static void __exit tbstream_exit(void)
+{
+	tb_unregister_service_driver(&tbstream_driver);
+	tb_configfs_unregister_group(&tbstream_group);
+	tb_unregister_property_dir("stream", tbstream_dir);
+	tb_property_free_dir(tbstream_dir);
+	class_unregister(&tbstream_class);
+	unregister_chrdev_region(tbstream_devt, TBSTREAM_DEV_MINORS);
+	ida_destroy(&tbstream_minors);
+}
+module_exit(tbstream_exit);
+
+MODULE_AUTHOR("Alan Borzeszkowski <alan.borzeszkowski@linux.intel.com>");
+MODULE_AUTHOR("Mika Westerberg <mika.westerberg@linux.intel.com>");
+MODULE_DESCRIPTION("Stream data over Thunderbolt/USB4 cable");
+MODULE_LICENSE("GPL");
-- 
2.50.1

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Greg KH]

On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> Introduce USB4STREAM protocol and Linux implementation. This allows two
> (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> through a character device without need to go through the network stack.
> 
> Any application that supports read(2) and write(2) in some form should
> be able to use the device without changes. The data is sent out to the
> other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> device is called /dev/tbstreamX where X is the minor number starting
> from 0.
> 
> All stream devices need to be configured first. This is done through
> ConfigFS interface. There can be multiple streams at the same time (this
> depends on number of DMA rings and available HopIDs) and a single stream
> supports traffic in both directions. For example there could be an
> application that uses one stream as control channel and another one as
> bi-directional data channel.
> 
> A real use-case for this is to take a backup as a part of recovery
> initramfs tooling (no need to setup networking or have ssh or similar
> tooling as part of the initramfs). Say we want to backup the disk of
> host1 to host2. First Thunderbolt/USB4 cable is connected between the
> hosts (there can be devices in the middle too) then the receiving side
> configures the stream:
> 
>   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
>   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
>   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
>   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> 
> We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> >From this point forward the /dev/tbstream0 can be used pretty much as
> regular file:
> 
>   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> 
> The host that is being backed up then configures the stream accordingly:
> 
>   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
>   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> 
> Here we take advantage of the fact that host2 also announces the active
> streams through XDomain properties so the name "backup" gives us the
> HopIDs. It is also possible to configure them manually in the same way
> we did for host2.
> 
> Then it is just a matter of copying the data over:
> 
>   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> 
> Similarly it is possible to transfer parts of the filesystem. For
> example copy contents of mydir over to the host2:
> 
>   host2 # gunzip < /dev/tbstream0 | tar xf -
>   host1 # tar cf - mydir | gzip > /dev/tbstream0
> 
> Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> to desktop (host2):
> 
>   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
>                          autovideosink
> 
>   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
>                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> 
> Once the streams are no longer needed they can be removed:
> 
>   host1 # cd /sys/kernel/config/thunderbolt/stream/
>   host1 # rmdir -p 0-503.0/backup
> 
>   host2 # cd /sys/kernel/config/thunderbolt/stream
>   host2 # rmdir -p 0-1.0/backup

Very cool, but shouldn't the above be in some documentation somewhere so
that people know how to use it?

And why do you need a whole major for this, why not just use a misc
device that it dynamically created for every new dev?

thanks,

greg k-h

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Mika Westerberg]

On Tue, Apr 28, 2026 at 05:57:37AM -0600, Greg KH wrote:
> On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > through a character device without need to go through the network stack.
> > 
> > Any application that supports read(2) and write(2) in some form should
> > be able to use the device without changes. The data is sent out to the
> > other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> > device is called /dev/tbstreamX where X is the minor number starting
> > from 0.
> > 
> > All stream devices need to be configured first. This is done through
> > ConfigFS interface. There can be multiple streams at the same time (this
> > depends on number of DMA rings and available HopIDs) and a single stream
> > supports traffic in both directions. For example there could be an
> > application that uses one stream as control channel and another one as
> > bi-directional data channel.
> > 
> > A real use-case for this is to take a backup as a part of recovery
> > initramfs tooling (no need to setup networking or have ssh or similar
> > tooling as part of the initramfs). Say we want to backup the disk of
> > host1 to host2. First Thunderbolt/USB4 cable is connected between the
> > hosts (there can be devices in the middle too) then the receiving side
> > configures the stream:
> > 
> >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
> >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
> >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
> >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> > 
> > We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> > >From this point forward the /dev/tbstream0 can be used pretty much as
> > regular file:
> > 
> >   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> > 
> > The host that is being backed up then configures the stream accordingly:
> > 
> >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
> >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> > 
> > Here we take advantage of the fact that host2 also announces the active
> > streams through XDomain properties so the name "backup" gives us the
> > HopIDs. It is also possible to configure them manually in the same way
> > we did for host2.
> > 
> > Then it is just a matter of copying the data over:
> > 
> >   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> > 
> > Similarly it is possible to transfer parts of the filesystem. For
> > example copy contents of mydir over to the host2:
> > 
> >   host2 # gunzip < /dev/tbstream0 | tar xf -
> >   host1 # tar cf - mydir | gzip > /dev/tbstream0
> > 
> > Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> > to desktop (host2):
> > 
> >   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
> >                          autovideosink
> > 
> >   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
> >                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> > 
> > Once the streams are no longer needed they can be removed:
> > 
> >   host1 # cd /sys/kernel/config/thunderbolt/stream/
> >   host1 # rmdir -p 0-503.0/backup
> > 
> >   host2 # cd /sys/kernel/config/thunderbolt/stream
> >   host2 # rmdir -p 0-1.0/backup
> 
> Very cool, but shouldn't the above be in some documentation somewhere so
> that people know how to use it?

Sure, I can add it part of the Documentation/admin-guide/thunderbolt.rs for
example.

> And why do you need a whole major for this, why not just use a misc
> device that it dynamically created for every new dev?

We do use this:

       ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
                                  "tbstream");

that should be dynamically allocated, no?

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Greg KH]

On Tue, Apr 28, 2026 at 02:03:14PM +0200, Mika Westerberg wrote:
> On Tue, Apr 28, 2026 at 05:57:37AM -0600, Greg KH wrote:
> > On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > > through a character device without need to go through the network stack.
> > > 
> > > Any application that supports read(2) and write(2) in some form should
> > > be able to use the device without changes. The data is sent out to the
> > > other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> > > device is called /dev/tbstreamX where X is the minor number starting
> > > from 0.
> > > 
> > > All stream devices need to be configured first. This is done through
> > > ConfigFS interface. There can be multiple streams at the same time (this
> > > depends on number of DMA rings and available HopIDs) and a single stream
> > > supports traffic in both directions. For example there could be an
> > > application that uses one stream as control channel and another one as
> > > bi-directional data channel.
> > > 
> > > A real use-case for this is to take a backup as a part of recovery
> > > initramfs tooling (no need to setup networking or have ssh or similar
> > > tooling as part of the initramfs). Say we want to backup the disk of
> > > host1 to host2. First Thunderbolt/USB4 cable is connected between the
> > > hosts (there can be devices in the middle too) then the receiving side
> > > configures the stream:
> > > 
> > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
> > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
> > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
> > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> > > 
> > > We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> > > >From this point forward the /dev/tbstream0 can be used pretty much as
> > > regular file:
> > > 
> > >   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> > > 
> > > The host that is being backed up then configures the stream accordingly:
> > > 
> > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
> > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> > > 
> > > Here we take advantage of the fact that host2 also announces the active
> > > streams through XDomain properties so the name "backup" gives us the
> > > HopIDs. It is also possible to configure them manually in the same way
> > > we did for host2.
> > > 
> > > Then it is just a matter of copying the data over:
> > > 
> > >   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> > > 
> > > Similarly it is possible to transfer parts of the filesystem. For
> > > example copy contents of mydir over to the host2:
> > > 
> > >   host2 # gunzip < /dev/tbstream0 | tar xf -
> > >   host1 # tar cf - mydir | gzip > /dev/tbstream0
> > > 
> > > Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> > > to desktop (host2):
> > > 
> > >   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
> > >                          autovideosink
> > > 
> > >   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
> > >                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> > > 
> > > Once the streams are no longer needed they can be removed:
> > > 
> > >   host1 # cd /sys/kernel/config/thunderbolt/stream/
> > >   host1 # rmdir -p 0-503.0/backup
> > > 
> > >   host2 # cd /sys/kernel/config/thunderbolt/stream
> > >   host2 # rmdir -p 0-1.0/backup
> > 
> > Very cool, but shouldn't the above be in some documentation somewhere so
> > that people know how to use it?
> 
> Sure, I can add it part of the Documentation/admin-guide/thunderbolt.rs for
> example.
> 
> > And why do you need a whole major for this, why not just use a misc
> > device that it dynamically created for every new dev?
> 
> We do use this:
> 
>        ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
>                                   "tbstream");
> 
> that should be dynamically allocated, no?

Yes, but you are using up a whole major number for this, and in reality
there's only going to be 1-2, maybe 4, different devices needed at once,
right?  So just use the miscdev interface instead?

thanks,

greg k-h

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Mika Westerberg]

On Tue, Apr 28, 2026 at 07:54:51AM -0600, Greg KH wrote:
> On Tue, Apr 28, 2026 at 02:03:14PM +0200, Mika Westerberg wrote:
> > On Tue, Apr 28, 2026 at 05:57:37AM -0600, Greg KH wrote:
> > > On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > > > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > > > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > > > through a character device without need to go through the network stack.
> > > > 
> > > > Any application that supports read(2) and write(2) in some form should
> > > > be able to use the device without changes. The data is sent out to the
> > > > other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> > > > device is called /dev/tbstreamX where X is the minor number starting
> > > > from 0.
> > > > 
> > > > All stream devices need to be configured first. This is done through
> > > > ConfigFS interface. There can be multiple streams at the same time (this
> > > > depends on number of DMA rings and available HopIDs) and a single stream
> > > > supports traffic in both directions. For example there could be an
> > > > application that uses one stream as control channel and another one as
> > > > bi-directional data channel.
> > > > 
> > > > A real use-case for this is to take a backup as a part of recovery
> > > > initramfs tooling (no need to setup networking or have ssh or similar
> > > > tooling as part of the initramfs). Say we want to backup the disk of
> > > > host1 to host2. First Thunderbolt/USB4 cable is connected between the
> > > > hosts (there can be devices in the middle too) then the receiving side
> > > > configures the stream:
> > > > 
> > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
> > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
> > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
> > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> > > > 
> > > > We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> > > > >From this point forward the /dev/tbstream0 can be used pretty much as
> > > > regular file:
> > > > 
> > > >   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> > > > 
> > > > The host that is being backed up then configures the stream accordingly:
> > > > 
> > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
> > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> > > > 
> > > > Here we take advantage of the fact that host2 also announces the active
> > > > streams through XDomain properties so the name "backup" gives us the
> > > > HopIDs. It is also possible to configure them manually in the same way
> > > > we did for host2.
> > > > 
> > > > Then it is just a matter of copying the data over:
> > > > 
> > > >   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> > > > 
> > > > Similarly it is possible to transfer parts of the filesystem. For
> > > > example copy contents of mydir over to the host2:
> > > > 
> > > >   host2 # gunzip < /dev/tbstream0 | tar xf -
> > > >   host1 # tar cf - mydir | gzip > /dev/tbstream0
> > > > 
> > > > Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> > > > to desktop (host2):
> > > > 
> > > >   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
> > > >                          autovideosink
> > > > 
> > > >   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
> > > >                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> > > > 
> > > > Once the streams are no longer needed they can be removed:
> > > > 
> > > >   host1 # cd /sys/kernel/config/thunderbolt/stream/
> > > >   host1 # rmdir -p 0-503.0/backup
> > > > 
> > > >   host2 # cd /sys/kernel/config/thunderbolt/stream
> > > >   host2 # rmdir -p 0-1.0/backup
> > > 
> > > Very cool, but shouldn't the above be in some documentation somewhere so
> > > that people know how to use it?
> > 
> > Sure, I can add it part of the Documentation/admin-guide/thunderbolt.rs for
> > example.
> > 
> > > And why do you need a whole major for this, why not just use a misc
> > > device that it dynamically created for every new dev?
> > 
> > We do use this:
> > 
> >        ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
> >                                   "tbstream");
> > 
> > that should be dynamically allocated, no?
> 
> Yes, but you are using up a whole major number for this, and in reality
> there's only going to be 1-2, maybe 4, different devices needed at once,
> right?  So just use the miscdev interface instead?

There could be 11 per host controller in Intel hardware (we have 12 DMA
rings, one of which is reserved for control traffic), and we have 2 host
conrollers in recent systems. Due to the dedicated flow control we use now
that's not possible but we are planning to make it to use shared flow
control instead which allows more.

Not sure if anybody ever will create that many, though.

Second thing is that we use cdev_device_add() to manage the char device and
the stream device as they are part of the same structure. I don't think
that can be done with miscdevice.

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Greg KH]

On Tue, Apr 28, 2026 at 04:11:48PM +0200, Mika Westerberg wrote:
> On Tue, Apr 28, 2026 at 07:54:51AM -0600, Greg KH wrote:
> > On Tue, Apr 28, 2026 at 02:03:14PM +0200, Mika Westerberg wrote:
> > > On Tue, Apr 28, 2026 at 05:57:37AM -0600, Greg KH wrote:
> > > > On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > > > > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > > > > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > > > > through a character device without need to go through the network stack.
> > > > > 
> > > > > Any application that supports read(2) and write(2) in some form should
> > > > > be able to use the device without changes. The data is sent out to the
> > > > > other side over a tunnel inside Thunderbolt/USB4 fabric. The character
> > > > > device is called /dev/tbstreamX where X is the minor number starting
> > > > > from 0.
> > > > > 
> > > > > All stream devices need to be configured first. This is done through
> > > > > ConfigFS interface. There can be multiple streams at the same time (this
> > > > > depends on number of DMA rings and available HopIDs) and a single stream
> > > > > supports traffic in both directions. For example there could be an
> > > > > application that uses one stream as control channel and another one as
> > > > > bi-directional data channel.
> > > > > 
> > > > > A real use-case for this is to take a backup as a part of recovery
> > > > > initramfs tooling (no need to setup networking or have ssh or similar
> > > > > tooling as part of the initramfs). Say we want to backup the disk of
> > > > > host1 to host2. First Thunderbolt/USB4 cable is connected between the
> > > > > hosts (there can be devices in the middle too) then the receiving side
> > > > > configures the stream:
> > > > > 
> > > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0
> > > > >   host2 # mkdir /sys/kernel/config/thunderbolt/stream/0-1.0/backup
> > > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/in_hopid
> > > > >   host2 # echo -1 > /sys/kernel/config/thunderbolt/stream/0-1.0/backup/out_hopid
> > > > > 
> > > > > We use automatic HopID allocation (writing -1 to HopIDs) for simplicity.
> > > > > >From this point forward the /dev/tbstream0 can be used pretty much as
> > > > > regular file:
> > > > > 
> > > > >   host2 # dd if=/dev/tbstream0 of=/tmp/host1.nvme0n1.backup-$(date +%F) bs=256k
> > > > > 
> > > > > The host that is being backed up then configures the stream accordingly:
> > > > > 
> > > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0
> > > > >   host1 # mkdir /sys/kernel/config/thunderbolt/stream/0-503.0/backup
> > > > > 
> > > > > Here we take advantage of the fact that host2 also announces the active
> > > > > streams through XDomain properties so the name "backup" gives us the
> > > > > HopIDs. It is also possible to configure them manually in the same way
> > > > > we did for host2.
> > > > > 
> > > > > Then it is just a matter of copying the data over:
> > > > > 
> > > > >   host1 # dd if=/dev/nvme0n1 of=/dev/tbstream0 bs=256k
> > > > > 
> > > > > Similarly it is possible to transfer parts of the filesystem. For
> > > > > example copy contents of mydir over to the host2:
> > > > > 
> > > > >   host2 # gunzip < /dev/tbstream0 | tar xf -
> > > > >   host1 # tar cf - mydir | gzip > /dev/tbstream0
> > > > > 
> > > > > Other end of the spectrum use-case is "borrowing" laptop (host1) camera
> > > > > to desktop (host2):
> > > > > 
> > > > >   host2 # gst-launch-1.0 filesrc location=/dev/tbstream0 ! jpegdec ! videoconvert ! \
> > > > >                          autovideosink
> > > > > 
> > > > >   host1 # gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080 ! \
> > > > >                          jpegenc quality=90 ! filesink location=/dev/tbstream0
> > > > > 
> > > > > Once the streams are no longer needed they can be removed:
> > > > > 
> > > > >   host1 # cd /sys/kernel/config/thunderbolt/stream/
> > > > >   host1 # rmdir -p 0-503.0/backup
> > > > > 
> > > > >   host2 # cd /sys/kernel/config/thunderbolt/stream
> > > > >   host2 # rmdir -p 0-1.0/backup
> > > > 
> > > > Very cool, but shouldn't the above be in some documentation somewhere so
> > > > that people know how to use it?
> > > 
> > > Sure, I can add it part of the Documentation/admin-guide/thunderbolt.rs for
> > > example.
> > > 
> > > > And why do you need a whole major for this, why not just use a misc
> > > > device that it dynamically created for every new dev?
> > > 
> > > We do use this:
> > > 
> > >        ret = alloc_chrdev_region(&tbstream_devt, 0, TBSTREAM_DEV_MINORS,
> > >                                   "tbstream");
> > > 
> > > that should be dynamically allocated, no?
> > 
> > Yes, but you are using up a whole major number for this, and in reality
> > there's only going to be 1-2, maybe 4, different devices needed at once,
> > right?  So just use the miscdev interface instead?
> 
> There could be 11 per host controller in Intel hardware (we have 12 DMA
> rings, one of which is reserved for control traffic), and we have 2 host
> conrollers in recent systems. Due to the dedicated flow control we use now
> that's not possible but we are planning to make it to use shared flow
> control instead which allows more.
> 
> Not sure if anybody ever will create that many, though.

Yeah, that's not many, and a bit of a waste of a full major number.

> Second thing is that we use cdev_device_add() to manage the char device and
> the stream device as they are part of the same structure. I don't think
> that can be done with miscdevice.

Not yet, but see the patches on the list for how to do that properly.
You will have issues with disconnect/open/close that you need to handle
very carefully, especially as you are a dynamic device.  See this
thread:
	https://lore.kernel.org/r/20260427134659.95181-1-tzungbi@kernel.org

thanks,

greg k-h

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Andrew Lunn]

On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> Introduce USB4STREAM protocol and Linux implementation. This allows two
> (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> through a character device without need to go through the network stack.

Is this mutually exclusive to networking, on a device?

   Andrew

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Mika Westerberg]

On Tue, Apr 28, 2026 at 05:08:55PM +0200, Andrew Lunn wrote:
> On Tue, Apr 28, 2026 at 09:22:09AM +0200, Mika Westerberg wrote:
> > Introduce USB4STREAM protocol and Linux implementation. This allows two
> > (or more) hosts to transfer data directly over Thunderbolt/USB4 cable
> > through a character device without need to go through the network stack.
> 
> Is this mutually exclusive to networking, on a device?

No - they can co-exist.

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Andrew Lunn]

> > Is this mutually exclusive to networking, on a device?
> 
> No - they can co-exist.

Do they have there own rings? So that ethtool -C on the netdev will
not conflict with your configfs setting of the interrupt coalescing
for the stream?

	Andrew

Re: [PATCH 9/9] thunderbolt: Add support for USB4STREAM

[Mika Westerberg]

On Tue, Apr 28, 2026 at 06:45:34PM +0200, Andrew Lunn wrote:
> > > Is this mutually exclusive to networking, on a device?
> > 
> > No - they can co-exist.
> 
> Do they have there own rings? So that ethtool -C on the netdev will
> not conflict with your configfs setting of the interrupt coalescing
> for the stream?

Yes they use their own rings.