* RE: flow control in vmbus ring buffers
From: Michael Kelley @ 2019-12-24 18:03 UTC (permalink / raw)
To: Roman Kagan, linux-hyperv@vger.kernel.org
In-Reply-To: <20191224162832.GA168681@rkaganb.sw.ru>
From: Roman Kagan <rkagan@virtuozzo.com> Sent: Tuesday, December 24, 2019 8:29 AM
>
> On Tue, Dec 24, 2019 at 10:56:08AM +0000, Roman Kagan wrote:
> > I'm trying to get my head around how the flow control in vmbus ring
> > buffers works.
> >
> > In particular, I'm failing to see how the following can be prevented:
> >
> >
> > producer | consumer
> > ==================================================================
> > read read_index |
> > (enough room for packet) |
> > write pending_send_sz = 0 |
> > write packet |
> > update write_index |
> > | read write_index
> > read read_index | read packet
> > (not enough room for packet) | update read_index (= write_index)
> > | read pending_send_sz = 0
> > write pending_send_sz = X | skip notification
> > go to sleep | go to sleep
> > stall
> >
> > Could anybody please shed some light on how it's supposed to work?
>
> Sorry to reply to myself, but looks like the answer is to re-read
> read_index on the producer side after setting pending_send_sz.
>
Are you seeing a problem in the Linux guest code in drivers/hv/ring_buffer.c?
The only time the Linux guest as the producer sets pending_send_sz is in the
hv_socket code. Or are you looking at the KVM code that does emulation of
Hyper-V, where KVM is acting as the producer?
Just curious as to what prompted your question. I did the most recent fixes a
couple years ago to the Linux guest code for handling the ring buffer interaction
with Hyper-V, and if there is still a problem, I'm super interested. :-)
Michael
^ permalink raw reply
* Re: flow control in vmbus ring buffers
From: Roman Kagan @ 2019-12-25 8:46 UTC (permalink / raw)
To: Michael Kelley; +Cc: linux-hyperv@vger.kernel.org
In-Reply-To: <CY4PR21MB06295F8CEE034AD045EEBE79D7290@CY4PR21MB0629.namprd21.prod.outlook.com>
On Tue, Dec 24, 2019 at 06:03:15PM +0000, Michael Kelley wrote:
> From: Roman Kagan <rkagan@virtuozzo.com> Sent: Tuesday, December 24, 2019 8:29 AM
> >
> > On Tue, Dec 24, 2019 at 10:56:08AM +0000, Roman Kagan wrote:
> > > I'm trying to get my head around how the flow control in vmbus ring
> > > buffers works.
> > >
> > > In particular, I'm failing to see how the following can be prevented:
> > >
> > >
> > > producer | consumer
> > > ==================================================================
> > > read read_index |
> > > (enough room for packet) |
> > > write pending_send_sz = 0 |
> > > write packet |
> > > update write_index |
> > > | read write_index
> > > read read_index | read packet
> > > (not enough room for packet) | update read_index (= write_index)
> > > | read pending_send_sz = 0
> > > write pending_send_sz = X | skip notification
> > > go to sleep | go to sleep
> > > stall
> > >
> > > Could anybody please shed some light on how it's supposed to work?
> >
> > Sorry to reply to myself, but looks like the answer is to re-read
> > read_index on the producer side after setting pending_send_sz.
> >
>
> Are you seeing a problem in the Linux guest code in drivers/hv/ring_buffer.c?
No, nothing in particular.
> The only time the Linux guest as the producer sets pending_send_sz is in the
> hv_socket code.
Indeed. This surprized me somewhat, but, well...
> Or are you looking at the KVM code that does emulation of Hyper-V,
> where KVM is acting as the producer?
Yes, at the QEMU code, more precisely. I wrote it a while back and now
have some spare cycles to at last submit it to mainline QEMU; while
reviewing it I noticed this part that I did wrong.
Thanks,
Roman.
^ permalink raw reply
* [Patch v3 1/2] PCI: hv: Decouple the func definition in hv_dr_state from VSP message
From: longli @ 2019-12-26 19:40 UTC (permalink / raw)
To: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
Lorenzo Pieralisi, Andrew Murray, Bjorn Helgaas, linux-hyperv,
linux-pci, linux-kernel
Cc: Long Li
From: Long Li <longli@microsoft.com>
hv_dr_state is used to find present PCI devices on the bus. The structure
reuses struct pci_function_description from VSP message to describe a device.
To prepare support for pci_function_description v2, we need to decouple this
dependence in hv_dr_state so it can work with both v1 and v2 VSP messages.
There is no functionality change.
Signed-off-by: Long Li <longli@microsoft.com>
---
Changes
v2: changed some spaces to tabs, changed failure code to -ENOMEM
v3: revised comment for function hv_pci_devices_present(), reformatted patch title
drivers/pci/controller/pci-hyperv.c | 101 +++++++++++++++++++---------
1 file changed, 70 insertions(+), 31 deletions(-)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index f1f300218fab..4452c6bae6cf 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -507,10 +507,24 @@ struct hv_dr_work {
struct hv_pcibus_device *bus;
};
+struct hv_pcidev_description {
+ u16 v_id; /* vendor ID */
+ u16 d_id; /* device ID */
+ u8 rev;
+ u8 prog_intf;
+ u8 subclass;
+ u8 base_class;
+ u32 subsystem_id;
+ union win_slot_encoding win_slot;
+ u32 ser; /* serial number */
+ u32 flags;
+ u16 virtual_numa_node;
+};
+
struct hv_dr_state {
struct list_head list_entry;
u32 device_count;
- struct pci_function_description func[0];
+ struct hv_pcidev_description func[0];
};
enum hv_pcichild_state {
@@ -527,7 +541,7 @@ struct hv_pci_dev {
refcount_t refs;
enum hv_pcichild_state state;
struct pci_slot *pci_slot;
- struct pci_function_description desc;
+ struct hv_pcidev_description desc;
bool reported_missing;
struct hv_pcibus_device *hbus;
struct work_struct wrk;
@@ -1862,7 +1876,7 @@ static void q_resource_requirements(void *context, struct pci_response *resp,
* Return: Pointer to the new tracking struct
*/
static struct hv_pci_dev *new_pcichild_device(struct hv_pcibus_device *hbus,
- struct pci_function_description *desc)
+ struct hv_pcidev_description *desc)
{
struct hv_pci_dev *hpdev;
struct pci_child_message *res_req;
@@ -1973,7 +1987,7 @@ static void pci_devices_present_work(struct work_struct *work)
{
u32 child_no;
bool found;
- struct pci_function_description *new_desc;
+ struct hv_pcidev_description *new_desc;
struct hv_pci_dev *hpdev;
struct hv_pcibus_device *hbus;
struct list_head removed;
@@ -2090,43 +2104,26 @@ static void pci_devices_present_work(struct work_struct *work)
put_hvpcibus(hbus);
kfree(dr);
}
-
/**
- * hv_pci_devices_present() - Handles list of new children
+ * hv_pci_start_relations_work() - Queue work to start device discovery
* @hbus: Root PCI bus, as understood by this driver
- * @relations: Packet from host listing children
+ * @dr: The list of children returned from host
*
- * This function is invoked whenever a new list of devices for
- * this bus appears.
+ * Return: 0 on success, -errno on failure
*/
-static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
- struct pci_bus_relations *relations)
+static int hv_pci_start_relations_work(struct hv_pcibus_device *hbus,
+ struct hv_dr_state *dr)
{
- struct hv_dr_state *dr;
struct hv_dr_work *dr_wrk;
- unsigned long flags;
bool pending_dr;
+ unsigned long flags;
dr_wrk = kzalloc(sizeof(*dr_wrk), GFP_NOWAIT);
if (!dr_wrk)
- return;
-
- dr = kzalloc(offsetof(struct hv_dr_state, func) +
- (sizeof(struct pci_function_description) *
- (relations->device_count)), GFP_NOWAIT);
- if (!dr) {
- kfree(dr_wrk);
- return;
- }
+ return -ENOMEM;
INIT_WORK(&dr_wrk->wrk, pci_devices_present_work);
dr_wrk->bus = hbus;
- dr->device_count = relations->device_count;
- if (dr->device_count != 0) {
- memcpy(dr->func, relations->func,
- sizeof(struct pci_function_description) *
- dr->device_count);
- }
spin_lock_irqsave(&hbus->device_list_lock, flags);
/*
@@ -2144,6 +2141,47 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
get_hvpcibus(hbus);
queue_work(hbus->wq, &dr_wrk->wrk);
}
+
+ return 0;
+}
+
+/**
+ * hv_pci_devices_present() - Handles list of new children
+ * @hbus: Root PCI bus, as understood by this driver
+ * @relations: Packet from host listing children
+ *
+ * This function processes a new list of devices on the bus. The list of
+ * devices is discoverd by VSP and sent to us via VSP message
+ * PCI_BUS_RELATIONS, whenever a new list of devices for this bus appears.
+ */
+static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
+ struct pci_bus_relations *relations)
+{
+ struct hv_dr_state *dr;
+ int i;
+
+ dr = kzalloc(offsetof(struct hv_dr_state, func) +
+ (sizeof(struct hv_pcidev_description) *
+ (relations->device_count)), GFP_NOWAIT);
+
+ if (!dr)
+ return;
+
+ dr->device_count = relations->device_count;
+ for (i = 0; i < dr->device_count; i++) {
+ dr->func[i].v_id = relations->func[i].v_id;
+ dr->func[i].d_id = relations->func[i].d_id;
+ dr->func[i].rev = relations->func[i].rev;
+ dr->func[i].prog_intf = relations->func[i].prog_intf;
+ dr->func[i].subclass = relations->func[i].subclass;
+ dr->func[i].base_class = relations->func[i].base_class;
+ dr->func[i].subsystem_id = relations->func[i].subsystem_id;
+ dr->func[i].win_slot = relations->func[i].win_slot;
+ dr->func[i].ser = relations->func[i].ser;
+ }
+
+ if (hv_pci_start_relations_work(hbus, dr))
+ kfree(dr);
}
/**
@@ -3018,7 +3056,7 @@ static void hv_pci_bus_exit(struct hv_device *hdev)
struct pci_packet teardown_packet;
u8 buffer[sizeof(struct pci_message)];
} pkt;
- struct pci_bus_relations relations;
+ struct hv_dr_state *dr;
struct hv_pci_compl comp_pkt;
int ret;
@@ -3030,8 +3068,9 @@ static void hv_pci_bus_exit(struct hv_device *hdev)
return;
/* Delete any children which might still exist. */
- memset(&relations, 0, sizeof(relations));
- hv_pci_devices_present(hbus, &relations);
+ dr = kzalloc(sizeof(*dr), GFP_KERNEL);
+ if (dr && hv_pci_start_relations_work(hbus, dr))
+ kfree(dr);
ret = hv_send_resources_released(hdev);
if (ret)
--
2.17.1
^ permalink raw reply related
* [Patch v3 2/2] PCI: hv: Add support for protocol 1.3 and support PCI_BUS_RELATIONS2
From: longli @ 2019-12-26 19:40 UTC (permalink / raw)
To: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
Lorenzo Pieralisi, Andrew Murray, Bjorn Helgaas, linux-hyperv,
linux-pci, linux-kernel
Cc: Long Li
In-Reply-To: <1577389241-108450-1-git-send-email-longli@linuxonhyperv.com>
From: Long Li <longli@microsoft.com>
Starting with Hyper-V PCI protocol version 1.3, the host VSP can send
PCI_BUS_RELATIONS2 and pass the vNUMA node information for devices on the bus.
The vNUMA node tells which guest NUMA node this device is on based on guest
VM configuration topology and physical device inforamtion.
The patch adds code to negotiate v1.3 and process PCI_BUS_RELATIONS2.
Signed-off-by: Long Li <longli@microsoft.com>
---
Changes
v2: Changed some spaces to tabs, added put_pcichild() after get_pcichild_wslot(), renamed pci_assign_numa_node() to hv_pci_assign_numa_node()
drivers/pci/controller/pci-hyperv.c | 109 ++++++++++++++++++++++++++++
1 file changed, 109 insertions(+)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 4452c6bae6cf..b03119f94cd7 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -63,6 +63,7 @@
enum pci_protocol_version_t {
PCI_PROTOCOL_VERSION_1_1 = PCI_MAKE_VERSION(1, 1), /* Win10 */
PCI_PROTOCOL_VERSION_1_2 = PCI_MAKE_VERSION(1, 2), /* RS1 */
+ PCI_PROTOCOL_VERSION_1_3 = PCI_MAKE_VERSION(1, 3), /* Vibranium */
};
#define CPU_AFFINITY_ALL -1ULL
@@ -72,6 +73,7 @@ enum pci_protocol_version_t {
* first.
*/
static enum pci_protocol_version_t pci_protocol_versions[] = {
+ PCI_PROTOCOL_VERSION_1_3,
PCI_PROTOCOL_VERSION_1_2,
PCI_PROTOCOL_VERSION_1_1,
};
@@ -124,6 +126,7 @@ enum pci_message_type {
PCI_RESOURCES_ASSIGNED2 = PCI_MESSAGE_BASE + 0x16,
PCI_CREATE_INTERRUPT_MESSAGE2 = PCI_MESSAGE_BASE + 0x17,
PCI_DELETE_INTERRUPT_MESSAGE2 = PCI_MESSAGE_BASE + 0x18, /* unused */
+ PCI_BUS_RELATIONS2 = PCI_MESSAGE_BASE + 0x19,
PCI_MESSAGE_MAXIMUM
};
@@ -169,6 +172,26 @@ struct pci_function_description {
u32 ser; /* serial number */
} __packed;
+enum pci_device_description_flags {
+ HV_PCI_DEVICE_FLAG_NONE = 0x0,
+ HV_PCI_DEVICE_FLAG_NUMA_AFFINITY = 0x1,
+};
+
+struct pci_function_description2 {
+ u16 v_id; /* vendor ID */
+ u16 d_id; /* device ID */
+ u8 rev;
+ u8 prog_intf;
+ u8 subclass;
+ u8 base_class;
+ u32 subsystem_id;
+ union win_slot_encoding win_slot;
+ u32 ser; /* serial number */
+ u32 flags;
+ u16 virtual_numa_node;
+ u16 reserved;
+} __packed;
+
/**
* struct hv_msi_desc
* @vector: IDT entry
@@ -304,6 +327,12 @@ struct pci_bus_relations {
struct pci_function_description func[0];
} __packed;
+struct pci_bus_relations2 {
+ struct pci_incoming_message incoming;
+ u32 device_count;
+ struct pci_function_description2 func[0];
+} __packed;
+
struct pci_q_res_req_response {
struct vmpacket_descriptor hdr;
s32 status; /* negative values are failures */
@@ -1417,6 +1446,7 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
break;
case PCI_PROTOCOL_VERSION_1_2:
+ case PCI_PROTOCOL_VERSION_1_3:
size = hv_compose_msi_req_v2(&ctxt.int_pkts.v2,
dest,
hpdev->desc.win_slot.slot,
@@ -1798,6 +1828,27 @@ static void hv_pci_remove_slots(struct hv_pcibus_device *hbus)
}
}
+/*
+ * Set NUMA node for the devices on the bus
+ */
+static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
+{
+ struct pci_dev *dev;
+ struct pci_bus *bus = hbus->pci_bus;
+ struct hv_pci_dev *hv_dev;
+
+ list_for_each_entry(dev, &bus->devices, bus_list) {
+ hv_dev = get_pcichild_wslot(hbus, devfn_to_wslot(dev->devfn));
+ if (!hv_dev)
+ continue;
+
+ if (hv_dev->desc.flags & HV_PCI_DEVICE_FLAG_NUMA_AFFINITY)
+ set_dev_node(&dev->dev, hv_dev->desc.virtual_numa_node);
+
+ put_pcichild(hv_dev);
+ }
+}
+
/**
* create_root_hv_pci_bus() - Expose a new root PCI bus
* @hbus: Root PCI bus, as understood by this driver
@@ -1820,6 +1871,7 @@ static int create_root_hv_pci_bus(struct hv_pcibus_device *hbus)
pci_lock_rescan_remove();
pci_scan_child_bus(hbus->pci_bus);
+ hv_pci_assign_numa_node(hbus);
pci_bus_assign_resources(hbus->pci_bus);
hv_pci_assign_slots(hbus);
pci_bus_add_devices(hbus->pci_bus);
@@ -2088,6 +2140,7 @@ static void pci_devices_present_work(struct work_struct *work)
*/
pci_lock_rescan_remove();
pci_scan_child_bus(hbus->pci_bus);
+ hv_pci_assign_numa_node(hbus);
hv_pci_assign_slots(hbus);
pci_unlock_rescan_remove();
break;
@@ -2184,6 +2237,46 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
kfree(dr);
}
+/**
+ * hv_pci_devices_present2() - Handles list of new children
+ * @hbus: Root PCI bus, as understood by this driver
+ * @relations2: Packet from host listing children
+ *
+ * This function is the v2 version of hv_pci_devices_present()
+ */
+static void hv_pci_devices_present2(struct hv_pcibus_device *hbus,
+ struct pci_bus_relations2 *relations)
+{
+ struct hv_dr_state *dr;
+ int i;
+
+ dr = kzalloc(offsetof(struct hv_dr_state, func) +
+ (sizeof(struct hv_pcidev_description) *
+ (relations->device_count)), GFP_NOWAIT);
+
+ if (!dr)
+ return;
+
+ dr->device_count = relations->device_count;
+ for (i = 0; i < dr->device_count; i++) {
+ dr->func[i].v_id = relations->func[i].v_id;
+ dr->func[i].d_id = relations->func[i].d_id;
+ dr->func[i].rev = relations->func[i].rev;
+ dr->func[i].prog_intf = relations->func[i].prog_intf;
+ dr->func[i].subclass = relations->func[i].subclass;
+ dr->func[i].base_class = relations->func[i].base_class;
+ dr->func[i].subsystem_id = relations->func[i].subsystem_id;
+ dr->func[i].win_slot = relations->func[i].win_slot;
+ dr->func[i].ser = relations->func[i].ser;
+ dr->func[i].flags = relations->func[i].flags;
+ dr->func[i].virtual_numa_node =
+ relations->func[i].virtual_numa_node;
+ }
+
+ if (hv_pci_start_relations_work(hbus, dr))
+ kfree(dr);
+}
+
/**
* hv_eject_device_work() - Asynchronously handles ejection
* @work: Work struct embedded in internal device struct
@@ -2289,6 +2382,7 @@ static void hv_pci_onchannelcallback(void *context)
struct pci_response *response;
struct pci_incoming_message *new_message;
struct pci_bus_relations *bus_rel;
+ struct pci_bus_relations2 *bus_rel2;
struct pci_dev_inval_block *inval;
struct pci_dev_incoming *dev_message;
struct hv_pci_dev *hpdev;
@@ -2356,6 +2450,21 @@ static void hv_pci_onchannelcallback(void *context)
hv_pci_devices_present(hbus, bus_rel);
break;
+ case PCI_BUS_RELATIONS2:
+
+ bus_rel2 = (struct pci_bus_relations2 *)buffer;
+ if (bytes_recvd <
+ offsetof(struct pci_bus_relations2, func) +
+ (sizeof(struct pci_function_description2) *
+ (bus_rel2->device_count))) {
+ dev_err(&hbus->hdev->device,
+ "bus relations v2 too small\n");
+ break;
+ }
+
+ hv_pci_devices_present2(hbus, bus_rel2);
+ break;
+
case PCI_EJECT:
dev_message = (struct pci_dev_incoming *)buffer;
--
2.17.1
^ permalink raw reply related
* [PATCH AUTOSEL 5.4 108/187] hv_netvsc: Fix tx_table init in rndis_set_subchannel()
From: Sasha Levin @ 2019-12-27 17:39 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Haiyang Zhang, Jakub Kicinski, Sasha Levin, linux-hyperv, netdev
In-Reply-To: <20191227174055.4923-1-sashal@kernel.org>
From: Haiyang Zhang <haiyangz@microsoft.com>
[ Upstream commit c39ea5cba5a2e97fc01b78c85208bf31383b399c ]
Host can provide send indirection table messages anytime after RSS is
enabled by calling rndis_filter_set_rss_param(). So the host provided
table values may be overwritten by the initialization in
rndis_set_subchannel().
To prevent this problem, move the tx_table initialization before calling
rndis_filter_set_rss_param().
Fixes: a6fb6aa3cfa9 ("hv_netvsc: Set tx_table to equal weight after subchannels open")
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/net/hyperv/rndis_filter.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index abaf8156d19d..e3d3c9097ff1 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -1165,6 +1165,9 @@ int rndis_set_subchannel(struct net_device *ndev,
wait_event(nvdev->subchan_open,
atomic_read(&nvdev->open_chn) == nvdev->num_chn);
+ for (i = 0; i < VRSS_SEND_TAB_SIZE; i++)
+ ndev_ctx->tx_table[i] = i % nvdev->num_chn;
+
/* ignore failures from setting rss parameters, still have channels */
if (dev_info)
rndis_filter_set_rss_param(rdev, dev_info->rss_key);
@@ -1174,9 +1177,6 @@ int rndis_set_subchannel(struct net_device *ndev,
netif_set_real_num_tx_queues(ndev, nvdev->num_chn);
netif_set_real_num_rx_queues(ndev, nvdev->num_chn);
- for (i = 0; i < VRSS_SEND_TAB_SIZE; i++)
- ndev_ctx->tx_table[i] = i % nvdev->num_chn;
-
return 0;
}
--
2.20.1
^ permalink raw reply related
* [PATCH net-next, 0/3] Add vmbus dev_num and enable netvsc async probing
From: Haiyang Zhang @ 2019-12-28 23:46 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
Add dev_num for vmbus device based on channel offer sequence.
Use this number for nic naming, and enable async probing.
Haiyang Zhang (3):
Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
Drivers: hv: vmbus: Add dev_num to sysfs
hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Documentation/ABI/stable/sysfs-bus-vmbus | 8 +++++++
drivers/hv/channel_mgmt.c | 40 ++++++++++++++++++++++++++++++--
drivers/hv/vmbus_drv.c | 13 +++++++++++
drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++---
include/linux/hyperv.h | 6 +++++
5 files changed, 80 insertions(+), 5 deletions(-)
--
1.8.3.1
^ permalink raw reply
* [PATCH net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Haiyang Zhang @ 2019-12-28 23:46 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577576793-113222-1-git-send-email-haiyangz@microsoft.com>
This number is set to the first available number, starting from zero,
when a vmbus device’s primary channel is offered.
It will be used for stable naming when Async probing is used.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
drivers/hv/channel_mgmt.c | 40 ++++++++++++++++++++++++++++++++++++++--
include/linux/hyperv.h | 6 ++++++
2 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 8eb1675..b14c6a2 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)
if (!channel)
return NULL;
+ channel->dev_num = HV_DEV_NUM_INVALID;
+
spin_lock_init(&channel->lock);
init_completion(&channel->rescind_event);
@@ -541,6 +543,38 @@ static void vmbus_add_channel_work(struct work_struct *work)
}
/*
+ * Get the first available device number of its type, then
+ * record it in the channel structure.
+ */
+static void hv_set_devnum(struct vmbus_channel *newchannel)
+{
+ struct vmbus_channel *channel;
+ unsigned int i = 0;
+ bool found;
+
+ BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex));
+
+next:
+ found = false;
+
+ list_for_each_entry(channel, &vmbus_connection.chn_list, listentry) {
+ if (i == channel->dev_num &&
+ guid_equal(&channel->offermsg.offer.if_type,
+ &newchannel->offermsg.offer.if_type)) {
+ found = true;
+ break;
+ }
+ }
+
+ if (found) {
+ i++;
+ goto next;
+ }
+
+ newchannel->dev_num = i;
+}
+
+/*
* vmbus_process_offer - Process the offer by creating a channel/device
* associated with this offer
*/
@@ -573,10 +607,12 @@ static void vmbus_process_offer(struct vmbus_channel *newchannel)
}
}
- if (fnew)
+ if (fnew) {
+ hv_set_devnum(newchannel);
+
list_add_tail(&newchannel->listentry,
&vmbus_connection.chn_list);
- else {
+ } else {
/*
* Check to see if this is a valid sub-channel.
*/
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 26f3aee..4f110c5 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -718,6 +718,8 @@ struct vmbus_device {
bool perf_device;
};
+#define HV_DEV_NUM_INVALID (-1)
+
struct vmbus_channel {
struct list_head listentry;
@@ -849,6 +851,10 @@ struct vmbus_channel {
*/
struct vmbus_channel *primary_channel;
/*
+ * Used for device naming based on channel offer sequence.
+ */
+ int dev_num;
+ /*
* Support per-channel state for use by vmbus drivers.
*/
void *per_channel_state;
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang @ 2019-12-28 23:46 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577576793-113222-1-git-send-email-haiyangz@microsoft.com>
The dev_num field in vmbus channel structure is assigned to the first
available number when the channel is offered. So netvsc driver uses it
for NIC naming based on channel offer sequence. Now re-enable the async
probing mode for faster probing.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index f3f9eb8..39c412f 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
struct net_device_context *net_device_ctx;
struct netvsc_device_info *device_info = NULL;
struct netvsc_device *nvdev;
+ char name[IFNAMSIZ];
int ret = -ENOMEM;
- net = alloc_etherdev_mq(sizeof(struct net_device_context),
- VRSS_CHANNEL_MAX);
+ snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
+ net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
+ NET_NAME_ENUM, ether_setup,
+ VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
+
if (!net)
goto no_net;
@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
net->max_mtu = ETH_DATA_LEN;
ret = register_netdevice(net);
+
+ if (ret == -EEXIST) {
+ pr_info("NIC name %s exists, request another name.\n",
+ net->name);
+ strlcpy(net->name, "eth%d", IFNAMSIZ);
+ ret = register_netdevice(net);
+ }
+
if (ret != 0) {
pr_err("Unable to register netdev.\n");
goto register_failed;
@@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev)
.suspend = netvsc_suspend,
.resume = netvsc_resume,
.driver = {
- .probe_type = PROBE_FORCE_SYNCHRONOUS,
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
},
};
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
From: Haiyang Zhang @ 2019-12-28 23:46 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577576793-113222-1-git-send-email-haiyangz@microsoft.com>
It's a number based on the vmbus device offer sequence.
Useful for device naming when using Async probing.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++
drivers/hv/vmbus_drv.c | 13 +++++++++++++
2 files changed, 21 insertions(+)
diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus b/Documentation/ABI/stable/sysfs-bus-vmbus
index 8e8d167..a42225d 100644
--- a/Documentation/ABI/stable/sysfs-bus-vmbus
+++ b/Documentation/ABI/stable/sysfs-bus-vmbus
@@ -49,6 +49,14 @@ Contact: Stephen Hemminger <sthemmin@microsoft.com>
Description: This NUMA node to which the VMBUS device is
attached, or -1 if the node is unknown.
+What: /sys/bus/vmbus/devices/<UUID>/dev_num
+Date: Dec 2019
+KernelVersion: 5.5
+Contact: Haiyang Zhang <haiyangz@microsoft.com>
+Description: A number based on the vmbus device offer sequence.
+ Useful for device naming when using Async probing.
+Users: Debugging tools and userspace drivers
+
What: /sys/bus/vmbus/devices/<UUID>/channels/<N>
Date: September. 2017
KernelVersion: 4.14
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 4ef5a66..fe7aefa 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -214,6 +214,18 @@ static ssize_t numa_node_show(struct device *dev,
static DEVICE_ATTR_RO(numa_node);
#endif
+static ssize_t dev_num_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct hv_device *hv_dev = device_to_hv_device(dev);
+
+ if (!hv_dev->channel)
+ return -ENODEV;
+
+ return sprintf(buf, "%d\n", hv_dev->channel->dev_num);
+}
+static DEVICE_ATTR_RO(dev_num);
+
static ssize_t server_monitor_pending_show(struct device *dev,
struct device_attribute *dev_attr,
char *buf)
@@ -598,6 +610,7 @@ static ssize_t driver_override_show(struct device *dev,
#ifdef CONFIG_NUMA
&dev_attr_numa_node.attr,
#endif
+ &dev_attr_dev_num.attr,
&dev_attr_server_monitor_pending.attr,
&dev_attr_client_monitor_pending.attr,
&dev_attr_server_monitor_latency.attr,
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Stephen Hemminger @ 2019-12-29 0:13 UTC (permalink / raw)
To: Haiyang Zhang
Cc: sashal, linux-hyperv, netdev, kys, sthemmin, olaf, vkuznets,
davem, linux-kernel
In-Reply-To: <1577576793-113222-4-git-send-email-haiyangz@microsoft.com>
On Sat, 28 Dec 2019 15:46:33 -0800
Haiyang Zhang <haiyangz@microsoft.com> wrote:
> - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> - VRSS_CHANNEL_MAX);
> + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
> + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> + NET_NAME_ENUM, ether_setup,
> + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> +
Naming is a hard problem, and best left to userspace.
By choosing ethN as a naming policy, you potentially run into naming
conflicts with other non netvsc devices like those passed through or
SR-IOV devices.
Better to have udev use dev_num and use something like envN or something.
Udev also handles SRIOV devices in later versions.
Fighting against systemd, netplan, etc is not going to be make friends.
^ permalink raw reply
* Re: [PATCH net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Stephen Hemminger @ 2019-12-29 0:20 UTC (permalink / raw)
To: Haiyang Zhang
Cc: sashal, linux-hyperv, netdev, kys, sthemmin, olaf, vkuznets,
davem, linux-kernel
In-Reply-To: <1577576793-113222-2-git-send-email-haiyangz@microsoft.com>
On Sat, 28 Dec 2019 15:46:31 -0800
Haiyang Zhang <haiyangz@microsoft.com> wrote:
> +
> +next:
> + found = false;
> +
> + list_for_each_entry(channel, &vmbus_connection.chn_list, listentry) {
> + if (i == channel->dev_num &&
> + guid_equal(&channel->offermsg.offer.if_type,
> + &newchannel->offermsg.offer.if_type)) {
> + found = true;
> + break;
> + }
> + }
> +
> + if (found) {
> + i++;
> + goto next;
> + }
> +
Overall, keeping track of dev_num is a good solution.
I prefer not having a loop coded with goto's. Why not
a nested loop. Also, there already is a search of the channel
list in vmbus_process_offer() so why is another lookup needed?
^ permalink raw reply
* RE: [PATCH net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Haiyang Zhang @ 2019-12-29 1:06 UTC (permalink / raw)
To: Stephen Hemminger
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, KY Srinivasan, Stephen Hemminger,
olaf@aepfle.de, vkuznets, davem@davemloft.net,
linux-kernel@vger.kernel.org
In-Reply-To: <20191228162002.3a603c8b@hermes.lan>
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Saturday, December 28, 2019 7:20 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: sashal@kernel.org; linux-hyperv@vger.kernel.org; netdev@vger.kernel.org;
> KY Srinivasan <kys@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; olaf@aepfle.de; vkuznets
> <vkuznets@redhat.com>; davem@davemloft.net; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable
> based on channel offer sequence
>
> On Sat, 28 Dec 2019 15:46:31 -0800
> Haiyang Zhang <haiyangz@microsoft.com> wrote:
>
> > +
> > +next:
> > + found = false;
> > +
> > + list_for_each_entry(channel, &vmbus_connection.chn_list, listentry) {
> > + if (i == channel->dev_num &&
> > + guid_equal(&channel->offermsg.offer.if_type,
> > + &newchannel->offermsg.offer.if_type)) {
> > + found = true;
> > + break;
> > + }
> > + }
> > +
> > + if (found) {
> > + i++;
> > + goto next;
> > + }
> > +
>
> Overall, keeping track of dev_num is a good solution.
>
> I prefer not having a loop coded with goto's. Why not
> a nested loop.
Sure, I can use a nested loop.
> Also, there already is a search of the channel
> list in vmbus_process_offer() so why is another lookup needed?
vmbus_process_offer() looks for the if_instance and if_type matches
to determine if this is a subchannel vs primary channel. The loop
terminates at different condition from hv_set_devnum().
So I didn't re-use the existing loop.
And this kind of search happens only during channel offering, and
doesn't impact performance much.
Thanks,
- Haiyang
^ permalink raw reply
* RE: [PATCH net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang @ 2019-12-29 1:23 UTC (permalink / raw)
To: Stephen Hemminger
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, KY Srinivasan, Stephen Hemminger,
olaf@aepfle.de, vkuznets, davem@davemloft.net,
linux-kernel@vger.kernel.org
In-Reply-To: <20191228161318.4501bb79@hermes.lan>
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Saturday, December 28, 2019 7:13 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: sashal@kernel.org; linux-hyperv@vger.kernel.org; netdev@vger.kernel.org;
> KY Srinivasan <kys@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; olaf@aepfle.de; vkuznets
> <vkuznets@redhat.com>; davem@davemloft.net; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer
> sequence and use async probe
>
> On Sat, 28 Dec 2019 15:46:33 -0800
> Haiyang Zhang <haiyangz@microsoft.com> wrote:
>
> > - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> > - VRSS_CHANNEL_MAX);
> > + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
> > + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> > + NET_NAME_ENUM, ether_setup,
> > + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> > +
>
> Naming is a hard problem, and best left to userspace.
> By choosing ethN as a naming policy, you potentially run into naming
> conflicts with other non netvsc devices like those passed through or
> SR-IOV devices.
If the dev_num based naming conflicts with existing device, it will fall back to
the previous scheme: "choose the next available eth* name by specifying eth%d".
See the fall back code path below:
if (ret == -EEXIST) {
pr_info("NIC name %s exists, request another name.\n",
net->name);
strlcpy(net->name, "eth%d", IFNAMSIZ);
ret = register_netdevice(net);
}
> Better to have udev use dev_num and use something like envN or something.
> Udev also handles SRIOV devices in later versions.
>
> Fighting against systemd, netplan, etc is not going to be make friends.
When netvsc was initially created, it uses "seth*" naming. But with strong requests
from many customers, we switched back to the regular "eth*" naming format.
The results of using dev_num based "eth*" names is same as what we are doing now --
The existing synchronous probing already generates "eth*" based on channel offer
sequence. With my patch, it still generates the same naming, but with asynchronous
probing.
So all the udev, or other user daemons, sees the same name for each device as before.
And, they can still set these names like what they do today.
Thanks,
- Haiyang
^ permalink raw reply
* [PATCH V2,net-next, 0/3] Add vmbus dev_num and enable netvsc async probing
From: Haiyang Zhang @ 2019-12-30 20:13 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
Add dev_num for vmbus device based on channel offer sequence.
Use this number for nic naming, and enable async probing.
Haiyang Zhang (3):
Drivers: hv: vmbus: Add a dev_num variable based on channel offer
sequence
Drivers: hv: vmbus: Add dev_num to sysfs
hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Documentation/ABI/stable/sysfs-bus-vmbus | 8 +++++++
drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++--
drivers/hv/vmbus_drv.c | 13 +++++++++++
drivers/net/hyperv/netvsc_drv.c | 18 ++++++++++++---
include/linux/hyperv.h | 6 +++++
5 files changed, 78 insertions(+), 5 deletions(-)
--
1.8.3.1
^ permalink raw reply
* [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
From: Haiyang Zhang @ 2019-12-30 20:13 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577736814-21112-1-git-send-email-haiyangz@microsoft.com>
It's a number based on the vmbus device offer sequence.
Useful for device naming when using Async probing.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++
drivers/hv/vmbus_drv.c | 13 +++++++++++++
2 files changed, 21 insertions(+)
diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus b/Documentation/ABI/stable/sysfs-bus-vmbus
index 8e8d167..a42225d 100644
--- a/Documentation/ABI/stable/sysfs-bus-vmbus
+++ b/Documentation/ABI/stable/sysfs-bus-vmbus
@@ -49,6 +49,14 @@ Contact: Stephen Hemminger <sthemmin@microsoft.com>
Description: This NUMA node to which the VMBUS device is
attached, or -1 if the node is unknown.
+What: /sys/bus/vmbus/devices/<UUID>/dev_num
+Date: Dec 2019
+KernelVersion: 5.5
+Contact: Haiyang Zhang <haiyangz@microsoft.com>
+Description: A number based on the vmbus device offer sequence.
+ Useful for device naming when using Async probing.
+Users: Debugging tools and userspace drivers
+
What: /sys/bus/vmbus/devices/<UUID>/channels/<N>
Date: September. 2017
KernelVersion: 4.14
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 4ef5a66..fe7aefa 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -214,6 +214,18 @@ static ssize_t numa_node_show(struct device *dev,
static DEVICE_ATTR_RO(numa_node);
#endif
+static ssize_t dev_num_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct hv_device *hv_dev = device_to_hv_device(dev);
+
+ if (!hv_dev->channel)
+ return -ENODEV;
+
+ return sprintf(buf, "%d\n", hv_dev->channel->dev_num);
+}
+static DEVICE_ATTR_RO(dev_num);
+
static ssize_t server_monitor_pending_show(struct device *dev,
struct device_attribute *dev_attr,
char *buf)
@@ -598,6 +610,7 @@ static ssize_t driver_override_show(struct device *dev,
#ifdef CONFIG_NUMA
&dev_attr_numa_node.attr,
#endif
+ &dev_attr_dev_num.attr,
&dev_attr_server_monitor_pending.attr,
&dev_attr_client_monitor_pending.attr,
&dev_attr_server_monitor_latency.attr,
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang @ 2019-12-30 20:13 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577736814-21112-1-git-send-email-haiyangz@microsoft.com>
The dev_num field in vmbus channel structure is assigned to the first
available number when the channel is offered. So netvsc driver uses it
for NIC naming based on channel offer sequence. Now re-enable the async
probing mode for faster probing.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index f3f9eb8..39c412f 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
struct net_device_context *net_device_ctx;
struct netvsc_device_info *device_info = NULL;
struct netvsc_device *nvdev;
+ char name[IFNAMSIZ];
int ret = -ENOMEM;
- net = alloc_etherdev_mq(sizeof(struct net_device_context),
- VRSS_CHANNEL_MAX);
+ snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
+ net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
+ NET_NAME_ENUM, ether_setup,
+ VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
+
if (!net)
goto no_net;
@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
net->max_mtu = ETH_DATA_LEN;
ret = register_netdevice(net);
+
+ if (ret == -EEXIST) {
+ pr_info("NIC name %s exists, request another name.\n",
+ net->name);
+ strlcpy(net->name, "eth%d", IFNAMSIZ);
+ ret = register_netdevice(net);
+ }
+
if (ret != 0) {
pr_err("Unable to register netdev.\n");
goto register_failed;
@@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev)
.suspend = netvsc_suspend,
.resume = netvsc_resume,
.driver = {
- .probe_type = PROBE_FORCE_SYNCHRONOUS,
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
},
};
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Haiyang Zhang @ 2019-12-30 20:13 UTC (permalink / raw)
To: sashal, linux-hyperv, netdev
Cc: haiyangz, kys, sthemmin, olaf, vkuznets, davem, linux-kernel
In-Reply-To: <1577736814-21112-1-git-send-email-haiyangz@microsoft.com>
This number is set to the first available number, starting from zero,
when a vmbus device’s primary channel is offered.
It will be used for stable naming when Async probing is used.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
Changes
V2:
Use nest loops in hv_set_devnum, instead of goto.
drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++++++++--
include/linux/hyperv.h | 6 ++++++
2 files changed, 42 insertions(+), 2 deletions(-)
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 8eb1675..00fa2db 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)
if (!channel)
return NULL;
+ channel->dev_num = HV_DEV_NUM_INVALID;
+
spin_lock_init(&channel->lock);
init_completion(&channel->rescind_event);
@@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct work_struct *work)
}
/*
+ * Get the first available device number of its type, then
+ * record it in the channel structure.
+ */
+static void hv_set_devnum(struct vmbus_channel *newchannel)
+{
+ struct vmbus_channel *channel;
+ int i = -1;
+ bool found;
+
+ BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex));
+
+ do {
+ i++;
+ found = false;
+
+ list_for_each_entry(channel, &vmbus_connection.chn_list,
+ listentry) {
+ if (i == channel->dev_num &&
+ guid_equal(&channel->offermsg.offer.if_type,
+ &newchannel->offermsg.offer.if_type)) {
+ found = true;
+ break;
+ }
+ }
+ } while (found);
+
+ newchannel->dev_num = i;
+}
+
+/*
* vmbus_process_offer - Process the offer by creating a channel/device
* associated with this offer
*/
@@ -573,10 +605,12 @@ static void vmbus_process_offer(struct vmbus_channel *newchannel)
}
}
- if (fnew)
+ if (fnew) {
+ hv_set_devnum(newchannel);
+
list_add_tail(&newchannel->listentry,
&vmbus_connection.chn_list);
- else {
+ } else {
/*
* Check to see if this is a valid sub-channel.
*/
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 26f3aee..4f110c5 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -718,6 +718,8 @@ struct vmbus_device {
bool perf_device;
};
+#define HV_DEV_NUM_INVALID (-1)
+
struct vmbus_channel {
struct list_head listentry;
@@ -849,6 +851,10 @@ struct vmbus_channel {
*/
struct vmbus_channel *primary_channel;
/*
+ * Used for device naming based on channel offer sequence.
+ */
+ int dev_num;
+ /*
* Support per-channel state for use by vmbus drivers.
*/
void *per_channel_state;
--
1.8.3.1
^ permalink raw reply related
* RE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Michael Kelley @ 2019-12-31 1:34 UTC (permalink / raw)
To: Haiyang Zhang, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <1577736814-21112-2-git-send-email-haiyangz@microsoft.com>
From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30, 2019 12:14 PM
>
> This number is set to the first available number, starting from zero,
> when a vmbus device's primary channel is offered.
Let's use "VMBus" as the capitalization in text.
> It will be used for stable naming when Async probing is used.
>
> Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> ---
> Changes
> V2:
> Use nest loops in hv_set_devnum, instead of goto.
>
> drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++++++++--
> include/linux/hyperv.h | 6 ++++++
> 2 files changed, 42 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> index 8eb1675..00fa2db 100644
> --- a/drivers/hv/channel_mgmt.c
> +++ b/drivers/hv/channel_mgmt.c
> @@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)
> if (!channel)
> return NULL;
>
> + channel->dev_num = HV_DEV_NUM_INVALID;
> +
> spin_lock_init(&channel->lock);
> init_completion(&channel->rescind_event);
>
> @@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct work_struct *work)
> }
>
> /*
> + * Get the first available device number of its type, then
> + * record it in the channel structure.
> + */
> +static void hv_set_devnum(struct vmbus_channel *newchannel)
> +{
> + struct vmbus_channel *channel;
> + int i = -1;
> + bool found;
> +
> + BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex));
> +
> + do {
> + i++;
> + found = false;
> +
> + list_for_each_entry(channel, &vmbus_connection.chn_list,
> + listentry) {
> + if (i == channel->dev_num &&
> + guid_equal(&channel->offermsg.offer.if_type,
> + &newchannel->offermsg.offer.if_type)) {
> + found = true;
> + break;
> + }
> + }
> + } while (found);
> +
> + newchannel->dev_num = i;
> +}
> +
It took me a little while to figure out what the above algorithm is doing.
Perhaps it would help to rename the "found" variable to "in_use", and add
this comment before the start of the "do" loop:
Iterate through each possible device number starting at zero. If the device
number is already in use for a device of this type, try the next device number
until finding one that is not in use. This approach selects the smallest
device number that is not in use, and so reuses any numbers that are freed
by devices that have been removed.
> +/*
> * vmbus_process_offer - Process the offer by creating a channel/device
> * associated with this offer
> */
> @@ -573,10 +605,12 @@ static void vmbus_process_offer(struct vmbus_channel
> *newchannel)
> }
> }
>
> - if (fnew)
> + if (fnew) {
> + hv_set_devnum(newchannel);
> +
> list_add_tail(&newchannel->listentry,
> &vmbus_connection.chn_list);
> - else {
> + } else {
> /*
> * Check to see if this is a valid sub-channel.
> */
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index 26f3aee..4f110c5 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -718,6 +718,8 @@ struct vmbus_device {
> bool perf_device;
> };
>
> +#define HV_DEV_NUM_INVALID (-1)
> +
> struct vmbus_channel {
> struct list_head listentry;
>
> @@ -849,6 +851,10 @@ struct vmbus_channel {
> */
> struct vmbus_channel *primary_channel;
> /*
> + * Used for device naming based on channel offer sequence.
> + */
> + int dev_num;
> + /*
> * Support per-channel state for use by vmbus drivers.
> */
> void *per_channel_state;
> --
> 1.8.3.1
^ permalink raw reply
* RE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
From: Michael Kelley @ 2019-12-31 1:34 UTC (permalink / raw)
To: Haiyang Zhang, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <1577736814-21112-3-git-send-email-haiyangz@microsoft.com>
From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30, 2019 12:14 PM
>
> It's a number based on the vmbus device offer sequence.
Let's use "VMBus" as the capitalization.
> Useful for device naming when using Async probing.
>
> Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> ---
> Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++
> drivers/hv/vmbus_drv.c | 13 +++++++++++++
> 2 files changed, 21 insertions(+)
>
> diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus
> b/Documentation/ABI/stable/sysfs-bus-vmbus
> index 8e8d167..a42225d 100644
> --- a/Documentation/ABI/stable/sysfs-bus-vmbus
> +++ b/Documentation/ABI/stable/sysfs-bus-vmbus
> @@ -49,6 +49,14 @@ Contact: Stephen Hemminger <sthemmin@microsoft.com>
> Description: This NUMA node to which the VMBUS device is
> attached, or -1 if the node is unknown.
>
> +What: /sys/bus/vmbus/devices/<UUID>/dev_num
> +Date: Dec 2019
> +KernelVersion: 5.5
> +Contact: Haiyang Zhang <haiyangz@microsoft.com>
> +Description: A number based on the vmbus device offer sequence.
The capitalization of "VMBus" is already inconsistent in this file in that
we have "VMBus" and "VMBUS" in the text. Let's not introduce another
capitalization -- use "VMBus".
> + Useful for device naming when using Async probing.
> +Users: Debugging tools and userspace drivers
> +
> What: /sys/bus/vmbus/devices/<UUID>/channels/<N>
> Date: September. 2017
> KernelVersion: 4.14
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 4ef5a66..fe7aefa 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -214,6 +214,18 @@ static ssize_t numa_node_show(struct device *dev,
> static DEVICE_ATTR_RO(numa_node);
> #endif
>
> +static ssize_t dev_num_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct hv_device *hv_dev = device_to_hv_device(dev);
> +
> + if (!hv_dev->channel)
> + return -ENODEV;
> +
> + return sprintf(buf, "%d\n", hv_dev->channel->dev_num);
> +}
> +static DEVICE_ATTR_RO(dev_num);
> +
> static ssize_t server_monitor_pending_show(struct device *dev,
> struct device_attribute *dev_attr,
> char *buf)
> @@ -598,6 +610,7 @@ static ssize_t driver_override_show(struct device *dev,
> #ifdef CONFIG_NUMA
> &dev_attr_numa_node.attr,
> #endif
> + &dev_attr_dev_num.attr,
> &dev_attr_server_monitor_pending.attr,
> &dev_attr_client_monitor_pending.attr,
> &dev_attr_server_monitor_latency.attr,
> --
> 1.8.3.1
^ permalink raw reply
* RE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Michael Kelley @ 2019-12-31 1:35 UTC (permalink / raw)
To: Haiyang Zhang, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <1577736814-21112-4-git-send-email-haiyangz@microsoft.com>
From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30, 2019 12:14 PM
>
> The dev_num field in vmbus channel structure is assigned to the first
Let's use "VMBus" in text.
> available number when the channel is offered. So netvsc driver uses it
> for NIC naming based on channel offer sequence. Now re-enable the async
> probing mode for faster probing.
>
> Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> ---
> drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
> 1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
> index f3f9eb8..39c412f 100644
> --- a/drivers/net/hyperv/netvsc_drv.c
> +++ b/drivers/net/hyperv/netvsc_drv.c
> @@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
> struct net_device_context *net_device_ctx;
> struct netvsc_device_info *device_info = NULL;
> struct netvsc_device *nvdev;
> + char name[IFNAMSIZ];
> int ret = -ENOMEM;
>
> - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> - VRSS_CHANNEL_MAX);
> + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
> + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> + NET_NAME_ENUM, ether_setup,
> + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> +
> if (!net)
> goto no_net;
>
> @@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
> net->max_mtu = ETH_DATA_LEN;
>
> ret = register_netdevice(net);
> +
> + if (ret == -EEXIST) {
> + pr_info("NIC name %s exists, request another name.\n",
> + net->name);
> + strlcpy(net->name, "eth%d", IFNAMSIZ);
> + ret = register_netdevice(net);
The message above could be confusing. "request another name" sounds
like a directive to the user/sysadmin, which I don't think it is. Perhaps
better would be "requesting another name", which says the system is
handling the collision automatically.
Also will this second call to register_netdevice() actually assign a numeric
name? If so, it would be really nice for the message to be output *after*
the second call to register_netdevice() and to show both the originally
selected name that collided as well as the new name. We sometimes get
into NIC naming issues with customers in Azure, and when a new name
has to be selected it will help debugging if we can show both the original
selection and the new selection. With both pieces of data, there's less
guessing about who did what regarding NIC naming.
Finally, having to choose a different name because of a collision does
*not* update the channel->dev_num value. Subsequent calls to
hv_set_devnum() will detect "in use" based on the originally assigned
dev_num value. I don't think that fundamentally breaks anything, but
I wondered if you had thought about that case. Maybe a comment here
to that effect would help a future reader of this code.
Michael
> + }
> +
> if (ret != 0) {
> pr_err("Unable to register netdev.\n");
> goto register_failed;
> @@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev)
> .suspend = netvsc_suspend,
> .resume = netvsc_resume,
> .driver = {
> - .probe_type = PROBE_FORCE_SYNCHRONOUS,
> + .probe_type = PROBE_PREFER_ASYNCHRONOUS,
> },
> };
>
> --
> 1.8.3.1
^ permalink raw reply
* Re: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Roman Kagan @ 2019-12-31 11:34 UTC (permalink / raw)
To: Haiyang Zhang
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, kys@microsoft.com, sthemmin@microsoft.com,
olaf@aepfle.de, vkuznets@redhat.com, davem@davemloft.net,
linux-kernel@vger.kernel.org
In-Reply-To: <1577736814-21112-4-git-send-email-haiyangz@microsoft.com>
On Mon, Dec 30, 2019 at 12:13:34PM -0800, Haiyang Zhang wrote:
> The dev_num field in vmbus channel structure is assigned to the first
> available number when the channel is offered. So netvsc driver uses it
> for NIC naming based on channel offer sequence. Now re-enable the async
> probing mode for faster probing.
>
> Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> ---
> drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
> 1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
> index f3f9eb8..39c412f 100644
> --- a/drivers/net/hyperv/netvsc_drv.c
> +++ b/drivers/net/hyperv/netvsc_drv.c
> @@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
> struct net_device_context *net_device_ctx;
> struct netvsc_device_info *device_info = NULL;
> struct netvsc_device *nvdev;
> + char name[IFNAMSIZ];
> int ret = -ENOMEM;
>
> - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> - VRSS_CHANNEL_MAX);
> + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
How is this supposed to work when there are other ethernet device types
on the system, which may claim the same device names?
> + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> + NET_NAME_ENUM, ether_setup,
> + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> +
> if (!net)
> goto no_net;
>
> @@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
> net->max_mtu = ETH_DATA_LEN;
>
> ret = register_netdevice(net);
> +
> + if (ret == -EEXIST) {
> + pr_info("NIC name %s exists, request another name.\n",
> + net->name);
> + strlcpy(net->name, "eth%d", IFNAMSIZ);
> + ret = register_netdevice(net);
> + }
IOW you want the device naming to be predictable, but don't guarantee
this?
I think the problem this patchset is trying to solve is much better
solved with a udev rule, similar to how it's done for PCI net devices.
And IMO the primary channel number, being a device's "hardware"
property, is more suited to be used in the device name, than this
completely ephemeral device number.
Thanks,
Roman.
> +
> if (ret != 0) {
> pr_err("Unable to register netdev.\n");
> goto register_failed;
> @@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev)
> .suspend = netvsc_suspend,
> .resume = netvsc_resume,
> .driver = {
> - .probe_type = PROBE_FORCE_SYNCHRONOUS,
> + .probe_type = PROBE_PREFER_ASYNCHRONOUS,
> },
> };
>
> --
> 1.8.3.1
>
^ permalink raw reply
* RE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Haiyang Zhang @ 2019-12-31 15:48 UTC (permalink / raw)
To: Michael Kelley, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: KY Srinivasan, Stephen Hemminger, olaf@aepfle.de, vkuznets,
davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <CY4PR21MB06294EA44916F9BD16138F94D7260@CY4PR21MB0629.namprd21.prod.outlook.com>
> -----Original Message-----
> From: Michael Kelley <mikelley@microsoft.com>
> Sent: Monday, December 30, 2019 8:35 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>; sashal@kernel.org; linux-
> hyperv@vger.kernel.org; netdev@vger.kernel.org
> Cc: Haiyang Zhang <haiyangz@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> olaf@aepfle.de; vkuznets <vkuznets@redhat.com>; davem@davemloft.net;
> linux-kernel@vger.kernel.org
> Subject: RE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num
> variable based on channel offer sequence
>
> From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30,
> 2019 12:14 PM
> >
> > This number is set to the first available number, starting from zero,
> > when a vmbus device's primary channel is offered.
>
> Let's use "VMBus" as the capitalization in text.
Sure I will.
>
> > It will be used for stable naming when Async probing is used.
> >
> > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > ---
> > Changes
> > V2:
> > Use nest loops in hv_set_devnum, instead of goto.
> >
> > drivers/hv/channel_mgmt.c | 38
> ++++++++++++++++++++++++++++++++++++--
> > include/linux/hyperv.h | 6 ++++++
> > 2 files changed, 42 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> > index 8eb1675..00fa2db 100644
> > --- a/drivers/hv/channel_mgmt.c
> > +++ b/drivers/hv/channel_mgmt.c
> > @@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)
> > if (!channel)
> > return NULL;
> >
> > + channel->dev_num = HV_DEV_NUM_INVALID;
> > +
> > spin_lock_init(&channel->lock);
> > init_completion(&channel->rescind_event);
> >
> > @@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct
> work_struct *work)
> > }
> >
> > /*
> > + * Get the first available device number of its type, then
> > + * record it in the channel structure.
> > + */
> > +static void hv_set_devnum(struct vmbus_channel *newchannel)
> > +{
> > + struct vmbus_channel *channel;
> > + int i = -1;
> > + bool found;
> > +
> > + BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex));
> > +
> > + do {
> > + i++;
> > + found = false;
> > +
> > + list_for_each_entry(channel, &vmbus_connection.chn_list,
> > + listentry) {
> > + if (i == channel->dev_num &&
> > + guid_equal(&channel->offermsg.offer.if_type,
> > + &newchannel->offermsg.offer.if_type)) {
> > + found = true;
> > + break;
> > + }
> > + }
> > + } while (found);
> > +
> > + newchannel->dev_num = i;
> > +}
> > +
>
> It took me a little while to figure out what the above algorithm is doing.
> Perhaps it would help to rename the "found" variable to "in_use", and add
> this comment before the start of the "do" loop:
>
> Iterate through each possible device number starting at zero. If the device
> number is already in use for a device of this type, try the next device number
> until finding one that is not in use. This approach selects the smallest
> device number that is not in use, and so reuses any numbers that are freed
> by devices that have been removed.
I will rename the variable, and add the code comments.
Thanks,
- Haiyang
^ permalink raw reply
* RE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
From: Haiyang Zhang @ 2019-12-31 15:50 UTC (permalink / raw)
To: Michael Kelley, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: KY Srinivasan, Stephen Hemminger, olaf@aepfle.de, vkuznets,
davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <CY4PR21MB06299FE9E4C2023C6D886C84D7260@CY4PR21MB0629.namprd21.prod.outlook.com>
> -----Original Message-----
> From: Michael Kelley <mikelley@microsoft.com>
> Sent: Monday, December 30, 2019 8:35 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>; sashal@kernel.org; linux-
> hyperv@vger.kernel.org; netdev@vger.kernel.org
> Cc: Haiyang Zhang <haiyangz@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> olaf@aepfle.de; vkuznets <vkuznets@redhat.com>; davem@davemloft.net;
> linux-kernel@vger.kernel.org
> Subject: RE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
>
> From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30,
> 2019 12:14 PM
> >
> > It's a number based on the vmbus device offer sequence.
>
> Let's use "VMBus" as the capitalization.
I will.
>
> > Useful for device naming when using Async probing.
> >
> > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > ---
> > Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++
> > drivers/hv/vmbus_drv.c | 13 +++++++++++++
> > 2 files changed, 21 insertions(+)
> >
> > diff --git a/Documentation/ABI/stable/sysfs-bus-vmbus
> > b/Documentation/ABI/stable/sysfs-bus-vmbus
> > index 8e8d167..a42225d 100644
> > --- a/Documentation/ABI/stable/sysfs-bus-vmbus
> > +++ b/Documentation/ABI/stable/sysfs-bus-vmbus
> > @@ -49,6 +49,14 @@ Contact: Stephen Hemminger
> <sthemmin@microsoft.com>
> > Description: This NUMA node to which the VMBUS device is
> > attached, or -1 if the node is unknown.
> >
> > +What: /sys/bus/vmbus/devices/<UUID>/dev_num
> > +Date: Dec 2019
> > +KernelVersion: 5.5
> > +Contact: Haiyang Zhang <haiyangz@microsoft.com>
> > +Description: A number based on the vmbus device offer sequence.
>
> The capitalization of "VMBus" is already inconsistent in this file in that
> we have "VMBus" and "VMBUS" in the text. Let's not introduce another
> capitalization -- use "VMBus".
I will use "VMBus".
Thanks,
- Haiyang
^ permalink raw reply
* RE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang @ 2019-12-31 16:01 UTC (permalink / raw)
To: Michael Kelley, sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: KY Srinivasan, Stephen Hemminger, olaf@aepfle.de, vkuznets,
davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <CY4PR21MB0629771D2D6FECB76339EFA0D7260@CY4PR21MB0629.namprd21.prod.outlook.com>
> -----Original Message-----
> From: Michael Kelley <mikelley@microsoft.com>
> Sent: Monday, December 30, 2019 8:35 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>; sashal@kernel.org; linux-
> hyperv@vger.kernel.org; netdev@vger.kernel.org
> Cc: Haiyang Zhang <haiyangz@microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; Stephen Hemminger <sthemmin@microsoft.com>;
> olaf@aepfle.de; vkuznets <vkuznets@redhat.com>; davem@davemloft.net;
> linux-kernel@vger.kernel.org
> Subject: RE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus
> offer sequence and use async probe
>
> From: Haiyang Zhang <haiyangz@microsoft.com> Sent: Monday, December 30,
> 2019 12:14 PM
> >
> > The dev_num field in vmbus channel structure is assigned to the first
>
> Let's use "VMBus" in text.
I will.
>
> > available number when the channel is offered. So netvsc driver uses it
> > for NIC naming based on channel offer sequence. Now re-enable the async
> > probing mode for faster probing.
> >
> > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > ---
> > drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
> > 1 file changed, 15 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
> > index f3f9eb8..39c412f 100644
> > --- a/drivers/net/hyperv/netvsc_drv.c
> > +++ b/drivers/net/hyperv/netvsc_drv.c
> > @@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
> > struct net_device_context *net_device_ctx;
> > struct netvsc_device_info *device_info = NULL;
> > struct netvsc_device *nvdev;
> > + char name[IFNAMSIZ];
> > int ret = -ENOMEM;
> >
> > - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> > - VRSS_CHANNEL_MAX);
> > + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
> > + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> > + NET_NAME_ENUM, ether_setup,
> > + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> > +
> > if (!net)
> > goto no_net;
> >
> > @@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
> > net->max_mtu = ETH_DATA_LEN;
> >
> > ret = register_netdevice(net);
> > +
> > + if (ret == -EEXIST) {
> > + pr_info("NIC name %s exists, request another name.\n",
> > + net->name);
> > + strlcpy(net->name, "eth%d", IFNAMSIZ);
> > + ret = register_netdevice(net);
>
> The message above could be confusing. "request another name" sounds
> like a directive to the user/sysadmin, which I don't think it is. Perhaps
> better would be "requesting another name", which says the system is
> handling the collision automatically.
>
> Also will this second call to register_netdevice() actually assign a numeric
> name? If so, it would be really nice for the message to be output *after*
> the second call to register_netdevice() and to show both the originally
> selected name that collided as well as the new name. We sometimes get
> into NIC naming issues with customers in Azure, and when a new name
> has to be selected it will help debugging if we can show both the original
> selection and the new selection. With both pieces of data, there's less
> guessing about who did what regarding NIC naming.
Good idea! I will include both new and old names in the log messages.
>
> Finally, having to choose a different name because of a collision does
> *not* update the channel->dev_num value. Subsequent calls to
> hv_set_devnum() will detect "in use" based on the originally assigned
> dev_num value. I don't think that fundamentally breaks anything, but
> I wondered if you had thought about that case. Maybe a comment here
> to that effect would help a future reader of this code.
That's correct. And I'm aware of this situation. But, the dev_num shouldn't
be changed, because we want it to keep track of the sequence of device
offering.
Avoid single or multiple collisions, we should use udev or other daemons to
set name of VF or other types of NICs to different formats, like "vf*", or "enP*",
etc. For distros have not done so already, we need to work with the distro vendors
to ensure udev or other settings are correctly included in Distros, and Azure images.
Thanks,
- Haiyang
^ permalink raw reply
* RE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang @ 2019-12-31 16:12 UTC (permalink / raw)
To: Roman Kagan
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, KY Srinivasan, Stephen Hemminger,
olaf@aepfle.de, vkuznets, davem@davemloft.net,
linux-kernel@vger.kernel.org
In-Reply-To: <20191231113440.GA380228@rkaganb.sw.ru>
> -----Original Message-----
> From: Roman Kagan <rkagan@virtuozzo.com>
> Sent: Tuesday, December 31, 2019 6:35 AM
> To: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: sashal@kernel.org; linux-hyperv@vger.kernel.org; netdev@vger.kernel.org;
> KY Srinivasan <kys@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; olaf@aepfle.de; vkuznets
> <vkuznets@redhat.com>; davem@davemloft.net; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus
> offer sequence and use async probe
>
> On Mon, Dec 30, 2019 at 12:13:34PM -0800, Haiyang Zhang wrote:
> > The dev_num field in vmbus channel structure is assigned to the first
> > available number when the channel is offered. So netvsc driver uses it
> > for NIC naming based on channel offer sequence. Now re-enable the
> > async probing mode for faster probing.
> >
> > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > ---
> > drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++---
> > 1 file changed, 15 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/hyperv/netvsc_drv.c
> > b/drivers/net/hyperv/netvsc_drv.c index f3f9eb8..39c412f 100644
> > --- a/drivers/net/hyperv/netvsc_drv.c
> > +++ b/drivers/net/hyperv/netvsc_drv.c
> > @@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,
> > struct net_device_context *net_device_ctx;
> > struct netvsc_device_info *device_info = NULL;
> > struct netvsc_device *nvdev;
> > + char name[IFNAMSIZ];
> > int ret = -ENOMEM;
> >
> > - net = alloc_etherdev_mq(sizeof(struct net_device_context),
> > - VRSS_CHANNEL_MAX);
> > + snprintf(name, IFNAMSIZ, "eth%d", dev->channel->dev_num);
>
> How is this supposed to work when there are other ethernet device types on the
> system, which may claim the same device names?
>
> > + net = alloc_netdev_mqs(sizeof(struct net_device_context), name,
> > + NET_NAME_ENUM, ether_setup,
> > + VRSS_CHANNEL_MAX, VRSS_CHANNEL_MAX);
> > +
> > if (!net)
> > goto no_net;
> >
> > @@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev,
> > net->max_mtu = ETH_DATA_LEN;
> >
> > ret = register_netdevice(net);
> > +
> > + if (ret == -EEXIST) {
> > + pr_info("NIC name %s exists, request another name.\n",
> > + net->name);
> > + strlcpy(net->name, "eth%d", IFNAMSIZ);
> > + ret = register_netdevice(net);
> > + }
>
> IOW you want the device naming to be predictable, but don't guarantee this?
>
> I think the problem this patchset is trying to solve is much better solved with a
> udev rule, similar to how it's done for PCI net devices.
> And IMO the primary channel number, being a device's "hardware"
> property, is more suited to be used in the device name, than this completely
> ephemeral device number.
The vmbus number can be affected by other types of devices and/or subchannel
offerings. They are not stable either. That's why this patch set keeps track of the
offering sequence within the same device type in a new variable "dev_num".
As in my earlier email, to avoid impact by other types of NICs, we should put them
into different naming formats, like "vf*", "enP*", etc. And yes, these can be done in
udev.
But for netvsc (synthetic) NICs, we still want the default naming format "eth*". And
the variable "dev_num" gives them the basis for stable naming with Async probing.
Thanks,
- Haiyang
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox