* Re: [PATCH net-next] rps: introduce a new sysctl switch rps_workaround_buggy_driver
From: David Miller @ 2012-04-05 10:44 UTC (permalink / raw)
To: raise.sail; +Cc: netdev
In-Reply-To: <4F7D7464.7040503@gmail.com>
From: Li Yu <raise.sail@gmail.com>
Date: Thu, 05 Apr 2012 18:31:00 +0800
>
> We encountered a buggy NIC driver or hardware/firmware, it keeps
> non-zero constant skb->rxhash for long time, so if we enabled RPS,
> the targeted CPU keeps same for long time too.
>
> This patch introduces a sysctl switch to workaround for such problem,
> if the switch was on, RPS core discards the skb->rxhash that is
> computed by NIC hardware.
>
> Hope this patch also can help others, thanks.
>
> Signed-off-by Li Yu <bingtian.ly@taobao.com>
No way, we fix the drivers not add workarounds like this.
^ permalink raw reply
* [net-next 0/3][pull request] Intel Wired LAN Driver Update
From: Jeff Kirsher @ 2012-04-05 10:47 UTC (permalink / raw)
To: davem; +Cc: Jeff Kirsher, netdev, gospo, sassmann, bhutchings
This series of patches contains updates for ixgbe only.
This patch set is a refactor of previous patches that export data
requested by our customers. The thermals are exposed via hwmon as
suggested by Michal Miroslaw and Ben Hutchings. Modeled on how Ben
did for the Solarfare driver. The other values are exported via
sysfs read only files.
The following are changes since commit 51c56b004e2c9a46207bb8a116589c2f84b92e5d:
net: remove k{un}map_skb_frag()
and are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next master
Don Skidmore (3):
ixgbe: add support functions to access thermal data
ixgbe: add hwmon interface to export thermal data
ixgbe: add syfs interface for to export read only driver information
drivers/net/ethernet/intel/Kconfig | 8 +
drivers/net/ethernet/intel/ixgbe/Makefile | 2 +-
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 26 +
drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c | 2 +
drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c | 2 +
drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 169 ++++
drivers/net/ethernet/intel/ixgbe/ixgbe_common.h | 13 +
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 +
drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c | 1035 +++++++++++++++++++++++
drivers/net/ethernet/intel/ixgbe/ixgbe_type.h | 39 +
drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c | 2 +
11 files changed, 1303 insertions(+), 1 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
--
1.7.7.6
^ permalink raw reply
* [net-next 1/3] ixgbe: add support functions to access thermal data
From: Jeff Kirsher @ 2012-04-05 10:47 UTC (permalink / raw)
To: davem; +Cc: Don Skidmore, netdev, gospo, sassmann, bhutchings, Jeff Kirsher
In-Reply-To: <1333622878-3855-1-git-send-email-jeffrey.t.kirsher@intel.com>
From: Don Skidmore <donald.c.skidmore@intel.com>
Some 82599 adapters contain thermal data that we can get to via
an i2c interface. These functions provide support to get at that
data. A following patch will export this data.
Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 171 +++++++++++++++++++++++
drivers/net/ethernet/intel/ixgbe/ixgbe_common.h | 13 ++
drivers/net/ethernet/intel/ixgbe/ixgbe_type.h | 39 +++++
3 files changed, 223 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index e598881..6c6c66e 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -3604,3 +3604,174 @@ void ixgbe_clear_tx_pending(struct ixgbe_hw *hw)
IXGBE_WRITE_REG(hw, IXGBE_GCR_EXT, gcr_ext);
IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg0);
}
+
+static const u8 ixgbe_emc_temp_data[4] = {
+ IXGBE_EMC_INTERNAL_DATA,
+ IXGBE_EMC_DIODE1_DATA,
+ IXGBE_EMC_DIODE2_DATA,
+ IXGBE_EMC_DIODE3_DATA
+};
+static const u8 ixgbe_emc_therm_limit[4] = {
+ IXGBE_EMC_INTERNAL_THERM_LIMIT,
+ IXGBE_EMC_DIODE1_THERM_LIMIT,
+ IXGBE_EMC_DIODE2_THERM_LIMIT,
+ IXGBE_EMC_DIODE3_THERM_LIMIT
+};
+
+/**
+ * ixgbe_get_ets_data - Extracts the ETS bit data
+ * @hw: pointer to hardware structure
+ * @ets_cfg: extected ETS data
+ * @ets_offset: offset of ETS data
+ *
+ * Returns error code.
+ **/
+static s32 ixgbe_get_ets_data(struct ixgbe_hw *hw, u16 *ets_cfg,
+ u16 *ets_offset)
+{
+ s32 status = 0;
+
+ status = hw->eeprom.ops.read(hw, IXGBE_ETS_CFG, ets_offset);
+ if (status)
+ goto out;
+
+ if ((*ets_offset == 0x0000) || (*ets_offset == 0xFFFF)) {
+ status = IXGBE_NOT_IMPLEMENTED;
+ goto out;
+ }
+
+ status = hw->eeprom.ops.read(hw, *ets_offset, ets_cfg);
+ if (status)
+ goto out;
+
+ if ((*ets_cfg & IXGBE_ETS_TYPE_MASK) != IXGBE_ETS_TYPE_EMC_SHIFTED) {
+ status = IXGBE_NOT_IMPLEMENTED;
+ goto out;
+ }
+
+out:
+ return status;
+}
+
+/**
+ * ixgbe_get_thermal_sensor_data - Gathers thermal sensor data
+ * @hw: pointer to hardware structure
+ *
+ * Returns the thermal sensor data structure
+ **/
+s32 ixgbe_get_thermal_sensor_data_generic(struct ixgbe_hw *hw)
+{
+ s32 status = 0;
+ u16 ets_offset;
+ u16 ets_cfg;
+ u16 ets_sensor;
+ u8 num_sensors;
+ u8 i;
+ struct ixgbe_thermal_sensor_data *data = &hw->mac.thermal_sensor_data;
+
+ /* Only support thermal sensors attached to 82599 physical port 0 */
+ if ((hw->mac.type != ixgbe_mac_82599EB) ||
+ (IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
+ status = IXGBE_NOT_IMPLEMENTED;
+ goto out;
+ }
+
+ status = ixgbe_get_ets_data(hw, &ets_cfg, &ets_offset);
+ if (status)
+ goto out;
+
+ num_sensors = (ets_cfg & IXGBE_ETS_NUM_SENSORS_MASK);
+ if (num_sensors > IXGBE_MAX_SENSORS)
+ num_sensors = IXGBE_MAX_SENSORS;
+
+ for (i = 0; i < num_sensors; i++) {
+ u8 sensor_index;
+ u8 sensor_location;
+
+ status = hw->eeprom.ops.read(hw, (ets_offset + 1 + i),
+ &ets_sensor);
+ if (status)
+ goto out;
+
+ sensor_index = ((ets_sensor & IXGBE_ETS_DATA_INDEX_MASK) >>
+ IXGBE_ETS_DATA_INDEX_SHIFT);
+ sensor_location = ((ets_sensor & IXGBE_ETS_DATA_LOC_MASK) >>
+ IXGBE_ETS_DATA_LOC_SHIFT);
+
+ if (sensor_location != 0) {
+ status = hw->phy.ops.read_i2c_byte(hw,
+ ixgbe_emc_temp_data[sensor_index],
+ IXGBE_I2C_THERMAL_SENSOR_ADDR,
+ &data->sensor[i].temp);
+ if (status)
+ goto out;
+ }
+ }
+out:
+ return status;
+}
+
+/**
+ * ixgbe_init_thermal_sensor_thresh_generic - Inits thermal sensor thresholds
+ * @hw: pointer to hardware structure
+ *
+ * Inits the thermal sensor thresholds according to the NVM map
+ * and save off the threshold and location values into mac.thermal_sensor_data
+ **/
+s32 ixgbe_init_thermal_sensor_thresh_generic(struct ixgbe_hw *hw)
+{
+ s32 status = 0;
+ u16 ets_offset;
+ u16 ets_cfg;
+ u16 ets_sensor;
+ u8 low_thresh_delta;
+ u8 num_sensors;
+ u8 therm_limit;
+ u8 i;
+ struct ixgbe_thermal_sensor_data *data = &hw->mac.thermal_sensor_data;
+
+ memset(data, 0, sizeof(struct ixgbe_thermal_sensor_data));
+
+ /* Only support thermal sensors attached to 82599 physical port 0 */
+ if ((hw->mac.type != ixgbe_mac_82599EB) ||
+ (IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
+ status = IXGBE_NOT_IMPLEMENTED;
+ goto out;
+ }
+
+ status = ixgbe_get_ets_data(hw, &ets_cfg, &ets_offset);
+ if (status)
+ goto out;
+
+ low_thresh_delta = ((ets_cfg & IXGBE_ETS_LTHRES_DELTA_MASK) >>
+ IXGBE_ETS_LTHRES_DELTA_SHIFT);
+ num_sensors = (ets_cfg & IXGBE_ETS_NUM_SENSORS_MASK);
+ if (num_sensors > IXGBE_MAX_SENSORS)
+ num_sensors = IXGBE_MAX_SENSORS;
+
+ for (i = 0; i < num_sensors; i++) {
+ u8 sensor_index;
+ u8 sensor_location;
+
+ hw->eeprom.ops.read(hw, (ets_offset + 1 + i), &ets_sensor);
+ sensor_index = ((ets_sensor & IXGBE_ETS_DATA_INDEX_MASK) >>
+ IXGBE_ETS_DATA_INDEX_SHIFT);
+ sensor_location = ((ets_sensor & IXGBE_ETS_DATA_LOC_MASK) >>
+ IXGBE_ETS_DATA_LOC_SHIFT);
+ therm_limit = ets_sensor & IXGBE_ETS_DATA_HTHRESH_MASK;
+
+ hw->phy.ops.write_i2c_byte(hw,
+ ixgbe_emc_therm_limit[sensor_index],
+ IXGBE_I2C_THERMAL_SENSOR_ADDR, therm_limit);
+
+ if (sensor_location == 0)
+ continue;
+
+ data->sensor[i].location = sensor_location;
+ data->sensor[i].caution_thresh = therm_limit;
+ data->sensor[i].max_op_thresh = therm_limit - low_thresh_delta;
+ }
+out:
+ return status;
+}
+
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
index d6d3432..f992777 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
@@ -107,6 +107,19 @@ void ixgbe_clear_tx_pending(struct ixgbe_hw *hw);
void ixgbe_set_rxpba_generic(struct ixgbe_hw *hw, int num_pb,
u32 headroom, int strategy);
+#define IXGBE_I2C_THERMAL_SENSOR_ADDR 0xF8
+#define IXGBE_EMC_INTERNAL_DATA 0x00
+#define IXGBE_EMC_INTERNAL_THERM_LIMIT 0x20
+#define IXGBE_EMC_DIODE1_DATA 0x01
+#define IXGBE_EMC_DIODE1_THERM_LIMIT 0x19
+#define IXGBE_EMC_DIODE2_DATA 0x23
+#define IXGBE_EMC_DIODE2_THERM_LIMIT 0x1A
+#define IXGBE_EMC_DIODE3_DATA 0x2A
+#define IXGBE_EMC_DIODE3_THERM_LIMIT 0x30
+
+s32 ixgbe_get_thermal_sensor_data_generic(struct ixgbe_hw *hw);
+s32 ixgbe_init_thermal_sensor_thresh_generic(struct ixgbe_hw *hw);
+
#define IXGBE_WRITE_REG(a, reg, value) writel((value), ((a)->hw_addr + (reg)))
#ifndef writeq
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
index ffa6679..8bd15c3 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
@@ -110,6 +110,26 @@
#define IXGBE_I2C_CLK_OUT 0x00000002
#define IXGBE_I2C_DATA_IN 0x00000004
#define IXGBE_I2C_DATA_OUT 0x00000008
+#define IXGBE_I2C_THERMAL_SENSOR_ADDR 0xF8
+#define IXGBE_EMC_INTERNAL_DATA 0x00
+#define IXGBE_EMC_INTERNAL_THERM_LIMIT 0x20
+#define IXGBE_EMC_DIODE1_DATA 0x01
+#define IXGBE_EMC_DIODE1_THERM_LIMIT 0x19
+#define IXGBE_EMC_DIODE2_DATA 0x23
+#define IXGBE_EMC_DIODE2_THERM_LIMIT 0x1A
+
+#define IXGBE_MAX_SENSORS 3
+
+struct ixgbe_thermal_diode_data {
+ u8 location;
+ u8 temp;
+ u8 caution_thresh;
+ u8 max_op_thresh;
+};
+
+struct ixgbe_thermal_sensor_data {
+ struct ixgbe_thermal_diode_data sensor[IXGBE_MAX_SENSORS];
+};
/* Interrupt Registers */
#define IXGBE_EICR 0x00800
@@ -1677,6 +1697,22 @@ enum {
#define IXGBE_PBANUM0_PTR 0x15
#define IXGBE_PBANUM1_PTR 0x16
#define IXGBE_FREE_SPACE_PTR 0X3E
+
+/* External Thermal Sensor Config */
+#define IXGBE_ETS_CFG 0x26
+#define IXGBE_ETS_LTHRES_DELTA_MASK 0x07C0
+#define IXGBE_ETS_LTHRES_DELTA_SHIFT 6
+#define IXGBE_ETS_TYPE_MASK 0x0038
+#define IXGBE_ETS_TYPE_SHIFT 3
+#define IXGBE_ETS_TYPE_EMC 0x000
+#define IXGBE_ETS_TYPE_EMC_SHIFTED 0x000
+#define IXGBE_ETS_NUM_SENSORS_MASK 0x0007
+#define IXGBE_ETS_DATA_LOC_MASK 0x3C00
+#define IXGBE_ETS_DATA_LOC_SHIFT 10
+#define IXGBE_ETS_DATA_INDEX_MASK 0x0300
+#define IXGBE_ETS_DATA_INDEX_SHIFT 8
+#define IXGBE_ETS_DATA_HTHRESH_MASK 0x00FF
+
#define IXGBE_SAN_MAC_ADDR_PTR 0x28
#define IXGBE_DEVICE_CAPS 0x2C
#define IXGBE_SERIAL_NUMBER_MAC_ADDR 0x11
@@ -2774,6 +2810,8 @@ struct ixgbe_mac_operations {
/* Manageability interface */
s32 (*set_fw_drv_ver)(struct ixgbe_hw *, u8, u8, u8, u8);
+ s32 (*get_thermal_sensor_data)(struct ixgbe_hw *);
+ s32 (*init_thermal_sensor_thresh)(struct ixgbe_hw *hw);
};
struct ixgbe_phy_operations {
@@ -2831,6 +2869,7 @@ struct ixgbe_mac_info {
bool orig_link_settings_stored;
bool autotry_restart;
u8 flags;
+ struct ixgbe_thermal_sensor_data thermal_sensor_data;
};
struct ixgbe_phy_info {
--
1.7.7.6
^ permalink raw reply related
* [net-next 2/3] ixgbe: add hwmon interface to export thermal data
From: Jeff Kirsher @ 2012-04-05 10:47 UTC (permalink / raw)
To: davem; +Cc: Don Skidmore, netdev, gospo, sassmann, bhutchings, Jeff Kirsher
In-Reply-To: <1333622878-3855-1-git-send-email-jeffrey.t.kirsher@intel.com>
From: Don Skidmore <donald.c.skidmore@intel.com>
Some of our adapters have thermal data available, this patch exports
this data via hwmon sysfs interface. Other sysfs read-only file follow in
another patch. Some of the functions introduced there take that into
account.
Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/Kconfig | 8 +
drivers/net/ethernet/intel/ixgbe/Makefile | 2 +-
drivers/net/ethernet/intel/ixgbe/ixgbe.h | 26 +++
drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c | 2 +
drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c | 2 +
drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 10 +-
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 +
drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c | 271 +++++++++++++++++++++++
drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c | 2 +
9 files changed, 322 insertions(+), 7 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 74215c0..546efe3 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -193,6 +193,14 @@ config IXGBE
To compile this driver as a module, choose M here. The module
will be called ixgbe.
+config IXGBE_HWMON
+ bool "Intel(R) 10GbE PCI Express adapters HWMON support"
+ default y
+ depends on IXGBE && HWMON && !(IXGBE=y && HWMON=m)
+ ---help---
+ Say Y if you want to expose the thermal sensor data on some of
+ our cards, via a hwmon sysfs interface.
+
config IXGBE_DCA
bool "Direct Cache Access (DCA) Support"
default y
diff --git a/drivers/net/ethernet/intel/ixgbe/Makefile b/drivers/net/ethernet/intel/ixgbe/Makefile
index 8be1d1b..eec1076 100644
--- a/drivers/net/ethernet/intel/ixgbe/Makefile
+++ b/drivers/net/ethernet/intel/ixgbe/Makefile
@@ -34,7 +34,7 @@ obj-$(CONFIG_IXGBE) += ixgbe.o
ixgbe-objs := ixgbe_main.o ixgbe_common.o ixgbe_ethtool.o \
ixgbe_82599.o ixgbe_82598.o ixgbe_phy.o ixgbe_sriov.o \
- ixgbe_mbx.o ixgbe_x540.o ixgbe_lib.o
+ ixgbe_mbx.o ixgbe_x540.o ixgbe_lib.o ixgbe_sysfs.o
ixgbe-$(CONFIG_IXGBE_DCB) += ixgbe_dcb.o ixgbe_dcb_82598.o \
ixgbe_dcb_82599.o ixgbe_dcb_nl.o
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
index 74e1921..7d99349 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
@@ -331,6 +331,26 @@ struct ixgbe_q_vector {
/* for dynamic allocation of rings associated with this q_vector */
struct ixgbe_ring ring[0] ____cacheline_internodealigned_in_smp;
};
+#ifdef CONFIG_IXGBE_HWMON
+
+#define IXGBE_HWMON_TYPE_LOC 0
+#define IXGBE_HWMON_TYPE_TEMP 1
+#define IXGBE_HWMON_TYPE_CAUTION 2
+#define IXGBE_HWMON_TYPE_MAX 3
+
+struct hwmon_attr {
+ struct device_attribute dev_attr;
+ struct ixgbe_hw *hw;
+ struct ixgbe_thermal_diode_data *sensor;
+ char name[12];
+};
+
+struct hwmon_buff {
+ struct device *device;
+ struct hwmon_attr *hwmon_list;
+ unsigned int n_hwmon;
+};
+#endif /* CONFIG_IXGBE_HWMON */
/*
* microsecond values for various ITR rates shifted by 2 to fit itr register
@@ -535,6 +555,10 @@ struct ixgbe_adapter {
u32 timer_event_accumulator;
u32 vferr_refcount;
+ struct kobject *info_kobj;
+#ifdef CONFIG_IXGBE_HWMON
+ struct hwmon_buff ixgbe_hwmon_buff;
+#endif /* CONFIG_IXGBE_HWMON */
};
struct ixgbe_fdir_filter {
@@ -633,6 +657,8 @@ extern int ixgbe_setup_tc(struct net_device *dev, u8 tc);
#endif
extern void ixgbe_tx_ctxtdesc(struct ixgbe_ring *, u32, u32, u32, u32);
extern void ixgbe_do_reset(struct net_device *netdev);
+extern void ixgbe_sysfs_exit(struct ixgbe_adapter *adapter);
+extern int ixgbe_sysfs_init(struct ixgbe_adapter *adapter);
#ifdef IXGBE_FCOE
extern void ixgbe_configure_fcoe(struct ixgbe_adapter *adapter);
extern int ixgbe_fso(struct ixgbe_ring *tx_ring,
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
index 56fd468..6175845 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
@@ -1277,6 +1277,8 @@ static struct ixgbe_mac_operations mac_ops_82598 = {
.set_fw_drv_ver = NULL,
.acquire_swfw_sync = &ixgbe_acquire_swfw_sync,
.release_swfw_sync = &ixgbe_release_swfw_sync,
+ .get_thermal_sensor_data = NULL,
+ .init_thermal_sensor_thresh = NULL,
};
static struct ixgbe_eeprom_operations eeprom_ops_82598 = {
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
index 9c14685..dee64d2 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
@@ -2119,6 +2119,8 @@ static struct ixgbe_mac_operations mac_ops_82599 = {
.set_vlan_anti_spoofing = &ixgbe_set_vlan_anti_spoofing,
.acquire_swfw_sync = &ixgbe_acquire_swfw_sync,
.release_swfw_sync = &ixgbe_release_swfw_sync,
+ .get_thermal_sensor_data = &ixgbe_get_thermal_sensor_data_generic,
+ .init_thermal_sensor_thresh = &ixgbe_init_thermal_sensor_thresh_generic,
};
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index 6c6c66e..e2b0519 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -3669,9 +3669,8 @@ s32 ixgbe_get_thermal_sensor_data_generic(struct ixgbe_hw *hw)
u8 i;
struct ixgbe_thermal_sensor_data *data = &hw->mac.thermal_sensor_data;
- /* Only support thermal sensors attached to 82599 physical port 0 */
- if ((hw->mac.type != ixgbe_mac_82599EB) ||
- (IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
+ /* Only support thermal sensors attached to physical port 0 */
+ if ((IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
status = IXGBE_NOT_IMPLEMENTED;
goto out;
}
@@ -3732,9 +3731,8 @@ s32 ixgbe_init_thermal_sensor_thresh_generic(struct ixgbe_hw *hw)
memset(data, 0, sizeof(struct ixgbe_thermal_sensor_data));
- /* Only support thermal sensors attached to 82599 physical port 0 */
- if ((hw->mac.type != ixgbe_mac_82599EB) ||
- (IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
+ /* Only support thermal sensors attached to physical port 0 */
+ if ((IXGBE_READ_REG(hw, IXGBE_STATUS) & IXGBE_STATUS_LAN_ID_1)) {
status = IXGBE_NOT_IMPLEMENTED;
goto out;
}
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 3e26b1f..983a3e0 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7150,6 +7150,10 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
e_dev_info("%s\n", ixgbe_default_device_descr);
cards_found++;
+
+ if (ixgbe_sysfs_init(adapter))
+ e_err(probe, "failed to allocate sysfs resources\n");
+
return 0;
err_register:
@@ -7196,6 +7200,8 @@ static void __devexit ixgbe_remove(struct pci_dev *pdev)
}
#endif
+ ixgbe_sysfs_exit(adapter);
+
#ifdef IXGBE_FCOE
if (adapter->flags & IXGBE_FLAG_FCOE_ENABLED)
ixgbe_cleanup_fcoe(adapter);
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
new file mode 100644
index 0000000..0882fd5
--- /dev/null
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
@@ -0,0 +1,271 @@
+/*******************************************************************************
+
+ Intel 10 Gigabit PCI Express Linux driver
+ Copyright(c) 1999 - 2011 Intel Corporation.
+
+ This program is free software; you can redistribute it and/or modify it
+ under the terms and conditions of the GNU General Public License,
+ version 2, as published by the Free Software Foundation.
+
+ This program is distributed in the hope it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ more details.
+
+ You should have received a copy of the GNU General Public License along with
+ this program; if not, write to the Free Software Foundation, Inc.,
+ 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+
+ The full GNU General Public License is included in this distribution in
+ the file called "COPYING".
+
+ Contact Information:
+ e1000-devel Mailing List <e1000-devel@lists.sourceforge.net>
+ Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-6497
+
+*******************************************************************************/
+
+#include "ixgbe.h"
+#include "ixgbe_common.h"
+#include "ixgbe_type.h"
+
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/sysfs.h>
+#include <linux/kobject.h>
+#include <linux/device.h>
+#include <linux/netdevice.h>
+#include <linux/hwmon.h>
+
+/*
+ * This file provides a sysfs interface to export information from the
+ * driver. The information presented is READ-ONLY.
+ */
+#ifdef CONFIG_IXGBE_HWMON
+
+/* hwmon callback functions */
+static ssize_t ixgbe_hwmon_show_location(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct hwmon_attr *ixgbe_attr = container_of(attr, struct hwmon_attr,
+ dev_attr);
+ return sprintf(buf, "loc%u\n",
+ ixgbe_attr->sensor->location);
+}
+
+static ssize_t ixgbe_hwmon_show_temp(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct hwmon_attr *ixgbe_attr = container_of(attr, struct hwmon_attr,
+ dev_attr);
+ unsigned int value;
+
+ /* reset the temp field */
+ ixgbe_attr->hw->mac.ops.get_thermal_sensor_data(ixgbe_attr->hw);
+
+ value = ixgbe_attr->sensor->temp;
+
+ /* display millidegree */
+ value *= 1000;
+
+ return sprintf(buf, "%u\n", value);
+}
+
+static ssize_t ixgbe_hwmon_show_cautionthresh(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct hwmon_attr *ixgbe_attr = container_of(attr, struct hwmon_attr,
+ dev_attr);
+ unsigned int value = ixgbe_attr->sensor->caution_thresh;
+
+ /* display millidegree */
+ value *= 1000;
+
+ return sprintf(buf, "%u\n", value);
+}
+
+static ssize_t ixgbe_hwmon_show_maxopthresh(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct hwmon_attr *ixgbe_attr = container_of(attr, struct hwmon_attr,
+ dev_attr);
+ unsigned int value = ixgbe_attr->sensor->max_op_thresh;
+
+ /* display millidegree */
+ value *= 1000;
+
+ return sprintf(buf, "%u\n", value);
+}
+
+/*
+ * ixgbe_add_hwmon_attr - Create hwmon attr table for a hwmon sysfs file.
+ * @ adapter: pointer to the adapter structure
+ * @ offset: offset in the eeprom sensor data table
+ * @ type: type of sensor data to display
+ *
+ * For each file we want in hwmon's sysfs interface we need a device_attribute
+ * This is included in our hwmon_attr struct that contains the references to
+ * the data structures we need to get the data to display.
+ */
+static int ixgbe_add_hwmon_attr(struct ixgbe_adapter *adapter,
+ unsigned int offset, int type) {
+ int rc;
+ unsigned int n_attr;
+ struct hwmon_attr *ixgbe_attr;
+
+ n_attr = adapter->ixgbe_hwmon_buff.n_hwmon;
+ ixgbe_attr = &adapter->ixgbe_hwmon_buff.hwmon_list[n_attr];
+
+ switch (type) {
+ case IXGBE_HWMON_TYPE_LOC:
+ ixgbe_attr->dev_attr.show = ixgbe_hwmon_show_location;
+ snprintf(ixgbe_attr->name, sizeof(ixgbe_attr->name),
+ "temp%u_label", offset);
+ break;
+ case IXGBE_HWMON_TYPE_TEMP:
+ ixgbe_attr->dev_attr.show = ixgbe_hwmon_show_temp;
+ snprintf(ixgbe_attr->name, sizeof(ixgbe_attr->name),
+ "temp%u_input", offset);
+ break;
+ case IXGBE_HWMON_TYPE_CAUTION:
+ ixgbe_attr->dev_attr.show = ixgbe_hwmon_show_cautionthresh;
+ snprintf(ixgbe_attr->name, sizeof(ixgbe_attr->name),
+ "temp%u_max", offset);
+ break;
+ case IXGBE_HWMON_TYPE_MAX:
+ ixgbe_attr->dev_attr.show = ixgbe_hwmon_show_maxopthresh;
+ snprintf(ixgbe_attr->name, sizeof(ixgbe_attr->name),
+ "temp%u_crit", offset);
+ break;
+ default:
+ rc = -EPERM;
+ return rc;
+ }
+
+ /* These always the same regardless of type */
+ ixgbe_attr->sensor =
+ &adapter->hw.mac.thermal_sensor_data.sensor[offset];
+ ixgbe_attr->hw = &adapter->hw;
+ ixgbe_attr->dev_attr.store = NULL;
+ ixgbe_attr->dev_attr.attr.mode = S_IRUGO;
+ ixgbe_attr->dev_attr.attr.name = ixgbe_attr->name;
+
+ rc = device_create_file(&adapter->pdev->dev,
+ &ixgbe_attr->dev_attr);
+
+ if (rc == 0)
+ ++adapter->ixgbe_hwmon_buff.n_hwmon;
+
+ return rc;
+}
+#endif /* CONFIG_IXGBE_HWMON */
+
+static void ixgbe_sysfs_del_adapter(struct ixgbe_adapter *adapter)
+{
+#ifdef CONFIG_IXGBE_HWMON
+ int i;
+#endif /* CONFIG_IXGBE_HWMON */
+
+ if (adapter == NULL)
+ return;
+#ifdef CONFIG_IXGBE_HWMON
+
+ for (i = 0; i < adapter->ixgbe_hwmon_buff.n_hwmon; i++) {
+ device_remove_file(&adapter->pdev->dev,
+ &adapter->ixgbe_hwmon_buff.hwmon_list[i].dev_attr);
+ }
+
+ kfree(adapter->ixgbe_hwmon_buff.hwmon_list);
+
+ if (adapter->ixgbe_hwmon_buff.device)
+ hwmon_device_unregister(adapter->ixgbe_hwmon_buff.device);
+#endif /* CONFIG_IXGBE_HWMON */
+
+ if (adapter->info_kobj != NULL)
+ kobject_put(adapter->info_kobj);
+}
+
+/* called from ixgbe_main.c */
+void ixgbe_sysfs_exit(struct ixgbe_adapter *adapter)
+{
+ ixgbe_sysfs_del_adapter(adapter);
+}
+
+/* called from ixgbe_main.c */
+int ixgbe_sysfs_init(struct ixgbe_adapter *adapter)
+{
+#ifdef CONFIG_IXGBE_HWMON
+ struct hwmon_buff *ixgbe_hwmon = &adapter->ixgbe_hwmon_buff;
+ unsigned int i;
+ int n_attrs;
+#endif /* CONFIG_IXGBE_HWMON */
+ struct net_device *netdev = adapter->netdev;
+ int rc = 0;
+
+ /* create info kobj and attribute listings in kobj */
+ adapter->info_kobj = kobject_create_and_add("info", &netdev->dev.kobj);
+ if (adapter->info_kobj == NULL) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+#ifdef CONFIG_IXGBE_HWMON
+ /* If this method isn't defined we don't support thermals */
+ if (adapter->hw.mac.ops.init_thermal_sensor_thresh == NULL) {
+ rc = -EPERM;
+ goto err;
+ }
+
+ /* Don't create thermal hwmon interface if no sensors present */
+ rc = adapter->hw.mac.ops.init_thermal_sensor_thresh(&adapter->hw);
+ if (rc)
+ goto err;
+
+ /*
+ * Allocation space for max attributs
+ * max num sensors * values (loc, temp, max, caution)
+ */
+ n_attrs = IXGBE_MAX_SENSORS * 4;
+ ixgbe_hwmon->hwmon_list = kcalloc(n_attrs, sizeof(struct hwmon_attr),
+ GFP_KERNEL);
+ if (!ixgbe_hwmon->hwmon_list) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ ixgbe_hwmon->device = hwmon_device_register(&adapter->pdev->dev);
+ if (IS_ERR(ixgbe_hwmon->device)) {
+ rc = PTR_ERR(ixgbe_hwmon->device);
+ goto err;
+ }
+
+ for (i = 0; i < IXGBE_MAX_SENSORS; i++) {
+ /*
+ * Only create hwmon sysfs entries for sensors that have
+ * meaningful data for.
+ */
+ if (adapter->hw.mac.thermal_sensor_data.sensor[i].location == 0)
+ continue;
+
+ /* Bail if any hwmon attr struct fails to initialize */
+ rc = ixgbe_add_hwmon_attr(adapter, i, IXGBE_HWMON_TYPE_CAUTION);
+ rc |= ixgbe_add_hwmon_attr(adapter, i, IXGBE_HWMON_TYPE_LOC);
+ rc |= ixgbe_add_hwmon_attr(adapter, i, IXGBE_HWMON_TYPE_TEMP);
+ rc |= ixgbe_add_hwmon_attr(adapter, i, IXGBE_HWMON_TYPE_MAX);
+ if (rc)
+ goto err;
+ }
+#endif /* CONFIG_IXGBE_HWMON */
+
+ goto exit;
+
+err:
+ ixgbe_sysfs_del_adapter(adapter);
+exit:
+ return rc;
+}
+
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
index 97a9914..f90ec07 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
@@ -849,6 +849,8 @@ static struct ixgbe_mac_operations mac_ops_X540 = {
.release_swfw_sync = &ixgbe_release_swfw_sync_X540,
.disable_rx_buff = &ixgbe_disable_rx_buff_generic,
.enable_rx_buff = &ixgbe_enable_rx_buff_generic,
+ .get_thermal_sensor_data = NULL,
+ .init_thermal_sensor_thresh = NULL,
};
static struct ixgbe_eeprom_operations eeprom_ops_X540 = {
--
1.7.7.6
^ permalink raw reply related
* [net-next 3/3] ixgbe: add syfs interface for to export read only driver information
From: Jeff Kirsher @ 2012-04-05 10:47 UTC (permalink / raw)
To: davem; +Cc: Don Skidmore, netdev, gospo, sassmann, bhutchings, Jeff Kirsher
In-Reply-To: <1333622878-3855-1-git-send-email-jeffrey.t.kirsher@intel.com>
From: Don Skidmore <donald.c.skidmore@intel.com>
This patch exports non-thermal (which was done via hwmon in an earlier
patch) data to syfs. All of the fields are read only as this interface
is to only export driver data.
Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c | 774 +++++++++++++++++++++++-
1 files changed, 769 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
index 0882fd5..ddcec9f 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
@@ -41,6 +41,762 @@
* This file provides a sysfs interface to export information from the
* driver. The information presented is READ-ONLY.
*/
+
+static struct net_device_stats *sysfs_get_stats(struct net_device *netdev)
+{
+ if (netdev == NULL)
+ return NULL;
+
+ /* only return the current stats */
+ return &netdev->stats;
+}
+
+static struct net_device *ixgbe_get_netdev(struct kobject *kobj)
+{
+ struct net_device *netdev;
+ struct kobject *parent = kobj->parent;
+ struct device *device_info_kobj;
+
+ if (kobj == NULL)
+ return NULL;
+
+ device_info_kobj = container_of(parent, struct device, kobj);
+ if (device_info_kobj == NULL)
+ return NULL;
+
+ netdev = container_of(device_info_kobj, struct net_device, dev);
+ return netdev;
+}
+
+static struct ixgbe_adapter *ixgbe_get_adapter(struct kobject *kobj)
+{
+ struct ixgbe_adapter *adapter;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return NULL;
+
+ adapter = netdev_priv(netdev);
+ return adapter;
+}
+
+static ssize_t ixgbe_fwbanner(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ int nvm_track_id;
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ nvm_track_id = (adapter->eeprom_verh << 16) | adapter->eeprom_verl;
+
+ return snprintf(buf, PAGE_SIZE, "0x%08x\n", nvm_track_id);
+}
+
+static ssize_t ixgbe_porttype(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ test_bit(__IXGBE_DOWN, &adapter->state));
+}
+
+static ssize_t ixgbe_portspeed(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ int speed = 0;
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ switch (adapter->link_speed) {
+ case IXGBE_LINK_SPEED_100_FULL:
+ speed = 1;
+ break;
+ case IXGBE_LINK_SPEED_1GB_FULL:
+ speed = 10;
+ break;
+ case IXGBE_LINK_SPEED_10GB_FULL:
+ speed = 100;
+ break;
+ }
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", speed);
+}
+
+static ssize_t ixgbe_wqlflag(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", adapter->wol);
+}
+
+static ssize_t ixgbe_xflowctl(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ struct ixgbe_hw *hw;
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", hw->fc.current_mode);
+}
+
+static ssize_t ixgbe_rxdrops(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n",
+ net_stats->rx_dropped);
+}
+
+static ssize_t ixgbe_rxerrors(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_errors);
+}
+
+static ssize_t ixgbe_rxupacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_TPR));
+}
+
+static ssize_t ixgbe_rxmpacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_MPRC));
+}
+
+static ssize_t ixgbe_rxbpacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_BPRC));
+}
+
+static ssize_t ixgbe_txupacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_TPT));
+}
+
+static ssize_t ixgbe_txmpacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_MPTC));
+}
+
+static ssize_t ixgbe_txbpacks(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_BPTC));
+}
+
+static ssize_t ixgbe_txerrors(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_errors);
+}
+
+static ssize_t ixgbe_txdrops(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_dropped);
+}
+
+static ssize_t ixgbe_rxframes(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_packets);
+}
+
+static ssize_t ixgbe_rxbytes(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_bytes);
+}
+
+static ssize_t ixgbe_txframes(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_packets);
+}
+
+static ssize_t ixgbe_txbytes(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device_stats *net_stats;
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ net_stats = sysfs_get_stats(netdev);
+ if (net_stats == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
+
+ return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_bytes);
+}
+
+static ssize_t ixgbe_linkstat(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ u32 link_speed;
+ bool link_up = false;
+ int bitmask = 0;
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ if (test_bit(__IXGBE_DOWN, &adapter->state))
+ bitmask |= 1;
+
+ if (hw->mac.ops.check_link)
+ hw->mac.ops.check_link(hw, &link_speed, &link_up, false);
+ else
+ /* always assume link is up, if no check link function */
+ link_up = true;
+
+ if (link_up)
+ bitmask |= 2;
+
+ return snprintf(buf, PAGE_SIZE, "0x%X\n", bitmask);
+}
+
+static ssize_t ixgbe_funcid(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ struct ixgbe_hw *hw;
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "0x%X\n", hw->bus.func);
+}
+
+static ssize_t ixgbe_funcvers(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%s\n", ixgbe_driver_version);
+}
+
+static ssize_t ixgbe_macburn(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "0x%X%X%X%X%X%X\n",
+ (unsigned int)hw->mac.perm_addr[0],
+ (unsigned int)hw->mac.perm_addr[1],
+ (unsigned int)hw->mac.perm_addr[2],
+ (unsigned int)hw->mac.perm_addr[3],
+ (unsigned int)hw->mac.perm_addr[4],
+ (unsigned int)hw->mac.perm_addr[5]);
+}
+
+static ssize_t ixgbe_macadmn(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ return snprintf(buf, PAGE_SIZE, "0x%X%X%X%X%X%X\n",
+ (unsigned int)hw->mac.addr[0],
+ (unsigned int)hw->mac.addr[1],
+ (unsigned int)hw->mac.addr[2],
+ (unsigned int)hw->mac.addr[3],
+ (unsigned int)hw->mac.addr[4],
+ (unsigned int)hw->mac.addr[5]);
+}
+
+static ssize_t ixgbe_maclla1(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_hw *hw;
+ u16 eeprom_buff[6];
+ int first_word = 0x37;
+ int word_count = 6;
+ int rc;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ hw = &adapter->hw;
+ if (hw == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
+
+ rc = hw->eeprom.ops.read_buffer(hw, first_word, word_count,
+ eeprom_buff);
+ if (rc)
+ return snprintf(buf, PAGE_SIZE, "error: reading buffer\n");
+
+ switch (hw->bus.func) {
+ case 0:
+ return snprintf(buf, PAGE_SIZE, "0x%04X%04X%04X\n",
+ eeprom_buff[0],
+ eeprom_buff[1],
+ eeprom_buff[2]);
+ case 1:
+ return snprintf(buf, PAGE_SIZE, "0x%04X%04X%04X\n",
+ eeprom_buff[3],
+ eeprom_buff[4],
+ eeprom_buff[5]);
+ }
+
+ return snprintf(buf, PAGE_SIZE, "unexpected port %d\n", hw->bus.func);
+}
+
+static ssize_t ixgbe_mtusize(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", netdev->mtu);
+}
+
+static ssize_t ixgbe_featflag(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int bitmask = 0;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ if (netdev->features & NETIF_F_RXCSUM)
+ bitmask |= 1;
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", bitmask);
+}
+
+static ssize_t ixgbe_lsominct(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "%d\n", 1);
+}
+
+static ssize_t ixgbe_prommode(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct net_device *netdev = ixgbe_get_netdev(kobj);
+
+ if (netdev == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no net device\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", netdev->flags & IFF_PROMISC);
+}
+
+static ssize_t ixgbe_txdscqsz(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", adapter->tx_ring[0]->count);
+}
+
+static ssize_t ixgbe_rxdscqsz(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", adapter->rx_ring[0]->count);
+}
+
+static ssize_t ixgbe_rxqavg(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int index;
+ int diff = 0;
+ u16 ntc;
+ u16 ntu;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ for (index = 0; index < adapter->num_rx_queues; index++) {
+ ntc = adapter->rx_ring[index]->next_to_clean;
+ ntu = adapter->rx_ring[index]->next_to_use;
+
+ if (ntc >= ntu)
+ diff += (ntc - ntu);
+ else
+ diff += (adapter->rx_ring[index]->count - ntu + ntc);
+ }
+
+ if (adapter->num_rx_queues <= 0)
+ return snprintf(buf, PAGE_SIZE,
+ "can't calculate, number of queues %d\n",
+ adapter->num_rx_queues);
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", diff/adapter->num_rx_queues);
+}
+
+static ssize_t ixgbe_txqavg(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ int index;
+ int diff = 0;
+ u16 ntc;
+ u16 ntu;
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ for (index = 0; index < adapter->num_tx_queues; index++) {
+ ntc = adapter->tx_ring[index]->next_to_clean;
+ ntu = adapter->tx_ring[index]->next_to_use;
+
+ if (ntc >= ntu)
+ diff += (ntc - ntu);
+ else
+ diff += (adapter->tx_ring[index]->count - ntu + ntc);
+ }
+
+ if (adapter->num_tx_queues <= 0)
+ return snprintf(buf, PAGE_SIZE,
+ "can't calculate, number of queues %d\n",
+ adapter->num_tx_queues);
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", diff/adapter->num_tx_queues);
+}
+
+static ssize_t ixgbe_iovotype(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return snprintf(buf, PAGE_SIZE, "2\n");
+}
+
+static ssize_t ixgbe_funcnbr(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", adapter->num_vfs);
+}
+
+static ssize_t ixgbe_pciebnbr(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
+
+ if (adapter == NULL)
+ return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
+
+ return snprintf(buf, PAGE_SIZE, "%d\n", adapter->pdev->bus->number);
+}
+
+/* Initialize the attributes */
+static struct kobj_attribute ixgbe_sysfs_fwbanner_attr =
+ __ATTR(fwbanner, 0444, ixgbe_fwbanner, NULL);
+static struct kobj_attribute ixgbe_sysfs_porttype_attr =
+ __ATTR(porttype, 0444, ixgbe_porttype, NULL);
+static struct kobj_attribute ixgbe_sysfs_portspeed_attr =
+ __ATTR(portspeed, 0444, ixgbe_portspeed, NULL);
+static struct kobj_attribute ixgbe_sysfs_wqlflag_attr =
+ __ATTR(wqlflag, 0444, ixgbe_wqlflag, NULL);
+static struct kobj_attribute ixgbe_sysfs_xflowctl_attr =
+ __ATTR(xflowctl, 0444, ixgbe_xflowctl, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxdrops_attr =
+ __ATTR(rxdrops, 0444, ixgbe_rxdrops, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxerrors_attr =
+ __ATTR(rxerrors, 0444, ixgbe_rxerrors, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxupacks_attr =
+ __ATTR(rxupacks, 0444, ixgbe_rxupacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxmpacks_attr =
+ __ATTR(rxmpacks, 0444, ixgbe_rxmpacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxbpacks_attr =
+ __ATTR(rxbpacks, 0444, ixgbe_rxbpacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_txupacks_attr =
+ __ATTR(txupacks, 0444, ixgbe_txupacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_txmpacks_attr =
+ __ATTR(txmpacks, 0444, ixgbe_txmpacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_txbpacks_attr =
+ __ATTR(txbpacks, 0444, ixgbe_txbpacks, NULL);
+static struct kobj_attribute ixgbe_sysfs_txerrors_attr =
+ __ATTR(txerrors, 0444, ixgbe_txerrors, NULL);
+static struct kobj_attribute ixgbe_sysfs_txdrops_attr =
+ __ATTR(txdrops, 0444, ixgbe_txdrops, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxframes_attr =
+ __ATTR(rxframes, 0444, ixgbe_rxframes, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxbytes_attr =
+ __ATTR(rxbytes, 0444, ixgbe_rxbytes, NULL);
+static struct kobj_attribute ixgbe_sysfs_txframes_attr =
+ __ATTR(txframes, 0444, ixgbe_txframes, NULL);
+static struct kobj_attribute ixgbe_sysfs_txbytes_attr =
+ __ATTR(txbytes, 0444, ixgbe_txbytes, NULL);
+static struct kobj_attribute ixgbe_sysfs_linkstat_attr =
+ __ATTR(linkstat, 0444, ixgbe_linkstat, NULL);
+static struct kobj_attribute ixgbe_sysfs_funcid_attr =
+ __ATTR(funcid, 0444, ixgbe_funcid, NULL);
+static struct kobj_attribute ixgbe_sysfs_funvers_attr =
+ __ATTR(funcvers, 0444, ixgbe_funcvers, NULL);
+static struct kobj_attribute ixgbe_sysfs_macburn_attr =
+ __ATTR(macburn, 0444, ixgbe_macburn, NULL);
+static struct kobj_attribute ixgbe_sysfs_macadmn_attr =
+ __ATTR(macadmn, 0444, ixgbe_macadmn, NULL);
+static struct kobj_attribute ixgbe_sysfs_maclla1_attr =
+ __ATTR(maclla1, 0444, ixgbe_maclla1, NULL);
+static struct kobj_attribute ixgbe_sysfs_mtusize_attr =
+ __ATTR(mtusize, 0444, ixgbe_mtusize, NULL);
+static struct kobj_attribute ixgbe_sysfs_featflag_attr =
+ __ATTR(featflag, 0444, ixgbe_featflag, NULL);
+static struct kobj_attribute ixgbe_sysfs_lsominct_attr =
+ __ATTR(lsominct, 0444, ixgbe_lsominct, NULL);
+static struct kobj_attribute ixgbe_sysfs_prommode_attr =
+ __ATTR(prommode, 0444, ixgbe_prommode, NULL);
+static struct kobj_attribute ixgbe_sysfs_txdscqsz_attr =
+ __ATTR(txdscqsz, 0444, ixgbe_txdscqsz, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxdscqsz_attr =
+ __ATTR(rxdscqsz, 0444, ixgbe_rxdscqsz, NULL);
+static struct kobj_attribute ixgbe_sysfs_txqavg_attr =
+ __ATTR(txqavg, 0444, ixgbe_txqavg, NULL);
+static struct kobj_attribute ixgbe_sysfs_rxqavg_attr =
+ __ATTR(rxqavg, 0444, ixgbe_rxqavg, NULL);
+static struct kobj_attribute ixgbe_sysfs_iovotype_attr =
+ __ATTR(iovotype, 0444, ixgbe_iovotype, NULL);
+static struct kobj_attribute ixgbe_sysfs_funcnbr_attr =
+ __ATTR(funcnbr, 0444, ixgbe_funcnbr, NULL);
+static struct kobj_attribute ixgbe_sysfs_pciebnbr_attr =
+ __ATTR(pciebnbr, 0444, ixgbe_pciebnbr, NULL);
+
+static struct attribute *attrs[] = {
+ &ixgbe_sysfs_fwbanner_attr.attr,
+ &ixgbe_sysfs_porttype_attr.attr,
+ &ixgbe_sysfs_portspeed_attr.attr,
+ &ixgbe_sysfs_wqlflag_attr.attr,
+ &ixgbe_sysfs_xflowctl_attr.attr,
+ &ixgbe_sysfs_rxdrops_attr.attr,
+ &ixgbe_sysfs_rxerrors_attr.attr,
+ &ixgbe_sysfs_rxupacks_attr.attr,
+ &ixgbe_sysfs_rxmpacks_attr.attr,
+ &ixgbe_sysfs_rxbpacks_attr.attr,
+ &ixgbe_sysfs_txdrops_attr.attr,
+ &ixgbe_sysfs_txerrors_attr.attr,
+ &ixgbe_sysfs_txupacks_attr.attr,
+ &ixgbe_sysfs_txmpacks_attr.attr,
+ &ixgbe_sysfs_txbpacks_attr.attr,
+ &ixgbe_sysfs_rxframes_attr.attr,
+ &ixgbe_sysfs_rxbytes_attr.attr,
+ &ixgbe_sysfs_txframes_attr.attr,
+ &ixgbe_sysfs_txbytes_attr.attr,
+ &ixgbe_sysfs_linkstat_attr.attr,
+ &ixgbe_sysfs_funcid_attr.attr,
+ &ixgbe_sysfs_funvers_attr.attr,
+ &ixgbe_sysfs_macburn_attr.attr,
+ &ixgbe_sysfs_macadmn_attr.attr,
+ &ixgbe_sysfs_maclla1_attr.attr,
+ &ixgbe_sysfs_mtusize_attr.attr,
+ &ixgbe_sysfs_featflag_attr.attr,
+ &ixgbe_sysfs_lsominct_attr.attr,
+ &ixgbe_sysfs_prommode_attr.attr,
+ &ixgbe_sysfs_txdscqsz_attr.attr,
+ &ixgbe_sysfs_rxdscqsz_attr.attr,
+ &ixgbe_sysfs_txqavg_attr.attr,
+ &ixgbe_sysfs_rxqavg_attr.attr,
+ &ixgbe_sysfs_iovotype_attr.attr,
+ &ixgbe_sysfs_funcnbr_attr.attr,
+ &ixgbe_sysfs_pciebnbr_attr.attr,
+ NULL
+};
+
+/* add attributes to a group */
+static struct attribute_group attr_group = {
+ .attrs = attrs,
+};
+
#ifdef CONFIG_IXGBE_HWMON
/* hwmon callback functions */
@@ -185,8 +941,11 @@ static void ixgbe_sysfs_del_adapter(struct ixgbe_adapter *adapter)
hwmon_device_unregister(adapter->ixgbe_hwmon_buff.device);
#endif /* CONFIG_IXGBE_HWMON */
- if (adapter->info_kobj != NULL)
+ if (adapter->info_kobj != NULL) {
+ sysfs_remove_group(adapter->info_kobj, &attr_group);
kobject_put(adapter->info_kobj);
+ adapter->info_kobj = NULL;
+ }
}
/* called from ixgbe_main.c */
@@ -213,17 +972,22 @@ int ixgbe_sysfs_init(struct ixgbe_adapter *adapter)
goto err;
}
+ rc = sysfs_create_group(adapter->info_kobj, &attr_group);
+ if (rc)
+ goto err;
+
#ifdef CONFIG_IXGBE_HWMON
/* If this method isn't defined we don't support thermals */
if (adapter->hw.mac.ops.init_thermal_sensor_thresh == NULL) {
- rc = -EPERM;
- goto err;
+ goto exit;
}
/* Don't create thermal hwmon interface if no sensors present */
rc = adapter->hw.mac.ops.init_thermal_sensor_thresh(&adapter->hw);
- if (rc)
- goto err;
+ if (rc) {
+ rc = 0;
+ goto exit;
+ }
/*
* Allocation space for max attributs
--
1.7.7.6
^ permalink raw reply related
* Re: [PATCH net-next] rps: introduce a new sysctl switch rps_workaround_buggy_driver
From: Eric Dumazet @ 2012-04-05 11:07 UTC (permalink / raw)
To: Li Yu; +Cc: netdev
In-Reply-To: <4F7D7464.7040503@gmail.com>
On Thu, 2012-04-05 at 18:31 +0800, Li Yu wrote:
> We encountered a buggy NIC driver or hardware/firmware, it keeps
> non-zero constant skb->rxhash for long time, so if we enabled RPS,
> the targeted CPU keeps same for long time too.
>
> This patch introduces a sysctl switch to workaround for such problem,
> if the switch was on, RPS core discards the skb->rxhash that is
> computed by NIC hardware.
>
> Hope this patch also can help others, thanks.
Really ?
to disable this driver rxhash, you should try :
ethtool -K eth0 rxhash off
^ permalink raw reply
* Re: [PATCH net-next V7 0/8] net/mlx4_en: DCB QoS support
From: Amir Vadai @ 2012-04-05 11:22 UTC (permalink / raw)
To: David Miller, john.r.fastabend, eric.dumazet
Cc: netdev, roland, yevgenyp, oren, amirv
In-Reply-To: <20120405.052828.1474884969422039237.davem@davemloft.net>
On 04/05/2012 12:28 PM, David Miller wrote:
> From: Amir Vadai<amirv@mellanox.com>
> Date: Thu, 5 Apr 2012 10:33:23 +0300
>
>> DCBX version 802.1qaz is supported.
>> User Priority (UP) is set in QP context instead of in WQE (QP Work Queue
>> Element), which means that all traffic from a queue will have the same UP.
>> UP is also set for untagged traffic to be able to classify such traffic too.
>>
>> Mapping from sk_prio to User Priority is done by sch_mqprio mapping. Although
>> confusingly sch_mqprio maps sk_prio to something called TC, it is not related
>> to DCBX's TC, and is interpreted by mlx4_en driver as UP.
>>
>> User can set maximal BW for an ETS TC. This could be done by a new optional
>> attribute added to DCB_ATTR_IEEE called DCBNL DCB_ATTR_IEEE_MAXRATE. It accept
>> an array of 8 64 bits values, 1 for every ETS TC. Units are in Kbps.
>
> All applied, thanks.
>
> Please double check that all the new locations that reference
> ip_tos2prio have a proper dependency on CONFIG_INET.
>
> THanks.
Thank you and to the reviewers.
And Yes, the dependencies are good:
The only new place that uses it is in cma.c which is part of rdma_cm.
rdma_cm depends on CONFIG_INFINIBAND_ADDR_TRANS that depends on
CONFIG_INET
- Amir
^ permalink raw reply
* wimax: i2400m - prevent a possible kernel bug (commit 4eee6a3a04)
From: Josh Boyer @ 2012-04-05 11:42 UTC (permalink / raw)
To: davem; +Cc: Phil Sutter, netdev
Hi Dave,
I noticed you just sent another round of net patches to the stable
trees, but I didn't see commit 4eee6a3a04 included. You might want to
include that in the next round, as we've seen it on a 3.3 kernel in
Fedora (rhbz 808603).
Just an FYI. I'd send it to stable myself, but I believe you prefer to
handle the flow of netdev patches to stable.
josh
^ permalink raw reply
* Re: [net-next 3/3] ixgbe: add syfs interface for to export read only driver information
From: Florian Fainelli @ 2012-04-05 12:19 UTC (permalink / raw)
To: Jeff Kirsher; +Cc: davem, Don Skidmore, netdev, gospo, sassmann, bhutchings
In-Reply-To: <1333622878-3855-4-git-send-email-jeffrey.t.kirsher@intel.com>
Hi,
Le 04/05/12 12:47, Jeff Kirsher a écrit :
> From: Don Skidmore<donald.c.skidmore@intel.com>
>
> This patch exports non-thermal (which was done via hwmon in an earlier
> patch) data to syfs. All of the fields are read only as this interface
> is to only export driver data.
>
> Signed-off-by: Don Skidmore<donald.c.skidmore@intel.com>
> Tested-by: Stephen Ko<stephen.s.ko@intel.com>
> Signed-off-by: Jeff Kirsher<jeffrey.t.kirsher@intel.com>
> ---
I only looked briefly at the exported attributes, but most of them can
already be queried using ethtool, can't they?
Also, if you want to go that way, you might want to make the sysfs
interface configurable through a Kconfig knob, and use one or several
macros to define attributes accessors and reduce the amount of lines of
code.
> drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c | 774 +++++++++++++++++++++++-
> 1 files changed, 769 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
> index 0882fd5..ddcec9f 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.c
> @@ -41,6 +41,762 @@
> * This file provides a sysfs interface to export information from the
> * driver. The information presented is READ-ONLY.
> */
> +
> +static struct net_device_stats *sysfs_get_stats(struct net_device *netdev)
> +{
> + if (netdev == NULL)
> + return NULL;
> +
> + /* only return the current stats */
> + return&netdev->stats;
> +}
> +
> +static struct net_device *ixgbe_get_netdev(struct kobject *kobj)
> +{
> + struct net_device *netdev;
> + struct kobject *parent = kobj->parent;
> + struct device *device_info_kobj;
> +
> + if (kobj == NULL)
> + return NULL;
> +
> + device_info_kobj = container_of(parent, struct device, kobj);
> + if (device_info_kobj == NULL)
> + return NULL;
> +
> + netdev = container_of(device_info_kobj, struct net_device, dev);
> + return netdev;
> +}
> +
> +static struct ixgbe_adapter *ixgbe_get_adapter(struct kobject *kobj)
> +{
> + struct ixgbe_adapter *adapter;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return NULL;
> +
> + adapter = netdev_priv(netdev);
> + return adapter;
> +}
> +
> +static ssize_t ixgbe_fwbanner(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + int nvm_track_id;
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + nvm_track_id = (adapter->eeprom_verh<< 16) | adapter->eeprom_verl;
> +
> + return snprintf(buf, PAGE_SIZE, "0x%08x\n", nvm_track_id);
> +}
> +
> +static ssize_t ixgbe_porttype(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n",
> + test_bit(__IXGBE_DOWN,&adapter->state));
> +}
> +
> +static ssize_t ixgbe_portspeed(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + int speed = 0;
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + switch (adapter->link_speed) {
> + case IXGBE_LINK_SPEED_100_FULL:
> + speed = 1;
> + break;
> + case IXGBE_LINK_SPEED_1GB_FULL:
> + speed = 10;
> + break;
> + case IXGBE_LINK_SPEED_10GB_FULL:
> + speed = 100;
> + break;
> + }
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", speed);
> +}
> +
> +static ssize_t ixgbe_wqlflag(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", adapter->wol);
> +}
> +
> +static ssize_t ixgbe_xflowctl(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + struct ixgbe_hw *hw;
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", hw->fc.current_mode);
> +}
> +
> +static ssize_t ixgbe_rxdrops(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n",
> + net_stats->rx_dropped);
> +}
> +
> +static ssize_t ixgbe_rxerrors(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_errors);
> +}
> +
> +static ssize_t ixgbe_rxupacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_TPR));
> +}
> +
> +static ssize_t ixgbe_rxmpacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_MPRC));
> +}
> +
> +static ssize_t ixgbe_rxbpacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_BPRC));
> +}
> +
> +static ssize_t ixgbe_txupacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_TPT));
> +}
> +
> +static ssize_t ixgbe_txmpacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_MPTC));
> +}
> +
> +static ssize_t ixgbe_txbpacks(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", IXGBE_READ_REG(hw, IXGBE_BPTC));
> +}
> +
> +static ssize_t ixgbe_txerrors(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_errors);
> +}
> +
> +static ssize_t ixgbe_txdrops(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_dropped);
> +}
> +
> +static ssize_t ixgbe_rxframes(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_packets);
> +}
> +
> +static ssize_t ixgbe_rxbytes(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->rx_bytes);
> +}
> +
> +static ssize_t ixgbe_txframes(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_packets);
> +}
> +
> +static ssize_t ixgbe_txbytes(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device_stats *net_stats;
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + net_stats = sysfs_get_stats(netdev);
> + if (net_stats == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net stats\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%lu\n", net_stats->tx_bytes);
> +}
> +
> +static ssize_t ixgbe_linkstat(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + u32 link_speed;
> + bool link_up = false;
> + int bitmask = 0;
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + if (test_bit(__IXGBE_DOWN,&adapter->state))
> + bitmask |= 1;
> +
> + if (hw->mac.ops.check_link)
> + hw->mac.ops.check_link(hw,&link_speed,&link_up, false);
> + else
> + /* always assume link is up, if no check link function */
> + link_up = true;
> +
> + if (link_up)
> + bitmask |= 2;
> +
> + return snprintf(buf, PAGE_SIZE, "0x%X\n", bitmask);
> +}
> +
> +static ssize_t ixgbe_funcid(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + struct ixgbe_hw *hw;
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "0x%X\n", hw->bus.func);
> +}
> +
> +static ssize_t ixgbe_funcvers(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + return snprintf(buf, PAGE_SIZE, "%s\n", ixgbe_driver_version);
> +}
> +
> +static ssize_t ixgbe_macburn(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "0x%X%X%X%X%X%X\n",
> + (unsigned int)hw->mac.perm_addr[0],
> + (unsigned int)hw->mac.perm_addr[1],
> + (unsigned int)hw->mac.perm_addr[2],
> + (unsigned int)hw->mac.perm_addr[3],
> + (unsigned int)hw->mac.perm_addr[4],
> + (unsigned int)hw->mac.perm_addr[5]);
> +}
> +
> +static ssize_t ixgbe_macadmn(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + return snprintf(buf, PAGE_SIZE, "0x%X%X%X%X%X%X\n",
> + (unsigned int)hw->mac.addr[0],
> + (unsigned int)hw->mac.addr[1],
> + (unsigned int)hw->mac.addr[2],
> + (unsigned int)hw->mac.addr[3],
> + (unsigned int)hw->mac.addr[4],
> + (unsigned int)hw->mac.addr[5]);
> +}
> +
> +static ssize_t ixgbe_maclla1(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_hw *hw;
> + u16 eeprom_buff[6];
> + int first_word = 0x37;
> + int word_count = 6;
> + int rc;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + hw =&adapter->hw;
> + if (hw == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no hw data\n");
> +
> + rc = hw->eeprom.ops.read_buffer(hw, first_word, word_count,
> + eeprom_buff);
> + if (rc)
> + return snprintf(buf, PAGE_SIZE, "error: reading buffer\n");
> +
> + switch (hw->bus.func) {
> + case 0:
> + return snprintf(buf, PAGE_SIZE, "0x%04X%04X%04X\n",
> + eeprom_buff[0],
> + eeprom_buff[1],
> + eeprom_buff[2]);
> + case 1:
> + return snprintf(buf, PAGE_SIZE, "0x%04X%04X%04X\n",
> + eeprom_buff[3],
> + eeprom_buff[4],
> + eeprom_buff[5]);
> + }
> +
> + return snprintf(buf, PAGE_SIZE, "unexpected port %d\n", hw->bus.func);
> +}
> +
> +static ssize_t ixgbe_mtusize(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", netdev->mtu);
> +}
> +
> +static ssize_t ixgbe_featflag(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + int bitmask = 0;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + if (netdev->features& NETIF_F_RXCSUM)
> + bitmask |= 1;
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", bitmask);
> +}
> +
> +static ssize_t ixgbe_lsominct(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + return snprintf(buf, PAGE_SIZE, "%d\n", 1);
> +}
> +
> +static ssize_t ixgbe_prommode(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct net_device *netdev = ixgbe_get_netdev(kobj);
> +
> + if (netdev == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no net device\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", netdev->flags& IFF_PROMISC);
> +}
> +
> +static ssize_t ixgbe_txdscqsz(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", adapter->tx_ring[0]->count);
> +}
> +
> +static ssize_t ixgbe_rxdscqsz(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", adapter->rx_ring[0]->count);
> +}
> +
> +static ssize_t ixgbe_rxqavg(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + int index;
> + int diff = 0;
> + u16 ntc;
> + u16 ntu;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + for (index = 0; index< adapter->num_rx_queues; index++) {
> + ntc = adapter->rx_ring[index]->next_to_clean;
> + ntu = adapter->rx_ring[index]->next_to_use;
> +
> + if (ntc>= ntu)
> + diff += (ntc - ntu);
> + else
> + diff += (adapter->rx_ring[index]->count - ntu + ntc);
> + }
> +
> + if (adapter->num_rx_queues<= 0)
> + return snprintf(buf, PAGE_SIZE,
> + "can't calculate, number of queues %d\n",
> + adapter->num_rx_queues);
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", diff/adapter->num_rx_queues);
> +}
> +
> +static ssize_t ixgbe_txqavg(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + int index;
> + int diff = 0;
> + u16 ntc;
> + u16 ntu;
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + for (index = 0; index< adapter->num_tx_queues; index++) {
> + ntc = adapter->tx_ring[index]->next_to_clean;
> + ntu = adapter->tx_ring[index]->next_to_use;
> +
> + if (ntc>= ntu)
> + diff += (ntc - ntu);
> + else
> + diff += (adapter->tx_ring[index]->count - ntu + ntc);
> + }
> +
> + if (adapter->num_tx_queues<= 0)
> + return snprintf(buf, PAGE_SIZE,
> + "can't calculate, number of queues %d\n",
> + adapter->num_tx_queues);
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", diff/adapter->num_tx_queues);
> +}
> +
> +static ssize_t ixgbe_iovotype(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + return snprintf(buf, PAGE_SIZE, "2\n");
> +}
> +
> +static ssize_t ixgbe_funcnbr(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", adapter->num_vfs);
> +}
> +
> +static ssize_t ixgbe_pciebnbr(struct kobject *kobj,
> + struct kobj_attribute *attr, char *buf)
> +{
> + struct ixgbe_adapter *adapter = ixgbe_get_adapter(kobj);
> +
> + if (adapter == NULL)
> + return snprintf(buf, PAGE_SIZE, "error: no adapter\n");
> +
> + return snprintf(buf, PAGE_SIZE, "%d\n", adapter->pdev->bus->number);
> +}
> +
> +/* Initialize the attributes */
> +static struct kobj_attribute ixgbe_sysfs_fwbanner_attr =
> + __ATTR(fwbanner, 0444, ixgbe_fwbanner, NULL);
> +static struct kobj_attribute ixgbe_sysfs_porttype_attr =
> + __ATTR(porttype, 0444, ixgbe_porttype, NULL);
> +static struct kobj_attribute ixgbe_sysfs_portspeed_attr =
> + __ATTR(portspeed, 0444, ixgbe_portspeed, NULL);
> +static struct kobj_attribute ixgbe_sysfs_wqlflag_attr =
> + __ATTR(wqlflag, 0444, ixgbe_wqlflag, NULL);
> +static struct kobj_attribute ixgbe_sysfs_xflowctl_attr =
> + __ATTR(xflowctl, 0444, ixgbe_xflowctl, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxdrops_attr =
> + __ATTR(rxdrops, 0444, ixgbe_rxdrops, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxerrors_attr =
> + __ATTR(rxerrors, 0444, ixgbe_rxerrors, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxupacks_attr =
> + __ATTR(rxupacks, 0444, ixgbe_rxupacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxmpacks_attr =
> + __ATTR(rxmpacks, 0444, ixgbe_rxmpacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxbpacks_attr =
> + __ATTR(rxbpacks, 0444, ixgbe_rxbpacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txupacks_attr =
> + __ATTR(txupacks, 0444, ixgbe_txupacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txmpacks_attr =
> + __ATTR(txmpacks, 0444, ixgbe_txmpacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txbpacks_attr =
> + __ATTR(txbpacks, 0444, ixgbe_txbpacks, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txerrors_attr =
> + __ATTR(txerrors, 0444, ixgbe_txerrors, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txdrops_attr =
> + __ATTR(txdrops, 0444, ixgbe_txdrops, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxframes_attr =
> + __ATTR(rxframes, 0444, ixgbe_rxframes, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxbytes_attr =
> + __ATTR(rxbytes, 0444, ixgbe_rxbytes, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txframes_attr =
> + __ATTR(txframes, 0444, ixgbe_txframes, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txbytes_attr =
> + __ATTR(txbytes, 0444, ixgbe_txbytes, NULL);
> +static struct kobj_attribute ixgbe_sysfs_linkstat_attr =
> + __ATTR(linkstat, 0444, ixgbe_linkstat, NULL);
> +static struct kobj_attribute ixgbe_sysfs_funcid_attr =
> + __ATTR(funcid, 0444, ixgbe_funcid, NULL);
> +static struct kobj_attribute ixgbe_sysfs_funvers_attr =
> + __ATTR(funcvers, 0444, ixgbe_funcvers, NULL);
> +static struct kobj_attribute ixgbe_sysfs_macburn_attr =
> + __ATTR(macburn, 0444, ixgbe_macburn, NULL);
> +static struct kobj_attribute ixgbe_sysfs_macadmn_attr =
> + __ATTR(macadmn, 0444, ixgbe_macadmn, NULL);
> +static struct kobj_attribute ixgbe_sysfs_maclla1_attr =
> + __ATTR(maclla1, 0444, ixgbe_maclla1, NULL);
> +static struct kobj_attribute ixgbe_sysfs_mtusize_attr =
> + __ATTR(mtusize, 0444, ixgbe_mtusize, NULL);
> +static struct kobj_attribute ixgbe_sysfs_featflag_attr =
> + __ATTR(featflag, 0444, ixgbe_featflag, NULL);
> +static struct kobj_attribute ixgbe_sysfs_lsominct_attr =
> + __ATTR(lsominct, 0444, ixgbe_lsominct, NULL);
> +static struct kobj_attribute ixgbe_sysfs_prommode_attr =
> + __ATTR(prommode, 0444, ixgbe_prommode, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txdscqsz_attr =
> + __ATTR(txdscqsz, 0444, ixgbe_txdscqsz, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxdscqsz_attr =
> + __ATTR(rxdscqsz, 0444, ixgbe_rxdscqsz, NULL);
> +static struct kobj_attribute ixgbe_sysfs_txqavg_attr =
> + __ATTR(txqavg, 0444, ixgbe_txqavg, NULL);
> +static struct kobj_attribute ixgbe_sysfs_rxqavg_attr =
> + __ATTR(rxqavg, 0444, ixgbe_rxqavg, NULL);
> +static struct kobj_attribute ixgbe_sysfs_iovotype_attr =
> + __ATTR(iovotype, 0444, ixgbe_iovotype, NULL);
> +static struct kobj_attribute ixgbe_sysfs_funcnbr_attr =
> + __ATTR(funcnbr, 0444, ixgbe_funcnbr, NULL);
> +static struct kobj_attribute ixgbe_sysfs_pciebnbr_attr =
> + __ATTR(pciebnbr, 0444, ixgbe_pciebnbr, NULL);
> +
> +static struct attribute *attrs[] = {
> + &ixgbe_sysfs_fwbanner_attr.attr,
> + &ixgbe_sysfs_porttype_attr.attr,
> + &ixgbe_sysfs_portspeed_attr.attr,
> + &ixgbe_sysfs_wqlflag_attr.attr,
> + &ixgbe_sysfs_xflowctl_attr.attr,
> + &ixgbe_sysfs_rxdrops_attr.attr,
> + &ixgbe_sysfs_rxerrors_attr.attr,
> + &ixgbe_sysfs_rxupacks_attr.attr,
> + &ixgbe_sysfs_rxmpacks_attr.attr,
> + &ixgbe_sysfs_rxbpacks_attr.attr,
> + &ixgbe_sysfs_txdrops_attr.attr,
> + &ixgbe_sysfs_txerrors_attr.attr,
> + &ixgbe_sysfs_txupacks_attr.attr,
> + &ixgbe_sysfs_txmpacks_attr.attr,
> + &ixgbe_sysfs_txbpacks_attr.attr,
> + &ixgbe_sysfs_rxframes_attr.attr,
> + &ixgbe_sysfs_rxbytes_attr.attr,
> + &ixgbe_sysfs_txframes_attr.attr,
> + &ixgbe_sysfs_txbytes_attr.attr,
> + &ixgbe_sysfs_linkstat_attr.attr,
> + &ixgbe_sysfs_funcid_attr.attr,
> + &ixgbe_sysfs_funvers_attr.attr,
> + &ixgbe_sysfs_macburn_attr.attr,
> + &ixgbe_sysfs_macadmn_attr.attr,
> + &ixgbe_sysfs_maclla1_attr.attr,
> + &ixgbe_sysfs_mtusize_attr.attr,
> + &ixgbe_sysfs_featflag_attr.attr,
> + &ixgbe_sysfs_lsominct_attr.attr,
> + &ixgbe_sysfs_prommode_attr.attr,
> + &ixgbe_sysfs_txdscqsz_attr.attr,
> + &ixgbe_sysfs_rxdscqsz_attr.attr,
> + &ixgbe_sysfs_txqavg_attr.attr,
> + &ixgbe_sysfs_rxqavg_attr.attr,
> + &ixgbe_sysfs_iovotype_attr.attr,
> + &ixgbe_sysfs_funcnbr_attr.attr,
> + &ixgbe_sysfs_pciebnbr_attr.attr,
> + NULL
> +};
> +
> +/* add attributes to a group */
> +static struct attribute_group attr_group = {
> + .attrs = attrs,
> +};
> +
> #ifdef CONFIG_IXGBE_HWMON
>
> /* hwmon callback functions */
> @@ -185,8 +941,11 @@ static void ixgbe_sysfs_del_adapter(struct ixgbe_adapter *adapter)
> hwmon_device_unregister(adapter->ixgbe_hwmon_buff.device);
> #endif /* CONFIG_IXGBE_HWMON */
>
> - if (adapter->info_kobj != NULL)
> + if (adapter->info_kobj != NULL) {
> + sysfs_remove_group(adapter->info_kobj,&attr_group);
> kobject_put(adapter->info_kobj);
> + adapter->info_kobj = NULL;
> + }
> }
>
> /* called from ixgbe_main.c */
> @@ -213,17 +972,22 @@ int ixgbe_sysfs_init(struct ixgbe_adapter *adapter)
> goto err;
> }
>
> + rc = sysfs_create_group(adapter->info_kobj,&attr_group);
> + if (rc)
> + goto err;
> +
> #ifdef CONFIG_IXGBE_HWMON
> /* If this method isn't defined we don't support thermals */
> if (adapter->hw.mac.ops.init_thermal_sensor_thresh == NULL) {
> - rc = -EPERM;
> - goto err;
> + goto exit;
> }
>
> /* Don't create thermal hwmon interface if no sensors present */
> rc = adapter->hw.mac.ops.init_thermal_sensor_thresh(&adapter->hw);
> - if (rc)
> - goto err;
> + if (rc) {
> + rc = 0;
> + goto exit;
> + }
>
> /*
> * Allocation space for max attributs
^ permalink raw reply
* Re: kernel BUG at include/net/netns/generic.h:41
From: Stanislav Kinsbursky @ 2012-04-05 12:21 UTC (permalink / raw)
To: Dave Jones; +Cc: Myklebust, Trond, Linux Kernel, netdev@vger.kernel.org
In-Reply-To: <1333390230.13718.34.camel@lade.trondhjem.org>
02.04.2012 22:10, Myklebust, Trond пишет:
> This looks related to the recent net namespace changes, so I'm adding
> Stanislav.
>
> Cheers
> Trond
>
> On Mon, 2012-04-02 at 13:26 -0400, Dave Jones wrote:
>> [added Trond to cc:]
>>
>> still seeing this in 3.4rc1
>>
Hello, Dave.
Could you reply with your kernel config to this letter?
--
Best regards,
Stanislav Kinsbursky
^ permalink raw reply
* Re: [RFC PATCH 0/1] NUMA aware scheduling per vhost thread patch
From: Michael S. Tsirkin @ 2012-04-05 12:28 UTC (permalink / raw)
To: Shirley Ma; +Cc: Jason Wang, netdev, kvm, tahm
In-Reply-To: <1332870183.6496.31.camel@oc3660625478.ibm.com>
On Tue, Mar 27, 2012 at 10:43:03AM -0700, Shirley Ma wrote:
> On Tue, 2012-03-27 at 18:09 +0800, Jason Wang wrote:
> > Hi:
> >
> > Thanks for the work and it looks very reasonable, some questions
> > below.
Yes I am happy to see the per-cpu work resurrected.
Some comments below.
> > On 03/23/2012 07:48 AM, Shirley Ma wrote:
> > > Sorry for being late to submit this patch. I have spent lots of time
> > > trying to find the best approach. This effort is still going on...
> > >
> > > This patch is built against net-next tree.
> > >
> > > This is an experimental RFC patch. The purpose of this patch is to
> > > address KVM networking scalability and NUMA scheduling issue.
> >
> > Need also test for non-NUMA machine, I see that you just choose the
> > cpu
> > that initiates the work for non-numa machine which seems sub optimal.
>
> Good suggestions. I don't have any non-numa systems. But KK run some
> tests on non-numa system. He could see around 20% performance gain for
> single VMs local host to guest. I hope we can run a full test on
> non-numa system.
>
> On non-numa system, the same per vhost-cpu thread will be always picked
> up consistently for a particular vq since all cores are on same cpu
> socket. So there will be two per-cpu vhost threads handle TX/RX
> simultaneously.
>
> > > The existing implementation of vhost creats a vhost thread
> > per-device
> > > (virtio_net) based. RX and TX work of a VMs per-device is handled by
> > > same vhost thread.
> > >
> > > One of the limitation of this implementation is with increasing the
> > > number VMs or the number of virtio-net interfces, more vhost threads
> > are
> > > created, it will consume more kernel resources, and induce more
> > threads
> > > context switches/scheduling overhead. We noticed that the KVM
> > network
> > > performance doesn't scale with increasing number of VMs.
> > >
> > > The other limitation is to have single vhost thread to process both
> > RX
> > > and TX, the work will be blocked. So we create this per cpu vhost
> > thread
> > > implementation. The number of vhost cpu threads is limited to the
> > number
> > > of cpus on the host.
> > >
> > > To address these limitations, we are propsing a per-cpu vhost thread
> > > model where the number of vhost threads are limited and equal to the
> > > number of online cpus on the host.
> >
> > The number of vhost thread needs more consideration. Consider that we
> > have a 1024 cores host with a card have 16 tx/rx queues, do we really
> > need 1024 vhost threads?
>
> In this case, we could add a module parameter to limit the number of
> cores/sockets to be used.
Hmm. And then which cores would we run on?
Also, is the parameter different between guests?
Another idea is to scale the # of threads on demand.
Sharing the same thread between guests is also an
interesting approach, if we did this then per-cpu
won't be so expensive but making this work well
with cgroups would be a challenge.
> > >
> > > Based on our testing experience, the vcpus can be scheduled across
> > cpu
> > > sockets even when the number of vcpus is smaller than the number of
> > > cores per cpu socket and there is no other activities besides KVM
> > > networking workload. We found that if vhost thread is scheduled on
> > the
> > > same socket as the work is received, the performance will be better.
> > >
> > > So in this per cpu vhost thread implementation, a vhost thread is
> > > selected dynamically based on where the TX/RX work is initiated. A
> > vhost
> > > thread on the same cpu socket is selected but not on the same cpu as
> > the
> > > vcpu/interrupt thread that initizated the TX/RX work.
> > >
> > > When we test this RFC patch, the other interesting thing we found is
> > the
> > > performance results also seem related to NIC flow steering. We are
> > > spending time on evaluate different NICs flow director
> > implementation
> > > now. We will enhance this patch based on our findings later.
> > >
> > > We have tried different scheduling: per-device based, per vq based
> > and
> > > per work type (tx_kick, rx_kick, tx_net, rx_net) based vhost
> > scheduling,
> > > we found that so far the per vq based scheduling is good enough for
> > now.
> >
> > Could you please explain more about those scheduling strategies? Does
> > per-device based means let a dedicated vhost thread to handle all
> > work
> > from that vhost device? As you mentioned, maybe an improvement of the
> > scheduling to take flow steering info (queue mapping, rxhash etc.) of
> > skb in host into account.
>
> Yes, per-device scheduling means one per-cpu vhost theads handle all
> works from one particular vhost-device.
>
> Yes, we think scheduling to take flow steering info would help
> performance. I am studying this now.
Did anything interesing turn up?
> > >
> > > We also tried different algorithm to select which cpu vhost thread
> > will
> > > running on a specific cpu socket: avg_load balance, and randomly...
> >
> > May worth to account the out-of-oder packet during the test as for a
> > single stream as different cpu/vhost/physical queue may be chose to
> > do
> > the packet transmission/reception?
>
> Good point. I haven't gone through all data yet. netstat output might
> tell us something.
>
> We used Intel 10G NIC to run all test. For a single steam test, Intel
> NIC receiving irq steers with same irq/queue which TX packets have been
> sent. So when we mask vcpus from same VM on one socket, we shouldn't hit
> packet out-of-order case. We might hit packet out of order when vcpus
> run across sockets.
>
> > >
> > > > From our test results, we found that the scalability has been
> > > significantly improved. And this patch is also helpful for small
> > packets
> > > performance.
> > >
> > > Hoever, we are seeing some regressions in a local guest to guest
> > > scenario on a 8 cpu NUMA system.
> > > In one case, 24 VMs 256 bytes tcp_stream test shows it has improved
> > from
> > > 810Mb/s to 9.1Gb/s. :)
> > > (We created two local VMs, and each VM has 2 vcpus. W/o this patch,
> > the
> > > number of threads is 4 vcpus + 2 vhosts = 6, w/i this patch is 4
> > vcpus +
> > > 8 vhosts = 12. It causes more context switches. When I change the
> > > scheduling to use 2-4 vhost threads, the regressions are gone. I am
> > > continue investigation on how to make small number of VMs, local
> > guest
> > > to gues performance better. Once I find the clue, I will share
> > here.)
So, that's one obvious reason. But there could be other explanations:
1. You explicitly mask out the same CPU. But if the socket
is very small (it's likely each socket is 2 CPUs or even 1 here),
this might limit the scheduler drastically.
2. If guest ends up running on the same socket, you cause
more IPIs which cause exists for the other guest.
> > >
> > > The cpu hotplug support hasn't in place yet. I will post it later.
Not yet done, right?
> > Another question is why not just using workqueue? It has full support
> > for cpu hotplug and allow more polices.
>
> Yes, it's good to use workqueue. I just did everything on top of current
> implementation so it's easy to compare/analyze the performance data.
>
> I remembered the vhost implementation changed from workqueue to thread
> for some reason. I couldn't recall the reason.
At the time the implementation didn't perform well with per-cpu
threads. We wanted a single thread so switched to use just that.
> > >
> > > Since we have per cpu vhost thread, each vhost thread will handle
> > > multiple vqs, so we will be able to reduce/remove vq notification
> > when
> > > the work is heavy loaded in future.
> >
> > Does this issue still exist if event index is used? If vhost does not
> > publish new used index, guest would not kick again.
>
> Since the vhost model has been changed to handle multiple VMs' vqs work,
> then it's not necessary to enable these VMs' vqs notification (published
> new used idex) where these vqs' future work will be processed on the
> same per-cpu vhost thread, as long as the per-cpu vhost thread is still
> running.
>
> > >
> > > Here is my test results for remote host to guest test: tcp_rrs,
> > udp_rrs,
> > > tcp_stream with guest has 2 vpus, host has two cpu socket, each
> > socket
> > > has 4 cores.
> > >
> > > TCP_STREAM 256 512 1K 2K 4K 8K 16K
> > > --------------------------------------------------------------------
> > > Original
> > >
> > H->Guest 2501 4238 4744 5256 7203 6975 5799 Patch
> > >
> > H->Guest 1676 2290 3149 8026 8439 8283 8216
> > >
> > > Original
> > >
> > Guest->H 744 1773 5675 1397 8207 7296 8117
> > > Patch
> > > Guest->Host 1041 1386 5407 7057 8298 8127 8241
> >
> > Looks like there's some noise in the result, the throughput of
> > "original
> > guest -> Host 2K" looks too low. And some strange is that I see
> > regressions of packet transmission of guest when testing this patch.
> > (
> > Guest to Local Host TCP_STREAM in a NUMA machine).
>
> Yes, since I didn't mask the vcpus on the same socket, it might come
> from packets out of order. I will rerun the test w/i masking vcpus on
> the same socket to see any difference.
Did anything interesting turn up?
> You can reference Tom's results. His test is more formal than mine.
>
> > >
> > > 60 instances TCP_RRs: Patch 150K trans/s vs. 91K trans/sec
> > > 65% improved with taskset vcpus on the same socket
> > > 60 instances UDP_RRs: Patch 172K trans/s vs. 103K trans/s
> > > 67% improved with taskset vcpus on the same socket
> > >
> > > Tom has run 1VM to 24 VMs test for different work. He will post it
> > here
> > > soon.
> > >
> > > If the host scheduler ensures that the VM's vcpus are not scheduled
> > to
> > > another socket (i.e. cpu mask the vcpus on same socket) then the
> > > performance will be better.
> > >
> > > Signed-off-by: Shirley Ma<xma@us.ibm.com>
> > > Signed-off-by: Krishna Kumar<krkumar2@in.ibm.com>
> > > Tested-by: Tom Lendacky<toml@us.ibm.com>
> > > ---
> > >
> > > drivers/vhost/net.c | 26 ++-
> > > drivers/vhost/vhost.c | 289
> > > +++++++++++++++++++++++----------
> > > drivers/vhost/vhost.h | 16 ++-
> > > 3 files changed, 232 insertions(+), 103 deletions(-)
> > >
> > > Thanks
> > > Shirley
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> >
Also a question: how does this interact with zero copy tx?
--
MST
^ permalink raw reply
* Re: [PATCH] tcp: allow splice() to build full TSO packets
From: Eric Dumazet @ 2012-04-05 13:05 UTC (permalink / raw)
To: David Miller
Cc: netdev, ncardwell, therbert, ycheng, hkchu, maze, maheshb,
ilpo.jarvinen, nanditad
In-Reply-To: <20120403.173614.962252876842659412.davem@davemloft.net>
On Tue, 2012-04-03 at 17:36 -0400, David Miller wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Tue, 03 Apr 2012 23:31:29 +0200
>
> > The code in tcp_sendmsg() and do_tcp_sendpages() is similar (actually
> > probably copy/pasted) but the thing is tcp_sendmsg() is called once per
> > sendmsg() call (and the push logic is OK at the end of it), while a
> > single splice() system call can call do_tcp_sendpages() 16 times (or
> > even more if pipe buffer was extended by fcntl(F_SETPIPE_SZ))
>
> Ok, so this means that in essence the tcp_mark_push should also only
> be done in the final sendpage call.
>
> And since I'm wholly convinced that the URG stuff is a complete
> "don't care" for this path, I'm convinced your patch is the right
> thing to do.
>
> Applied to 'net' and queued up for -stable, thanks Eric.
Hmm, thinking again about this, I did more tests and it appears we need
to differentiate the SPLICE_F_MORE flag (user request) and the internal
marker provided by splice logic (handling a batch of pages)
A program doing splice(... SPLICE_F_MORE) should really call tcp_push()
at the end of its work.
Thanks
[PATCH] tcp: tcp_sendpages() should call tcp_push() once
commit 2f533844242 (tcp: allow splice() to build full TSO packets) added
a regression for splice() calls using SPLICE_F_MORE.
We need to call tcp_flush() at the end of the last page processed in
tcp_sendpages(), or else transmits can be deferred and future sends
stall.
Add a new internal flag, MSG_SENDPAGE_NOTLAST, acting like MSG_MORE, but
with different semantic.
For all sendpage() providers, its a transparent change. Only
sock_sendpage() and tcp_sendpages() can differentiate the two different
flags provided by pipe_to_sendpage()
Reported-by: Tom Herbert <therbert@google.com>
Cc: Nandita Dukkipati <nanditad@google.com>
Cc: Neal Cardwell <ncardwell@google.com>
Cc: Tom Herbert <therbert@google.com>
Cc: Yuchung Cheng <ycheng@google.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Maciej Żenczykowski <maze@google.com>
Cc: Mahesh Bandewar <maheshb@google.com>
Cc: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
Signed-off-by: Eric Dumazet <eric.dumazet@gmail>com>
---
fs/splice.c | 5 ++++-
include/linux/socket.h | 2 +-
net/ipv4/tcp.c | 2 +-
net/socket.c | 6 +++---
4 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/fs/splice.c b/fs/splice.c
index 5f883de..f847684 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -30,6 +30,7 @@
#include <linux/uio.h>
#include <linux/security.h>
#include <linux/gfp.h>
+#include <linux/socket.h>
/*
* Attempt to steal a page from a pipe buffer. This should perhaps go into
@@ -690,7 +691,9 @@ static int pipe_to_sendpage(struct pipe_inode_info *pipe,
if (!likely(file->f_op && file->f_op->sendpage))
return -EINVAL;
- more = (sd->flags & SPLICE_F_MORE) || sd->len < sd->total_len;
+ more = (sd->flags & SPLICE_F_MORE) ? MSG_MORE : 0;
+ if (sd->len < sd->total_len)
+ more |= MSG_SENDPAGE_NOTLAST;
return file->f_op->sendpage(file, buf->page, buf->offset,
sd->len, &pos, more);
}
diff --git a/include/linux/socket.h b/include/linux/socket.h
index da2d3e2..b84bbd4 100644
--- a/include/linux/socket.h
+++ b/include/linux/socket.h
@@ -265,7 +265,7 @@ struct ucred {
#define MSG_NOSIGNAL 0x4000 /* Do not generate SIGPIPE */
#define MSG_MORE 0x8000 /* Sender will send more */
#define MSG_WAITFORONE 0x10000 /* recvmmsg(): block until 1+ packets avail */
-
+#define MSG_SENDPAGE_NOTLAST 0x20000 /* sendpage() internal : not the last page */
#define MSG_EOF MSG_FIN
#define MSG_CMSG_CLOEXEC 0x40000000 /* Set close_on_exit for file
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 2ff6f45..5d54ed3 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -860,7 +860,7 @@ wait_for_memory:
}
out:
- if (copied && !(flags & MSG_MORE))
+ if (copied && !(flags & MSG_SENDPAGE_NOTLAST))
tcp_push(sk, flags, mss_now, tp->nonagle);
return copied;
diff --git a/net/socket.c b/net/socket.c
index 484cc69..851edcd 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -811,9 +811,9 @@ static ssize_t sock_sendpage(struct file *file, struct page *page,
sock = file->private_data;
- flags = !(file->f_flags & O_NONBLOCK) ? 0 : MSG_DONTWAIT;
- if (more)
- flags |= MSG_MORE;
+ flags = (file->f_flags & O_NONBLOCK) ? MSG_DONTWAIT : 0;
+ /* more is a combination of MSG_MORE and MSG_SENDPAGE_NOTLAST */
+ flags |= more;
return kernel_sendpage(sock, page, offset, size, flags);
}
^ permalink raw reply related
* Re: [PATCH] m68k/atari: EtherNEC - rewrite to use mainstream ne.c, take two
From: Paul Gortmaker @ 2012-04-05 13:24 UTC (permalink / raw)
To: Geert Uytterhoeven; +Cc: Michael Schmitz, linux-m68k, netdev
In-Reply-To: <CAMuHMdWSzG4npwjpJ=cwNFKhyBZjvgPn1sp6nZ2re6QsRWPpXg@mail.gmail.com>
On 12-04-05 05:28 AM, Geert Uytterhoeven wrote:
> On Wed, Apr 4, 2012 at 22:46, Paul Gortmaker
> <paul.gortmaker@windriver.com> wrote:
>> On 12-04-01 04:49 AM, Michael Schmitz wrote:
>>>> And on re-reading the comments in the other part of the patch, i.e.
>>>> "...emulates the card interrupt via a timer" --perhaps the driver
>>>> should be just fixed to support generic netpoll, instead of adding
>>>> an arch specific thing that amounts to netpoll. Then anyone can
>>>> attempt to limp along and use one of these ancient relics w/o IRQ.
>>>>
>>> Here's take two of my patch to convert the m68k Atari ROM port Ethernet
>>> driver to use mainstream ne.c, with minimal changes to the core NE2000
>>> code.
>>>
>>> In particular:
>>>
>>> Changes to core net code:
>>> * add a platform specific IRQ flag, so ne.c can share a hardware or
>>> timer interrupt with some other interrupt source.
>>>
>>> Changes to arch/m68k code:
>>> * register the 8390 platform device on Atari only if the hardware is present
>>> * retain the old driver (atari_ethernec.c in Geert's tree) under a
>>> different config option, to be removed soon.
>>>
>>> Regarding your suggestion that netpoll be used instead of a dedicated
>>> timer interrupt: no changes to ne.c or 8390p.c are required to use
>>> netpoll, it all works out of the box. All that is needed to use the
>>> driver with netpoll is setting the device interrupt to some source that
>>> can be registered, and enabling CONFIG_NETPOLL. Interrupt rate and hence
>>> throughput is lower with netpoll though, which is why I still prefer the
>>> dedicated timer option.
>>
>> How much lower? Enough to matter? Implicit in that question is
>> the assumption that this is largely a hobbyist platform and nobody
>> is using it in a closet to route gigabytes of traffic.
>
> One other thing we could do is increase CONFIG_HZ to 250.
>
>> Also, the only advantage to modifying ne.c is to allow dumping
>> the old driver. What is the "remove soon" plan? Any reason
>> for it to not be synchronous? That would eliminate the Kconfig
>> churn and the introduction of the _OLD option. Modifying ne.c
>> and then deciding to keep the old driver because it is "faster"
>> would make this change pointless.
>
> From my point of view, "remove soon" means it will never hit mainline.
Can you clarify what "it" is? It isn't clear to me if you
mean the _removal_ will never hit mainline, or the transient
renamed "old" driver will never hit mainline.
If the former, then there is no point pursuing this any further
as I said above.
If the latter, then the commit sent out for review should have
no instances of this "renaming to old" related changes.
Thanks,
Paul.
>
>>> diff --git a/drivers/net/ethernet/8390/8390.h
>>> b/drivers/net/ethernet/8390/8390.h
>>> index ef325ff..9416245 100644
>>> --- a/drivers/net/ethernet/8390/8390.h
>>> +++ b/drivers/net/ethernet/8390/8390.h
>>> @@ -32,6 +32,14 @@ extern void ei_poll(struct net_device *dev);
>>> extern void eip_poll(struct net_device *dev);
>>> #endif
>>>
>>> +/* Some platforms may need special IRQ flags */
>>> +#if IS_ENABLED(CONFIG_ATARI_ETHERNEC)
>>> +# define EI_IRQ_FLAGS IRQF_SHARED
>>> +#endif
>>> +
>>> +#ifndef EI_IRQ_FLAGS
>>> +# define EI_IRQ_FLAGS 0
>>> +#endif
>>
>> This seems more klunky than it needs to be. If we assume that anyone
>> building ne.c on atari is hence trying to drive an ethernec device
>> than it can just be
>>
>> #ifdef CONFIG_ATARI
>> #define EI_IRQ_FLAGS IRQF_SHARED
>> #else
>> #define EI_IRQ_FLAGS 0
>> #endif
>
> Indeed, with a small modification (keep multi-platform kernels in mind):
>
> #ifdef CONFIG_ATARI
> #define EI_IRQ_FLAGS (MACH_IS_ATARI ? IRQF_SHARED : 0)
> #else
> #define EI_IRQ_FLAGS 0
> #endif
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
^ permalink raw reply
* [PATCH 1/1] bonding: properly unset current_arp_slave on slave link up
From: Veaceslav Falico @ 2012-04-05 13:47 UTC (permalink / raw)
To: netdev; +Cc: Jay Vosburgh, Andy Gospodarek, linux-kernel
When a slave comes up, we're unsetting the current_arp_slave without
removing active flags from it, which can lead to situations where we have
more than one slave with active flags in active-backup mode.
To avoid this situation we must remove the active flags from a slave before
removing it as a current_arp_slave.
Signed-off-by: Veaceslav Falico <vfalico@redhat.com>
---
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 0c76186..ad731a1 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -3001,7 +3001,11 @@ static void bond_ab_arp_commit(struct bonding *bond, int delta_in_ticks)
trans_start + delta_in_ticks)) ||
bond->curr_active_slave != slave) {
slave->link = BOND_LINK_UP;
- bond->current_arp_slave = NULL;
+ if (bond->current_arp_slave) {
+ bond_set_slave_inactive_flags(
+ bond->current_arp_slave);
+ bond->current_arp_slave = NULL;
+ }
pr_info("%s: link status definitely up for interface %s.\n",
bond->dev->name, slave->dev->name);
^ permalink raw reply related
* Re: [PATCH] m68k/atari: EtherNEC - rewrite to use mainstream ne.c, take two
From: Geert Uytterhoeven @ 2012-04-05 14:21 UTC (permalink / raw)
To: Paul Gortmaker; +Cc: Michael Schmitz, linux-m68k, netdev
In-Reply-To: <4F7D9D1B.3000801@windriver.com>
Hi Paul,
On Thu, Apr 5, 2012 at 15:24, Paul Gortmaker
<paul.gortmaker@windriver.com> wrote:
> On 12-04-05 05:28 AM, Geert Uytterhoeven wrote:
>> On Wed, Apr 4, 2012 at 22:46, Paul Gortmaker
>> <paul.gortmaker@windriver.com> wrote:
>>> On 12-04-01 04:49 AM, Michael Schmitz wrote:
>>>>> And on re-reading the comments in the other part of the patch, i.e.
>>>>> "...emulates the card interrupt via a timer" --perhaps the driver
>>>>> should be just fixed to support generic netpoll, instead of adding
>>>>> an arch specific thing that amounts to netpoll. Then anyone can
>>>>> attempt to limp along and use one of these ancient relics w/o IRQ.
>>>>>
>>>> Here's take two of my patch to convert the m68k Atari ROM port Ethernet
>>>> driver to use mainstream ne.c, with minimal changes to the core NE2000
>>>> code.
>>>>
>>>> In particular:
>>>>
>>>> Changes to core net code:
>>>> * add a platform specific IRQ flag, so ne.c can share a hardware or
>>>> timer interrupt with some other interrupt source.
>>>>
>>>> Changes to arch/m68k code:
>>>> * register the 8390 platform device on Atari only if the hardware is present
>>>> * retain the old driver (atari_ethernec.c in Geert's tree) under a
>>>> different config option, to be removed soon.
>>>>
>>>> Regarding your suggestion that netpoll be used instead of a dedicated
>>>> timer interrupt: no changes to ne.c or 8390p.c are required to use
>>>> netpoll, it all works out of the box. All that is needed to use the
>>>> driver with netpoll is setting the device interrupt to some source that
>>>> can be registered, and enabling CONFIG_NETPOLL. Interrupt rate and hence
>>>> throughput is lower with netpoll though, which is why I still prefer the
>>>> dedicated timer option.
>>>
>>> How much lower? Enough to matter? Implicit in that question is
>>> the assumption that this is largely a hobbyist platform and nobody
>>> is using it in a closet to route gigabytes of traffic.
>>
>> One other thing we could do is increase CONFIG_HZ to 250.
>>
>>> Also, the only advantage to modifying ne.c is to allow dumping
>>> the old driver. What is the "remove soon" plan? Any reason
>>> for it to not be synchronous? That would eliminate the Kconfig
>>> churn and the introduction of the _OLD option. Modifying ne.c
>>> and then deciding to keep the old driver because it is "faster"
>>> would make this change pointless.
>>
>> From my point of view, "remove soon" means it will never hit mainline.
>
> Can you clarify what "it" is? It isn't clear to me if you
> mean the _removal_ will never hit mainline, or the transient
> renamed "old" driver will never hit mainline.
>
> If the former, then there is no point pursuing this any further
> as I said above.
>
> If the latter, then the commit sent out for review should have
> no instances of this "renaming to old" related changes.
Sorry for being unclear. The latter (i.e. the old driver will
never hit mainline).
And Michael's patch against ne.c should indeed not touch the old
driver, as it doesn't exist in mainline.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: ipv6: tunnel: hang when destroying ipv6 tunnel
From: Tetsuo Handa @ 2012-04-05 14:29 UTC (permalink / raw)
To: levinsasha928, garlick, ericvh
Cc: oleg, eric.dumazet, davem, kuznet, jmorris, yoshfuji, kaber,
netdev, linux-kernel, davej
In-Reply-To: <CA+1xoqe+T=iUtUeFdeqmRvMZ7TMCgkLs_1QYv-4YNtn-jCoMCQ@mail.gmail.com>
> Tetsuo Handa wrote:
> > Maybe you can get more useful information with below untested printk() patch.
> >
> > diff --git a/net/9p/client.c b/net/9p/client.c
> > index b23a17c..2dd447a 100644
> > --- a/net/9p/client.c
> > +++ b/net/9p/client.c
> > @@ -734,7 +734,9 @@ p9_client_rpc(struct p9_client *c, int8_t type, const char *fmt, ...)
> > } else
> > sigpending = 0;
> >
> > + printk("%u:Calling %pS\n", current->pid, c->trans_mod->request);
> > err = c->trans_mod->request(c, req);
> > + printk("%u:%pS = %d\n", current->pid, c->trans_mod->request, err);
> > if (err < 0) {
> > if (err != -ERESTARTSYS && err != -EFAULT)
> > c->status = Disconnected;
> > @@ -742,8 +744,10 @@ p9_client_rpc(struct p9_client *c, int8_t type, const char *fmt, ...)
> > }
> > again:
> > /* Wait for the response */
> > + printk("%u:req->status = %u\n", current->pid, req->status);
> > err = wait_event_interruptible(*req->wq,
> > req->status >= REQ_STATUS_RCVD);
> > + printk("%u:wait = %d\n", current->pid, err);
> >
> > if ((err == -ERESTARTSYS) && (c->status == Connected)
> > && (type == P9_TFLUSH)) {
> >
Sasha Levin wrote:
> Heya,
>
> The output from the printk confirmed that there are several threads
> waiting for RPC to complete, with the last two having and odd 'wait'
> result. This is just before the hang:
>
> [ 809.165663] 19964:Calling p9_virtio_request+0x0/0x200
> [ 809.166951] 19964:p9_virtio_request+0x0/0x200 = 0
> [ 809.167878] 19964:req->status = 3
> [ 809.803535] 19957:Calling p9_virtio_request+0x0/0x200
> [ 809.804506] 19957:p9_virtio_request+0x0/0x200 = 0
> [ 809.805332] 19957:req->status = 3
> [ 809.868591] 19955:Calling p9_virtio_request+0x0/0x200
> [ 809.869493] 19955:p9_virtio_request+0x0/0x200 = 0
> [ 809.870331] 19955:req->status = 3
> [ 811.364554] 19985:Calling p9_virtio_request+0x0/0x200
> [ 811.365498] 19985:p9_virtio_request+0x0/0x200 = 0
> [ 811.366386] 19985:req->status = 3
> [ 811.458600] 19999:wait = -512
> [ 811.459171] 19999:Calling p9_virtio_request+0x0/0x200
> [ 811.459992] 19999:p9_virtio_request+0x0/0x200 = 0
> [ 811.460822] 19999:req->status = 3
> [ 811.472175] 19994:wait = -512
> [ 811.472943] 19994:Calling p9_virtio_request+0x0/0x200
> [ 811.474195] 19994:p9_virtio_request+0x0/0x200 = 0
> [ 811.474955] 19994:req->status = 3
> [... Hang 120 sec later here]
>
Good. -512 is -ERESTARTSYS, and this hang occurs after -ERESTARTSYS is
returned. It indicates that c->trans_mod->request() is interrupted by signal.
Since c->trans_mod->request is pointing at p9_virtio_request, the location
returning that error would be
254 static int
255 p9_virtio_request(struct p9_client *client, struct p9_req_t *req)
256 {
257 int err;
258 int in, out;
259 unsigned long flags;
260 struct virtio_chan *chan = client->trans;
261
262 p9_debug(P9_DEBUG_TRANS, "9p debug: virtio request\n");
263
264 req->status = REQ_STATUS_SENT;
265 req_retry:
266 spin_lock_irqsave(&chan->lock, flags);
267
268 /* Handle out VirtIO ring buffers */
269 out = pack_sg_list(chan->sg, 0,
270 VIRTQUEUE_NUM, req->tc->sdata, req->tc->size);
271
272 in = pack_sg_list(chan->sg, out,
273 VIRTQUEUE_NUM, req->rc->sdata, req->rc->capacity);
274
275 err = virtqueue_add_buf(chan->vq, chan->sg, out, in, req->tc,
276 GFP_ATOMIC);
277 if (err < 0) {
278 if (err == -ENOSPC) {
279 chan->ring_bufs_avail = 0;
280 spin_unlock_irqrestore(&chan->lock, flags);
281 err = wait_event_interruptible(*chan->vc_wq,
282 chan->ring_bufs_avail);
here.
283 if (err == -ERESTARTSYS)
284 return err;
285
286 p9_debug(P9_DEBUG_TRANS, "Retry virtio request\n");
287 goto req_retry;
288 } else {
289 spin_unlock_irqrestore(&chan->lock, flags);
290 p9_debug(P9_DEBUG_TRANS,
291 "virtio rpc add_buf returned failure\n");
292 return -EIO;
293 }
294 }
295 virtqueue_kick(chan->vq);
296 spin_unlock_irqrestore(&chan->lock, flags);
297
298 p9_debug(P9_DEBUG_TRANS, "virtio request kicked\n");
299 return 0;
300 }
Comparing 3.3.1 and linux-next in my environment, there are several changes.
# diff -ur linux-3.3.1/drivers/virtio/ linux-next/drivers/virtio/ | diffstat
config.c | 1
virtio_balloon.c | 14 ----------
virtio_pci.c | 74 +++++--------------------------------------------------
3 files changed, 8 insertions(+), 81 deletions(-)
# diff -urp linux-3.3.1/fs/9p/ linux-next/fs/9p/ | diffstat
v9fs.c | 16 ++++++++--------
vfs_super.c | 5 ++---
2 files changed, 10 insertions(+), 11 deletions(-)
# diff -ur linux-3.3.1/net/9p/ linux-next/net/9p/ | diffstat
client.c | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
Most suspicious change is net/9p/client.c because it is changing handling of
ERESTARTSYS case.
--- linux-3.3.1/net/9p/client.c
+++ linux-next/net/9p/client.c
@@ -740,10 +740,18 @@
c->status = Disconnected;
goto reterr;
}
+again:
/* Wait for the response */
err = wait_event_interruptible(*req->wq,
req->status >= REQ_STATUS_RCVD);
+ if ((err == -ERESTARTSYS) && (c->status == Connected)
+ && (type == P9_TFLUSH)) {
+ sigpending = 1;
+ clear_thread_flag(TIF_SIGPENDING);
+ goto again;
+ }
+
if (req->status == REQ_STATUS_ERROR) {
p9_debug(P9_DEBUG_ERROR, "req_status error %d\n", req->t_err);
err = req->t_err;
@@ -1420,6 +1428,7 @@
int err;
struct p9_client *clnt;
struct p9_req_t *req;
+ int retries = 0;
if (!fid) {
pr_warn("%s (%d): Trying to clunk with NULL fid\n",
@@ -1428,7 +1437,9 @@
return 0;
}
- p9_debug(P9_DEBUG_9P, ">>> TCLUNK fid %d\n", fid->fid);
+again:
+ p9_debug(P9_DEBUG_9P, ">>> TCLUNK fid %d (try %d)\n", fid->fid,
+ retries);
err = 0;
clnt = fid->clnt;
@@ -1444,8 +1455,14 @@
error:
/*
* Fid is not valid even after a failed clunk
+ * If interrupted, retry once then give up and
+ * leak fid until umount.
*/
- p9_fid_destroy(fid);
+ if (err == -ERESTARTSYS) {
+ if (retries++ == 0)
+ goto again;
+ } else
+ p9_fid_destroy(fid);
return err;
}
EXPORT_SYMBOL(p9_client_clunk);
@@ -1470,7 +1487,10 @@
p9_free_req(clnt, req);
error:
- p9_fid_destroy(fid);
+ if (err == -ERESTARTSYS)
+ p9_client_clunk(fid);
+ else
+ p9_fid_destroy(fid);
return err;
}
EXPORT_SYMBOL(p9_client_remove);
Maybe commit a314f274 "net/9p: don't allow Tflush to be interrupted" or nearby.
By the way, have you already tried 3.4-rc1?
In my environment, there is no difference between linux-next and 3.4-rc1.
# diff -ur linux-3.4.0-rc1/net/9p/ linux-next/net/9p/
# diff -ur linux-3.4.0-rc1/drivers/virtio/ linux-next/drivers/virtio/
# diff -ur linux-3.4.0-rc1/fs/9p/ linux-next/fs/9p/
^ permalink raw reply
* Re: ipv6: tunnel: hang when destroying ipv6 tunnel
From: Tetsuo Handa @ 2012-04-05 14:34 UTC (permalink / raw)
To: levinsasha928, garlick, ericvh
Cc: oleg, eric.dumazet, davem, kuznet, jmorris, yoshfuji, kaber,
netdev, linux-kernel, davej, penguin-kernel
In-Reply-To: <201204052329.EBH52139.FFOLQMOJSVHOFt@I-love.SAKURA.ne.jp>
Tetsuo Handa wrote:
> Good. -512 is -ERESTARTSYS, and this hang occurs after -ERESTARTSYS is
> returned. It indicates that c->trans_mod->request() is interrupted by signal.
> Since c->trans_mod->request is pointing at p9_virtio_request, the location
> returning that error would be
(...snipped...)
> 281 err = wait_event_interruptible(*chan->vc_wq,
> 282 chan->ring_bufs_avail);
>
> here.
Oops. Not p9_virtio_request().
It is p9_client_rpc(). I misread the output lines.
> > > + printk("%u:req->status = %u\n", current->pid, req->status);
> > > err = wait_event_interruptible(*req->wq,
> > > req->status >= REQ_STATUS_RCVD);
> > > + printk("%u:wait = %d\n", current->pid, err);
But anyway, I think this is interupt related bug.
^ permalink raw reply
* Server Rental Service in Hong Kong
From: boris @ 2012-04-05 14:13 UTC (permalink / raw)
Dear All,
We have our own datacenter in Hong Kong & provide email/application/web rental service to clients.We are APNIC member & provide clean IP to clients.
Dell? PowerEdge? EnterpriseRack Mount Server
-Intel(R) Xeon(R) E3-1240 Processor (3.3GHz, 8M Cache, Turbo, 4C/8T, 80W)
-8GB RAM, 2x4GB, 1333MHz, DDR-3, Dual Ranked UDIMMs
-500GB, 3.5", 6Gbps SAS x 2
-Raid 1 Mirroring Protection
-Remote KVM (iDRAC6 Enterprise)
Dell(TM) PowerEdge(TM) R410 Rack Mount Server
-Intel(R) Quad Core E5606 Xeon(R) CPU, 2.13GHz, 4M Cache, 4.86 GT/s QPI
-4GB Memory (2x2GB), 1333MHz Dual Ranked RDIMMs Fully-Buffered
-500GB 7.2K RPM SATAII 3.5" Hard Drive x 2
-iDRAC6 Enterprise or Express (Remote KVM Management)
Every Dedicated Server Hosting Solution Also Includes:
Software Specification
- CentOS / Fedora / Debian / FreeBSD / Ubuntu / Redhat Linux
- Full root-level access
- Data Center Facilities
- Shared Local & International Bandwidth
- 2 IP Addresses Allocation
- Un-interruptible Power Supply (UPS) backed up by private diesel generator
- FM200¡§based fire suppression system
- 24x7 CRAC Air Conditioning and Humidity Control
- 24x7 Security Control
- 24x7 Remote Hand Service
Pls send us email for further information.Thanks,
Boris
boris@cloudluca.com
If you do not wish to further receive this event message, email "borislamsv2@gmail.com" to unsubscribe this message or remove your email from the list.
^ permalink raw reply
* Re: [PATCH 5/9] ipvs: use adaptive pause in master thread
From: Pablo Neira Ayuso @ 2012-04-05 15:05 UTC (permalink / raw)
To: Julian Anastasov
Cc: Simon Horman, lvs-devel, netdev, netfilter-devel, Wensong Zhang
In-Reply-To: <alpine.LFD.2.00.1204032258540.1995@ja.ssi.bg>
Hi Julian,
On Wed, Apr 04, 2012 at 12:16:15AM +0300, Julian Anastasov wrote:
>
> Hello,
>
> On Mon, 2 Apr 2012, Pablo Neira Ayuso wrote:
>
> > > From: Julian Anastasov <ja@ssi.bg>
> > >
> > > High rate of sync messages in master can lead to
> > > overflowing the socket buffer and dropping the messages.
> > > Instead of using fixed pause try to adapt to the sync
> > > rate, so that we do not wakeup too often while at the same
> > > time staying below the socket buffer limit.
> >
> > I see. You are avoiding the congestion in the socket queue by putting
> > the pressure in your sync_buff queue (I don't find any limit in
> > the code for the amount of memory that it may consume btw).
> >
> > Please, correct me if I'm wrong, but from this I can se you're
> > assuming that there's always room in the syn_buff queue to store
> > messages. This is valid if the high peak of traffic lasts for short
> > time. However, if it lasts long, the worker thread may not be able to
> > consume all the messages in time under high stress situation that
> > lasts long and the sync_buffer may keep growing more and more.
>
> Agreed, packet processing does not check for any
> limits for the sync_queues lists. This can be addressed
> additionally, may be there can be some wakeup if threshold
> is reached, so that we wake up the master thread early.
> But there is always the problem that we do not know how
> deep/free is the socket buffer, after how much time the
> master thread will wakeup... We have to select some arbitrary
> number again.
I see, see below.
> > Moreover, the backup may receive sync messages talking about old
> > states that are not useful anymore to recover the load-balancing in
> > case of failover.
>
> Can you explain more? Are you referring to the
> 2-second buffer delay?
I was refering to the situation in which you decide to delay sync
messages due to congestion. State sync needs to be soft real-time to
ensure that the backup is one copy-equivalence of the master (at
least, close to it, even if that's not possible). Otherwise, the
backup may contain useless out-of-date states.
> > One more concern, please see below.
>
> > > - schedule_timeout_interruptible(HZ);
> > > + /* Max packets to send at once */
> > > + if (count > 200)
> > > + pause = max(1, (pause - HZ / 20));
> > > + else if (count < 20)
> > > + pause = min(HZ / 4, (pause + HZ / 20));
> > > + schedule_timeout_interruptible(sb ? 1 : pause);
> >
> > This rate-limits the amount of times the worker is woken up.
> >
> > Still, this seems to me like an ad-hoc congestion solution. There's no
> > justification why those numbers has been chosed, eg. why do we assume
> > that we're congested if we've reached 200 packets in one single loop?
>
> 200 is arbitrary, 20% of txqueuelen, a way to avoid
> large bursts. It is the 'pause = max(1, (pause >> 1));' part
> that also matters because we try to increase the wakeup rate, so
> that socket buffer is below 50%. We assume sync traffic rate
> does not change much but we are still prepared to avoid
> drops while pause > 1 jiffie. If sync rate is too high may be
> we should wakeup the master thread in sb_queue_tail.
>
> The above logic should be read like this:
>
> - limit bursts to min(200, half of guessed socket buffer size)
> - limit pause to HZ/4 jiffies in idle periods
> - if socket buffer is full try again after 1 jiffie and
> increase the wakeups twice, so that we do not hit the
> socket limit again
I think you can control when the kernel thread is woken up with a
counting semaphore. The counter of that semaphore will be initially
set to zero. Then, you can up() the semaphore once per new buffer
that you enqueue to the sender.
feeder:
add message to sync buffer
if buffer full:
enqueue buffer to sender_thread
up(s)
sender_thread:
while (1) {
down(s)
retrieve message from queue
send message
}
It seems to me like the classical producer/consumer problem that you
can resolve with semaphores.
Thus, you don't need to poll periodically to check if there's
messages. You won't need to adapt "pause" time.
> > To me, congestion control is a complicated thing (there is plenty of
> > literature for TCP avoid it). I'm not sure how many patches will
> > follow after this one to try to improve your congestion control
> > solution.
> >
> > So, my question is, are you sure you want to enter this domain?
>
> We should start with something. Better ideas are
> always welcome.
>
> > IMO, better to stick to some simple solution, ie. just drop messages
> > if the worker thread receives a high peak of work, than trying to
> > define some sort of rate-limit solution based on assumptions that are
> > not generic enough for every setup.
>
> Before now we dropped messages due to large fixed sleep time.
> They were dropped when socket buffer was full. Dropping messages is
> not nice just because we decided to sleep longer than needed.
> With the change we still drop messages but not before pause
> reaches 1 jiffie. As we can not use lower sleep time only
> additional wakeup in sb_queue_tail can keep up with higher
> sync rates. May be we should decide to drop messages only
> when some max limit for packets in sync_queues is reached.
Under congestion the situation is complicated. At some point you'll
end up dropping messages.
You may want to increase the socket queue to delay the moment at which
we start dropping messages. You can expose the socke buffer length via
/proc interface I guess (not sure if you're already doing that or
suggesting to use the global socket buffer length).
You also can define some mechanism to reduce the amount of events,
some state filtering so you only propagate important states.
Some partially reliable protocol, so the backup can request messages
that got lost in a smart way would can also in handy. Basically, the
master only retransmits the current state, not the whole sequence of
messages (this is good under congestion, since you save messages).
I implement that in conntrackd, but that's more complex solution,
of course. I'd start with something simple.
^ permalink raw reply
* [PATCH 1/2] igb: fix rtnl race in PM resume path
From: Benjamin Poirier @ 2012-04-05 15:11 UTC (permalink / raw)
To: Jeff Kirsher
Cc: Emil Tantilov, e1000-devel, Bruce Allan, Jesse Brandeburg,
linux-kernel, John Fastabend, John Ronciak, netdev,
David S. Miller
Since the caller (PM resume code) is not the one holding rtnl, when taking the
'else' branch rtnl may be released at any moment, thereby defeating the whole
purpose of this code block.
Signed-off-by: Benjamin Poirier <bpoirier@suse.de>
---
drivers/net/ethernet/intel/igb/igb_main.c | 20 ++++++--------------
1 files changed, 6 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index f022ff7..4854ab6 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1083,9 +1083,12 @@ msi_only:
adapter->flags |= IGB_FLAG_HAS_MSI;
out:
/* Notify the stack of the (possibly) reduced queue counts. */
+ rtnl_lock();
netif_set_real_num_tx_queues(adapter->netdev, adapter->num_tx_queues);
- return netif_set_real_num_rx_queues(adapter->netdev,
- adapter->num_rx_queues);
+ err = netif_set_real_num_rx_queues(adapter->netdev,
+ adapter->num_rx_queues);
+ rtnl_unlock();
+ return err;
}
/**
@@ -6646,18 +6649,7 @@ static int igb_resume(struct device *dev)
pci_enable_wake(pdev, PCI_D3hot, 0);
pci_enable_wake(pdev, PCI_D3cold, 0);
- if (!rtnl_is_locked()) {
- /*
- * shut up ASSERT_RTNL() warning in
- * netif_set_real_num_tx/rx_queues.
- */
- rtnl_lock();
- err = igb_init_interrupt_scheme(adapter);
- rtnl_unlock();
- } else {
- err = igb_init_interrupt_scheme(adapter);
- }
- if (err) {
+ if (igb_init_interrupt_scheme(adapter)) {
dev_err(&pdev->dev, "Unable to allocate memory for queues\n");
return -ENOMEM;
}
--
1.7.7
------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply related
* [PATCH 2/2] ixgbe: add missing rtnl_lock in PM resume path
From: Benjamin Poirier @ 2012-04-05 15:11 UTC (permalink / raw)
To: Jeff Kirsher
Cc: Emil Tantilov, e1000-devel, Bruce Allan, Jesse Brandeburg,
linux-kernel, John Fastabend, John Ronciak, netdev,
David S. Miller
In-Reply-To: <1333638664-17062-1-git-send-email-bpoirier@suse.de>
Upon resume from standby, ixgbe may trigger the ASSERT_RTNL() in
netif_set_real_num_tx_queues(). The call stack is:
netif_set_real_num_tx_queues
ixgbe_set_num_queues
ixgbe_init_interrupt_scheme
ixgbe_resume
Signed-off-by: Benjamin Poirier <bpoirier@suse.de>
---
Unlike patch 1 for igb, we cannot lock rtnl closer to the point where
netif_set_real_num_tx_queues() is called because ixgbe_init_interrupt_scheme()
is also called by the dcb code, already under rtnl.
---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 3e26b1f..ff6edab 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -4836,7 +4836,9 @@ static int ixgbe_resume(struct pci_dev *pdev)
pci_wake_from_d3(pdev, false);
+ rtnl_lock();
err = ixgbe_init_interrupt_scheme(adapter);
+ rtnl_unlock();
if (err) {
e_dev_err("Cannot initialize interrupts for device\n");
return err;
--
1.7.7
------------------------------------------------------------------------------
Better than sec? Nothing is better than sec when it comes to
monitoring Big Data applications. Try Boundary one-second
resolution app monitoring today. Free.
http://p.sf.net/sfu/Boundary-dev2dev
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply related
* Re: [RFC PATCH 0/1] NUMA aware scheduling per cpu vhost thread patch
From: Shirley Ma @ 2012-04-05 15:22 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: Jason Wang, netdev, kvm, tahm, vivek
In-Reply-To: <20120405122824.GB31982@redhat.com>
On Thu, 2012-04-05 at 15:28 +0300, Michael S. Tsirkin wrote:
> On Tue, Mar 27, 2012 at 10:43:03AM -0700, Shirley Ma wrote:
> > On Tue, 2012-03-27 at 18:09 +0800, Jason Wang wrote:
> > > Hi:
> > >
> > > Thanks for the work and it looks very reasonable, some questions
> > > below.
>
> Yes I am happy to see the per-cpu work resurrected.
> Some comments below.
Glad to see you have time on reviewing this.
> > > On 03/23/2012 07:48 AM, Shirley Ma wrote:
> > > > Sorry for being late to submit this patch. I have spent lots of
> time
> > > > trying to find the best approach. This effort is still going
> on...
> > > >
> > > > This patch is built against net-next tree.
> > > >
> > > > This is an experimental RFC patch. The purpose of this patch is
> to
> > > > address KVM networking scalability and NUMA scheduling issue.
> > >
> > > Need also test for non-NUMA machine, I see that you just choose
> the
> > > cpu
> > > that initiates the work for non-numa machine which seems sub
> optimal.
> >
> > Good suggestions. I don't have any non-numa systems. But KK run some
> > tests on non-numa system. He could see around 20% performance gain
> for
> > single VMs local host to guest. I hope we can run a full test on
> > non-numa system.
> >
> > On non-numa system, the same per vhost-cpu thread will be always
> picked
> > up consistently for a particular vq since all cores are on same cpu
> > socket. So there will be two per-cpu vhost threads handle TX/RX
> > simultaneously.
> >
> > > > The existing implementation of vhost creats a vhost thread
> > > per-device
> > > > (virtio_net) based. RX and TX work of a VMs per-device is
> handled by
> > > > same vhost thread.
> > > >
> > > > One of the limitation of this implementation is with increasing
> the
> > > > number VMs or the number of virtio-net interfces, more vhost
> threads
> > > are
> > > > created, it will consume more kernel resources, and induce more
> > > threads
> > > > context switches/scheduling overhead. We noticed that the KVM
> > > network
> > > > performance doesn't scale with increasing number of VMs.
> > > >
> > > > The other limitation is to have single vhost thread to process
> both
> > > RX
> > > > and TX, the work will be blocked. So we create this per cpu
> vhost
> > > thread
> > > > implementation. The number of vhost cpu threads is limited to
> the
> > > number
> > > > of cpus on the host.
> > > >
> > > > To address these limitations, we are propsing a per-cpu vhost
> thread
> > > > model where the number of vhost threads are limited and equal to
> the
> > > > number of online cpus on the host.
> > >
> > > The number of vhost thread needs more consideration. Consider that
> we
> > > have a 1024 cores host with a card have 16 tx/rx queues, do we
> really
> > > need 1024 vhost threads?
> >
> > In this case, we could add a module parameter to limit the number of
> > cores/sockets to be used.
>
> Hmm. And then which cores would we run on?
> Also, is the parameter different between guests?
> Another idea is to scale the # of threads on demand.
If we are able to pass number of guests/vcpus info to vhost, we can
scale the vhost threads. Any API to get this info?
> Sharing the same thread between guests is also an
> interesting approach, if we did this then per-cpu
> won't be so expensive but making this work well
> with cgroups would be a challenge.
Yes, I am comparing vhost thread pool to share among guests approach
with per-cpu vhost approach now.
It's challenge to work with cgroups anyway.
>
> > > >
> > > > Based on our testing experience, the vcpus can be scheduled
> across
> > > cpu
> > > > sockets even when the number of vcpus is smaller than the number
> of
> > > > cores per cpu socket and there is no other activities besides
> KVM
> > > > networking workload. We found that if vhost thread is scheduled
> on
> > > the
> > > > same socket as the work is received, the performance will be
> better.
> > > >
> > > > So in this per cpu vhost thread implementation, a vhost thread
> is
> > > > selected dynamically based on where the TX/RX work is initiated.
> A
> > > vhost
> > > > thread on the same cpu socket is selected but not on the same
> cpu as
> > > the
> > > > vcpu/interrupt thread that initizated the TX/RX work.
> > > >
> > > > When we test this RFC patch, the other interesting thing we
> found is
> > > the
> > > > performance results also seem related to NIC flow steering. We
> are
> > > > spending time on evaluate different NICs flow director
> > > implementation
> > > > now. We will enhance this patch based on our findings later.
> > > >
> > > > We have tried different scheduling: per-device based, per vq
> based
> > > and
> > > > per work type (tx_kick, rx_kick, tx_net, rx_net) based vhost
> > > scheduling,
> > > > we found that so far the per vq based scheduling is good enough
> for
> > > now.
> > >
> > > Could you please explain more about those scheduling strategies?
> Does
> > > per-device based means let a dedicated vhost thread to handle all
> > > work
> > > from that vhost device? As you mentioned, maybe an improvement of
> the
> > > scheduling to take flow steering info (queue mapping, rxhash etc.)
> of
> > > skb in host into account.
> >
> > Yes, per-device scheduling means one per-cpu vhost theads handle all
> > works from one particular vhost-device.
> >
> > Yes, we think scheduling to take flow steering info would help
> > performance. I am studying this now.
>
> Did anything interesing turn up?
Not yet, still investigating.
>
> > > >
> > > > We also tried different algorithm to select which cpu vhost
> thread
> > > will
> > > > running on a specific cpu socket: avg_load balance, and
> randomly...
> > >
> > > May worth to account the out-of-oder packet during the test as for
> a
> > > single stream as different cpu/vhost/physical queue may be chose
> to
> > > do
> > > the packet transmission/reception?
> >
> > Good point. I haven't gone through all data yet. netstat output
> might
> > tell us something.
> >
> > We used Intel 10G NIC to run all test. For a single steam test,
> Intel
> > NIC receiving irq steers with same irq/queue which TX packets have
> been
> > sent. So when we mask vcpus from same VM on one socket, we shouldn't
> hit
> > packet out-of-order case. We might hit packet out of order when
> vcpus
> > run across sockets.
> >
> > > >
> > > > > From our test results, we found that the scalability has been
> > > > significantly improved. And this patch is also helpful for small
> > > packets
> > > > performance.
> > > >
> > > > Hoever, we are seeing some regressions in a local guest to guest
> > > > scenario on a 8 cpu NUMA system.
> > > > In one case, 24 VMs 256 bytes tcp_stream test shows it has
> improved
> > > from
> > > > 810Mb/s to 9.1Gb/s. :)
> > > > (We created two local VMs, and each VM has 2 vcpus. W/o this
> patch,
> > > the
> > > > number of threads is 4 vcpus + 2 vhosts = 6, w/i this patch is 4
> > > vcpus +
> > > > 8 vhosts = 12. It causes more context switches. When I change
> the
> > > > scheduling to use 2-4 vhost threads, the regressions are gone. I
> am
> > > > continue investigation on how to make small number of VMs, local
> > > guest
> > > > to gues performance better. Once I find the clue, I will share
> > > here.)
>
> So, that's one obvious reason. But there could be other explanations:
> 1. You explicitly mask out the same CPU. But if the socket
> is very small (it's likely each socket is 2 CPUs or even 1 here),
> this might limit the scheduler drastically.
Only if we limit guest vcpus on same socket. The default host schedules
vcpus across sockets.
> 2. If guest ends up running on the same socket, you cause
> more IPIs which cause exists for the other guest.
I used different approaches to schedule vhost thread: 1. check loadavg
on a particular cpu; 2. randomly pick up a cpu, the performance didn't
make much difference in a small amount of VMs.
On Tom's 1-24 VMs scalability test, it had impressive results when
amount of VMs are increased compared to existing approach. So it might
not be a big issue.
> > > >
> > > > The cpu hotplug support hasn't in place yet. I will post it
> later.
>
> Not yet done, right?
Done now, under testing.
> > > Another question is why not just using workqueue? It has full
> support
> > > for cpu hotplug and allow more polices.
> >
> > Yes, it's good to use workqueue. I just did everything on top of
> current
> > implementation so it's easy to compare/analyze the performance data.
> >
> > I remembered the vhost implementation changed from workqueue to
> thread
> > for some reason. I couldn't recall the reason.
>
> At the time the implementation didn't perform well with per-cpu
> threads. We wanted a single thread so switched to use just that.
>
> > > >
> > > > Since we have per cpu vhost thread, each vhost thread will
> handle
> > > > multiple vqs, so we will be able to reduce/remove vq
> notification
> > > when
> > > > the work is heavy loaded in future.
> > >
> > > Does this issue still exist if event index is used? If vhost does
> not
> > > publish new used index, guest would not kick again.
> >
> > Since the vhost model has been changed to handle multiple VMs' vqs
> work,
> > then it's not necessary to enable these VMs' vqs notification
> (published
> > new used idex) where these vqs' future work will be processed on the
> > same per-cpu vhost thread, as long as the per-cpu vhost thread is
> still
> > running.
> >
> > > >
> > > > Here is my test results for remote host to guest test: tcp_rrs,
> > > udp_rrs,
> > > > tcp_stream with guest has 2 vpus, host has two cpu socket, each
> > > socket
> > > > has 4 cores.
> > > >
> > > > TCP_STREAM 256 512 1K 2K 4K 8K
> 16K
> > > >
> --------------------------------------------------------------------
> > > > Original
> > > >
> > > H->Guest 2501 4238 4744 5256 7203 6975 5799
> Patch
> > > >
> > > H->Guest 1676 2290 3149 8026 8439 8283
> 8216
> > > >
> > > > Original
> > > >
> > > Guest->H 744 1773 5675 1397 8207 7296
> 8117
> > > > Patch
> > > > Guest->Host 1041 1386 5407 7057 8298 8127
> 8241
> > >
> > > Looks like there's some noise in the result, the throughput of
> > > "original
> > > guest -> Host 2K" looks too low. And some strange is that I see
> > > regressions of packet transmission of guest when testing this
> patch.
> > > (
> > > Guest to Local Host TCP_STREAM in a NUMA machine).
> >
> > Yes, since I didn't mask the vcpus on the same socket, it might come
> > from packets out of order. I will rerun the test w/i masking vcpus
> on
> > the same socket to see any difference.
>
> Did anything interesting turn up?
Haven't had time to focus on single stream result yet.
>
> > You can reference Tom's results. His test is more formal than mine.
> >
> > > >
> > > > 60 instances TCP_RRs: Patch 150K trans/s vs. 91K trans/sec
> > > > 65% improved with taskset vcpus on the same socket
> > > > 60 instances UDP_RRs: Patch 172K trans/s vs. 103K trans/s
> > > > 67% improved with taskset vcpus on the same socket
> > > >
> > > > Tom has run 1VM to 24 VMs test for different work. He will post
> it
> > > here
> > > > soon.
> > > >
> > > > If the host scheduler ensures that the VM's vcpus are not
> scheduled
> > > to
> > > > another socket (i.e. cpu mask the vcpus on same socket) then the
> > > > performance will be better.
> > > >
> > > > Signed-off-by: Shirley Ma<xma@us.ibm.com>
> > > > Signed-off-by: Krishna Kumar<krkumar2@in.ibm.com>
> > > > Tested-by: Tom Lendacky<toml@us.ibm.com>
> > > > ---
> > > >
> > > > drivers/vhost/net.c | 26 ++-
> > > > drivers/vhost/vhost.c | 289
> > > > +++++++++++++++++++++++----------
> > > > drivers/vhost/vhost.h | 16 ++-
> > > > 3 files changed, 232 insertions(+), 103 deletions(-)
> > > >
> > > > Thanks
> > > > Shirley
> > > >
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe
> netdev" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> > >
> > >
>
> Also a question: how does this interact with zero copy tx?
Yes, I tested this with zero copy tx. The vhost thread which handles tx
work has been significantly reduced.
Thanks
Shirley
^ permalink raw reply
* Re: kernel BUG at include/net/netns/generic.h:41
From: Stanislav Kinsbursky @ 2012-04-05 16:08 UTC (permalink / raw)
To: Myklebust, Trond, Dave Jones; +Cc: Linux Kernel, netdev@vger.kernel.org
In-Reply-To: <1333390230.13718.34.camel@lade.trondhjem.org>
02.04.2012 22:10, Myklebust, Trond пишет:
> This looks related to the recent net namespace changes, so I'm adding
> Stanislav.
>
Hi, Trond.
Looks like we stepped into exactly that situation, what Al Viro was talking on
LFS about early registering of file system before other module "init" operations
completed.
I.e. SUNRPC registers it's file system, and then we have races between
register_pernet_subsys() call and mount of SUNRPC PipeFS.
BTW, these races can be triggered only if mount of SUNRPC PipeFS is called prior
to SUNRPC module insertion is completed. In spite of it, I'll send the patch soon.
But I'm not 100% sure, that this patch will solve Dave's problem.
So, there are few questions to you, Dave:
1) can this happen (PipeFS mount attempt before SUNPRC module insertion has
returned any value) in your case?
2) how often do you experience this problem? Not every boot, I suppose?
3) And is it easy for you to reproduce?
--
Best regards,
Stanislav Kinsbursky
^ permalink raw reply
* Re: kernel BUG at include/net/netns/generic.h:41
From: Dave Jones @ 2012-04-05 16:12 UTC (permalink / raw)
To: Stanislav Kinsbursky
Cc: Myklebust, Trond, Linux Kernel, netdev@vger.kernel.org
In-Reply-To: <4F7DC37F.7090404@parallels.com>
On Thu, Apr 05, 2012 at 08:08:31PM +0400, Stanislav Kinsbursky wrote:
> So, there are few questions to you, Dave:
> 1) can this happen (PipeFS mount attempt before SUNPRC module insertion has
> returned any value) in your case?
I don't know enough of what systemd is doing to answer this.
> 2) how often do you experience this problem? Not every boot, I suppose?
> 3) And is it easy for you to reproduce?
It happens most the time, but not every time.
Dave
^ permalink raw reply
* Re: kernel BUG at include/net/netns/generic.h:41
From: Stanislav Kinsbursky @ 2012-04-05 16:14 UTC (permalink / raw)
To: Dave Jones, Myklebust, Trond, Linux Kernel,
netdev@vger.kernel.org
In-Reply-To: <20120405161241.GA23658@redhat.com>
05.04.2012 20:12, Dave Jones пишет:
> On Thu, Apr 05, 2012 at 08:08:31PM +0400, Stanislav Kinsbursky wrote:
>
> > So, there are few questions to you, Dave:
> > 1) can this happen (PipeFS mount attempt before SUNPRC module insertion has
> > returned any value) in your case?
>
> I don't know enough of what systemd is doing to answer this.
>
> > 2) how often do you experience this problem? Not every boot, I suppose?
> > 3) And is it easy for you to reproduce?
>
> It happens most the time, but not every time.
>
Ok, I'll send the patch soon.
Hope, it will help.
--
Best regards,
Stanislav Kinsbursky
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox