From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
To: Can Net <netcan1996@gmail.com>,
dvorax.fuxbrumer@linux.intel.com, anthony.l.nguyen@intel.com
Cc: intel-wired-lan@lists.osuosl.org
Subject: Re: [Intel-wired-lan] use-after-free during reset bug report for igc driver at linux 6.1.0-6
Date: Mon, 20 Mar 2023 17:56:03 -0700 [thread overview]
Message-ID: <87355yq5wc.fsf@intel.com> (raw)
In-Reply-To: <CAHN_zNRo0y+NRw_6B2tGv8EnS5KnqnzsPYNs3wF+Pk1rS3hSNQ@mail.gmail.com>
Hi,
+CC intel-wired-lan
Can Net <netcan1996@gmail.com> writes:
> Hello developers!
> I find a use-after-free during reset bug at Linux 6.1.0-6(Debian testing)
> My i225-v NUC connected to an ac66u router 1Gbps port, sometimes it caused
> that router reboot, and later i225-v igc driver down caused kernel to hang.
> I searched the kernel patch that repaired it(
> https://github.com/torvalds/linux/commit/56ea7ed103b46970e171eb1c95916f393d64eeff),
> but it reappear.
>
> thank you for your time.
>
> here is my kernel log:
> ```
[...]
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: NIC Link is Down
> Mar 18 12:38:38 intel-n100 kernel: virt-br0: port 1(enp2s0) entered
> disabled state
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: Register Dump
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: Register Name
> Value
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: CTRL
> 181c0641
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: STATUS
> 40280691
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: CTRL_EXT
> 100000c0
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: MDIC
> 18017949
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: ICR
> 00000000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RCTL
> 0440803a
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RDLEN[0-3]
> 00001000 00001000 00001000 00001000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RDH[0-3]
> 00000014 00000018 000000c5 000000fe
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RDT[0-3]
> 00000013 00000017 000000c4 000000fd
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RXDCTL[0-3]
> 02040808 02040808 02040808 02040808
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RDBAL[0-3]
> ffffb000 ffffa000 ffff9000 ffff8000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: RDBAH[0-3]
> 00000000 00000000 00000000 00000000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TCTL
> a503f0fa
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TDBAL[0-3]
> fffff000 ffffe000 ffffd000 ffffc000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TDBAH[0-3]
> 00000000 00000000 00000000 00000000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TDLEN[0-3]
> 00001000 00001000 00001000 00001000
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TDH[0-3]
> 00000017 0000005d 00000019 00000028
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TDT[0-3]
> 00000036 0000006f 00000029 0000003f
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: TXDCTL[0-3]
> 02100108 02100108 02100108 02100108
> Mar 18 12:38:38 intel-n100 kernel: igc 0000:02:00.0 enp2s0: Reset adapter
> Mar 18 12:38:41 intel-n100 kernel: igc 0000:02:00.0 enp2s0: NIC Link is Up
> 1000 Mbps Full Duplex, Flow Control: RX/TX
> Mar 18 12:38:41 intel-n100 kernel: virt-br0: port 1(enp2s0) entered
> blocking state
> Mar 18 12:38:41 intel-n100 kernel: virt-br0: port 1(enp2s0) entered
> forwarding state
So, enp2s0 is part of a bridge, wonder if it there's anything to do with
the crash.
Did you manage to get a more or less consistent way to reproduce this?
> Mar 18 12:38:45 intel-n100 kernel: ------------[ cut here ]------------
> Mar 18 12:38:45 intel-n100 kernel: refcount_t: underflow; use-after-free.
> Mar 18 12:38:45 intel-n100 kernel: WARNING: CPU: 3 PID: 7219 at
> lib/refcount.c:28 refcount_warn_saturate+0xba/0x110
Interesting that it happened 4 seconds after the link was up.
> Mar 18 12:38:45 intel-n100 kernel: Modules linked in: veth xt_nat xt_tcpudp
> xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_algo
> xt_addrtype nft_compat br_netfilter rfcomm snd_seq_dummy snd_hrtimer
> snd_seq wireguard libchacha20poly1305 chacha_x86_64 poly1305_x86_64
> curve25519_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel
> vhost_net vhost vhost_iotlb tap tun qrtr overlay cmac algif_hash
> algif_skcipher af_alg bnep binfmt_misc nls_ascii nls_cp437 vfat fat ext4
> intel_rapl_msr mbcache intel_rapl_common jbd2 x86_pkg_temp_thermal
> intel_powerclamp coretemp kvm_intel snd_sof_pci_intel_tgl
> snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation kvm
> soundwire_cadence iwlmvm snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp
> irqbypass snd_sof snd_sof_utils mac80211 snd_soc_hdac_hda
> ghash_clmulni_intel snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi
> snd_hda_codec_hdmi snd_soc_core btusb snd_compress btrtl soundwire_bus
> libarc4 btbcm btintel btmtk snd_hda_intel
> Mar 18 12:38:45 intel-n100 kernel: snd_intel_dspcfg bluetooth mei_hdcp
> snd_intel_sdw_acpi snd_hda_codec pmt_telemetry snd_usb_audio pmt_class
> aesni_intel crypto_simd snd_hda_core cryptd jitterentropy_rng rapl
> snd_usbmidi_lib iwlwifi bridge snd_hwdep intel_cstate snd_rawmidi
> snd_seq_device stp llc mc sha512_ssse3 intel_uncore pcspkr wmi_bmof snd_pcm
> sha512_generic ctr iTCO_wdt snd_timer intel_pmc_bxt iTCO_vendor_support
> drbg cfg80211 ansi_cprng ee1004 snd ecdh_generic watchdog ecc mei_me
> soundcore crc16 mei rfkill intel_vsec nft_chain_nat nf_nat intel_pmc_core
> intel_hid acpi_pad sparse_keymap acpi_tad sg joydev nft_ct nf_conntrack
> evdev nf_defrag_ipv6 nf_defrag_ipv4 nf_tables libcrc32c crc32c_generic msr
> parport_pc nfnetlink ppdev lp parport dm_mod loop fuse efi_pstore configfs
> efivarfs ip_tables x_tables autofs4 hid_generic usbhid hid zfs(POE)
> zunicode(POE) zzstd(OE) zlua(OE) zavl(POE) icp(POE) zcommon(POE)
> znvpair(POE) spl(OE) sd_mod uas usb_storage i915 nvme ahci sdhci_pci
> libahci cqhci sdhci libata
> Mar 18 12:38:45 intel-n100 kernel: nvme_core t10_pi drm_buddy i2c_algo_bit
> drm_display_helper igc cec crc64_rocksoft crc64 crc_t10dif rc_core xhci_pci
> crct10dif_generic crct10dif_pclmul crct10dif_common scsi_mod ttm xhci_hcd
> drm_kms_helper ptp mmc_core crc32_pclmul usbcore pps_core drm i2c_i801
> i2c_smbus crc32c_intel usb_common scsi_common fan video wmi
> pinctrl_alderlake button
> Mar 18 12:38:45 intel-n100 kernel: CPU: 3 PID: 7219 Comm: transmission-da
> Tainted: P OE 6.1.0-6-amd64 #1 Debian 6.1.15-1
> Mar 18 12:38:45 intel-n100 kernel: Hardware name: /, BIOS 5.27 12/21/2022
> Mar 18 12:38:45 intel-n100 kernel: RIP:
> 0010:refcount_warn_saturate+0xba/0x110
> Mar 18 12:38:45 intel-n100 kernel: Code: 01 01 e8 df 5e 4a 00 0f 0b c3 cc
> cc cc cc 80 3d 38 86 ad 01 00 75 85 48 c7 c7 50 aa d4 b7 c6 05 28 86 ad 01
> 01 e8 bc 5e 4a 00 <0f> 0b c3 cc cc cc cc 80 3d 13 86 ad 01 00 0f 85 5e ff
> ff ff 48 c7
> Mar 18 12:38:45 intel-n100 kernel: RSP: 0000:ffffac66e0783cb8 EFLAGS:
> 00010282
> Mar 18 12:38:45 intel-n100 kernel: RAX: 0000000000000000 RBX:
> ffff94d25681a600 RCX: 0000000000000000
> Mar 18 12:38:45 intel-n100 kernel: RDX: 0000000000000101 RSI:
> ffffffffb7d332d6 RDI: 00000000ffffffff
> Mar 18 12:38:45 intel-n100 kernel: RBP: ffffac66c0f7d1b0 R08:
> 0000000000000000 R09: ffffac66e0783b40
> Mar 18 12:38:45 intel-n100 kernel: R10: 0000000000000003 R11:
> ffffffffb88d43a8 R12: ffffac66c120e578
> Mar 18 12:38:45 intel-n100 kernel: R13: 0000000000000233 R14:
> ffff94d2529e0bc0 R15: 00000000ffffff19
> Mar 18 12:38:45 intel-n100 kernel: FS: 00007fc48bd876c0(0000)
> GS:ffff94d99fb80000(0000) knlGS:0000000000000000
> Mar 18 12:38:45 intel-n100 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
> 0000000080050033
> Mar 18 12:38:45 intel-n100 kernel: CR2: 000055b0939bc074 CR3:
> 00000001300e0001 CR4: 0000000000772ee0
> Mar 18 12:38:45 intel-n100 kernel: PKRU: 55555554
> Mar 18 12:38:45 intel-n100 kernel: Call Trace:
> Mar 18 12:38:45 intel-n100 kernel: <TASK>
> Mar 18 12:38:45 intel-n100 kernel: tcp_wfree+0x15b/0x180
> Mar 18 12:38:45 intel-n100 kernel: skb_release_head_state+0x20/0x90
> Mar 18 12:38:45 intel-n100 kernel: napi_consume_skb+0xc2/0xf0
> Mar 18 12:38:45 intel-n100 kernel: igc_poll+0x7f5/0x16d0 [igc]
> Mar 18 12:38:45 intel-n100 kernel: __napi_poll+0x28/0x160
> Mar 18 12:38:45 intel-n100 kernel: net_rx_action+0x2a5/0x360
> Mar 18 12:38:45 intel-n100 kernel: ? __raise_softirq_irqoff+0x16/0x70
> Mar 18 12:38:46 intel-n100 kernel: __do_softirq+0xed/0x2fe
> Mar 18 12:38:46 intel-n100 kernel: ? handle_edge_irq+0x9f/0x240
> Mar 18 12:38:46 intel-n100 kernel: __irq_exit_rcu+0xc7/0x130
> Mar 18 12:38:46 intel-n100 kernel: common_interrupt+0x61/0xd0
> Mar 18 12:38:46 intel-n100 kernel: asm_common_interrupt+0x22/0x40
> Mar 18 12:38:46 intel-n100 kernel: RIP: 0033:0x7fc48e1e5676
> Mar 18 12:38:46 intel-n100 kernel: Code: 48 8b 74 24 10 48 b8 cf f7 53 e3
> a5 9b c4 20 48 f7 e9 48 c1 f9 3f 48 c1 fa 07 29 ca 48 8b 44 24 28 64 48 2b
> 04 25 28 00 00 00 <75> 1c 48 89 f0 48 83 c4 38 c3 31 f6 48 89 e7 e8 86 a3
> fa ff 48 8b
> Mar 18 12:38:46 intel-n100 kernel: RSP: 002b:00007fc48bd866a0 EFLAGS:
> 00000246
> Mar 18 12:38:46 intel-n100 kernel: RAX: 0000000000000000 RBX:
> 00007fc4780c4bd0 RCX: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: RDX: 00000000000c6dd5 RSI:
> 0000000000000096 RDI: 00000000013d7a45
> Mar 18 12:38:46 intel-n100 kernel: RBP: 00007fc478c8dec0 R08:
> 0000000000000000 R09: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: R10: 00007ffe3a7ee080 R11:
> 000000000000ba3e R12: 00007fc48e2088e0
> Mar 18 12:38:46 intel-n100 kernel: R13: 00007fc478c8e9d0 R14:
> 0000000000000000 R15: 00007fc478c8dec0
> Mar 18 12:38:46 intel-n100 kernel: </TASK>
> Mar 18 12:38:46 intel-n100 kernel: ---[ end trace 0000000000000000 ]---
> Mar 18 12:38:46 intel-n100 kernel: ------------[ cut here ]------------
> Mar 18 12:38:46 intel-n100 kernel: WARNING: CPU: 3 PID: 7219 at
> drivers/iommu/dma-iommu.c:1038 iommu_dma_unmap_page+0x79/0x90
> Mar 18 12:38:46 intel-n100 kernel: Modules linked in: veth xt_nat xt_tcpudp
> xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_algo
> xt_addrtype nft_compat br_netfilter rfcomm snd_seq_dummy snd_hrtimer
> snd_seq wireguard libchacha20poly1305 chacha_x86_64 poly1305_x86_64
> curve25519_x86_64 libcurve25519_generic libchacha ip6_udp_tunnel udp_tunnel
> vhost_net vhost vhost_iotlb tap tun qrtr overlay cmac algif_hash
> algif_skcipher af_alg bnep binfmt_misc nls_ascii nls_cp437 vfat fat ext4
> intel_rapl_msr mbcache intel_rapl_common jbd2 x86_pkg_temp_thermal
> intel_powerclamp coretemp kvm_intel snd_sof_pci_intel_tgl
> snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation kvm
> soundwire_cadence iwlmvm snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp
> irqbypass snd_sof snd_sof_utils mac80211 snd_soc_hdac_hda
> ghash_clmulni_intel snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi
> snd_hda_codec_hdmi snd_soc_core btusb snd_compress btrtl soundwire_bus
> libarc4 btbcm btintel btmtk snd_hda_intel
> Mar 18 12:38:46 intel-n100 kernel: snd_intel_dspcfg bluetooth mei_hdcp
> snd_intel_sdw_acpi snd_hda_codec pmt_telemetry snd_usb_audio pmt_class
> aesni_intel crypto_simd snd_hda_core cryptd jitterentropy_rng rapl
> snd_usbmidi_lib iwlwifi bridge snd_hwdep intel_cstate snd_rawmidi
> snd_seq_device stp llc mc sha512_ssse3 intel_uncore pcspkr wmi_bmof snd_pcm
> sha512_generic ctr iTCO_wdt snd_timer intel_pmc_bxt iTCO_vendor_support
> drbg cfg80211 ansi_cprng ee1004 snd ecdh_generic watchdog ecc mei_me
> soundcore crc16 mei rfkill intel_vsec nft_chain_nat nf_nat intel_pmc_core
> intel_hid acpi_pad sparse_keymap acpi_tad sg joydev nft_ct nf_conntrack
> evdev nf_defrag_ipv6 nf_defrag_ipv4 nf_tables libcrc32c crc32c_generic msr
> parport_pc nfnetlink ppdev lp parport dm_mod loop fuse efi_pstore configfs
> efivarfs ip_tables x_tables autofs4 hid_generic usbhid hid zfs(POE)
> zunicode(POE) zzstd(OE) zlua(OE) zavl(POE) icp(POE) zcommon(POE)
> znvpair(POE) spl(OE) sd_mod uas usb_storage i915 nvme ahci sdhci_pci
> libahci cqhci sdhci libata
> Mar 18 12:38:46 intel-n100 kernel: nvme_core t10_pi drm_buddy i2c_algo_bit
> drm_display_helper igc cec crc64_rocksoft crc64 crc_t10dif rc_core xhci_pci
> crct10dif_generic crct10dif_pclmul crct10dif_common scsi_mod ttm xhci_hcd
> drm_kms_helper ptp mmc_core crc32_pclmul usbcore pps_core drm i2c_i801
> i2c_smbus crc32c_intel usb_common scsi_common fan video wmi
> pinctrl_alderlake button
> Mar 18 12:38:46 intel-n100 kernel: CPU: 3 PID: 7219 Comm: transmission-da
> Tainted: P W OE 6.1.0-6-amd64 #1 Debian 6.1.15-1
> Mar 18 12:38:46 intel-n100 kernel: Hardware name: /, BIOS 5.27 12/21/2022
> Mar 18 12:38:46 intel-n100 kernel: RIP: 0010:iommu_dma_unmap_page+0x79/0x90
> Mar 18 12:38:46 intel-n100 kernel: Code: 2b 48 3b 28 72 26 48 3b 68 08 73
> 20 4d 89 f8 44 89 f1 4c 89 ea 48 89 ee 48 89 df 5b 5d 41 5c 41 5d 41 5e 41
> 5f e9 47 69 a7 ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 cc cc cc cc 66
> 0f 1f 44 00
> Mar 18 12:38:46 intel-n100 kernel: RSP: 0000:ffffac66e0783ce8 EFLAGS:
> 00010246
> Mar 18 12:38:46 intel-n100 kernel: RAX: 0000000000000000 RBX:
> ffff94d24263e0d0 RCX: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: RDX: 0000000000000000 RSI:
> 0000000000000000 RDI: 0000000000000001
> Mar 18 12:38:46 intel-n100 kernel: RBP: ffffac66c0f7d1b0 R08:
> 0000000000000002 R09: 0000000080100007
> Mar 18 12:38:46 intel-n100 kernel: R10: 0000000000000003 R11:
> ffffffffb88d43a8 R12: 00000000ffbce0ea
> Mar 18 12:38:46 intel-n100 kernel: R13: 0000000000000000 R14:
> 0000000000000001 R15: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: FS: 00007fc48bd876c0(0000)
> GS:ffff94d99fb80000(0000) knlGS:0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
> 0000000080050033
> Mar 18 12:38:46 intel-n100 kernel: CR2: 000055b0939bc074 CR3:
> 00000001300e0001 CR4: 0000000000772ee0
> Mar 18 12:38:46 intel-n100 kernel: PKRU: 55555554
> Mar 18 12:38:46 intel-n100 kernel: Call Trace:
> Mar 18 12:38:46 intel-n100 kernel: <TASK>
> Mar 18 12:38:46 intel-n100 kernel: igc_poll+0x14d/0x16d0 [igc]
> Mar 18 12:38:46 intel-n100 kernel: __napi_poll+0x28/0x160
> Mar 18 12:38:46 intel-n100 kernel: net_rx_action+0x2a5/0x360
> Mar 18 12:38:46 intel-n100 kernel: ? __raise_softirq_irqoff+0x16/0x70
> Mar 18 12:38:46 intel-n100 kernel: __do_softirq+0xed/0x2fe
> Mar 18 12:38:46 intel-n100 kernel: ? handle_edge_irq+0x9f/0x240
> Mar 18 12:38:46 intel-n100 kernel: __irq_exit_rcu+0xc7/0x130
> Mar 18 12:38:46 intel-n100 kernel: common_interrupt+0x61/0xd0
> Mar 18 12:38:46 intel-n100 kernel: asm_common_interrupt+0x22/0x40
> Mar 18 12:38:46 intel-n100 kernel: RIP: 0033:0x7fc48e1e5676
> Mar 18 12:38:46 intel-n100 kernel: Code: 48 8b 74 24 10 48 b8 cf f7 53 e3
> a5 9b c4 20 48 f7 e9 48 c1 f9 3f 48 c1 fa 07 29 ca 48 8b 44 24 28 64 48 2b
> 04 25 28 00 00 00 <75> 1c 48 89 f0 48 83 c4 38 c3 31 f6 48 89 e7 e8 86 a3
> fa ff 48 8b
> Mar 18 12:38:46 intel-n100 kernel: RSP: 002b:00007fc48bd866a0 EFLAGS:
> 00000246
> Mar 18 12:38:46 intel-n100 kernel: RAX: 0000000000000000 RBX:
> 00007fc4780c4bd0 RCX: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: RDX: 00000000000c6dd5 RSI:
> 0000000000000096 RDI: 00000000013d7a45
> Mar 18 12:38:46 intel-n100 kernel: RBP: 00007fc478c8dec0 R08:
> 0000000000000000 R09: 0000000000000000
> Mar 18 12:38:46 intel-n100 kernel: R10: 00007ffe3a7ee080 R11:
> 000000000000ba3e R12: 00007fc48e2088e0
> Mar 18 12:38:46 intel-n100 kernel: R13: 00007fc478c8e9d0 R14:
> 0000000000000000 R15: 00007fc478c8dec0
> Mar 18 12:38:46 intel-n100 kernel: </TASK>
> Mar 18 12:38:46 intel-n100 kernel: ---[ end trace 0000000000000000 ]---
> Mar 18 12:38:46 intel-n100 kernel: ------------[ cut here ]------------
> Mar 18 12:38:46 intel-n100 kernel: kernel BUG at
> lib/dynamic_queue_limits.c:27!
> ```
>
The crash log from the router side was not really legible, perhaps
something in the formatting was lost? But I don't think it's that
useful, even if I can see it also crashing. Or is your router also using
i225 (and I missed that)?
Cheers,
--
Vinicius
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
next parent reply other threads:[~2023-03-21 0:56 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAHN_zNRo0y+NRw_6B2tGv8EnS5KnqnzsPYNs3wF+Pk1rS3hSNQ@mail.gmail.com>
2023-03-21 0:56 ` Vinicius Costa Gomes [this message]
2023-03-21 14:23 ` [Intel-wired-lan] use-after-free during reset bug report for igc driver at linux 6.1.0-6 Can Net
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87355yq5wc.fsf@intel.com \
--to=vinicius.gomes@intel.com \
--cc=anthony.l.nguyen@intel.com \
--cc=dvorax.fuxbrumer@linux.intel.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=netcan1996@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox