From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933342Ab1JYMeW (ORCPT ); Tue, 25 Oct 2011 08:34:22 -0400 Received: from fgwmail5.fujitsu.co.jp ([192.51.44.35]:33342 "EHLO fgwmail5.fujitsu.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933008Ab1JYMeT (ORCPT ); Tue, 25 Oct 2011 08:34:19 -0400 X-SecurityPolicyCheck-FJ: OK by FujitsuOutboundMailChecker v1.3.1 Message-ID: <4EA6ACBA.20200@jp.fujitsu.com> Date: Tue, 25 Oct 2011 21:34:02 +0900 From: Kenji Kaneshige User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Yinghai Lu CC: Bjorn Helgaas , Jesse Barnes , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH 1/8] pci: Make sriov work with hotplug removal References: <4E9A3092.4080309@oracle.com> <4E9A3408.9080905@oracle.com> <4E9C6F0E.40501@oracle.com> <4E9CAB21.2020302@oracle.com> <4E9DB11A.7030505@oracle.com> In-Reply-To: <4E9DB11A.7030505@oracle.com> Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org (2011/10/19 2:02), Yinghai Lu wrote: > On 10/18/2011 09:49 AM, Bjorn Helgaas wrote: >> On Mon, Oct 17, 2011 at 4:24 PM, Yinghai Lu wrote: >>> On 10/17/2011 03:12 PM, Bjorn Helgaas wrote: >>>> >>>> Maybe this is the best we can do, but it still doesn't seem ideal, and >>>> it's certainly not obvious when reading the code. It doesn't seem >>>> right for the driver ->remove() method to be calling >>>> pci_destroy_dev(). Won't the core data structures be corrupted if a >>>> defective driver doesn't call pci_disable_sriov()? Seems like we >>>> could end up with a device that's been physically removed, but still >>>> has pci_dev structs hanging around. >>> >>> i did add some print out in >>> pci_stop_bus_device >>> when stop PF, that function is called for those VFs. >>> >>> also driver have to call pci_disable_sriov() and that is current design. >> >> Yep. But I don't have to like the current design :) It doesn't seem >> as robust as it could be. >> >> It took me a long time to puzzle out what was happening here. Here's >> some possible changelog text that would have saved me a lot of time: >> >> The PCI hot-remove path calls pci_stop_bus_devices() via >> pci_remove_bus_device(). >> >> pci_stop_bus_devices() traverses the bus->devices list (point A below), >> stopping each device in turn, which calls the driver remove() method. When >> the device is an SR-IOV PF, the driver calls pci_disable_sriov(), which >> also uses pci_remove_bus_device() to remove the VF devices from the >> bus->devices list (point B). >> >> pci_remove_bus_device >> pci_stop_bus_device >> pci_stop_bus_devices(subordinate) >> list_for_each(bus->devices)<-- A >> pci_stop_bus_device(PF) >> ... >> driver->remove >> pci_disable_sriov >> ... >> pci_remove_bus_device(VF) >> <-- B >> >> At B, we're changing the same list we're iterating through at A, so when >> the driver remove() method returns, the pci_stop_bus_devices() iterator has >> a pointer to a list entry that has already been freed. >> >> This patch avoids the problem by building a separate list of all PFs on >> the bus and traversing that at A instead of the bus->devices list. > > yes. I have one question. I think pci_stop_bus_devices() is called only when bridge device is removed. Are you trying to hot-remove the bridge device that has SR-IOV capable devices on the subordinate bus? Regards, Kenji Kaneshige