* Fwd: How to determine which device crashes udev in boot?
From: iwillallways forget1 @ 2016-03-15 1:39 UTC (permalink / raw)
To: linux-hotplug
Greg KH, thank you for replying to me, since your bot on your
documented e-mail address kept deleting my message and kept telling me
to resend if I really intended to mail you, whereupon it automatically
deleted the resends, etc.
Does anyone know why my real e-mail address n0diamond attract yahoo
dotty co dotty jp iw disliked by vger.kernel.org? What spam does it
accuse me of spamming?
Greg KH wrote:
> I (Norman Diamond) wrote:
>> I'm trying to narrow which device in an NEC VF-6 causes Linux boots to
>> hang, permanently, in udevadm trigger and udevadm settle. Many Google
>> searches found many pages but no tutorials on how to debug udev this
>> way.
>> I downloaded fresh copies of Porteus 3.1 32-bit and 64-bit.
>
> What kernel versions are these?
3.17.4. I had the same problem with kernel 3.4.something and probably
some older ones, but that's probabky beside the point because the
question of how to track down the device is a udev question.
>> Porteus 3.1 64-bit boots fine, udev finds everything, x Windows works
>> (I told you Windows rulez)
(Oh yeah, I created a throwaway account on outlook.com but
vger.kernel.org accused my brand new account of being a spammer too,
and I forgot to delete that part when trying to send from gmail.com)
>> etc. Even on this NEC VF-6.
>>
>> Porteus 3.1 32-bit boots fine on everything except this NEC VF-6, x
>> Windows works, etc.
>>
>> Porteus 3.1 32-bit hangs when starting udev on this NEC VF-6.
>>
>> Booting with "3 nohotplug" works (3 tells Porteus what run level to
>> use, and nohotplug is observed by both the kernel and Porteus). In
>> text mode I can do some amount of experiments. I don't know how to do
>> meaningful experiments, to try to track down which device causes the
>> hang.
>
> Try looking at the documentation of udevadm, you can manually enable the
> 'coldplug' options there to narrow down what hardware is causing the
> problem.
That's exactly what I want to do. Several Google searches today were
as futile as dozens of Google searches during the past week. As far
as I can tell, udevadm trigger --actiond (the same as in rc.udev)
adds everything, and there is some way to exclude an individual device
but I haven't found how to specify a device correctly to get it
excluded.
--max-childs=1 and stuff like that have not helped.
> Odds are, you have a kernel driver that doesn't like the
> hardware and it is getting loaded automatically by udev.
Yes, that's why I'm asking how to track down which device it is.
If the problematic device is something not needed for my employer's
product, I can try to blacklist the device if I can figure out or get
help to do it. But first I have to find out which device it is.
Some devices obviously have drivers running already before udevd and
udevadm trigger get started. But for example if I type startx then
the video driver gets loaded dynamically and the keyboard and mouse
die dynamically, so I have to press the power switch. So I think the
kernel hang comes from some other driver getting loaded dynamically
after udevd starts and udevadm triggers everything, but I can't find
what device it is.
Some device has a working driver for Porteus's 64-bit kernel but a
crashing driver for Porteus's 32-bit kernel, and the crash is so hard
that it doesn't even start flashing two keyboard lights. Also this
being text mode (when I type udevadm not startx), the crash is so hard
that it doesn't print an oops.
greg k-h
^ permalink raw reply
* Re: Fwd: How to determine which device crashes udev in boot?
From: Greg KH @ 2016-03-15 3:16 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <CAD4SBxBhtLmMqiUmCFZ7i8q1aQ2d6Hrse=k_BBz_2VGcJQSmcg@mail.gmail.com>
On Tue, Mar 15, 2016 at 10:39:39AM +0900, iwillallways forget1 wrote:
> Some device has a working driver for Porteus's 64-bit kernel but a
> crashing driver for Porteus's 32-bit kernel, and the crash is so hard
> that it doesn't even start flashing two keyboard lights. Also this
> being text mode (when I type udevadm not startx), the crash is so hard
> that it doesn't print an oops.
Use a serial console, that way you can see the oops happen there.
good luck!
greg k-h
^ permalink raw reply
* Fwd: Fwd: How to determine which device crashes udev in boot?
From: iwillallways forget1 @ 2016-03-17 8:54 UTC (permalink / raw)
To: linux-hotplug
Greg K-H replied to me:
>> Some device has a working driver for Porteus's 64-bit kernel but a
>> crashing driver for Porteus's 32-bit kernel, and the crash is so hard
>> that it doesn't even start flashing two keyboard lights. Also this
>> being text mode (when I type udevadm not startx), the crash is so hard
>> that it doesn't print an oops.
>
> Use a serial console, that way you can see the oops happen there.
Perhaps I should mention that I had
tail -f /var/log/messages >/mnt/sdb1/what 2>&1 &
running in the background before running udevd and udevadm trigger.
The log included udev stuff until the crash, but did not include the oops.
I think I own two USB-to-serial adapters and a serial reverse cable.
I'll have to hunt for them and bring them to my employer.
Do you think the oops will get out over a USB-to-serial adapter
before the kernel crashes hard?
(Several years ago I saved virtual serial output to a real file under
VMware and diagnosed an oops, but the NEC VF-6 is a real notebook with
some device that a VMware virtual PC doesn't have.)
^ permalink raw reply
* Re: Fwd: Fwd: How to determine which device crashes udev in boot?
From: Greg KH @ 2016-03-17 16:56 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <CAD4SBxAoUczieAGCkjzMfYrPru4P+KJiVwA-TDkrdDS+osk_Lw@mail.gmail.com>
On Thu, Mar 17, 2016 at 05:54:10PM +0900, iwillallways forget1 wrote:
> Greg K-H replied to me:
> >> Some device has a working driver for Porteus's 64-bit kernel but a
> >> crashing driver for Porteus's 32-bit kernel, and the crash is so hard
> >> that it doesn't even start flashing two keyboard lights. Also this
> >> being text mode (when I type udevadm not startx), the crash is so hard
> >> that it doesn't print an oops.
> >
> > Use a serial console, that way you can see the oops happen there.
>
> Perhaps I should mention that I had
> tail -f /var/log/messages >/mnt/sdb1/what 2>&1 &
> running in the background before running udevd and udevadm trigger.
> The log included udev stuff until the crash, but did not include the oops.
Yes, that will not be saved as the kernel crashed.
> I think I own two USB-to-serial adapters and a serial reverse cable.
> I'll have to hunt for them and bring them to my employer.
> Do you think the oops will get out over a USB-to-serial adapter
> before the kernel crashes hard?
No, usb serial usually does not work for capturing oops messages because
the kernel crashes and USB needs interrupts to be running properly to
deliver data.
Try a "real" serial connection if the device has one.
As you know what kernel drivers are needed for this hardware (it works
in 64bit mode), just load them by hand, one by one, in 32bit mode and
see what one dies.
good luck,
greg k-h
^ permalink raw reply
* Fwd: Fwd: Fwd: How to determine which device crashes udev in boot?
From: iwillallways forget1 @ 2016-03-31 5:13 UTC (permalink / raw)
To: linux-hotplug
Greg K-H replied to me:
>>>> Some device has a working driver for Porteus's 64-bit kernel but a
>>>> crashing driver for Porteus's 32-bit kernel, and the crash is so hard
>>>> that it doesn't even start flashing two keyboard lights. Also this
>>>> being text mode (when I type udevadm not startx), the crash is so hard
>>>> that it doesn't print an oops.
[...]
> No, usb serial usually does not work for capturing oops messages because
> the kernel crashes and USB needs interrupts to be running properly to
> deliver data.
It's a notebook PC and doesn't have a real serial port.
It does have a real PCMCIA slot, which will be the subject of a new thread.
> As you know what kernel drivers are needed for this hardware (it works
> in 64bit mode), just load them by hand, one by one, in 32bit mode and
> see what one dies.
All modprobes succeeded. A new thread will discuss the problem.
I still never figured out how to make udev ignore a device. Of course
I see debugging options (which didn't help) and options to ignore
devices that have certain attributes or properties, but never got the
syntax right to ignore the PCMCIA slot or bridge. For the moment I no
longer need to know, but if there's a tutorial somewhere please say.
^ permalink raw reply
* /lib/udev/pcmcia-socket-startup makes 32-bit Linux crash hard
From: iwillallways forget1 @ 2016-03-31 5:30 UTC (permalink / raw)
To: linux-hotplug
Porteus 3.1 uses kernel 3.17.4.
64-bit Porteus 3.1 works fine on an NEC VF-6.
32-bit Porteus 3.1 hangs in udev during boot. It's not a 120-second
delay for udevadm settle, it's a hard crash which doesn't even display
an oops in text mode, doesn't even flash two keyboard lamps, and
doesn't even respond to the Caps Lock or Num Lock keys or
Ctrl+Alt+Delete. It needs a press of the power button.
32-bit Porteus 3.1, when booted with nohotplug, works until I try
running rc.udev start. Then the hang is the same as always.
Porteus is a live Linux distribution and has a rootcopy directory. I
copied file /lib/udev/rules.d/60-pcmcia.rules to the appropriate
directory in a Porteus USB stick, and edited it, so this would
override the built-in 60-pcmcia.rules file. I commented out these
lines:
SUBSYSTEM="pcmcia_socket", \
RUN+="/lib/udev/pcmcia-socket-startup"
So, /lib/udev/pcmcia-socket-startup works in 64-bit mode but makes
32-bit Linux die.
When those lines are commented out, the yenta drivers don't crash,
though I don't know if they'll work when a 32-bit Cardbus card or
16-bit PCMCIA card is actually inserted
I don't know yet if the problem is particular to this PC model, or
particular to the PCMCIA slot, or more widespread.
0a:01.0 CardBus bridge [0607]: Ricoh Co Ltd RL5c476 II [1180:0476] (rev b6)
Subsystem: NEC Corporation Device [1033:88ec]
Kernel driver in use: yenta_cardbus
Kernel modules: yenta_socket
^ permalink raw reply
* Fwd: /lib/udev/pcmcia-socket-startup makes 32-bit Linux crash hard
From: iwillallways forget1 @ 2016-04-04 3:21 UTC (permalink / raw)
To: linux-hotplug
Porteus 3.1 uses kernel 3.17.4.
64-bit Porteus 3.1 works fine on an NEC VF-6.
32-bit Porteus 3.1 hangs in udev during boot. It's not a 120-second
delay for udevadm settle, it's a hard crash which doesn't even display
an oops in text mode, doesn't even flash two keyboard lamps, and
doesn't even respond to the Caps Lock or Num Lock keys or
Ctrl+Alt+Delete. It needs a press of the power button.
It appears that the particular device is highly relevant:
0a:01.0 CardBus bridge [0607]: Ricoh Co Ltd RL5c476 II [1180:0476] (rev b6)
Subsystem: NEC Corporation Device [1033:88ec]
Kernel driver in use: yenta_cardbus
Kernel modules: yenta_socket
I tested two other PCs with pcmcia slots made by different manufacturers.
One was cardbus only, and had no problem.
One was cardbus + 16-bit pcmcia, and had no problem, not made by Ricoh.
So the particular Ricoh controller looks relevant.
In /lib/udev/rules.d/60-pcmcia.rules I commented out these two lines:
SUBSYSTEM="pcmcia_socket", \
RUN+="/lib/udev/pcmcia-socket-startup"
After commenting them out, Linux boots with no problem.
Sorry to repeat for the nth time, but 64-bit Linux doesn't have this probem
but 32-bit Linux's version of pcmcia-socket-startup kills the kernel hard
on this very same PC.
In a separate message I'll discuss two intermittent problems when a 16-bit
pcmcia card is actually used. But these won't affect many people because
no one, not even me, uses a 16-bit pcmcia card for anything other than testing.
^ permalink raw reply
* pcmcia oopses
From: iwillallways forget1 @ 2016-04-04 3:30 UTC (permalink / raw)
To: linux-hotplug
Here are two oopses from kernel 4.0.4, 32 bits, on a PC with a pcmcia
slot that can handle both cardbux and 16-bit pcmcia cards.
As mentioned in my other thread, I commented out udev's call to
pcmcia-socket-startup so booting completes.
This first one occurs intermittently during boot before udev starts.
It is non-fatal and I can even use a pcmcia device.
[ 7.900418] ------------[ cut here ]------------
[ 7.906355] WARNING: CPU: 0 PID: 979 at kernel/sched/core.c:7302
__might_sleep+0x66/0x82()
[ 7.912393] do not call blocking ops when !TASK_RUNNING; state=1
set at [<c13de002>] pccardd+0xce/0x313
[ 7.918479] usb usb3: SerialNumber: 0000:00:1a.0
[ 7.924591] Modules linked in:
[ 7.930622] hub 3-0:1.0: USB hub found
[ 7.936383] CPU: 0 PID: 979 Comm: pccardd Not tainted 4.0.4-xxxxxxxxxxxx #1
[ 7.942209] Hardware name: NEC PC-VY20MFD5RJX6/AK3ML, BIOS NOTE
BIOS Version /554A0800 10/31/2008
[ 7.948217] 00000000 00000000 f4167ea4 c152c76a f4167ee4 f4167ed4
c103598d c17ab6bc
[ 7.954333] f4167f00 000003d3 c17ab587 00001c86 c104e13e c104e13e
c13de002 00000001
[ 7.960462] 00000000 f4167eec c10359d2 00000009 f4167ee4 c17ab6bc
f4167f00 f4167f20
[ 7.966596] Call Trace:
[ 7.972620] [<c152c76a>] dump_stack+0x49/0x73
[ 7.978687] [<c103598d>] warn_slowpath_common+0x98/0xaf
[ 7.984760] [<c104e13e>] ? __might_sleep+0x66/0x82
[ 7.990843] [<c104e13e>] ? __might_sleep+0x66/0x82
[ 7.996874] [<c13de002>] ? pccardd+0xce/0x313
[ 8.002815] [<c10359d2>] warn_slowpath_fmt+0x2e/0x30
[ 8.008752] [<c104e13e>] __might_sleep+0x66/0x82
[ 8.014631] [<c13de002>] ? pccardd+0xce/0x313
[ 8.020461] [<c13de002>] ? pccardd+0xce/0x313
[ 8.026190] [<c152f8a7>] mutex_lock+0x17/0x2f
[ 8.031906] [<c13de06e>] pccardd+0x13a/0x313
[ 8.037645] [<c152f1d5>] ? preempt_schedule+0x1f/0x21
[ 8.043388] [<c12a3378>] ? ___preempt_schedule+0x8/0xc
[ 8.049121] [<c13ddf34>] ? socket_complete_resume+0x2e/0x2e
[ 8.054850] [<c13ddf34>] ? socket_complete_resume+0x2e/0x2e
[ 8.060528] [<c104928d>] kthread+0x8c/0x91
[ 8.066266] [<c1531681>] ret_from_kernel_thread+0x21/0x30
[ 8.071978] [<c1049201>] ? __kthread_parkme+0x6e/0x6e
[ 8.077663] hub 3-0:1.0: 2 ports detected
[ 8.083527] uhci_hcd 0000:00:1a.1: UHCI Host Controller
[ 8.089150] uhci_hcd 0000:00:1a.1: new USB bus registered, assigned
bus number 4
[ 8.094822] uhci_hcd 0000:00:1a.1: detected 2 ports
[ 8.100476] uhci_hcd 0000:00:1a.1: irq 21, io base 0x00001860
[ 8.106204] usb usb4: New USB device found, idVendor\x1d6b, idProduct\001
[ 8.111938] usb usb4: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[ 8.117672] usb usb4: Product: UHCI Host Controller
[ 8.123353] usb usb4: Manufacturer: Linux 4.0.4-xxxxxxxxxxxx uhci_hcd
[ 8.129331] usb usb4: SerialNumber: 0000:00:1a.1
[ 8.135174] hub 4-0:1.0: USB hub found
[ 8.140799] hub 4-0:1.0: 2 ports detected
[ 8.146632] uhci_hcd 0000:00:1a.2: UHCI Host Controller
[ 8.152280] uhci_hcd 0000:00:1a.2: new USB bus registered, assigned
bus number 5
[ 8.157945] uhci_hcd 0000:00:1a.2: detected 2 ports
[ 8.163558] uhci_hcd 0000:00:1a.2: irq 19, io base 0x00001880
[ 8.169172] usb usb5: New USB device found, idVendor\x1d6b, idProduct\001
[ 8.174784] usb usb5: New USB device strings: Mfr=3, Product=2,
SerialNumber=1
[ 8.180377] usb 2-2: new high-speed USB device number 2 using ehci-pci
[ 8.185970] usb usb5: Product: UHCI Host Controller
[ 8.191420] usb usb5: Manufacturer: Linux 4.0.4-xxxxxxxxxxxx uhci_hcd
[ 8.196775] usb usb5: SerialNumber: 0000:00:1a.2
[ 8.202294] hub 5-0:1.0: USB hub found
[ 8.207640] hub 5-0:1.0: 2 ports detected
[ 8.213132] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[ 8.218301] ---[ end trace 711724ffae446cbc ]---
The second one is also intermittent and is fatal.
The oops scrolled off the screen before two keyboard lights started flashing.
I think the driver was Qlogicfas.
I inserted a 16-bit pcmcia SCSI adapter, attached to a vintager CD-ROM drive.
The kernel recognized the drive.
I inserted a CD-R that was burned during this millennium and I guess the drive
couldn't read it, and a driver probably Qlogicfas couldn't handle the
error report.
Sorry, I only saw it once and didn't want to spend half an hour writing down the
tail end of the oops report.
^ permalink raw reply
* Re: pcmcia oopses
From: Greg KH @ 2016-04-04 14:49 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <CAD4SBxDzgHSdQmHr2Qk--X-9_Nz5upJWQ4r7HJfSoUMrcXCfXw@mail.gmail.com>
On Mon, Apr 04, 2016 at 12:30:52PM +0900, iwillallways forget1 wrote:
> Here are two oopses from kernel 4.0.4, 32 bits, on a PC with a pcmcia
> slot that can handle both cardbux and 16-bit pcmcia cards.
> As mentioned in my other thread, I commented out udev's call to
> pcmcia-socket-startup so booting completes.
Try sending this to the pcmcia maintainer and mailing list, as found in
the kernel MAINTAINERS file. They are the ones that can help you out
the best here, not the hotplug list.
thanks,
greg k-h
^ permalink raw reply
* udev: USB Printers not printing [solved]
From: Max Euer @ 2016-04-19 16:38 UTC (permalink / raw)
To: linux-hotplug
Dear Sirs!
I am no computer expert at all, only an ordinary Linux user for 16 years.
I noticed that with the introduction of udev a couple of years ago some things
stopped working:
notably my USB printer: lpq: queue-name:no daemon present.
This was not the case when udev was prevented from starting at boot time.
Well, you may say 'Why's he clinging on to this old dilapidated lpr thingy?'
I won't bore you with the reason why.
I could not find a solution on the internet but a hint in the udev documentation
made me do this:
creating a device-node usblp0 in /lib/udev/devices/ did the trick.
I might also mention that even the cd-player device did not work with an
ordinary program like alsaplayer for the same reason: /dev/cdrom not being present.
[udev created a cdrom1 but it is not used]
Remains the question of how to create those device-nodes.
I could only do it by booting without udev by doing
cp -a /dev/usblp0{cdrom} /lib/udev/devices/
[MAKEDEV refuses to run when udev is on]
I won't mention other strange behaviours like udev renaming my only eth card
from eth0 to eth1 but that was easy to fix.[pardon me for having it mentioned
anyway]
Hoping this information has been of mild interest to you but for an ordinary
user those common devices not working are a real pain in the ass;
I remain, with compliments
Yours truly
--
Max Euer - Oud Lemiers 18 - NL6295AT Lemiers -
T (NL 06)1840 3128 - F (NL 084)716 4118 - E m.euer@zonnet.nl
^ permalink raw reply
* rules for interface naming for on-board cards
From: Gadre Nayan @ 2016-05-02 12:56 UTC (permalink / raw)
To: kernelnewbies
Hi,
I am trying to understand the way interface names are given.
So one way is hard coding in net_device->name.
If the name contains a %d format string, the first available device
name with the given base is used; assigned numbers start at zero.
On my base machine, I have 1 NIC card which is onboard, rest I can
remove and plug.
For the onboard card I get the interfaces as em1-4, while all the
other cards I get names as p1p2, p3p4 etc.
Does Udev know the difference.
How are these names given, definitely there must be some configuration
which is used to distinguish on-board card vs removable.
Thanks.
^ permalink raw reply
* Re: rules for interface naming for on-board cards
From: Mandeep Sandhu @ 2016-05-02 16:52 UTC (permalink / raw)
To: kernelnewbies
In-Reply-To: <CAKJ7aR4LY6fxtT3Pwo_nz0+5TwE86u68BDnMw2EHAkSu=Q5NNg@mail.gmail.com>
>
> If the name contains a %d format string, the first available device
> name with the given base is used; assigned numbers start at zero.
>
> On my base machine, I have 1 NIC card which is onboard, rest I can
> remove and plug.
> For the onboard card I get the interfaces as em1-4, while all the
> other cards I get names as p1p2, p3p4 etc.
This looks like systemd's way of naming network interfaces (aka
predictable network interface names). Are you running a systemd
enabled system?
If so, look at this systemd guide for the naming convention:
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
HTH,
-mandeep
>
> Does Udev know the difference.
>
> How are these names given, definitely there must be some configuration
> which is used to distinguish on-board card vs removable.
>
> Thanks.
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply
* Re: rules for interface naming for on-board cards
From: Gadre Nayan @ 2016-05-03 4:28 UTC (permalink / raw)
To: kernelnewbies
In-Reply-To: <CAC+QLdTcFSqump4hD4mP67zWJSqcNVJR0aL+aDx0MqL_conCqw@mail.gmail.com>
Hi,
Thanks. I see that in my system for onboard devices I get a sysfs
attribute "acpi_index", which is not available for removable cards.
So currently existence and non existence of this file will be
sufficient to detect.
Thanks.
On Mon, May 2, 2016 at 10:22 PM, Mandeep Sandhu
<mandeepsandhu.chd@gmail.com> wrote:
>>
>> If the name contains a %d format string, the first available device
>> name with the given base is used; assigned numbers start at zero.
>>
>> On my base machine, I have 1 NIC card which is onboard, rest I can
>> remove and plug.
>> For the onboard card I get the interfaces as em1-4, while all the
>> other cards I get names as p1p2, p3p4 etc.
>
> This looks like systemd's way of naming network interfaces (aka
> predictable network interface names). Are you running a systemd
> enabled system?
>
> If so, look at this systemd guide for the naming convention:
>
> https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
>
> HTH,
> -mandeep
>
>
>>
>> Does Udev know the difference.
>>
>> How are these names given, definitely there must be some configuration
>> which is used to distinguish on-board card vs removable.
>>
>> Thanks.
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies@kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply
* monitoring udevd
From: SZÉPE Viktor @ 2016-05-07 17:11 UTC (permalink / raw)
To: linux-hotplug
Hello!
I am trying to monitor udevd (by monit).
Could you help me to make up a command that generates a real kernel event
which does nothing but proves that udevd is functioning (like a ping
... pong message)
All I have is
/bin/udevadm trigger --verbose --dry-run --typefivices --subsystem-match=rtc
But it also works when I kill udevd.
Thank you.
All the best!
SZÉPE Viktor
--
+36-20-4242498 sms@szepe.net skype: szepe.viktor
Budapest, III. kerület
--
To unsubscribe from this list: send the line "unsubscribe linux-hotplug" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Use the lastest linux version test hotplug funtion, Print a lagre number of call trace on the stage
From: Peter Yang(杨相坤) @ 2016-06-07 6:39 UTC (permalink / raw)
To: linux-hotplug
ICBIaSAsDQoNCiAgIFJlY2VudGx5LCBJIHVzZWQgdGhlIGxhdGVzdCBsaW51eCB2ZXJzaW9uKExp
bnV4IDQuNi4xPGh0dHBzOi8vZ2l0Lmtlcm5lbC5vcmcvY2dpdC9saW51eC9rZXJuZWwvZ2l0L3N0
YWJsZS9saW51eC1zdGFibGUuZ2l0L2NvbW1pdC8/aD1saW51eC00LjYueT4pIHRvIHRlc3QgdGhl
IGhvdC1wbHVnIGZ1bmN0aW9uLA0KDQpPbiB0aGUgc3RhZ2Ugb2YgobBwaHlzaWNhbCByZW1vdmUg
bWVtb3J5obEsIGl0IHdpbGwgcHJpbnQgYSBsYXJnZSBudW1iZXIgb2YgQ2FsbCBUcmFjZS4NCg0K
dGhlIG9wZXJhdGUgcHJvY2VzcyBpcyBsb2dpY2FsIHJlbW92ZShjcHUvbWVtb3J5KSAtLS0+cGh5
c2ljYWwgcmVtb3ZlLS0tPnBoeXNpY2FsIGFkZC0tLS0+bG9naWNhbCBhZGQuDQoNCkFuZCBJIGRv
bid0IGtvd24gaG93IGl0IGhhcHBlbnMsIHBsZWFzZSBoZWxwIG1lLCB0aGFua3MhDQoNCkFuZCBo
ZXJlIGlzIHRoZSBjYWxsIHRyYWNlIGZvciB0aGlzIHZlcnNpb246DQoNClsgIDc3Ni42MDExMjJd
IFdvcmtxdWV1ZToga2FjcGlfaG90cGx1ZyBhY3BpX2hvdHBsdWdfd29ya19mbg0KWyAgNzc2LjYw
MTEyMl0gIDAwMDAwMDAwMDAwMDAyODYgMDAwMDAwMDA2YjVhMzFhZSBmZmZmODgyYTU1NDEzYTQw
IGZmZmZmZmZmODEzMzMxN2YNClsgIDc3Ni42MDExMjJdICBmZmZmZWEwMGFkYmQ4MDAwIGZmZmZm
ZmZmODFhMTIxYzggZmZmZjg4MmE1NTQxM2E2OCBmZmZmZmZmZjgxMjA2MGNiDQpbICA3NzYuNjAx
MTIyXSAgMDAwMDAwMDAwMDAwMDgwMCBmZmZmZWEwMGFkYmQ4MDAwIDAwMDAwMDAwMDAwMDAwMDEg
ZmZmZjg4MmE1NTQxM2FiOA0KDQpbICA3NzYuNjAxMTIyXSBDYWxsIFRyYWNlOg0KWyAgNzc2LjYw
MTEyMl0gIFs8ZmZmZmZmZmY4MTMzMzE3Zj5dIGR1bXBfc3RhY2srMHg2My8weDg0DQpbICA3NzYu
NjAxMTIyXSAgWzxmZmZmZmZmZjgxMjA2MGNiPl0gYmFkX3BhZ2UrMHgxMTQvMHgxMzANClsgIDc3
Ni42MDExMjJdICBbPGZmZmZmZmZmODExODlkNDk+XSBmcmVlX3BhZ2VzX3ByZXBhcmUrMHgyOTkv
MHgyZDANClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODEwYjAwMDE+XSA/IGNwdXNldF9jcHVf
aW5hY3RpdmUrMHg2MS8weDEwMA0KWyAgNzc2LjYwMTEyMl0gIFs8ZmZmZmZmZmY4MTBiODllMz5d
ID8gcHV0X3ByZXZfZW50aXR5KzB4MzMvMHg3MTANClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZm
ODExOGMxNzE+XSBmcmVlX2hvdF9jb2xkX3BhZ2UrMHgzMS8weDE1MA0KWyAgNzc2LjYwMTEyMl0g
IFs8ZmZmZmZmZmY4MTE4YzM1NT5dIF9fZnJlZV9wYWdlcysweDI1LzB4MzANClsgIDc3Ni42MDEx
MjJdICBbPGZmZmZmZmZmODE2OTZiZDM+XSBmcmVlX3BhZ2V0YWJsZSsweDZmLzB4YjQNClsgIDc3
Ni42MDExMjJdICBbPGZmZmZmZmZmODE2OTc0MTA+XSByZW1vdmVfcGFnZXRhYmxlKzB4Mzc5LzB4
N2ZmDQpbICA3NzYuNjAxMTIyXSAgWzxmZmZmZmZmZjgxNjkzOGIwPl0gdm1lbW1hcF9mcmVlKzB4
MTAvMHgyMA0KWyAgNzc2LjYwMTEyMl0gIFs8ZmZmZmZmZmY4MTFkZjMyOT5dIHNwYXJzZV9yZW1v
dmVfb25lX3NlY3Rpb24rMHgxNDkvMHgxODANClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODEx
ZWIxYTc+XSBfX3JlbW92ZV9wYWdlcysweDJlNy8weDRlMA0KWyAgNzc2LjYwMTEyMl0gIFs8ZmZm
ZmZmZmY4MTY5MzkxZT5dIGFyY2hfcmVtb3ZlX21lbW9yeSsweDVlLzB4YjANClsgIDc3Ni42MDEx
MjJdICBbPGZmZmZmZmZmODE2OTU2Y2M+XSByZW1vdmVfbWVtb3J5KzB4OGMvMHhjMA0KWyAgNzc2
LjYwMTEyMl0gIFs8ZmZmZmZmZmY4MTNkZjk0Nz5dIGFjcGlfbWVtb3J5X2RldmljZV9yZW1vdmUr
MHg3OS8weGE1DQpbICA3NzYuNjAxMTIyXSAgWzxmZmZmZmZmZjgxM2FlODk3Pl0gYWNwaV9idXNf
dHJpbSsweDVhLzB4OGQNClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODEzYjAzNDk+XSBhY3Bp
X2RldmljZV9ob3RwbHVnKzB4MWI3LzB4NDE4DQpbICA3NzYuNjAxMTIyXSAgWzxmZmZmZmZmZjgx
M2E4ZWY1Pl0gYWNwaV9ob3RwbHVnX3dvcmtfZm4rMHgxZS8weDI5DQpbICA3NzYuNjAxMTIyXSAg
WzxmZmZmZmZmZjgxMDk5N2EyPl0gcHJvY2Vzc19vbmVfd29yaysweDE1Mi8weDQwMA0KWyAgNzc2
LjYwMTEyMl0gIFs8ZmZmZmZmZmY4MTA5YTA5NT5dIHdvcmtlcl90aHJlYWQrMHgxMjUvMHg0YjAN
ClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODE2OWIyODI+XSA/IF9fc2NoZWR1bGUrMHgyYjIv
MHg4ZTANClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODEwOTlmNzA+XSA/IHJlc2N1ZXJfdGhy
ZWFkKzB4MzgwLzB4MzgwDQpbICA3NzYuNjAxMTIyXSAgWzxmZmZmZmZmZjgxMDlmYmU4Pl0ga3Ro
cmVhZCsweGQ4LzB4ZjANClsgIDc3Ni42MDExMjJdICBbPGZmZmZmZmZmODE2OWY2MDI+XSByZXRf
ZnJvbV9mb3JrKzB4MjIvMHg0MA0KWyAgNzc2LjYwMTEyMl0gIFs8ZmZmZmZmZmY4MTA5ZmIxMD5d
ID8ga3RocmVhZF9wYXJrKzB4NjAvMHg2MA0KDQpbICA3NzYuNzg5OTI4XSBEaXNhYmxpbmcgbG9j
ayBkZWJ1Z2dpbmcgZHVlIHRvIGtlcm5lbCB0YWludA0KWyAgNzc2Ljc5NTI0N10gQlVHOiBCYWQg
cGFnZSBzdGF0ZSBpbiBwcm9jZXNzIGt3b3JrZXIvdTUxMDowICBwZm46MmI2ZjYwMQ0KWyAgNzc2
LjgwMTc4MF0gcGFnZTpmZmZmZWEwMGFkYmQ4MDQwIGNvdW50OjAgbWFwY291bnQ6MCBtYXBwaW5n
OiAgICAgICAgICAobnVsbCkgaW5kZXg6MHgwDQpbICA3NzYuODA5ODA2XSBmbGFnczogMHgyZmZm
ZmY4MDAwMDgwMChwcml2YXRlKQ0KWyAgNzc2LjgxNDA4OF0gcGFnZSBkdW1wZWQgYmVjYXVzZTog
UEFHRV9GTEFHU19DSEVDS19BVF9GUkVFIGZsYWcocykgc2V0DQpbICA3NzYuODIwNTMwXSBiYWQg
YmVjYXVzZSBvZiBmbGFnczogMHg4MDAocHJpdmF0ZSkNClsgIDc3Ni44MjUxNjZdIE1vZHVsZXMg
bGlua2VkIGluOiB0Y3BfbHAgYm5lcCBibHVldG9vdGggcmZraWxsIGZ1c2UgeHRfYWRkcnR5cGUg
YnJfbmV0ZmlsdGVyIG5mX2Nvbm50cmFja19pcHY0IG5mX2RlZnJhZ19pcHY0IG5mX25hdCB4dF9j
b25udHJhY2sgbmZfY29ubnRyYWNrIGlwdF9SRUpFQ1QgbmZfcmVqZWN0X2lwdjQgaXB0YWJsZV9m
aWx0ZXIgaXBfdGFibGVzIGRtX3RoaW5fcG9vbCB0dW4gZG1fcGVyc2lzdGVudF9kYXRhIGRtX2Jp
b19wcmlzb24gZG1fYnVmaW8gbG9vcCBicmlkZ2Ugc3RwIGxsYyBycGNyZG1hIHN1bnJwYyBpYl9p
c2VydCBpc2NzaV90YXJnZXRfbW9kIGliX2lzZXIgbGliaXNjc2kgc2NzaV90cmFuc3BvcnRfaXNj
c2kgaWJfc3JwdCB0YXJnZXRfY29yZV9tb2QgeDg2X3BrZ190ZW1wX3RoZXJtYWwgY29yZXRlbXAg
a3ZtX2ludGVsIGt2bSBpYl9zcnAgc2NzaV90cmFuc3BvcnRfc3JwIGliX2lwb2liIGlycWJ5cGFz
cyBjcmN0MTBkaWZfcGNsbXVsIHJkbWFfdWNtIGNyYzMyX3BjbG11bCBjcmMzMmNfaW50ZWwgaWJf
dWNtIGdoYXNoX2NsbXVsbmlfaW50ZWwgaWJfdXZlcmJzIGliX3VtYWQgYWVzbmlfaW50ZWwgcmRt
YV9jbSBpYl9jbSBscncgaXdfY20gZ2YxMjhtdWwgZ2x1ZV9oZWxwZXIgYWJsa19oZWxwZXIgY3J5
cHRkIGlUQ09fd2R0IGlwbWlfZGV2aW50ZiBpVENPX3ZlbmRvcl9zdXBwb3J0IG1laV9tZSBzYl9l
ZGFjIHBjc3BrciBscGNfaWNoIGkyY19pODAxIGlwbWlfc2kgbWZkX2NvcmUgaXBtaV9tc2doYW5k
bGVyIGVkYWNfY29yZSBtZWkgaW9hdGRtYSBzaHBjaHAgdWlucHV0IGJpbmZtdF9taXNjIG1seDRf
aWIgaWJfc2EgaWJfbWFkIG1seDRfZW4gaWJfY29yZSB2eGxhbiBpcDZfdWRwX3R1bm5lbCB1ZHBf
dHVubmVsIGliX2FkZHIgeGZzIGxpYmNyYzMyYyBzZF9tb2QgYXN0IGRybV9rbXNfaGVscGVyIHN5
c2NvcHlhcmVhIHN5c2ZpbGxyZWN0IHN5c2ltZ2JsdCBmYl9zeXNfZm9wcyBpZ2IgdHRtIGFoY2kg
cHRwIG1seDRfY29yZSBsaWJhaGNpIGRybSBwcHNfY29yZSBsaWJhdGEgZGNhIG1lZ2FyYWlkX3Nh
cyBpMmNfYWxnb19iaXQgaTJjX2NvcmUgZG1fbWlycm9yIGRtX3JlZ2lvbl9oYXNoIGRtX2xvZyBk
bV9tb2QNClsgIDc3Ni45MjUzNDNdIENQVTogMSBQSUQ6IDExMzQ5IENvbW06IGt3b3JrZXIvdTUx
MDowIFRhaW50ZWQ6IEcgICAgQiAgICAgSSAgICAgNC42LjAta2FuYXMtcmMzLng4Nl82NCsgIzEN
ClsgIDc3Ni45MjYxNDNdIEhhcmR3YXJlIG5hbWU6IEluc3B1ciBUUzg2MC9UUzg2MCwgQklPUyBU
Uzg2MF8yLjAuMyAyMDE1LzA3LzAyDQpbICA3NzYuOTI2MTQzXSBXb3JrcXVldWU6IGthY3BpX2hv
dHBsdWcgYWNwaV9ob3RwbHVnX3dvcmtfZm4NClsgIDc3Ni45MjYxNDNdICAwMDAwMDAwMDAwMDAw
Mjg2IDAwMDAwMDAwNmI1YTMxYWUgZmZmZjg4MmE1NTQxM2E0MCBmZmZmZmZmZjgxMzMzMTdmDQpb
ICA3NzYuOTI2MTQzXSAgZmZmZmVhMDBhZGJkODA0MCBmZmZmZmZmZjgxYTEyMWM4IGZmZmY4ODJh
NTU0MTNhNjggZmZmZmZmZmY4MTIwNjBjYg0KWyAgNzc2LjkyNjE0M10gIDAwMDAwMDAwMDAwMDA4
MDAgZmZmZmVhMDBhZGJkODA0MCAwMDAwMDAwMDAwMDAwMDAxIGZmZmY4ODJhNTU0MTNhYjgNCg0K
VGhhbmtzLiA6KQ0KDQoNCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCg0K
^ permalink raw reply
* Assign BAR address for pci device after hotplug
From: Chitturi, Divakar @ 2016-08-02 21:11 UTC (permalink / raw)
To: linux-hotplug
Hi
Below are two scenarios for which i would like to understand the right way to assign the BAR address for my PCI device
Device details:
pci switch upstrem port connected to root complex
pci switch downstream port 1 connected to my_pci_device
my_pci_device is FPGA
FPGA has bunch of controllers and other logic
Driver details:
Have a dedicated pci kernel driver for my_device
Enabled pcihp kernel driver
At power on, BIOS assigns/reserves specific region of memory for my pci device.
during the linux boot process, pci core will assign the bar and other initialization and call my
probe function in the driver. probe function will perform some initialization which includes reading
the BAR and setting some registers in IO space of the device.
Scn1:
At some point the device is removed and reinserted
Scn2:
At some point similar device is inserted in a new pci slot
In both the scenarios explained about, BIOS is not involved as system was never turned off completely. So
1. how do we obtain the BAR address for the device.
2. Should i assign it explicitly?
3. if yes, should it be done in my driver ( in both the case i am assuming driver's probe function will be called )
4. can i use pci_assign_resource() in drivers probe function
5. is something like below the right way to do it ?
static int my_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
if( pci_resource_start (pdev,BAR0) <= 0) {
pr_debug(" Assign BAR0 \n");
ret = pci_assign_resource(pdev,BAR0);
if( 0 > ret) {
dev_err(&pdev->dev, " Failed to assign resource \n");
return ret;
}
}
ret = pci_enable_device(pdev);
ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
base_addr = pci_iomap(pdev, BAR0, 0);
}
Thanks
Divakar
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: Greg KH @ 2016-08-03 5:12 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> Hi
>
> Below are two scenarios for which i would like to understand the right
> way to assign the BAR address for my PCI device
>
> Device details:
> pci switch upstrem port connected to root complex
> pci switch downstream port 1 connected to my_pci_device
> my_pci_device is FPGA
> FPGA has bunch of controllers and other logic
>
> Driver details:
> Have a dedicated pci kernel driver for my_device
> Enabled pcihp kernel driver
Which pci hotplug driver? Just the PCI Hotplug core isn't going to do
anything, you need a controller driver for your specific PCI hotplug
hardware.
> At power on, BIOS assigns/reserves specific region of memory for my pci device.
> during the linux boot process, pci core will assign the bar and other initialization and call my
> probe function in the driver. probe function will perform some initialization which includes reading
> the BAR and setting some registers in IO space of the device.
>
> Scn1:
> At some point the device is removed and reinserted
>
> Scn2:
> At some point similar device is inserted in a new pci slot
How are they added/removed without a hotplug controller?
> In both the scenarios explained about, BIOS is not involved as system was never turned off completely. So
> 1. how do we obtain the BAR address for the device.
What architecture is this? x86? If so, it just uses whatever the BIOS
said was there for the last time.
> 2. Should i assign it explicitly?
Yes.
> 3. if yes, should it be done in my driver ( in both the case i am assuming driver's probe function will be called )
No, in your pci hotplug driver, you need to handle it there.
> 4. can i use pci_assign_resource() in drivers probe function
Nope.
> 5. is something like below the right way to do it ?
>
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> if( pci_resource_start (pdev,BAR0) <= 0) {
> pr_debug(" Assign BAR0 \n");
> ret = pci_assign_resource(pdev,BAR0);
> if( 0 > ret) {
> dev_err(&pdev->dev, " Failed to assign resource \n");
> return ret;
> }
> }
>
> ret = pci_enable_device(pdev);
> ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
> base_addr = pci_iomap(pdev, BAR0, 0);
> }
You need to write a pci hotplug controller driver to manage the
resources and assign them properly. That's its responsibility, see the
PCI hotplug specification for all of the details...
good luck!
greg k-h
^ permalink raw reply
* RE: Assign BAR address for pci device after hotplug
From: Chitturi, Divakar @ 2016-08-04 17:09 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
Thanks for your comments.
I am not sure if I can attach a picture to explain the HW config. Please find my comments for your questions inline.
Thanks
Divakar
-----Original Message-----
From: linux-hotplug-owner@vger.kernel.org [mailto:linux-hotplug-owner@vger.kernel.org] On Behalf Of Greg KH
Sent: Tuesday, August 02, 2016 10:12 PM
Cc: linux-hotplug@vger.kernel.org
Subject: Re: Assign BAR address for pci device after hotplug
On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> Hi
>
> Below are two scenarios for which i would like to understand the right
> way to assign the BAR address for my PCI device
>
> Device details:
> pci switch upstrem port connected to root complex pci switch
> downstream port 1 connected to my_pci_device my_pci_device is FPGA
> FPGA has bunch of controllers and other logic
>
> Driver details:
> Have a dedicated pci kernel driver for my_device Enabled pcihp kernel
> driver
Which pci hotplug driver? Just the PCI Hotplug core isn't going to do anything, you need a controller driver for your specific PCI hotplug hardware.
There is a hotplug/hotswap controller in the plx switch but this is not being used. The hp signals are no connect. However the is a kernel driver loaded for this switch/bridge.
The power control for the downstream devices is handled by a master FPGA (apart from the fpga in downstream port of pcieswitch ) in the system. So the master fpga acts like a hotplug controller ?
If that's the case, then there is no driver for the master fpga. I plan to develop one. Even in this case what is/are the functions to reassign the bar for the downstream devices.
> At power on, BIOS assigns/reserves specific region of memory for my pci device.
> during the linux boot process, pci core will assign the bar and other
> initialization and call my probe function in the driver. probe
> function will perform some initialization which includes reading the BAR and setting some registers in IO space of the device.
>
> Scn1:
> At some point the device is removed and reinserted
>
> Scn2:
> At some point similar device is inserted in a new pci slot
How are they added/removed without a hotplug controller?
> In both the scenarios explained about, BIOS is not involved as system
> was never turned off completely. So 1. how do we obtain the BAR address for the device.
What architecture is this? x86? If so, it just uses whatever the BIOS said was there for the last time.
Yes this is x86. When a new device (downstream fpga of pcie switch) is inserted after linux boot, and the BIOS has reserved the memory region for the BUS on which new device sits, who's responsibility is to assign the BAR to the new device?
If it's the Hotplug controller drivers responsibility, then what are the functions that needs to be invoked.
> 2. Should i assign it explicitly?
Yes.
> 3. if yes, should it be done in my driver ( in both the case i am
> assuming driver's probe function will be called )
No, in your pci hotplug driver, you need to handle it there.
I would like to understand how to handle in the pci hotplug driver
> 4. can i use pci_assign_resource() in drivers probe function
Nope.
Right now by using this function in the probe function of the kernel driver for the new device (downstream fpga of pcie switch ) inserted ( scn2 ) it is able to obtain the BAR and I can access the IO space and read /write.
However I am not convinced if this is the right way to do it.
> 5. is something like below the right way to do it ?
>
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id
> *id) {
> if( pci_resource_start (pdev,BAR0) <= 0) {
> pr_debug(" Assign BAR0 \n");
> ret = pci_assign_resource(pdev,BAR0);
> if( 0 > ret) {
> dev_err(&pdev->dev, " Failed to assign resource \n");
> return ret;
> }
> }
>
> ret = pci_enable_device(pdev);
> ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
> base_addr = pci_iomap(pdev, BAR0, 0); }
You need to write a pci hotplug controller driver to manage the resources and assign them properly. That's its responsibility, see the PCI hotplug specification for all of the details...
Can you point me to a sample which handles the BAR assignment in hotplug scenario for me to understand better?
good luck!
greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-hotplug" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: Greg KH @ 2016-08-05 10:53 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
On Thu, Aug 04, 2016 at 05:09:21PM +0000, Chitturi, Divakar wrote:
> Thanks for your comments.
> I am not sure if I can attach a picture to explain the HW config.
> Please find my comments for your questions inline.
I can't distinguish your responses from my responses, please use an
email client that does this correctly and I'll be glad to respond.
thanks,
greg k-h
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: divakar @ 2016-08-09 0:32 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
Hi Greg,
Thanks for your comments. I managed to install a client which will do
the auto format. Hopefully this is better than before. Please find my
responses inline. Also added extra questions and information about the
setup.
Questions:
1. Since the modified_my_driver_probe is working, is that the recommended
way of doing it? I couldn't find any sample drivers using it. So wanted
your guidance
2. There is no hotplug controller as far is know off in the setup (
assuming native x86 soc has root port which has inbuilt hp controller
and pcieport driver takes care of it). Am i missing something?
3. When card 2 is inserted there is no hotplug event. Not sure why.
Should anything be done from driver side ? I just started looking into
the pcie switch eeprom config space settings to check if there is any
mask for the event notification , assuming if masks cleared, pcieport
driver can be notified by pciehp similar to card 1 insertion.
HW details:
x86 ( Soc) -> pcie root port 0 -> Device 0 (FPGA_0 )
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 1 -> Device 1
(FPGA_1 ) ]
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 2 -> Device 2 ]
x86 ( Soc) -> pcie root port 2 -> pcie-switch_2_port 1 -> [ pcie-
switch_3_port 1 -> Device 3 ( FPGA_2) ]
x86 ( Soc) -> pcie root port 2 -> pcie-switch_2_port 1 -> [ pcie-
switch_3_port 2 -> Device 4 ]
card 1- pcie-switch_1, Device 1 and 2 are on the same physical pcb card
which can be hotplugged ( inserted or removed )
card 2- pcie-switch_3, Device 3 and 4 are on the same physical pcb card
which can be hotplugged ( inserted or removed )
Drivers details:
- FPGA_0-2 are devices have "my_pci_driver"
- Device 2 and Device 4 are third party devices which have their drivers
- All pcie ports has handled by "pcieport" driver (output of lspci -vvv)
- pciehp core driver is enabled
Sequence of operations:
- Boot up
- Insmod my_pci_driver, third party device driver
- Insert card 1 ( output sequence explained below happens )
- Insert card 2 ( nothing happens )
- issue /sys/pci/bus/rescan command ( output sequence explained below
happens again )
Output sequence:
- pcieport driver see an event and will start enabling downstream
devices
- my_pci_driver and third party device drivers probe functions get
called
- third party driver probe function completes and the BAR is assigned (
output of lspci -vvv -s B:D:F )
- my_pci_driver probe function throws out an error and the probe fails.
error as below
pcieport 0000:54:05.0: enabling device (0000 -> 0002)
my-driver 0000:56:00.3: can't enable device: BAR 0 [mem
0x00000000-0x000fffff 64bit pref] not claimed
my-driver 0000:56:00.3: Failed to enable My device rc=-22
my-driver: probe of 0000:56:00.3 failed with error -22
Modify driver:
- when i add the pci_reassign_resource function before the
pci_enable_device i nolonger see the error
- BAR is assigned in the lspci -vvv output
my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id
*id)
{
ret = pci_enable_device(pdev);
if (ret) {
dev_err(&pdev->dev,"Failed to enable my device
rc=%d\n",ret);
goto err_pci_enable;
}
ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
if (ret) {
dev_err(&pdev->dev, "Failed to request PCI regions
rc=%d\n", ret);
goto err_pci_req;
}
base_addr = pci_iomap(pdev, 0, 0);
if (IS_ERR(base_addr)) {
dev_err(&pdev->dev, "Failed to iomap PCI BAR %d\n", 0);
goto err_pci_iomap;
}
...
...
...
}
modified_my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id
*id)
{
/* Enable hotplug support
TODO: Check if this is right way to do it */
if( pci_resource_start (pdev,0) <= 0) {
ret = pci_assign_resource(pdev,0);
if( 0 > ret) {
dev_err(&pdev->dev, " Failed to assign resource
\n");
return ret;
}
}
ret = pci_enable_device(pdev);
if (ret) {
dev_err(&pdev->dev,"Failed to enable my device
rc=%d\n",ret);
goto err_pci_enable;
}
ret = pci_request_regions(pdev, dev_driver_string(&pdev->dev));
if (ret) {
dev_err(&pdev->dev, "Failed to request PCI regions
rc=%d\n", ret);
goto err_pci_req;
}
base_addr = pci_iomap(pdev, 0, 0);
if (IS_ERR(base_addr)) {
dev_err(&pdev->dev, "Failed to iomap PCI BAR %d\n", 0);
goto err_pci_iomap;
}
}
-Divakar
On Wednesday, August 03, 2016 07:12:16 AM Greg KH wrote:
> On Tue, Aug 02, 2016 at 09:11:09PM +0000, Chitturi, Divakar wrote:
> > Hi
> >
> > Below are two scenarios for which i would like to understand the
> > right way to assign the BAR address for my PCI device
> >
> > Device details:
> > pci switch upstrem port connected to root complex
> > pci switch downstream port 1 connected to my_pci_device
> > my_pci_device is FPGA
> > FPGA has bunch of controllers and other logic
> >
> > Driver details:
> > Have a dedicated pci kernel driver for my_device
> > Enabled pcihp kernel driver
>
> Which pci hotplug driver? Just the PCI Hotplug core isn't going to do
> anything, you need a controller driver for your specific PCI hotplug
> hardware.
>
> > At power on, BIOS assigns/reserves specific region of memory for my
> > pci device. during the linux boot process, pci core will assign the
> > bar and other initialization and call my probe function in the
> > driver. probe function will perform some initialization which
> > includes reading the BAR and setting some registers in IO space of
> > the device.
> >
> > Scn1:
> > At some point the device is removed and reinserted
> >
> > Scn2:
> > At some point similar device is inserted in a new pci slot
>
> How are they added/removed without a hotplug controller?
>
> > In both the scenarios explained about, BIOS is not involved as
> > system was never turned off completely. So 1. how do we obtain the
> > BAR address for the device.
>
> What architecture is this? x86? If so, it just uses whatever the
> BIOS said was there for the last time.
Yes this is x86 CPU. BIOS has reserved the memory region for the bus
slot on which new device sits during boot. When a new device (downstream
fpga of pcie switch) is inserted and i issue the "echo 1 >
/sys/bus/pci/rescan/ the device shows up in lscpi -t but no BAR is
assigned.
>
> > 2. Should i assign it explicitly?
>
> Yes.
>
> > 3. if yes, should it be done in my driver ( in both the case i am
> > assuming driver's probe function will be called )
> No, in your pci hotplug driver, you need to handle it there.
>
> > 4. can i use pci_assign_resource() in drivers probe function
>
> Nope.
>
> > 5. is something like below the right way to do it ?
> >
> > static int my_probe(struct pci_dev *pdev, const struct pci_device_id
> > *id) {
> >
> > if( pci_resource_start (pdev,BAR0) <= 0) {
> >
> > pr_debug(" Assign BAR0 \n");
> >
> > ret = pci_assign_resource(pdev,BAR0);
> >
> > if( 0 > ret) {
> >
> > dev_err(&pdev->dev, " Failed to assign resource
> > \n");
> > return ret;
> >
> > }
> >
> > }
> >
> > ret = pci_enable_device(pdev);
> > ret = pci_request_regions(pdev,
> > dev_driver_string(&pdev->dev));
> > base_addr = pci_iomap(pdev, BAR0, 0);
> >
> > }
>
> You need to write a pci hotplug controller driver to manage the
> resources and assign them properly. That's its responsibility, see
> the PCI hotplug specification for all of the details...
Right now by using this function in the probe function of the kernel
driver for the new device (downstream fpga of pcie switch ) inserted (
scn2 ) it is able to obtain the BAR and I can access the IO space and
read /write.
However I am not convinced if this is the right way to do it.
>
> good luck!
>
> greg k-h
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-hotplug" in the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: Greg KH @ 2016-08-09 0:49 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
On Mon, Aug 08, 2016 at 05:32:42PM -0700, divakar wrote:
> Hi Greg,
>
> Thanks for your comments. I managed to install a client which will do
> the auto format. Hopefully this is better than before. Please find my
> responses inline. Also added extra questions and information about the
> setup.
>
> Questions:
> 1. Since the modified_my_driver_probe is working, is that the recommended
> way of doing it? I couldn't find any sample drivers using it. So wanted
> your guidance
I dont understand, what "modified" probe function?
> 2. There is no hotplug controller as far is know off in the setup (
> assuming native x86 soc has root port which has inbuilt hp controller
> and pcieport driver takes care of it). Am i missing something?
Then your hardware does not support PCI hotplug, so I don't recommend
trying to use it.
Seriously, this is a hardware and BIOS requirement to support this, if
you don't have that hardware, you can't really work around it in the
kernel. Go buy some real hardware for this please.
> 3. When card 2 is inserted there is no hotplug event. Not sure why.
Why would there be? You don't have hardware that notices this.
good luck,
greg k-h
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: divakar @ 2016-08-09 17:00 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
Hi ,
The modified probe function was the one with the pci_reassign_resource
function.
modified_my_driver_probe:
static int my_probe(struct pci_dev *pdev, const struct pci_device_id
*id)
{
/* Enable hotplug support
TODO: Check if this is right way to do it */
if( pci_resource_start (pdev,0) <= 0) {
ret = pci_assign_resource(pdev,0);
if( 0 > ret) {
dev_err(&pdev->dev, " Failed to assign resource
\n");
return ret;
}
}
ret = pci_enable_device(pdev);
}
Thanks for your comments on the hardware. I hope my previous message
with the output log snippets came through. My observation of the events
happening seem to suggest there is a hotplug controller as i can see the
difference between with and without enabling "pciehp". Also
pci_resource_start in the probe function of the endpoint device driver
seem to work. Hence my persistence in understanding the behavior.
Thanks
On Tuesday, August 09, 2016 02:49:39 AM Greg KH wrote:
> On Mon, Aug 08, 2016 at 05:32:42PM -0700, divakar wrote:
> > Hi Greg,
> >
> > Thanks for your comments. I managed to install a client which will
> > do
> > the auto format. Hopefully this is better than before. Please find
> > my
> > responses inline. Also added extra questions and information about
> > the setup.
> >
> > Questions:
> > 1. Since the modified_my_driver_probe is working, is that the
> > recommended way of doing it? I couldn't find any sample drivers
> > using it. So wanted your guidance
>
> I dont understand, what "modified" probe function?
>
> > 2. There is no hotplug controller as far is know off in the setup (
> > assuming native x86 soc has root port which has inbuilt hp
> > controller
> > and pcieport driver takes care of it). Am i missing something?
>
> Then your hardware does not support PCI hotplug, so I don't recommend
> trying to use it.
>
> Seriously, this is a hardware and BIOS requirement to support this, if
> you don't have that hardware, you can't really work around it in the
> kernel. Go buy some real hardware for this please.
>
> > 3. When card 2 is inserted there is no hotplug event. Not sure why.
>
> Why would there be? You don't have hardware that notices this.
>
> good luck,
>
> greg k-h
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: Greg KH @ 2016-08-09 17:18 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
A: Top-posting.
Q: What is the most annoying thing in e-mail?
A: No.
Q: Should I include quotations after my reply?
http://daringfireball.net/2007/07/on_top
On Tue, Aug 09, 2016 at 10:00:22AM -0700, divakar wrote:
> Hi ,
>
> The modified probe function was the one with the pci_reassign_resource
> function.
Don't do that. That's for the pci core to use only.
>
> modified_my_driver_probe:
> static int my_probe(struct pci_dev *pdev, const struct pci_device_id
> *id)
> {
>
> /* Enable hotplug support
> TODO: Check if this is right way to do it */
> if( pci_resource_start (pdev,0) <= 0) {
> ret = pci_assign_resource(pdev,0);
> if( 0 > ret) {
> dev_err(&pdev->dev, " Failed to assign resource
> \n");
> return ret;
> }
> }
>
> ret = pci_enable_device(pdev);
>
> }
>
> Thanks for your comments on the hardware. I hope my previous message
> with the output log snippets came through. My observation of the events
> happening seem to suggest there is a hotplug controller as i can see the
> difference between with and without enabling "pciehp". Also
> pci_resource_start in the probe function of the endpoint device driver
> seem to work. Hence my persistence in understanding the behavior.
If the pciehp driver does not bind to your hardware, then it will not
work properly.
Again, PCI hotplug requires a PCI hotplug controller, or special
firmware/bios support for it. You can't do it properly without it, read
the specification for all of the nasty details.
good luck!
greg k-h
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: divakar @ 2016-08-09 18:48 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
May be i missed one piece of information. I thought i had that in my
inline response but just realized i missed it.
Device 0 ( FPGA 0), has custom logic which will detect the presence of
the card 1 and card 2 and controls the power switches to the same. So If
this makes fpga0 a "hotplug controller " then i can add the resource
management logic/code for devices on card 1 and card 2 to this driver.
Currently fpga_0 driver does some init for some of the devices on
itself. can i follow any hp controller driver example to do the above or
is there any specific driver/eg i can follow? noted your point on reading
the spec for the details and i am on it.
HW details:
x86 ( Soc) -> pcie root port 0 -> Device 0 (FPGA_0 )
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 1 -> Device 1
(FPGA_1 ) ]
x86 ( Soc) -> pcie root port 1 -> [ pcie-switch_1_port 2 -> Device 2 ]
x86 ( Soc) -> pcie root port 2 -> pcie-switch_2_port 1 -> [ pcie-
switch_3_port 1 -> Device 3 ( FPGA_2) ]
x86 ( Soc) -> pcie root port 2 -> pcie-switch_2_port 1 -> [ pcie-
switch_3_port 2 -> Device 4 ]
Thanks for your patience
Divakar
^ permalink raw reply
* Re: Assign BAR address for pci device after hotplug
From: Greg KH @ 2016-08-09 18:59 UTC (permalink / raw)
To: linux-hotplug
In-Reply-To: <AT5PR84MB0177723391A9CA3EAEE43C0088050@AT5PR84MB0177.NAMPRD84.PROD.OUTLOOK.COM>
On Tue, Aug 09, 2016 at 11:48:59AM -0700, divakar wrote:
> May be i missed one piece of information. I thought i had that in my
> inline response but just realized i missed it.
>
> Device 0 ( FPGA 0), has custom logic which will detect the presence of
> the card 1 and card 2 and controls the power switches to the same. So If
> this makes fpga0 a "hotplug controller " then i can add the resource
> management logic/code for devices on card 1 and card 2 to this driver.
> Currently fpga_0 driver does some init for some of the devices on
> itself. can i follow any hp controller driver example to do the above or
> is there any specific driver/eg i can follow? noted your point on reading
> the spec for the details and i am on it.
You might get away with it for your limited hardware platform, but all
bets are off if you plug it into something else.
Again, PCI resource management is handled by the firmware/bios and the
pci hotplug controller. You can try to fake it out, with limited
success, but really, why do you want to do so? Just use a real machine,
that's what the pci hotplug hardware was designed for. It's much
cheaper than trying to work around it in the end, trust me...
greg k-h
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox