Who's doing what with cpu/memory/node hotplug?

Who's doing what with cpu/memory/node hotplug?

[Paul Jackson]

Could someone explain to me who's doing what with cpu, memory and node hotplug?

In particular, I haven't noticed much interaction between some cpu
hotplug work that Ashok has been posting on linux-ia64 and/or lkml, and
either of the email lists lhns-devel or linux-hotplug-devel.  And I also
just noticed the thread on lkml by Keiichiro, with comments from Dave,
mentioning Matthew, hinting at additional interactions that might be
desirable between various efforts here.  I even see some SGI work here.

Perhaps if I had been following this for a while, it would all make sense.
But I haven't been, and it doesn't.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Dave Hansen]

On Wed, 2004-05-12 at 20:51, Paul Jackson wrote:
> Could someone explain to me who's doing what with cpu, memory and node
> hotplug?
>
> And I also just noticed the thread on lkml by Keiichiro, with comments
> from Dave, mentioning Matthew, hinting at additional interactions that
> might be desirable between various efforts here.  I even see some SGI
> work here. 

I'm working on a little corner of memory hotplug: resizing existing
zones.  I'll be doing it on hardware that is both NUMA and flat SMP
systems, with no removal of entire NUMA nodes, at least not yet.  

We've decided to use CONFIG_NONLINEAR, which has been discussed quite a
bit before.  It allows you to take discontiguous, nonlinear physical
memory, and rearrange it so that it can be managed in linear pieces. 
Basically, this lets us keep the struct zone virtually unchanged because
we can map any physical memory to any pfn that we like (in certain size
sections, of course).  This is done for x86, and we're working on
ppc64.  This code has the potential to replace much of
CONFIG_DISCONTIGMEM.

The second bit of work that I have my eyes on are the sysfs interfaces
which will end up being the hotplug memory API to userspace.  I hope to
keep these as simple as possible inside the kernel, exposing enough
information to userspace that it can do most of the work.  For instance,
I certainly hope that an offline procedure for an entire node would
consist of more than one call into the kernel.  offline each memory
section ... offline each cpu ... offline the struct node ...

Although I haven't thought about it much, I would imagine that most of
the node hotplug work will involve very similar operations to the cpu
hotplug work: going through all of the for_each_node() loops and adding
some locking along with some node_possible() vs. node_online() checks.  

-- Dave



-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Ashok Raj]

On Wed, May 12, 2004 at 08:51:07PM -0700, Paul Jackson wrote:
> Could someone explain to me who's doing what with cpu, memory and node hotplug?
Paul back in action :-)
> 
> In particular, I haven't noticed much interaction between some cpu
> hotplug work that Ashok has been posting on linux-ia64 and/or lkml, and
> either of the email lists lhns-devel or linux-hotplug-devel.  And I also

You dont see any interaction between the cpu hotplug and the one tokunaga-san
is posting because they can be done independently. So here is the scope,
i will try to keep it short and sweet.!

CPU hotplug - Ashok(intel for IA64) Primarily does logical cpuonline/offline
	support to work with base support released by rusty from 2.6.5
	PPC64 - Joel Schoop, Vatsa (IBM), x86 (Rusty/Vatsa)
	ACPI support for cpu hotplug - Anil Keshavamurty (intel) base line 
	emulation and ACPI patches are ready, we are reviewing it internally 
	and it should make its way to acpi lists sometime in the next 2 weeks.
	(on major changes.. just extensions to acpi already in base kernel)
Memory hotplug - Dave Hansen et all (mostly IBM) + Matt Tolentino (Intel)
		Goto-san (fujitsu) + Takahashi/Iwamoto San (VA Linux)
		disscussed in lhms (sourceforgelist)
	Matt is also co-ordinating ACPI based memory hotplug support for 
	the work that he requires as necessary.

Node Hotplug support - Tokunaga-san (fujitsu) mostly concentrating on 
		ACPI based numa node hotplug. This work is independent
of rest of the hotplug (atleast mostly) and would invoke other module
hotplug code as required.  (roughly @10000 ft level)

Iam not sure what specifically you are looking for, please post your specific 
request to the appropriate lists and iam sure you will get hooked up.

lhcs - cpu hotplug list
lhms - memory hotplug
lhns - newly launched node hotplug

all of them hosted from sourceforge.

Hope this helps.
> just noticed the thread on lkml by Keiichiro, with comments from Dave,
> mentioning Matthew, hinting at additional interactions that might be
> desirable between various efforts here.  I even see some SGI work here.
> 
> Perhaps if I had been following this for a while, it would all make sense.
> But I haven't been, and it doesn't.


Cheers,
ashok


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Paul Jackson]

> Iam not sure what specifically you are looking for ...

I was looking for the short and sweet overview such as you just
presented ;).  Thanks.

I need to start following this hotplug work because it is next door to
the cpu/memory placement work that I am most focused on, such as
cpusets, led by Simon Derr of Bull, sitting on top of sched_setaffinity
and Andi Kleen's numa work.  A cpuset is a set of cpu and memory
resources that several tasks share, perhaps exclusively (all other
tasks kept out).

At some point, questions such as "what happens when a node is unplugged
from a cpuset" will need to be answered.  And even earlier, "what happens
when a cpu is unplugged that was in a tasks cpus_allowed, or a memory
node unplugged, that was in a vma's list of allowed memory zones."

Or perhaps the answers to these questions are already known ??

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Keiichiro Tokunaga]

Ashok Raj has already talked about LHNS and me :)  However,
I'd like to add something.

> Node Hotplug support - Tokunaga-san (fujitsu) mostly concentrating on 
> 		ACPI based numa node hotplug. This work is independent
> of rest of the hotplug (atleast mostly) and would invoke other module
> hotplug code as required.  (roughly @10000 ft level)

I'm rather focusing on ACPI based container device hotplug.
A scope of LHNS includes a container device that contains
IO devices.  I think such a devices couldn't be handled by NUMA
hotplug.

Thanks,
Kei


On Wed, 12 May 2004 20:51:07 -0700
Paul Jackson <pj@sgi.com> wrote:

> Could someone explain to me who's doing what with cpu, memory and node hotplug?
> 
> In particular, I haven't noticed much interaction between some cpu
> hotplug work that Ashok has been posting on linux-ia64 and/or lkml, and
> either of the email lists lhns-devel or linux-hotplug-devel.  And I also
> just noticed the thread on lkml by Keiichiro, with comments from Dave,
> mentioning Matthew, hinting at additional interactions that might be
> desirable between various efforts here.  I even see some SGI work here.
> 
> Perhaps if I had been following this for a while, it would all make sense.
> But I haven't been, and it doesn't.
> 
> -- 
>                           I won't rest till it's the best ...
>                           Programmer, Linux Scalability
>                           Paul Jackson <pj@sgi.com> 1.650.933.1373


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Dave Hansen]

On Wed, 2004-05-12 at 23:35, Keiichiro Tokunaga wrote:
> > Node Hotplug support - Tokunaga-san (fujitsu) mostly concentrating on 
> > 		ACPI based numa node hotplug. This work is independent
> > of rest of the hotplug (atleast mostly) and would invoke other module
> > hotplug code as required.  (roughly @10000 ft level)
> 
> I'm rather focusing on ACPI based container device hotplug.
> A scope of LHNS includes a container device that contains
> IO devices.  I think such a devices couldn't be handled by NUMA
> hotplug.

Why not?  We already export the information about which PCI busses are
attached to which NUMA nodes.  How are I/O devices different from CPU or
memory?  

BTW, what *is* NUMA hotplug? :)

-- Dave



-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Paul Jackson]

Ok - thanks for the additional explanation, sir.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Jack Steiner]

On Wed, May 12, 2004 at 09:23:35PM -0700, Dave Hansen wrote:
> On Wed, 2004-05-12 at 20:51, Paul Jackson wrote:
> > Could someone explain to me who's doing what with cpu, memory and node
> > hotplug?
> >
> > And I also just noticed the thread on lkml by Keiichiro, with comments
> > from Dave, mentioning Matthew, hinting at additional interactions that
> > might be desirable between various efforts here.  I even see some SGI
> > work here. 
> 
> I'm working on a little corner of memory hotplug: resizing existing
> zones.  I'll be doing it on hardware that is both NUMA and flat SMP
> systems, with no removal of entire NUMA nodes, at least not yet.  
> 
> We've decided to use CONFIG_NONLINEAR, which has been discussed quite a
> bit before.  It allows you to take discontiguous, nonlinear physical
> memory, and rearrange it so that it can be managed in linear pieces. 
> Basically, this lets us keep the struct zone virtually unchanged because
> we can map any physical memory to any pfn that we like (in certain size
> sections, of course).  This is done for x86, and we're working on
> ppc64.  This code has the potential to replace much of
> CONFIG_DISCONTIGMEM.

Where can I find a copy of the latest CONFIG_NONLINEAR patch? I recall
one that was posted by Dave McCracken in early Apr. Is that the one I should
review?

When we did the initial implementation of CONFIG_DISCONTIGMEM, we
looked briefly at the CONFIG_NONLINEAR patch (or idea) that was
floating around at that time. The patch may have changed so some of 
my initial concerns may no longer apply, but the early patch would
not have performed very well on the SGI hardware. 

The SGI architecture has an absurdly sparse address space. The smallest
memory block is 64MB but the max physical address is 49 bits (500TB).  
IIRC, this resulted in some very large tables used to convert between 
logical & physical addresses. Because of the size of these tables, cache
misses would be common on references to these tables. Is this still a 
valid concern???

Anyway, I'd like to study the latest patch & see how well it would work on our
hardware......


-- 
Thanks

Jack Steiner (steiner@sgi.com)          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.




-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Dave Hansen]

On Thu, 2004-05-13 at 07:00, Jack Steiner wrote:
> Where can I find a copy of the latest CONFIG_NONLINEAR patch? I recall
> one that was posted by Dave McCracken in early Apr. Is that the one I should
> review?

There are a few bugfixes on top of it since then, but most of it remains
untouched.  

> When we did the initial implementation of CONFIG_DISCONTIGMEM, we
> looked briefly at the CONFIG_NONLINEAR patch (or idea) that was
> floating around at that time. The patch may have changed so some of 
> my initial concerns may no longer apply, but the early patch would
> not have performed very well on the SGI hardware. 
> 
> The SGI architecture has an absurdly sparse address space. The smallest
> memory block is 64MB but the max physical address is 49 bits (500TB).  
> IIRC, this resulted in some very large tables used to convert between 
> logical & physical addresses. Because of the size of these tables, cache
> misses would be common on references to these tables. Is this still a 
> valid concern???

Our address space on ppc64 isn't that sparse, but we may be dealing with
memory sections that are as small as 16MB.  Our problem is that any of
those 16MB sections of memory is removable, which means that we are
going to get some sparse data structures if a machine isn't populated
with a significant portion of its capacity of RAM.

But, one cool/horrifying thing is that it appears that ppc64 is somewhat
dynamic about what these section sizes are.  On power4, they're fixed at
256MB, but on power5, they can go as small as 16MB, determined at boot. 
But, we don't really know what the size is going to be until we're
booted, and we run the same binaries everywhere.

We're probably going to need something other than statically allocated
nonlinear arrays eventually.  As you eluded to, there are 2 things that
combined will make nonlinear bad on the cache: sparse physical addresses
spaces and small memory sections.  However, small sections aren't really
all that common, especailly on big machines.  Most of the machines that
we deal with are going to be presenting their memory in pretty
contiguous regions.  If we can make nonlinear more dynamic, we could
shrink the arrays in all of the common cases, leaving only the
pathological memory removal, or machines that really *do* dole out
memory in 64MB sections, which are probably virutalized anyway. 

I don't doubt that your architecture allows memory addition in units as
small as 64MB, but what is it in practice?  

You could always sparsely allocate the nonlinear arrays, and handle
faults on them like you do mem_map ;)

-- Dave



-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Jack Steiner]

On Thu, May 13, 2004 at 09:01:48AM -0700, Dave Hansen wrote:
> On Thu, 2004-05-13 at 07:00, Jack Steiner wrote:
> > Where can I find a copy of the latest CONFIG_NONLINEAR patch? I recall
> > one that was posted by Dave McCracken in early Apr. Is that the one I should
> > review?
> 
> There are a few bugfixes on top of it since then, but most of it remains
> untouched.  

Ok, I grab that patch & take a look.


> 
> > When we did the initial implementation of CONFIG_DISCONTIGMEM, we
> > looked briefly at the CONFIG_NONLINEAR patch (or idea) that was
> > floating around at that time. The patch may have changed so some of 
> > my initial concerns may no longer apply, but the early patch would
> > not have performed very well on the SGI hardware. 
> > 
> > The SGI architecture has an absurdly sparse address space. The smallest
> > memory block is 64MB but the max physical address is 49 bits (500TB).  
> > IIRC, this resulted in some very large tables used to convert between 
> > logical & physical addresses. Because of the size of these tables, cache
> > misses would be common on references to these tables. Is this still a 
> > valid concern???
> 
> Our address space on ppc64 isn't that sparse, but we may be dealing with
> memory sections that are as small as 16MB.  Our problem is that any of
> those 16MB sections of memory is removable, which means that we are
> going to get some sparse data structures if a machine isn't populated
> with a significant portion of its capacity of RAM.
> 
> But, one cool/horrifying thing is that it appears that ppc64 is somewhat
> dynamic about what these section sizes are.  On power4, they're fixed at
> 256MB, but on power5, they can go as small as 16MB, determined at boot. 
> But, we don't really know what the size is going to be until we're
> booted, and we run the same binaries everywhere.
> 
> We're probably going to need something other than statically allocated
> nonlinear arrays eventually.  As you eluded to, there are 2 things that
> combined will make nonlinear bad on the cache: sparse physical addresses
> spaces and small memory sections.  However, small sections aren't really
> all that common, especailly on big machines.  Most of the machines that
> we deal with are going to be presenting their memory in pretty
> contiguous regions.  If we can make nonlinear more dynamic, we could
> shrink the arrays in all of the common cases, leaving only the
> pathological memory removal, or machines that really *do* dole out
> memory in 64MB sections, which are probably virutalized anyway. 
> 
> I don't doubt that your architecture allows memory addition in units as
> small as 64MB, but what is it in practice?  

It really is 64MB - at least that is the largest power of 2 that
evenly divides into the size of a chunk of memory.

We currently support 512MB, 1GB & 2GB DIMMs. Memory is populated in
multiples of 4 DIMMs. That means that memory will PHYSICALLY be
in chunks of 2GB, 4 GB or 8GB of contiguous memory.  The memory
on a node consists of up to 4 chunks of memory located at 
offset 0, 16GB, 32GB & 48GB of the nodes memory space. 

The memory directory (used for tracking coherency) is allocated within
each chunk of memory. The directory occupied the top 1/32 of the chunk.
This memory is not accessible to software - a bus error occurs if
software tries to access the directory. 1/32 of a 2GB chunk of memory
is where I got 64MB. Maybe there is another way to handle the
"missing" 64MB of a 2GB chunk of memory.

The other oddity of the SN architecture is that there is no memory
at physical address 0. Physical memory on node 0 starts at 384GB.



> 
> You could always sparsely allocate the nonlinear arrays, and handle
> faults on them like you do mem_map ;)
> 
> -- Dave

-- 
Thanks

Jack Steiner (steiner@sgi.com)          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.




-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Keiichiro Tokunaga]

On Wed, 12 May 2004 23:50:58 -0700
Dave Hansen <haveblue@us.ibm.com> wrote:

> On Wed, 2004-05-12 at 23:35, Keiichiro Tokunaga wrote:
> > > Node Hotplug support - Tokunaga-san (fujitsu) mostly concentrating on 
> > > 		ACPI based numa node hotplug. This work is independent
> > > of rest of the hotplug (atleast mostly) and would invoke other module
> > > hotplug code as required.  (roughly @10000 ft level)
> > 
> > I'm rather focusing on ACPI based container device hotplug.
> > A scope of LHNS includes a container device that contains
> > IO devices.  I think such a devices couldn't be handled by NUMA
> > hotplug.
> 
> Why not?  We already export the information about which PCI busses are
> attached to which NUMA nodes.  How are I/O devices different from CPU or
> memory?  

I have just one concern.
How about a container device that contains IO devices only?
  - Does it count as a NUMA node?
  - Is it having a NUMA node ID?

> BTW, what *is* NUMA hotplug? :)

I'm sorry, that would have to be "NUMA node hotplug"...

Thanks,
Kei


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Who's doing what with cpu/memory/node hotplug?

[Russ Anderson]

Keiichiro Tokunaga wrote:
> Dave Hansen <haveblue@us.ibm.com> wrote:
> > On Wed, 2004-05-12 at 23:35, Keiichiro Tokunaga wrote:
> > > 
> > > I'm rather focusing on ACPI based container device hotplug.
> > > A scope of LHNS includes a container device that contains
> > > IO devices.  I think such a devices couldn't be handled by NUMA
> > > hotplug.
> > 
> > Why not?  We already export the information about which PCI busses are
> > attached to which NUMA nodes.  How are I/O devices different from CPU or
> > memory?  
> 
> I have just one concern.
> How about a container device that contains IO devices only?
>   - Does it count as a NUMA node?
>   - Is it having a NUMA node ID?

Yes, it would count as NUMA node with a NUMA ID.

So a NUMA node could have CPUs, memory, and I/O,
where each of those may or may not be present.

There could be a node without any CPU, memory, or I/O.
An example would be an I/O only node with PCI devices
that are hot plug.  The PCI devices may or may not be
connected at any point in time.

And Yes, SGI does have I/O only nodes (and memory only nodes).

Thanks,
-- 
Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          rja@sgi.com


-------------------------------------------------------
This SF.Net email is sponsored by: SourceForge.net Broadband
Sign-up now for SourceForge Broadband and get the fastest
6.0/768 connection for only $19.95/mo for the first 3 months!
http://ads.osdn.com/?ad_id%62&alloc_ida84&op=click
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel