From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f47.google.com (mail-qg0-f47.google.com [209.85.192.47]) by kanga.kvack.org (Postfix) with ESMTP id 9E6FE6B0005 for ; Tue, 12 Apr 2016 06:02:25 -0400 (EDT) Received: by mail-qg0-f47.google.com with SMTP id j35so11196359qge.0 for ; Tue, 12 Apr 2016 03:02:25 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id x67si7581987qhe.81.2016.04.12.03.02.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2016 03:02:24 -0700 (PDT) Date: Tue, 12 Apr 2016 12:02:15 +0200 From: Jesper Dangaard Brouer Subject: [LSF/MM TOPIC] Ideas for SLUB allocator Message-ID: <20160412120215.000283c7@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: lsf@lists.linux-foundation.org, linux-mm , Rik van Riel , Christoph Lameter , Joonsoo Kim , Pekka Enberg , David Rientjes , Andrew Morton , js1304@gmail.com Cc: lsf-pc@lists.linux-foundation.org Hi Rik, I have another topic, which is very MM-specific. I have some ideas for improving SLUB allocator further, after my work on implementing the slab bulk APIs. Maybe you can give me a small slot, I only have 7 guidance slides. Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved in slab development... I've already published the preliminary slides here: http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f179.google.com (mail-pf0-f179.google.com [209.85.192.179]) by kanga.kvack.org (Postfix) with ESMTP id B82626B025F for ; Tue, 12 Apr 2016 09:37:31 -0400 (EDT) Received: by mail-pf0-f179.google.com with SMTP id e128so14068860pfe.3 for ; Tue, 12 Apr 2016 06:37:31 -0700 (PDT) Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTP id q75si10039136pfq.207.2016.04.12.06.37.30 for ; Tue, 12 Apr 2016 06:37:31 -0700 (PDT) Date: Tue, 12 Apr 2016 09:37:28 -0400 From: Matthew Wilcox Subject: Re: [Lsf] [LSF/MM TOPIC] Ideas for SLUB allocator Message-ID: <20160412133728.GM2781@linux.intel.com> References: <20160412120215.000283c7@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160412120215.000283c7@redhat.com> Sender: owner-linux-mm@kvack.org List-ID: To: Jesper Dangaard Brouer Cc: lsf@lists.linux-foundation.org, linux-mm , Rik van Riel , Christoph Lameter , Joonsoo Kim , Pekka Enberg , David Rientjes , Andrew Morton , js1304@gmail.com, lsf-pc@lists.linux-foundation.org On Tue, Apr 12, 2016 at 12:02:15PM +0200, Jesper Dangaard Brouer wrote: > Hi Rik, > > I have another topic, which is very MM-specific. > > I have some ideas for improving SLUB allocator further, after my work > on implementing the slab bulk APIs. Maybe you can give me a small > slot, I only have 7 guidance slides. Or else I hope we/I can talk > about these ideas in a hallway track with Christoph and others involved > in slab development... > > I've already published the preliminary slides here: > http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp The current bulk API returns the pointers in an array. What the radix tree would like is the ability to bulk allocate from a slab and chain the allocations through an offset. See __radix_tree_preload() in lib/radix-tree.c. I don't know if this is a common thing to do elsewhere in the kernel. Obviously, radix-tree could allocate the array on the stack and set up the chain itself, but I would think it would be just as easy for slab to do it itself and save the stack space. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f53.google.com (mail-qg0-f53.google.com [209.85.192.53]) by kanga.kvack.org (Postfix) with ESMTP id 430CD6B0005 for ; Tue, 12 Apr 2016 11:03:58 -0400 (EDT) Received: by mail-qg0-f53.google.com with SMTP id c6so18127684qga.1 for ; Tue, 12 Apr 2016 08:03:58 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id j138si24678719qhc.103.2016.04.12.08.03.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2016 08:03:57 -0700 (PDT) Date: Tue, 12 Apr 2016 17:03:47 +0200 From: Jesper Dangaard Brouer Subject: Re: [Lsf] [LSF/MM TOPIC] Ideas for SLUB allocator Message-ID: <20160412170347.4e21f5d3@redhat.com> In-Reply-To: <20160412133728.GM2781@linux.intel.com> References: <20160412120215.000283c7@redhat.com> <20160412133728.GM2781@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Matthew Wilcox Cc: lsf@lists.linux-foundation.org, linux-mm , Rik van Riel , Christoph Lameter , Joonsoo Kim , Pekka Enberg , David Rientjes , Andrew Morton , js1304@gmail.com, lsf-pc@lists.linux-foundation.org On Tue, 12 Apr 2016 09:37:28 -0400 Matthew Wilcox wrote: > On Tue, Apr 12, 2016 at 12:02:15PM +0200, Jesper Dangaard Brouer wrote: > > Hi Rik, > > > > I have another topic, which is very MM-specific. > > > > I have some ideas for improving SLUB allocator further, after my work > > on implementing the slab bulk APIs. Maybe you can give me a small > > slot, I only have 7 guidance slides. Or else I hope we/I can talk > > about these ideas in a hallway track with Christoph and others involved > > in slab development... > > > > I've already published the preliminary slides here: > > http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp > > The current bulk API returns the pointers in an array. What the > radix tree would like is the ability to bulk allocate from a slab and > chain the allocations through an offset. See __radix_tree_preload() > in lib/radix-tree.c. I don't know if this is a common thing to do > elsewhere in the kernel. Obviously, radix-tree could allocate the array > on the stack and set up the chain itself, but I would think it would be > just as easy for slab to do it itself and save the stack space. It does look like a good candidate for bulk alloc in __radix_tree_preload(). Especially because you have an annoying preempt_disable() and preempt_enable() interaction loop, and reloading of this_cpu_ptr. And RADIX_TREE_PRELOAD_SIZE==21 is not excessive alloc bulking. Considering local_irq's are disabled during the bulk alloc. The allocator is delivering "raw" memory, thus it does not know anything about the callers data structure, and shouldn't. (I have considered delivering bulk objects single linked via offset 0, as this already happens internally in SLUB, but I decided against it) Looking closer at your specific data structures, they are also a bit complicated to deliver "linked" easily... struct radix_tree_preload { int nr; /* nodes->private_data points to next preallocated node */ struct radix_tree_node *nodes; }; struct radix_tree_node { unsigned int path; /* Offset in parent & height from the bottom */ unsigned int count; union { struct { /* Used when ascending tree */ struct radix_tree_node *parent; /* For tree user */ void *private_data; }; /* Used when freeing node */ struct rcu_head rcu_head; }; /* For tree user */ struct list_head private_list; void __rcu *slots[RADIX_TREE_MAP_SIZE]; unsigned long tags[RADIX_TREE_MAX_TAGS][RADIX_TREE_TAG_LONGS]; }; -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f174.google.com (mail-io0-f174.google.com [209.85.223.174]) by kanga.kvack.org (Postfix) with ESMTP id 10BA06B0005 for ; Tue, 12 Apr 2016 12:01:45 -0400 (EDT) Received: by mail-io0-f174.google.com with SMTP id u185so34018552iod.3 for ; Tue, 12 Apr 2016 09:01:45 -0700 (PDT) Received: from resqmta-ch2-01v.sys.comcast.net (resqmta-ch2-01v.sys.comcast.net. [2001:558:fe21:29:69:252:207:33]) by mx.google.com with ESMTPS id 68si22799258iow.126.2016.04.12.09.01.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2016 09:01:44 -0700 (PDT) Date: Tue, 12 Apr 2016 11:01:42 -0500 (CDT) From: Christoph Lameter Subject: Re: [LSF/MM TOPIC] Ideas for SLUB allocator In-Reply-To: <20160412120215.000283c7@redhat.com> Message-ID: References: <20160412120215.000283c7@redhat.com> Content-Type: text/plain; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Jesper Dangaard Brouer Cc: lsf@lists.linux-foundation.org, linux-mm , Rik van Riel , Joonsoo Kim , Pekka Enberg , David Rientjes , Andrew Morton , js1304@gmail.com, lsf-pc@lists.linux-foundation.org On Tue, 12 Apr 2016, Jesper Dangaard Brouer wrote: > I have some ideas for improving SLUB allocator further, after my work > on implementing the slab bulk APIs. Maybe you can give me a small > slot, I only have 7 guidance slides. Or else I hope we/I can talk > about these ideas in a hallway track with Christoph and others involved > in slab development... I will be there. > I've already published the preliminary slides here: > http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp Re Autotuning: SLUB obj per page: SLUB can combine pages of different orders in a slab cache so this would be possible. per CPU freelist per page: Could we drop the per cpu partial lists if this works? Clearing memory: Could exploit the fact that the page is zero on alloc and also zap when no object in the page is in use? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f44.google.com (mail-qg0-f44.google.com [209.85.192.44]) by kanga.kvack.org (Postfix) with ESMTP id 88E156B0005 for ; Tue, 12 Apr 2016 14:13:53 -0400 (EDT) Received: by mail-qg0-f44.google.com with SMTP id j35so23144658qge.0 for ; Tue, 12 Apr 2016 11:13:53 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a4si25407428qga.6.2016.04.12.11.13.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2016 11:13:52 -0700 (PDT) Message-ID: <1460484828.7134.4.camel@redhat.com> Subject: Re: [LSF/MM TOPIC] Ideas for SLUB allocator From: Rik van Riel Date: Tue, 12 Apr 2016 14:13:48 -0400 In-Reply-To: References: <20160412120215.000283c7@redhat.com> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-Nsgc35t3Flj6nAMjns6c" Mime-Version: 1.0 Sender: owner-linux-mm@kvack.org List-ID: To: Christoph Lameter , Jesper Dangaard Brouer Cc: lsf@lists.linux-foundation.org, linux-mm , Joonsoo Kim , Pekka Enberg , David Rientjes , Andrew Morton , js1304@gmail.com, lsf-pc@lists.linux-foundation.org --=-Nsgc35t3Flj6nAMjns6c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2016-04-12 at 11:01 -0500, Christoph Lameter wrote: > On Tue, 12 Apr 2016, Jesper Dangaard Brouer wrote: >=20 > > I have some ideas for improving SLUB allocator further, after my > > work > > on implementing the slab bulk APIs.=C2=A0=C2=A0Maybe you can give me a = small > > slot, I only have 7 guidance slides.=C2=A0=C2=A0Or else I hope we/I can= talk > > about these ideas in a hallway track with Christoph and others > > involved > > in slab development... >=20 > I will be there. >=20 > > I've already published the preliminary slides here: > > =C2=A0http://people.netfilter.org/hawk/presentations/MM-summit2016/slab= _ > > mm_summit2016.odp >=20 > Re Autotuning: SLUB obj per page: > SLUB can combine pages of different orders in a slab cache so > this would > be possible. >=20 > per CPU freelist per page: > Could we drop the per cpu partial lists if this works? >=20 > Clearing memory: > Could exploit the fact that the page is zero on alloc and also > zap > when no object in the page is in use? Between the SLUB things both of you want to discuss, do you think one 30 minute slot will be enough to start with, or should we schedule a whole hour? We have some free slots left on the second day, where discussions can overflow if necessary. --=20 All rights reversed --=-Nsgc35t3Flj6nAMjns6c Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAABAgAGBQJXDTrdAAoJEM553pKExN6Den8H/3fhk4JithGqhJr9atIlw6FW KUux8bwgePE37BTaflnPVk4t8nHBEwcOmf41De5VlAic37Oo7dWnkoRsebLWIojz y9GBoCeSAsn/4IpgBAXCkcLp+rGoPvT2XBXDI8QmN3D80Od79dUdSSHng3xxix8X JczqimYVqtumsXmC/+Z8S0QJblK5ZbzkFoeXbjXz3FJumZcssh49cZgWlzOgHt0g 50QCu2+AS0cMb7zavWqxM1P65szW6SvvQebrxGtTaW82qlzhKhT99xAMl4YTdV7X 7uftQHXSea+Hq5myi+ZYwGThOwyfURWTzc0M9msJRssNcXMZU3r/YhV5yI8iASw= =bp3c -----END PGP SIGNATURE----- --=-Nsgc35t3Flj6nAMjns6c-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f47.google.com (mail-qg0-f47.google.com [209.85.192.47]) by kanga.kvack.org (Postfix) with ESMTP id CC3AD6B0005 for ; Tue, 12 Apr 2016 17:14:42 -0400 (EDT) Received: by mail-qg0-f47.google.com with SMTP id f52so27464880qga.3 for ; Tue, 12 Apr 2016 14:14:42 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id b32si17484979qgb.109.2016.04.12.14.14.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Apr 2016 14:14:42 -0700 (PDT) Date: Tue, 12 Apr 2016 23:14:35 +0200 From: Jesper Dangaard Brouer Subject: Re: [Lsf] [LSF/MM TOPIC] Ideas for SLUB allocator Message-ID: <20160412231435.3cbf3aeb@redhat.com> In-Reply-To: <1460484828.7134.4.camel@redhat.com> References: <20160412120215.000283c7@redhat.com> <1460484828.7134.4.camel@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: To: Rik van Riel Cc: brouer@redhat.com, Christoph Lameter , Pekka Enberg , js1304@gmail.com, lsf@lists.linux-foundation.org, linux-mm , David Rientjes , Joonsoo Kim , lsf-pc@lists.linux-foundation.org, Andrew Morton On Tue, 12 Apr 2016 14:13:48 -0400 Rik van Riel wrote: > On Tue, 2016-04-12 at 11:01 -0500, Christoph Lameter wrote: > > On Tue, 12 Apr 2016, Jesper Dangaard Brouer wrote: > > =20 > > > I have some ideas for improving SLUB allocator further, after my > > > work > > > on implementing the slab bulk APIs.=C2=A0=C2=A0Maybe you can give me = a small > > > slot, I only have 7 guidance slides.=C2=A0=C2=A0Or else I hope we/I c= an talk > > > about these ideas in a hallway track with Christoph and others > > > involved > > > in slab development... =20 > >=20 > > I will be there. > > =20 > > > I've already published the preliminary slides here: > > > =C2=A0http://people.netfilter.org/hawk/presentations/MM-summit2016/sl= ab_ > > > mm_summit2016.odp =20 > >=20 > > Re Autotuning: SLUB obj per page: > > SLUB can combine pages of different orders in a slab cache so > > this would > > be possible. > >=20 > > per CPU freelist per page: > > Could we drop the per cpu partial lists if this works? > >=20 > > Clearing memory: > > Could exploit the fact that the page is zero on alloc and also > > zap > > when no object in the page is in use? =20 >=20 > Between the SLUB things both of you want to > discuss, do you think one 30 minute slot will > be enough to start with, or should we schedule > a whole hour? >=20 > We have some free slots left on the second day, > where discussions can overflow if necessary. 30 min slot is fine by me :-) --=20 Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org