From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751844AbcELAG2 (ORCPT ); Wed, 11 May 2016 20:06:28 -0400 Received: from mail-bl2on0090.outbound.protection.outlook.com ([65.55.169.90]:7424 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751129AbcELAGY (ORCPT ); Wed, 11 May 2016 20:06:24 -0400 Authentication-Results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=caviumnetworks.com; Message-ID: <5733C8F5.6090206@caviumnetworks.com> Date: Wed, 11 May 2016 17:06:13 -0700 From: David Daney User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7 MIME-Version: 1.0 To: Catalin Marinas , "Rafael J. Wysocki" CC: David Daney , Will Deacon , , Mark Rutland , Tony Luck , Fenghua Yu , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , , Len Brown , Rob Herring , Frank Rowand , Grant Likely , Robert Moore , Lv Zheng , Hanjun Guo , Marc Zyngier , , , , Robert Richter , Ganapatrao Kulkarni , , David Daney , Jon Masters Subject: Re: [PATCH v6 13/14] arm64, acpi, numa: NUMA support based on SRAT and SLIT References: <1461780436-27182-1-git-send-email-ddaney.cavm@gmail.com> <1461780436-27182-14-git-send-email-ddaney.cavm@gmail.com> <20160511103929.GC3051@e104818-lin.cambridge.arm.com> In-Reply-To: <20160511103929.GC3051@e104818-lin.cambridge.arm.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [64.2.3.194] X-ClientProxiedBy: SN1PR0701CA0070.namprd07.prod.outlook.com (10.163.126.38) To CY1PR07MB2134.namprd07.prod.outlook.com (10.164.112.12) X-MS-Office365-Filtering-Correlation-Id: de0d407e-0784-43f8-6216-08d379f93f06 X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2134;2:uPXkfzxV0uEwNjfcOKf3Ns1lXZ/wgVOKsWqbh7yqnwdV8I+o5vZ09UA+XuXvuc0YUM1brd/vlsb2c8JA1dwdirsu2MitunLfjCMCBHsZgtziRhSVmlkc5ppRrL7VItUIUZqKe4k+U0TcFI6QIEvCcRFfZW0iGob1ZJEK4AJZ5k5QRzIbOk0chlafIPJZD/Ah;3:2fyp2cmS2lPnw5cCabhukH3+wwDCKFpo5eHi99kO9XL7HojARHP+wAzKGoP+ttByfXOE1K47Y6PV0LssD94J7Bxg/JdCOxtJ9R536xPS6apGOmfctLeRlwnUL5T8y0/2;25:UTRdPFVaMcgmG8yghZU2mqlXdhj9H+1nxHZEsFvLEGvSPovctZ0DeOdqWYmqVJpFv+wsoaqpSWsRYa6B0Gg6o9qJcRI1EUksWYubeB1D1EBcsJJyWqscb62vt8KSaNodfKxym1rn5XYTT11eyD9P4Q0CJFU46LZuYNdIeH6xDS1BuTp9cKDIyBJ48U6xCmmW6RM3jmzWwpFWX8UC6PfAQ+DbdIll1ZnxHAzVHXTL30ru7htuHwNodnxqMYy7toOUUruuVIgLXUDAM1zYJFguT/zVhJRRJX4Inqw5+ZlDTImItVRQTT3biDQ4dzgyv29ZSPqCPpUwUW1NwUEw9JWGdQel+ZX58TH96lgx96tVovQNgJLuDHXxk/kzVDRFr230Eicfuuby1PH+WbW0djpvJn8zo1tIY4mKMbb1uCCExxw= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2134; X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2134;20:QueIxhuDBVOyaQnFVOq5UOiOyJSGf2y9S/7V7LkKq32J4DXZbGqK66DauXA/LdCx9V88iFVlKHc3eBVN+GW+FGc/pEk441EvFI6cPFdi63Rzcwqgb1lSiOVYfc1RTe/rVG7A8n+i2voSUatg+Ihg0DTbENksPOv23LPdDYCH6gz7IK0iwsmbdIny0WS5Z3EXCcvQvrqR/788K+hG30oHqUZ6u1mZIPcUf3yzgi2DDOaiFdGdDNfMcT0Jow1HTNnUJp+P0ychqdlYePOGmI4TTsK3EfcSnQvxCfxiqYee61aeOVFv2Pud4gp8vaBtmnZYQxOat05Kd4YWJn9D7OJdbDBOhYhwcf4RdnODZ6M0Smb4oSV2H3SwM8Gq/5koB23ig1fvpLdoXdEfh6b28MSYyQODVD3M+r4m8GAoN8I34nDB2c+16CWM42AjPTMw1KnPf0myfjZBIylMLRxcY1mMsaQoFqHUGRfe09U3HIxuN6fiU+To3eCGVoRCZv2kMN0cARoJxks5IErIgbgSDN13resANHl+/R0tRGvCD1hOQcVMAPVK2x+sa/d21uGeVDo1JHWXK7QfLaR+UFYMZLV+MA7Y8q5OJKhIwxKnCwY3vis= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:CY1PR07MB2134;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2134; X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2134;4:uMqiWsJMdqleCVYMadORoJ1N4CMVJwbGtJqtFRtPlq09hUCsLZWCriQY2AzenrHEo3TgVLwkxSquZcRXxZ617CUMstCOatGE1Y0486/fRpD/dCRASpU87/tpaSXylv3T0nicLfrdFyDeQQPYwx7Tsq/bWMf1SaewY/OtbedfoV5k/t3Gbhfu5vywrB+LDhuNbB+SVw2rPAkEQKslffBscr3cTS5nEYB3ieFzLp0Og85YdVYh/Mhsdn7gPm7J3nrIIv2tXVZctLdZAzbkOKe1vA7XnBmLpeTlJpcXlqiL8FwS6AfQ12n14jPf2C5OcHzuKhWS8y0qpqAd9Xq5E1CC7fqrGenpXHLgzBQB/bPUysDKu8mgP1LlTguUPyydymwp X-Forefront-PRVS: 0940A19703 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(24454002)(377454003)(586003)(3846002)(6116002)(23756003)(47776003)(65956001)(33656002)(2950100001)(65816999)(87266999)(50986999)(54356999)(76176999)(4326007)(2906002)(92566002)(59896002)(81166006)(5004730100002)(5008740100001)(4001350100001)(230700001)(66066001)(189998001)(5001770100001)(53416004)(36756003)(42186005)(50466002)(77096005)(83506001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR07MB2134;H:dl.caveonetworks.com;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;CY1PR07MB2134;23:IaF1jsa28EiEACoIcm7UJv5BfFC0BXXjEGTKXPU?= =?iso-8859-1?Q?jhSoTc/xYZN+dvPLcPeG4Gbe9Gmzuko0NlUZcyLEuNxgUgiE0XblIr0f3G?= =?iso-8859-1?Q?CAw8wneHoh0aoEVPP/Qr7aSaFczEwvJPuEMhvKuJd9mxjMmwhZ1Ror4rPc?= =?iso-8859-1?Q?GAet+bxIWfIL9KpGiA2iARqp0skPSDgQa5fqk89EyRLQQ0XPsDQVPH1Oy1?= =?iso-8859-1?Q?NIe/pR71iTqYQ9AVLqFrooyFihyXXkkLXU7JN2FTgHDLfL4TzjOnpqcxbd?= =?iso-8859-1?Q?halEJ3gPvyUH6BYVgmIn74Wai00I2PLJV2CM01oBiNssMFDe/agSXPTkU+?= =?iso-8859-1?Q?+bFWOf05My3eJt/Ezosza7zayoy0p/2r6uC1eSI5DGUtwH/mr6Bj3tpHpJ?= =?iso-8859-1?Q?QLhUlkSQ42OA3fmhfvgokzqs/CZTmAuBNRUPg+smV9m6YWMZpeZfa1ECq2?= =?iso-8859-1?Q?TT+8cBP/kCY5D7qSk9jEHm1BpOI0fIjpjka89S8nG236zKyhl1X6sRxJpa?= =?iso-8859-1?Q?y/yhl5WIOklsAe5T/Q75GvIEtoAizxh39Fa4dplB8KGwYkCY1XbFE2BeZV?= =?iso-8859-1?Q?xgOMeTOpXNStdff4906d4SYlfjjtxNnAdsn7gd7tuNa/Gk3PIJ0StP9V9K?= =?iso-8859-1?Q?RHBBFqb1L+meCD8SHDQ5I3DO3Nt4P7zFWd5xd2R1/6IlHJL/OGuH7I3Y8t?= =?iso-8859-1?Q?IhNOE9jgnzuIKXE76Kkr/5pQeHkNy6l61aThUj06dPLwz0tKQUMgiX/IUP?= =?iso-8859-1?Q?KIxNjb8LuFgTRG059uqLBkzU+QZQX9CBF1WTRXAeCT2eExAfg0nMsNpiMl?= =?iso-8859-1?Q?atb2aZQWcwRqyJyq+0WGevgicvA9fBoig+i5Ntlmu2P3gLYU6iXSxX9GcS?= =?iso-8859-1?Q?D74EpIqMKbXxQ3zuK8N1Uu448NlmXrH/8/Cw/cGKiLZzwO746o1G6ihyCp?= =?iso-8859-1?Q?ah8a9FRNWAHeBFYDXLiezH8IMYMRYEgVApM2uk7Pjo5h/oBzpSr3dTscKV?= =?iso-8859-1?Q?EqR4rJRk+DVy7GQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2134;5:hFQ35KHOsap3BLUrAA3cjoG1vsDeS2Jyf1f54WOwmZ4skQJBk4aoDMG7XvQCf8nLbvUNeN2sBePAtw5SPTp6bD5s4yYZhCUGaCt5Ns5L8izRjFGOZHDQGE5uN24zfKIkIoB0NOnnWJyFmu8jhvTgbw==;24:7VkOK8Am4On0eQuJBdQqkNemWVsQ3bPkRyaD514D5AL4cUNOR1ToNCRaJeCtFUSJrznPYDNkqDycJARyFuInX0quf+czJ1HrbkDNu3xfueI=;7:z0hmEmzTLzdgn5YzDP1W8v0Xe3gFdZ7RWwI7wZhxqjxmStLRMJlj+LMY2fL1+Ir5C43LDXGtquh8iUPOGzkfisNACT4Tp61nGla6EQK1yZMFySke5VycrG1e9k2mKEpNrZxvohd32Da+jck9OUfRgJjdi5uEy8jNZQTOrUY0KwnDyeUUbgqUcZ35BWT8kyiv SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2016 00:06:17.8954 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2134 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/11/2016 03:39 AM, Catalin Marinas wrote: > On Wed, Apr 27, 2016 at 11:07:15AM -0700, David Daney wrote: >> +static int __init get_mpidr_in_madt(int acpi_id, u64 *mpidr) >> +{ >> + unsigned long madt_end, entry; >> + struct acpi_table_madt *madt; >> + acpi_size tbl_size; >> + >> + if (ACPI_FAILURE(acpi_get_table_with_size(ACPI_SIG_MADT, 0, >> + (struct acpi_table_header **)&madt, &tbl_size))) >> + return -ENODEV; >> + >> + entry = (unsigned long)madt; >> + madt_end = entry + madt->header.length; >> + >> + /* Parse all entries looking for a match. */ >> + entry += sizeof(struct acpi_table_madt); >> + while (entry + sizeof(struct acpi_subtable_header) < madt_end) { >> + struct acpi_subtable_header *header = >> + (struct acpi_subtable_header *)entry; >> + >> + if (header->type == ACPI_MADT_TYPE_GENERIC_INTERRUPT) { >> + struct acpi_madt_generic_interrupt *gicc = >> + container_of(header, >> + struct acpi_madt_generic_interrupt, header); >> + >> + if ((gicc->flags & ACPI_MADT_ENABLED) && >> + (gicc->uid == acpi_id)) { >> + *mpidr = gicc->arm_mpidr; >> + early_acpi_os_unmap_memory(madt, tbl_size); >> + return 0; >> + } >> + } >> + entry += header->length; >> + } >> + >> + early_acpi_os_unmap_memory(madt, tbl_size); >> + return -ENODEV; >> +} >> + >> +/* Callback for Proximity Domain -> ACPI processor UID mapping */ >> +void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) >> +{ >> + int pxm, node; >> + u64 mpidr; >> + >> + if (srat_disabled()) >> + return; >> + >> + if (pa->header.length < sizeof(struct acpi_srat_gicc_affinity)) { >> + pr_err("SRAT: Invalid SRAT header length: %d\n", >> + pa->header.length); >> + bad_srat(); >> + return; >> + } >> + >> + if (!(pa->flags & ACPI_SRAT_GICC_ENABLED)) >> + return; >> + >> + if (cpus_in_srat >= NR_CPUS) { >> + pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n", >> + NR_CPUS); >> + return; >> + } >> + >> + pxm = pa->proximity_domain; >> + node = acpi_map_pxm_to_node(pxm); >> + >> + if (node == NUMA_NO_NODE || node >= MAX_NUMNODES) { >> + pr_err("SRAT: Too many proximity domains %d\n", pxm); >> + bad_srat(); >> + return; >> + } >> + >> + if (get_mpidr_in_madt(pa->acpi_processor_uid, &mpidr)) { >> + pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n", >> + pxm, pa->acpi_processor_uid); >> + bad_srat(); >> + return; >> + } > > I wonder whether you could replace the get_mpidr_in_madt() function with > something like acpi_get_phys_id(). It looks like get_mpidr_in_madt() > duplicates functionality already available elsewhere. > I just tried that, and it doesn't work. The problem is that this code is being run very early in the boot, and kmalloc cannot be used. acpi_get_phys_id() and its ilk can only be used once we have working kmalloc. We need to extract the NUMA information early like this precisely because it is needed to initializing the slab system Notice that we are using early_acpi_os_unmap_memory() et al. in get_mpidr_in_madt() explicitly for this reason. In summary: I don't think we need another revision of this patch, it is like this for a good reason. David Daney