From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40992C43387 for ; Sun, 16 Dec 2018 19:47:08 +0000 (UTC) Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E4BE72082F for ; Sun, 16 Dec 2018 19:47:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="QPrOJ2CM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E4BE72082F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=fail smtp.mailfrom=kernelnewbies-bounces@kernelnewbies.org Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.91) (envelope-from ) id 1gYcNL-0002Bf-O3; Sun, 16 Dec 2018 14:46:55 -0500 Received: from mail-qt1-x841.google.com ([2607:f8b0:4864:20::841]) by shelob.surriel.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1gYcNJ-0002BZ-LK for kernelnewbies@kernelnewbies.org; Sun, 16 Dec 2018 14:46:53 -0500 Received: by mail-qt1-x841.google.com with SMTP id r14so11953014qtp.1 for ; Sun, 16 Dec 2018 11:46:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=date:from:to:subject:message-id:mail-followup-to:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=UdYVx5GGtu9r8tifhjI5FSvucXiHocbZJsPJay6zmRo=; b=QPrOJ2CMw4KLEZUKOz+KHEE0XJ8WIfoi+s80wQlgaCQsi68ZVxeH9lBOzblQEIcS+b uWQuqemyR+qYhk/97rjr1IeobQmVyO7imeVsqz9cR4kKelFSnEAJQZ7Oh3WZwctdlJmL P8Q0wmDVFVBMTc6qNejQVwwzDmGpb5GgIqtBA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=UdYVx5GGtu9r8tifhjI5FSvucXiHocbZJsPJay6zmRo=; b=SeS2v8l5hpTP3ww/Mo/qGA7NI0KOqY6GeApQbRn3nSnAO7CSw+3sCKEjlEE87j0JBD FN1rbxr4CH6srsj+JgLrHhB2mrSZkZECm2F9jg0emuFJEPqfos4nn8Ms0qlftUcWhGGx rc8CeOjNWBfB+1VYPf9J+4PwPCjmhX7bnOrOcBL6+yqHRrFbj2cakbIBI2cBre/mx+2G yg1FvN7PDmWsww1+hJ0gGuEUyrfdtCHDTUwwOV47KnrFBRrd+f46P80zAXRwlaEr4T6T 8ZIFu/TaC8XKBJviXbf38JG7TN90hHrLFPukhAn4XwfORZMp+YLMTY7KNvP9qR7BaVga Fy+A== X-Gm-Message-State: AA+aEWY7HRYlWRZX5G7N0kMP3celT88ITw1rxZFpo/F+xigYbcfjntEr uh3mF2stICaN0LAwaCogtZIW/Q== X-Google-Smtp-Source: AFSGD/W7PoX6zk8G11BJN61HtUd4XAloiIQ5pkJbv44mWeyVyy0tMPVr9cM59qKDlLe3rtT5yLqOuA== X-Received: by 2002:aed:3a22:: with SMTP id n31mr10901212qte.29.1544989612318; Sun, 16 Dec 2018 11:46:52 -0800 (PST) Received: from pure.paranoia.local ([198.144.156.49]) by smtp.gmail.com with ESMTPSA id h31sm2014690qtk.5.2018.12.16.11.46.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 16 Dec 2018 11:46:51 -0800 (PST) Date: Sun, 16 Dec 2018 14:46:49 -0500 From: Konstantin Ryabitsev To: Joey Pabalinas , Linux Kernel Mailing List , kernelnewbies@kernelnewbies.org, Linus Torvalds , Greg Kroah-Hartman Subject: Re: [RFC] LKML Archive in Maildir Format Message-ID: <20181216194649.GA7732@pure.paranoia.local> Mail-Followup-To: Joey Pabalinas , Linux Kernel Mailing List , kernelnewbies@kernelnewbies.org, Linus Torvalds , Greg Kroah-Hartman References: <20181216190639.6safwjqwdphkce67@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20181216190639.6safwjqwdphkce67@gmail.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kernelnewbies-bounces@kernelnewbies.org On Sun, Dec 16, 2018 at 09:06:39AM -1000, Joey Pabalinas wrote: > I spent a lot of time trying to find an LKML archive in Maildir format > that I could use for local searches with nutmuch or something, but all > the links I was able to find were all dead. > > I ended up just compiling one myself and I currently host it at: > > https://alyptik.org/lkml.tar.xz You seem to have duplicated a lot of effort that has already been done to compile the archive on lore.kernel.org. > It's possible I'm the only weirdo who finds this kind of thing useful, but > I figured I should share it just in case I'm not. The maildir format is kind of terrible for LKML, because having millions of messages in a single directory is very hard on the underlying FS. If you break it up into multiple folders, then it becomes difficult to search. This is the main reason why we have chosen to go with the public-inbox format, which solves both of these problems and allows for a very efficient archive updating and replication using git. > It's about 1.1 million files, I was wondering if anyone had an idea of a > better way to host this? I've tried Github and GitLab, but they don't > appreciate repos with that many files, hah. Like I said, you seem to be going down the road we've already tried and rejected. :) -K _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies