From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pete Zaitcev Subject: Design challenges in chunkd self-checking Date: Tue, 22 Dec 2009 14:41:11 -0700 Message-ID: <20091222144111.789a5b91@redhat.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: Sender: hail-devel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii" To: Project Hail List I'm looking into adding self-checking to chunkd. This involves basically a process that re-reads everything stored in the chunkserver and verifies that it's still ok. Nothing can be simpler, right? So, current problems for which I'd like input are: - Scheduling and deconflicting with normal operation. Run "genisofs" in your Fedora desktop and your Firefox is DEAD. It is also the reason why everyone does rpm -e mlocate the first thing after the installation. The effect of massive data access blowing away caches is very drastic in a regular Linux. So, I have to have a good way to keep self-checkig from interfering with normal service of a chunkserver. Also, need to save power instead of burning it on re-reading data. - Consistency. Returning wrong checksums for an object that is being updated may lead to us deciding to drop a perfectly good object, which is unacceptable (especially when redundancy is impaired already). So, I need some kind of locking, or logging, or invalidation... -- Pete