All of lore.kernel.org
 help / color / mirror / Atom feed
From: fitzboy <fitzboy@iparadigms.com>
To: Avi Kivity <avi@argo.co.il>
Cc: linux-kernel@vger.kernel.org
Subject: Re: tuning for large files in xfs
Date: Mon, 22 May 2006 17:40:07 -0700	[thread overview]
Message-ID: <447259E7.8050706@iparadigms.com> (raw)
In-Reply-To: <447212B5.1010208@argo.co.il>

 >
 > What's your testing methodology?
 >

here is my code... pretty simple, opens the file and reads around to a 
random block, 32k worth (note, doing a 2k read doesn't make much of a 
difference, only marginal, from 6.7ms per seek down to 6.1).

int main (int argc, char* argv[]) {
   char buffer[256*1024];
   int fd = open(argv[1],O_RDONLY);
   struct stat s;
   fstat(fd,&s);
   s.st_size=s.st_size-(s.st_size%256*1024);
   initTimers();
   srandom(startSec);
   long long currentPos;
   int currentSize=32*1024;
   int startTime=getTimeMS();
   for (int currentRead = 0;currentRead<10000;currentRead++) {
     currentPos=random();
     currentPos=currentPos*currentSize;
     currentPos=currentPos%s.st_size;
     if (pread(fd,buffer,currentSize,currentPos) != currentSize)
     std::cout<<"couldn't read in"<<std::endl;
   }
   std::cout << "blocksize of 
"<<currentSize<<"took"<<getTimeMS()-startTime<<" ms"<<std::endl;
}

 > You can try to measure the amount of seeks going to the disk by using
 > iostat, and see if that matches your test program.
 >

I used iostat and found exactly what I was expecting: 10,000 rounds x 16 
   (number of 2k blocks in a 32k read) x 4 (number of 512 blocks per 2k 
block) = 640,000 reads, which is what iostat told me. so now my question 
remains, if each seek is supposed to average 3.5ms, how come I am seeing 
an average of 6-7ms?


  reply	other threads:[~2006-05-23  0:40 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2006-05-22 18:57 tuning for large files in xfs fitzboy
2006-05-22 19:15 ` Avi Kivity
2006-05-22 19:32   ` fitzboy
2006-05-22 19:36     ` Avi Kivity
2006-05-23  0:40       ` fitzboy [this message]
2006-05-23  8:05         ` Avi Kivity
2006-05-23 20:21           ` fitzboy
2006-05-24  8:12             ` Avi Kivity
2006-05-22 22:22 ` Miquel van Smoorenburg
2006-05-22 22:30 ` Matheus Izvekov
2006-05-23  0:42   ` fitzboy
2006-05-23  1:07     ` Matheus Izvekov
2006-05-22 22:51 ` Nathan Scott
2006-05-23  0:49   ` fitzboy
2006-05-23  1:59     ` Nathan Scott
2006-05-24  1:41       ` fitzboy
2006-05-24  2:23         ` Nathan Scott
2006-05-25 19:15           ` fitzboy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=447259E7.8050706@iparadigms.com \
    --to=fitzboy@iparadigms.com \
    --cc=avi@argo.co.il \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.