Using DMA in read/write, setting block size for I/O

Using DMA in read/write, setting block size for I/O

[Karthik Sarangan]

In my application, I have the following code.

int main(void)
{
    char *pcBuffer;

    posix_memalign((void **) pcBuffer, 512, 262144);
    int ifd = open("/dev/sdb", O_DIRECT | O_RDWR);
    long lLen;

    lLen = read(ifd, pcBuffer, 262144);
   
    close(ifd);
    return 0;
}

Will the underlying block device read a single 256KB block from the hdd 
into pcBuffer
or will it read 256KB as a set of smaller blocks?

Since the buffer is memory aligned will it enable DMA?

scsi disk driver is adaptec aic79xx.o
distro is RedHat Enterprise Linux WS 4 (kernel-2.6.9-11)

Re: Using DMA in read/write, setting block size for I/O

[Arjan van de Ven]

> Will the underlying block device read a single 256KB block from the hdd 
> into pcBuffer
> or will it read 256KB as a set of smaller blocks?

that depends, not all pieces of hardware can do transfers that big (or
at least advertise to the kernel that they can); if that's the case the
kernel will chop it up. Note:t he kernel will not *guarantee* that it'll
be one io either way. So don't depend on it for correctness!
Yet of course the kernel will try to optimize as good as possible


> Since the buffer is memory aligned will it enable DMA?

doesn't matter; DMA will always happen if the HW is capable of it. Since
you use O_DIRECT there will also not be a copy (which would happen if
you didn't use O_DIRECT)

Re: Using DMA in read/write, setting block size for I/O

[Arjan van de Ven]

On Wed, 2005-10-05 at 11:53 +0530, Karthik Sarangan wrote:
> Arjan van de Ven wrote:
> > <>that depends, not all pieces of hardware can do transfers that big (or
> > at least advertise to the kernel that they can); if that's the case the
> > kernel will chop it up. Note:t he kernel will not *guarantee* that it'll
> > be one io either way. So don't depend on it for correctness!
> > Yet of course the kernel will try to optimize as good as possible
> Is there a way to find out whether my hardware supports such huge DMA?

max_sectors in the host template for scsi

>  If it does how do I set it to 256KB chunk?

that ought to be automatic really once that's there

Re: Using DMA in read/write, setting block size for I/O -> max_sectors

[Karthik Sarangan]

Arjan van de Ven wrote:

>max_sectors in the host template for scsi

Can I set this parameter during startup for scsi_mod.ko during load?
I actually needed a 256KB transfer size only for my second scsi disk @
/dev/sdb.

Is there some similar load time parameter for the sd_mod.ko module? How
to set it?

Re: Using DMA in read/write, setting block size for I/O -> max_sectors

[Arjan van de Ven]

On Wed, 2005-10-05 at 15:13 +0530, Karthik Sarangan wrote:
> Arjan van de Ven wrote:
> 
> >max_sectors in the host template for scsi
> 
> Can I set this parameter during startup for scsi_mod.ko during load?

no it's a property of the hardware and as such the DRIVER has to set it.
Not all hw can deal with really big sizes, so the driver is supposed to
set what the hw is capable of.

> I actually needed a 256KB transfer size 

needed in what sense? 2x 128Kb isn't too bad either anyway :)

Re: Using DMA in read/write, setting block size for I/O -> max_sectors

[Karthik Sarangan]

Arjan van de Ven wrote:

> no it's a property of the hardware and as such the DRIVER has to set it.
> Not all hw can deal with really big sizes, so the driver is supposed to
> set what the hw is capable of.
> 
so I need to send some ioctl to the driver to set the transfer size or 
find something in the /proc/scsi area, isn't it?

Re: Using DMA in read/write, setting block size for I/O

[linux-os (Dick Johnson)]

On Tue, 4 Oct 2005, Karthik Sarangan wrote:

> In my application, I have the following code.
>
> int main(void)
> {
>    char *pcBuffer;
>
>    posix_memalign((void **) pcBuffer, 512, 262144);
>    int ifd = open("/dev/sdb", O_DIRECT | O_RDWR);
>    long lLen;
>
>    lLen = read(ifd, pcBuffer, 262144);
>
>    close(ifd);
>    return 0;
> }
>
> Will the underlying block device read a single 256KB block from the hdd
> into pcBuffer

No. It might read 512 bytes! You can't assume that a read will
always return the value requested. That's a bug. You need code
that will make as many reads as necessary to satisfy your
request.

> or will it read 256KB as a set of smaller blocks?
>

You (your code) may have to read multiple times. The
kernel code may even read ahead.

> Since the buffer is memory aligned will it enable DMA?
>
> scsi disk driver is adaptec aic79xx.o

This Adaptec SCSI driver always uses DMA.

> distro is RedHat Enterprise Linux WS 4 (kernel-2.6.9-11)
> -


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.