IO engine flags: FIO_RAWIO vs FIO_MEMALIGN

IO engine flags: FIO_RAWIO vs FIO_MEMALIGN

[Steven Lang]

Hello,

What is the intended difference between these two flags?  The former
seems to carry some specific semantic meaning while the latter seems
like it is merely a statement of requirement.  However, within the
code they seem to be used synonymously, to the point that if an IO
engine uses FIO_RAWIO and not FIO_MEMALIGN it would potentially crash.

From memory.c:196
        if (td->o.odirect || td->o.mem_align ||
            (td->io_ops->flags & FIO_MEMALIGN)) {
                total_mem += page_mask;
                if (td->o.mem_align && td->o.mem_align > page_size)
                        total_mem += td->o.mem_align - page_size;
        }

From fio.c:830
        if (td->o.odirect || td->o.mem_align ||
            (td->io_ops->flags & FIO_RAWIO))
                p = PAGE_ALIGN(td->orig_buffer) + td->o.mem_align;

These two pieces of code seem like they are a pair that should always
execute together, yet they have slightly different conditions.  The
result seems like if FIO_RAWIO is used without FIO_MEMALIGN, it will
try to page align the buffer (Which of course involves just padding it
to the next alignment) without the actual memory available to do so,
resulting in the IO buffer overwriting other parts of the heap.

If this is changed one way or the other, however, it would result in
an unused flag.  The condition in memory.c is the only place
FIO_MEMALIGN appears, and FIO_RAWIO only makes two other
appearances...

From init.c:320
        if (o->bs_unaligned && (o->odirect || td->io_ops->flags & FIO_RAWIO))
                log_err("fio: bs_unaligned may not work with raw io\n");
From init.c:585
        if (td->o.odirect)
                td->io_ops->flags |= FIO_RAWIO;

The former doesn't actually do anything but print a message and run
anyway (And should probably use the same form of the flag as the other
two anyway), and the latter is redundant since the same check for
FIO_RAWIO/FIO_MEMALIGN also checks for o.direct.

- Steven

Re: IO engine flags: FIO_RAWIO vs FIO_MEMALIGN

[Jens Axboe]

On 2011-07-02 03:11, Steven Lang wrote:
> Hello,
> 
> What is the intended difference between these two flags?  The former
> seems to carry some specific semantic meaning while the latter seems
> like it is merely a statement of requirement.  However, within the
> code they seem to be used synonymously, to the point that if an IO
> engine uses FIO_RAWIO and not FIO_MEMALIGN it would potentially crash.
> 
> From memory.c:196
>         if (td->o.odirect || td->o.mem_align ||
>             (td->io_ops->flags & FIO_MEMALIGN)) {
>                 total_mem += page_mask;
>                 if (td->o.mem_align && td->o.mem_align > page_size)
>                         total_mem += td->o.mem_align - page_size;
>         }
> 
> From fio.c:830
>         if (td->o.odirect || td->o.mem_align ||
>             (td->io_ops->flags & FIO_RAWIO))
>                 p = PAGE_ALIGN(td->orig_buffer) + td->o.mem_align;
> 
> These two pieces of code seem like they are a pair that should always
> execute together, yet they have slightly different conditions.  The
> result seems like if FIO_RAWIO is used without FIO_MEMALIGN, it will
> try to page align the buffer (Which of course involves just padding it
> to the next alignment) without the actual memory available to do so,
> resulting in the IO buffer overwriting other parts of the heap.
> 
> If this is changed one way or the other, however, it would result in
> an unused flag.  The condition in memory.c is the only place
> FIO_MEMALIGN appears, and FIO_RAWIO only makes two other
> appearances...
> 
> From init.c:320
>         if (o->bs_unaligned && (o->odirect || td->io_ops->flags & FIO_RAWIO))
>                 log_err("fio: bs_unaligned may not work with raw io\n");
> From init.c:585
>         if (td->o.odirect)
>                 td->io_ops->flags |= FIO_RAWIO;
> 
> The former doesn't actually do anything but print a message and run
> anyway (And should probably use the same form of the flag as the other
> two anyway), and the latter is redundant since the same check for
> FIO_RAWIO/FIO_MEMALIGN also checks for o.direct.

Sorry for missing this email. Yes, the two flags are a bit of a mix. I
think what used to be the case was that there used to be other
differences. But your analysis looks correct, FIO_RAWIO without MEMALIGN
would be buggy at this point.

Feel free to send a patch to unify and cleanup those flags.

-- 
Jens Axboe