Why Seperate Log Files?

Why Seperate Log Files?

[Jon Tango]

When I use fio in 2.1.8 the write_iops_log command creates a singular log file, even with 8 workers. With fio in the latest versions, however, it creates eight separate log files. Is there a way to get fio to generate a log file for all 8 workers, instead of 8 separate files? That is untenable. I had to go back to 2.1.8 to get a useable output. It's not like the data outputs are very friendly anyway, but with one file at least that data can easily be parsed and worked with. With a different file for each worker it is just ridiculous. 
How do we get one output for all workers?
It would also be nice if we could just get one output with all of the numbers for the different workers combined, but that may be asking a bit much. Just getting them all in a workable file would be nice :) 

-----Original Message-----
From: Jon Tango [mailto:cheerios123@outlook.com] 
Sent: Tuesday, September 30, 2014 4:14 AM
To: 'Andrey Kuzmin'
Cc: 'Jens Axboe'; 'Sitsofe Wheeler'; fio@vger.kernel.org
Subject: RE: Limit LBA Range


The same thing could be done by partitioning the SSD in the operating system with the desired partition size, and then running a time-based fio job against the partition.

Basically, fio options offset/size should yield the same net result, but I'm always unsure on what is the actual effect of mixing size and time_based in a single job, and whether fio will wrap around the specified size to the specified offset if the job is time-based. HOWTO could have used some clarification on size/time_based interaction.

Regards,
Andrey

On Sep 30, 2014 11:41 AM, "Sitsofe Wheeler" <sitsofe@gmail.com> wrote:
>
> On 30 September 2014 07:34, Jon Tango <cheerios123@outlook.com> wrote:
> >
> > The taskfile is this:
>
> I should have been more specific - you need to show both the _vdbench_ 
> parameter file that you are comparing to in addition to showing your 
> fio job file.
>
> --
> Sitsofe |http://sucs.org/~sits/
> --
> To unsubscribe from this list: send the line "unsubscribe fio" in the 
> body of a message tomajordomo@vger.kernel.org More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

I'm really wanting to avoid the filesystem, they create so much interference. I am focusing on just testing the raw device :)

Re: Why Seperate Log Files?

[Jens Axboe]

On 05/10/2015 09:36 PM, Jon Tango wrote:
> When I use fio in 2.1.8 the write_iops_log command creates a singular log file, even with 8 workers. With fio in the latest versions, however, it creates eight separate log files. Is there a way to get fio to generate a log file for all 8 workers, instead of 8 separate files? That is untenable. I had to go back to 2.1.8 to get a useable output. It's not like the data outputs are very friendly anyway, but with one file at least that data can easily be parsed and worked with. With a different file for each worker it is just ridiculous.
> How do we get one output for all workers?
> It would also be nice if we could just get one output with all of the numbers for the different workers combined, but that may be asking a bit much. Just getting them all in a workable file would be nice :)

Lets back this up a bit. Your bug reporting _sucks_. Constructive 
criticism - look it up. You were even lazy enough to just reply to an 
old, totally unrelated email, instead of starting a new thread.

So lets try this again. I'll even offer you some free hints. Find out 
where the change happened, see if there's reasoning there. If not, 
suggest a way to improve the situation. Or even just, constructively, 
suggest why the current behavior is a problem and offer your opinion on 
how it could be improved to make your life easier. Nobody on this list 
owes you anything, you need to earn it.

-- 
Jens Axboe

RE: Why Seperate Log Files?

[Jon Tango]

Well, I can honestly admit that I deserved that. Sorry, I had wrote that after having stressed out for like 5 hours trying to get the outputs correct using a variety of commands with the new version. It wasn’t until I switched back to the old version that I realized somewhere along the line the output format had changed. I will re-submit this with some context. 

-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Monday, May 11, 2015 11:21 AM
To: Jon Tango; 'Andrey Kuzmin'
Cc: 'Sitsofe Wheeler'; fio@vger.kernel.org
Subject: Re: Why Seperate Log Files?

On 05/10/2015 09:36 PM, Jon Tango wrote:
> When I use fio in 2.1.8 the write_iops_log command creates a singular log file, even with 8 workers. With fio in the latest versions, however, it creates eight separate log files. Is there a way to get fio to generate a log file for all 8 workers, instead of 8 separate files? That is untenable. I had to go back to 2.1.8 to get a useable output. It's not like the data outputs are very friendly anyway, but with one file at least that data can easily be parsed and worked with. With a different file for each worker it is just ridiculous.
> How do we get one output for all workers?
> It would also be nice if we could just get one output with all of the 
> numbers for the different workers combined, but that may be asking a 
> bit much. Just getting them all in a workable file would be nice :)

Lets back this up a bit. Your bug reporting _sucks_. Constructive criticism - look it up. You were even lazy enough to just reply to an old, totally unrelated email, instead of starting a new thread.

So lets try this again. I'll even offer you some free hints. Find out where the change happened, see if there's reasoning there. If not, suggest a way to improve the situation. Or even just, constructively, suggest why the current behavior is a problem and offer your opinion on how it could be improved to make your life easier. Nobody on this list owes you anything, you need to earn it.

--
Jens Axboe

Re: Why Seperate Log Files?

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 607 bytes --]


On May 11, 2015, at 4:28 PM, Jon Tango <cheerios123@outlook.com 
<mailto:cheerios123@outlook.com>> wrote:

> Well, I can honestly admit that I deserved that. Sorry, I had wrote 
> that after having stressed out for like 5 hours trying to get the 
> outputs correct using a variety of commands with the new version. It 
> wasn’t until I switched back to the old version that I realized 
> somewhere along the line the output format had changed. I will 
> re-submit this with some context.

I'm nicer than you deserved:

http://git.kernel.dk/?p=fio.git;a=commit;h=3a5db9201bf7b1717f63cb8f7dbd025903117f70


[-- Attachment #2: Type: text/html, Size: 1181 bytes --]

Re: Why Seperate Log Files?

[Jens Axboe]

On May 11, 2015, at 4:28 PM, Jon Tango <cheerios123@outlook.com 
<mailto:cheerios123@outlook.com>> wrote:

> Well, I can honestly admit that I deserved that. Sorry, I had wrote 
> that after having stressed out for like 5 hours trying to get the 
> outputs correct using a variety of commands with the new version. It 
> wasn’t until I switched back to the old version that I realized 
> somewhere along the line the output format had changed. I will 
> re-submit this with some context.

I'm nicer than you deserved:

http://git.kernel.dk/?p=fio.git;a=commit;h=3a5db9201bf7b1717f63cb8f7dbd025903117f70