Tracking flash IO activity?

Tracking flash IO activity?

[valent.turkovic]

Hi,
I'm using low cost devices like home wifi routers for few diy projects
using OpenWrt.

These devices use mtd partitions, but I don't have enough knowledge on
how to track IO activity, especially number of writes.

Can you please point me in the right direction? Thanks.

Re: Tracking flash IO activity?

[valent.turkovic]

I have used iostat but it is not showing any changes on /dev/mtdblock5
which is mounted as /overlay jffs2 partition on this device I'm using.
I have also checked stats in /proc/diskstat, and they also only change
for /dev/mmcblk0p1 which is usb device, but not for any of
/dev/mtdblockX partitions.



> through fio and iostats , just starts an io through dd gets status through
> iostats, i have not used but i know.
>
> On Sun, May 17, 2015 at 1:51 AM, valent.turkovic@gmail.com
> <valent.turkovic@gmail.com> wrote:
>>
>> Hi,
>> I'm using low cost devices like home wifi routers for few diy projects
>> using OpenWrt.
>>
>> These devices use mtd partitions, but I don't have enough knowledge on
>> how to track IO activity, especially number of writes.
>>
>> Can you please point me in the right direction? Thanks.
>>
>> ______________________________________________________
>> Linux MTD discussion mailing list
>> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
>

Re: Tracking flash IO activity?

[Brian Norris]

On Sat, May 16, 2015 at 10:41:53PM +0200, valent.turkovic@gmail.com wrote:
> I have used iostat but it is not showing any changes on /dev/mtdblock5
> which is mounted as /overlay jffs2 partition on this device I'm using.
> I have also checked stats in /proc/diskstat, and they also only change
> for /dev/mmcblk0p1 which is usb device, but not for any of
> /dev/mtdblockX partitions.

JFFS2 doesn't actually use mtdblock. At most, it uses it as a
placeholder, which tells it to actually use the corresponding /dev/mtdX
device:

http://www.linux-mtd.infradead.org/faq/jffs2.html#L_mtdblock

> > through fio and iostats , just starts an io through dd gets status through
> > iostats, i have not used but i know.

Those are all for monitoring block devices. MTDs are not block devices:

http://www.linux-mtd.infradead.org/faq/general.html#L_mtd_what

I don't know of any similar IO stats for MTDs, sorry.

Brian

> > On Sun, May 17, 2015 at 1:51 AM, valent.turkovic@gmail.com
> > <valent.turkovic@gmail.com> wrote:
> >>
> >> Hi,
> >> I'm using low cost devices like home wifi routers for few diy projects
> >> using OpenWrt.
> >>
> >> These devices use mtd partitions, but I don't have enough knowledge on
> >> how to track IO activity, especially number of writes.
> >>
> >> Can you please point me in the right direction? Thanks.
> >>
> >> ______________________________________________________
> >> Linux MTD discussion mailing list
> >> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> >
> >
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Tracking flash IO activity?

[valent.turkovic]

On 16 May 2015 at 22:53, Brian Norris <computersforpeace@gmail.com> wrote:
> On Sat, May 16, 2015 at 10:41:53PM +0200, valent.turkovic@gmail.com wrote:
>> I have used iostat but it is not showing any changes on /dev/mtdblock5
>> which is mounted as /overlay jffs2 partition on this device I'm using.
>> I have also checked stats in /proc/diskstat, and they also only change
>> for /dev/mmcblk0p1 which is usb device, but not for any of
>> /dev/mtdblockX partitions.
>
> JFFS2 doesn't actually use mtdblock. At most, it uses it as a
> placeholder, which tells it to actually use the corresponding /dev/mtdX
> device:

Thanks Brian, I'll keep that in mind.

> http://www.linux-mtd.infradead.org/faq/jffs2.html#L_mtdblock
>
>> > through fio and iostats , just starts an io through dd gets status through
>> > iostats, i have not used but i know.
>
> Those are all for monitoring block devices. MTDs are not block devices:
>
> http://www.linux-mtd.infradead.org/faq/general.html#L_mtd_what
>
> I don't know of any similar IO stats for MTDs, sorry.


I tried digging trough /sys and checked out few files from
/sys/devices/platform/ath79-spi/spi_master/spi0/spi0.0/mtd/ and
/sys/block/mtdblock5/ and most I found out that this is NOR type of
memory and that it has 65536 size of eraseblock.

I also found mtd-utils:
http://processors.wiki.ti.com/index.php/MTD_Utilities, but I don't see
any tool in mtd-utils that shows stats of how many eraseblocks were
written and read.

Could I access jffs2 stats somehow for /overlay partition?

Re: Tracking flash IO activity?

[Richard Weinberger]

On Sat, May 16, 2015 at 10:53 PM, Brian Norris
<computersforpeace@gmail.com> wrote:
> I don't know of any similar IO stats for MTDs, sorry.

I wonder how hard it would be to integrate mtd into our current block IO stats.
It mtd is too low level we could also do it with UBI.

-- 
Thanks,
//richard

Re: Tracking flash IO activity?

[valent.turkovic]

Here is some interesting info I found using mtdinfo tool:

# mtdinfo /dev/mtd5
mtd5
Name:                           rootfs_data
Type:                           nor
Eraseblock size:                65536 bytes, 64.0 KiB
Amount of eraseblocks:          104 (6815744 bytes, 6.5 MiB)
Minimum input/output unit size: 1 byte
Sub-page size:                  1 byte
Character device major/minor:   90:10
Bad blocks are allowed:         false
Device is writable:             true

So I see there are 104 eraseblocks on this partition. Interesting so
see finaly some numbers. Now how to see how many times these
eraseblock were issued "erase eraseblock" command?

Re: Tracking flash IO activity?

[valent.turkovic]

So nobody from this mailing list knows how to track IO usage on mtd
partitions? How do you track usage in your embedded projects? Is
anybody tracking how much mtd devices are being erased and written?

Re: Tracking flash IO activity?

[Brian Norris]

On Sat, May 16, 2015 at 11:22:52PM +0200, Richard Weinberger wrote:
> On Sat, May 16, 2015 at 10:53 PM, Brian Norris
> <computersforpeace@gmail.com> wrote:
> > I don't know of any similar IO stats for MTDs, sorry.
> 
> I wonder how hard it would be to integrate mtd into our current block IO stats.
> It mtd is too low level we could also do it with UBI.

I'm not sure if blk-io's iostats would be sufficiently informative for
MTD/UBI anyway. Wouldn't we care about at least one extra
non-block-device stat: the erase stats?

Also (and this may be more like what Valent is asking for, though he
mentions JFFS2, not UBIFS) is there a readily-available tool for reading
the erase counter information from a UBI volume?

Brian

Re: Tracking flash IO activity?

[Richard Weinberger]

Am 18.05.2015 um 19:57 schrieb Brian Norris:
> On Sat, May 16, 2015 at 11:22:52PM +0200, Richard Weinberger wrote:
>> On Sat, May 16, 2015 at 10:53 PM, Brian Norris
>> <computersforpeace@gmail.com> wrote:
>>> I don't know of any similar IO stats for MTDs, sorry.
>>
>> I wonder how hard it would be to integrate mtd into our current block IO stats.
>> It mtd is too low level we could also do it with UBI.
> 
> I'm not sure if blk-io's iostats would be sufficiently informative for
> MTD/UBI anyway. Wouldn't we care about at least one extra
> non-block-device stat: the erase stats?

This coin has two sides.
I agree that for MTD we have three interesting counters, read, write and erase
which would deserve their own tool. Maybe something like mtdtop or mtdstat?
But it would be nice if well known tools like iostat(1) could work too.

> Also (and this may be more like what Valent is asking for, though he
> mentions JFFS2, not UBIFS) is there a readily-available tool for reading
> the erase counter information from a UBI volume?

Not really. I have a very hacky tool to inspect UBI Images.
Mainly to debug fastmap issues. If I find some spare time to
polish it I'll happily publish it.

Thanks,
//richard

Re: Tracking flash IO activity?

[nick]

On 2015-05-18 03:12 PM, Richard Weinberger wrote:
> Am 18.05.2015 um 19:57 schrieb Brian Norris:
>> On Sat, May 16, 2015 at 11:22:52PM +0200, Richard Weinberger wrote:
>>> On Sat, May 16, 2015 at 10:53 PM, Brian Norris
>>> <computersforpeace@gmail.com> wrote:
>>>> I don't know of any similar IO stats for MTDs, sorry.
>>>
>>> I wonder how hard it would be to integrate mtd into our current block IO stats.
>>> It mtd is too low level we could also do it with UBI.
>>
>> I'm not sure if blk-io's iostats would be sufficiently informative for
>> MTD/UBI anyway. Wouldn't we care about at least one extra
>> non-block-device stat: the erase stats?
> 
> This coin has two sides.
> I agree that for MTD we have three interesting counters, read, write and erase
> which would deserve their own tool. Maybe something like mtdtop or mtdstat?
> But it would be nice if well known tools like iostat(1) could work too.
> 
>> Also (and this may be more like what Valent is asking for, though he
>> mentions JFFS2, not UBIFS) is there a readily-available tool for reading
>> the erase counter information from a UBI volume?
> 
> Not really. I have a very hacky tool to inspect UBI Images.
> Mainly to debug fastmap issues. If I find some spare time to
> polish it I'll happily publish it.
> 
> Thanks,
> //richard
> 

Richard,
I feel that not having a tool like this is a very bad idea, it
may be one of the major issues I or others may  have with ubifs 
as a file system choice for flash based systems. You stated that
the tool you have is hacky, does this mean in terms of bugs or 
coding standards? If the tool works and is fairly stable why not
push to the repo for mtd-tools and mark it as experimental for now.
This way the mtd developers can start working on it and allow you
time to do other more important things with your time as a maintainer.
Nick
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

Re: Tracking flash IO activity?

[Richard Weinberger]

Am 18.05.2015 um 21:22 schrieb nick:
> I feel that not having a tool like this is a very bad idea, it
> may be one of the major issues I or others may  have with ubifs 
> as a file system choice for flash based systems. You stated that
> the tool you have is hacky, does this mean in terms of bugs or 
> coding standards? If the tool works and is fairly stable why not
> push to the repo for mtd-tools and mark it as experimental for now.
> This way the mtd developers can start working on it and allow you
> time to do other more important things with your time as a maintainer.

Hacky in terms of, it will kill your hamster and makes your girlfriend
cry if you use it not correctly. ;-)

Thanks,
//richard

Re: Tracking flash IO activity?

[nick]

On 2015-05-18 03:37 PM, Richard Weinberger wrote:
> Am 18.05.2015 um 21:22 schrieb nick:
>> I feel that not having a tool like this is a very bad idea, it
>> may be one of the major issues I or others may  have with ubifs 
>> as a file system choice for flash based systems. You stated that
>> the tool you have is hacky, does this mean in terms of bugs or 
>> coding standards? If the tool works and is fairly stable why not
>> push to the repo for mtd-tools and mark it as experimental for now.
>> This way the mtd developers can start working on it and allow you
>> time to do other more important things with your time as a maintainer.
> 
> Hacky in terms of, it will kill your hamster and makes your girlfriend
> cry if you use it not correctly. ;-)
> 
> Thanks,
> //richard
> 
If that is the case, I am aware my rep here isn't that good but would you some help
with making it better as a maintainer you probably won't get around to this for a 
while, no offence :).
Nick

Re: Tracking flash IO activity?

[Richard Weinberger]

Am 18.05.2015 um 21:40 schrieb nick:
> If that is the case, I am aware my rep here isn't that good but would you some help
> with making it better as a maintainer you probably won't get around to this for a 
> while, no offence :).

Honestly, if you want to get a grip on UBI, create such a tool on your own.
A EC dumper can be done in a few lines of C (or any other language).

Thanks,
//richard

Re: Tracking flash IO activity?

[nick]

On 2015-05-18 03:44 PM, Richard Weinberger wrote:
> Am 18.05.2015 um 21:40 schrieb nick:
>> If that is the case, I am aware my rep here isn't that good but would you some help
>> with making it better as a maintainer you probably won't get around to this for a 
>> while, no offence :).
> 
> Honestly, if you want to get a grip on UBI, create such a tool on your own.
> A EC dumper can be done in a few lines of C (or any other language).
> 
> Thanks,
> //richard
> 
I have busy for the last while in terms of doing networking and kvm stack work/learning.
Further more I was going to help test the mtd stack but am short on cash at the moment
so buying a board is a little hard now :(. 
Thanks for the Advice,
Nick