* Fwd: UBIFS recovery taking too long [not found] <CACwUX0PEYGsNMDvy85EsrApWzNU3iTyY-1KHMaj1ay0xxKX6RQ@mail.gmail.com> @ 2013-12-09 20:53 ` David Mosberger-Tang 2013-12-09 21:30 ` Shuah Khan 0 siblings, 1 reply; 4+ messages in thread From: David Mosberger-Tang @ 2013-12-09 20:53 UTC (permalink / raw) To: LKML I've had no luck getting any response from the linux-mtd mailing list regarding the issue reported below. I think it is a very serious issue since it can easily render an embedded system unusable. --david ---------- Forwarded message ---------- From: David Mosberger-Tang <dmosberger@gmail.com> Date: Thu, Nov 7, 2013 at 11:44 AM Subject: UBIFS recovery taking too long To: "linux-mtd@lists.infradead.org" <linux-mtd@lists.infradead.org> We recently encountered a strange issue where UBIFS recovery is suddenly taking a lot longer than usual (v3.7-based kernel). In our case, recovery took about 30 seconds. This caused a serious problem because our watchdog timer was set to 15 seconds, effectively rendering the devices unbootable. Does anybody know what triggers this slow recovery mode? Also, how can we calculate a worst-case recovery time for a given flash chip-size. Thanks, --david ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: UBIFS recovery taking too long 2013-12-09 20:53 ` Fwd: UBIFS recovery taking too long David Mosberger-Tang @ 2013-12-09 21:30 ` Shuah Khan 2013-12-10 7:25 ` Adrian Hunter 0 siblings, 1 reply; 4+ messages in thread From: Shuah Khan @ 2013-12-09 21:30 UTC (permalink / raw) To: David Mosberger-Tang, dedekind1, adrian.hunter; +Cc: LKML Adding ubifs maintainers. -- Shuah On Mon, Dec 9, 2013 at 1:53 PM, David Mosberger-Tang <dmosberger@gmail.com> wrote: > I've had no luck getting any response from the linux-mtd mailing list > regarding the issue reported below. > I think it is a very serious issue since it can easily render an > embedded system unusable. > > --david > > ---------- Forwarded message ---------- > From: David Mosberger-Tang <dmosberger@gmail.com> > Date: Thu, Nov 7, 2013 at 11:44 AM > Subject: UBIFS recovery taking too long > To: "linux-mtd@lists.infradead.org" <linux-mtd@lists.infradead.org> > > > We recently encountered a strange issue where UBIFS recovery is > suddenly taking a lot longer than usual (v3.7-based kernel). In our > case, recovery took about 30 seconds. This caused a serious problem > because our watchdog timer was set to 15 seconds, effectively > rendering the devices unbootable. > > Does anybody know what triggers this slow recovery mode? Also, how > can we calculate a worst-case recovery time for a given flash > chip-size. > > Thanks, > > --david > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: UBIFS recovery taking too long 2013-12-09 21:30 ` Shuah Khan @ 2013-12-10 7:25 ` Adrian Hunter 2013-12-10 18:41 ` David Mosberger-Tang 0 siblings, 1 reply; 4+ messages in thread From: Adrian Hunter @ 2013-12-10 7:25 UTC (permalink / raw) To: Shuah Khan; +Cc: David Mosberger-Tang, dedekind1, LKML On 09/12/13 23:30, Shuah Khan wrote: > Adding ubifs maintainers. > > -- Shuah > > On Mon, Dec 9, 2013 at 1:53 PM, David Mosberger-Tang > <dmosberger@gmail.com> wrote: >> I've had no luck getting any response from the linux-mtd mailing list >> regarding the issue reported below. >> I think it is a very serious issue since it can easily render an >> embedded system unusable. >> >> --david >> >> ---------- Forwarded message ---------- >> From: David Mosberger-Tang <dmosberger@gmail.com> >> Date: Thu, Nov 7, 2013 at 11:44 AM >> Subject: UBIFS recovery taking too long >> To: "linux-mtd@lists.infradead.org" <linux-mtd@lists.infradead.org> >> >> >> We recently encountered a strange issue where UBIFS recovery is >> suddenly taking a lot longer than usual (v3.7-based kernel). In our >> case, recovery took about 30 seconds. This caused a serious problem >> because our watchdog timer was set to 15 seconds, effectively >> rendering the devices unbootable. >> >> Does anybody know what triggers this slow recovery mode? Also, how >> can we calculate a worst-case recovery time for a given flash >> chip-size. Slowness is probably caused by trying to make free space. Was the file system very full? A smaller journal might help. Probably the only way to determine worst-case recovery time is to test it. Worst case conditions are likely to be a full journal and a nearly full file system. Ideally you want to capture a file system image that exhibits the slow behaviour, then you can enable debug messages and see what it is doing. >> >> Thanks, >> >> --david >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ > > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: UBIFS recovery taking too long 2013-12-10 7:25 ` Adrian Hunter @ 2013-12-10 18:41 ` David Mosberger-Tang 0 siblings, 0 replies; 4+ messages in thread From: David Mosberger-Tang @ 2013-12-10 18:41 UTC (permalink / raw) To: Adrian Hunter; +Cc: Shuah Khan, dedekind1, LKML Adrian, On Tue, Dec 10, 2013 at 12:25 AM, Adrian Hunter <adrian.hunter@intel.com> wrote: > Slowness is probably caused by trying to make free space. Was the file > system very full? A smaller journal might help. That's possible. We have not been able to reproduce the issue in house but a customer had it happen on several devices in a row (3 or 4 of them, I believe). I don't think we are very sensitive to how long it takes we just need to know an upper bound on the recovery time. > Probably the only way to determine worst-case recovery time is to test it. > Worst case conditions are likely to be a full journal and a nearly full file > system. I have not found a lot of UBIFS stresstest tools or even tools to display what state UBIFS is in. Can you point me to something? I assume lots of small writes would fill up the journal? We'd be very happy to test this more but as things stands, it's very hard to tell how full the filesystem even is (due to compression) and I'm not sure how to check on the state of the journal. > Ideally you want to capture a file system image that exhibits the slow > behaviour, then you can enable debug messages and see what it is doing. OK, the two devices we've seen this initially have been restored so we can't find out from those. However, the customer affected by this issue may have another device or two they haven't sent back yet. We might be able to snapshot the root filesystem on one of those. --david >>> >>> Thanks, >>> >>> --david >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> Please read the FAQ at http://www.tux.org/lkml/ >> >> > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-12-10 18:41 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CACwUX0PEYGsNMDvy85EsrApWzNU3iTyY-1KHMaj1ay0xxKX6RQ@mail.gmail.com>
2013-12-09 20:53 ` Fwd: UBIFS recovery taking too long David Mosberger-Tang
2013-12-09 21:30 ` Shuah Khan
2013-12-10 7:25 ` Adrian Hunter
2013-12-10 18:41 ` David Mosberger-Tang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox