RE: UFS failures with v5.13-rc4

RE: UFS failures with v5.13-rc4

[Alim Akhtar]

+ codeaurora guys ( Can and Asutosh)



> -----Original Message-----
> From: Felipe Balbi <balbi@kernel.org>
> Sent: 03 June 2021 19:56
> To: Alim Akhtar <alim.akhtar@samsung.com>; Avri Altman
> <avri.altman@wdc.com>; James E.J. Bottomley <jejb@linux.ibm.com>;
> Martin K. Petersen <martin.petersen@oracle.com>; linux-
> scsi@vger.kernel.org; linux-arm-msm@vger.kernel.org
> Subject: UFS failures with v5.13-rc4
> 
> 
> Hi guys,
> 
> Today I managed to trigger some pretty bad faults with with v5.13-rc4
> running on SM8150 (Snapdragon) and iozone.
> 
> It seems the problem only happens when using bfq iosched, as I couldn't
> trigger it with mq-deadline.
> 
> Here's the relevant snippet of console logs, full dmesg.txt attached.
> 
> # echo bfq > /sys/block/sdg/queue/scheduler # iozone -i 0 -R -t 8 -s 1G -r
> 16M -b /tmp/iozone-bfq.xls -F /mnt/file0 /mnt/file1 /mnt/file2 /mnt/file3
> /mnt/file4 /mnt/file5 /mnt/file6 /mnt/file7
>         Iozone: Performance Test of File I/O
>                 Version $Revision: 3.489 $
>                 Compiled for 64 bit mode.
>                 Build: linux
> 
>         Contributors:William Norcott, Don Capps, Isom Crawford, Kirby
Collins
>                      Al Slater, Scott Rhine, Mike Wisner, Ken Goss
>                      Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain
CYR,
>                      Randy Dunlap, Mark Montague, Dan Million, Gavin
Brebner,
>                      Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave
Boone,
>                      Erik Habbinga, Kris Strecker, Walter Wong, Joshua
Root,
>                      Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren
Sawyer,
>                      Vangel Bojaxhi, Ben England, Vikentsi Lapa,
>                      Alexey Skidanov, Sudhir Kumar.
> 
>         Run began: Thu Jan  1 02:29:50 1970
> 
>         Excel chart generation enabled
>         File size set to 1048576 kB
>         Record Size 16384 kB
>         Command line used: iozone -i 0 -R -t 8 -s 1G -r 16M -b
/tmp/iozone-
> bfq.xls -F /mnt/file0 /mnt/file1 /mnt/file2 /mnt/file3 /mnt/file4
/mnt/file5
> /mnt/file6 /mnt/fil
> e7
>         Output is in kBytes/sec
>         Time Resolution = 0.000001 seconds.
>         Processor cache size set to 1024 kBytes.
>         Processor cache line size set to 32 bytes.
>         File stride size set to 17 * record size.
>         Throughput test with 8 processes
>         Each process writes a 1048576 kByte file in 16384 kByte records [
> 2475.173128] ufshcd-qcom 1d84000.ufshc: __ufshcd_issue_tm_cmd: task
> management cmd 0x80 timed-out [ 2475.182205] ufshcd-qcom
> 1d84000.ufshc: ufshcd_try_to_abort_task: no response from device. tag = 0,
> err -110 [ 2475.252049] ufshcd-qcom 1d84000.ufshc: Controller enable
failed [
> 2475.317224] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2475.382038] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2475.447318] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2475.453450] ufshcd-qcom 1d84000.ufshc: ufshcd_host_reset_and_restore:
> Host init failed -5 [ 2475.522779] ufshcd-qcom 1d84000.ufshc: Controller
> enable failed [ 2475.587983] ufshcd-qcom 1d84000.ufshc: Controller enable
> failed [ 2475.653027] ufshcd-qcom 1d84000.ufshc: Controller enable failed
[
> 2475.719555] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2475.725690] ufshcd-qcom 1d84000.ufshc: ufshcd_host_reset_and_restore:
> Host init failed -5 [ 2475.795173] ufshcd-qcom 1d84000.ufshc: Controller
> enable failed [ 2475.861232] ufshcd-qcom 1d84000.ufshc: Controller enable
> failed [ 2475.926035] ufshcd-qcom 1d84000.ufshc: Controller enable failed
[
> 2475.991184] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2475.997382] ufshcd-qcom 1d84000.ufshc: ufshcd_host_reset_and_restore:
> Host init failed -5 [ 2476.064948] ufshcd-qcom 1d84000.ufshc: Controller
> enable failed [ 2476.129955] ufshcd-qcom 1d84000.ufshc: Controller enable
> failed [ 2476.194502] ufshcd-qcom 1d84000.ufshc: Controller enable failed
[
> 2476.259851] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2476.265962] ufshcd-qcom 1d84000.ufshc: ufshcd_host_reset_and_restore:
> Host init failed -5 [ 2476.334428] ufshcd-qcom 1d84000.ufshc: Controller
> enable failed [ 2476.399788] ufshcd-qcom 1d84000.ufshc: Controller enable
> failed [ 2476.465468] ufshcd-qcom 1d84000.ufshc: Controller enable failed
[
> 2476.529965] ufshcd-qcom 1d84000.ufshc: Controller enable failed [
> 2476.536086] ufshcd-qcom 1d84000.ufshc: ufshcd_host_reset_and_restore:
> Host init failed -5 [ 2476.544523] ufshcd-qcom 1d84000.ufshc:
> ufshcd_err_handler: reset and restore failed with err -5 [ 2476.606521] sd
> 0:0:0:6: [sdg] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x07
> driverbyte=0x00 cmd_age=0s [ 2476.615917] sd 0:0:0:6: [sdg] tag#0 CDB:
> opcode=0x2a 2a 00 00 35 b5 7c 00 00 10 00 [ 2476.623715]
> blk_update_request: I/O error, dev sdg, sector 28158944 op 0x1:(WRITE)
> flags 0x4000 phys_seg 16 prio class 0

Re: UFS failures with v5.13-rc4

[Bart Van Assche]

On 6/3/21 7:25 AM, Felipe Balbi wrote:
> Today I managed to trigger some pretty bad faults with with v5.13-rc4
> running on SM8150 (Snapdragon) and iozone.
> 
> It seems the problem only happens when using bfq iosched, as I couldn't
> trigger it with mq-deadline.

Both the UFS driver and the BFQ scheduler have been changed 
significantly recently. It would help if this issue could be bisected. 
That will reveal whether the root cause is in the UFS driver or in the 
BFQ scheduler.

Thanks,

Bart.