Re: 4.15.14 crash with iscsi target and dvd

Re: 4.15.14 crash with iscsi target and dvd

[Richard Weinberger]

On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> I reported this before but noone responded.

Because you're sending only to LKML.
CC'ing storage folks.

> I have an iscsi target setup with /dev/sr[012] using pscsi.  On the
> initiator, I mount only 1 disc.  Then I issue find -type f | xargs cat >
> /dev/null  Then after a few seconds, I get 2 oops and the system has to b=
e
> hard reset.
>
> I noticed if I cat /dev/sr1 from the initiator, it doesn't crash.  I was
> also able to burn without crashing.
>
> Here's the OOPS:
> [2692.733468] WARNING: CPU: 8 PID: 0 at /usr/src/linux/dist/4.15.14-nobkl=
cd/drivers/scsi/scsi_lib.c:1068 scsi_init_io+0x111/0x1a0 [scsi_mod]
> [2692.734154] Modules linked in: dm_thin_pool dm_persistent_data dm_bio_p=
rison dm_bufio raid456 async_raid6_recov async_memcpy async_pq async_xor as=
ync_tx xor raid6_pq libcrc32c crc32c_generic md_mod dm_crypt algif_skcipher=
 af_alg dm_mod dax ext4 crc16 mbcache jbd2 af_packet iscsi_target_mod tcm_l=
oop vhost_scsi vhost target_core_file target_core_iblock target_core_pscsi =
target_core_mod nfsd exportfs dummy bridge stp llc ib_iser rdma_cm iw_cm ib=
_cm ib_core ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi netco=
nsole configfs sr_mod cdrom adt7475 hwmon_vid sd_mod sg coretemp x86_pkg_te=
mp_thermal kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_int=
el ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic nou=
veau video led_class drm_kms_helper cfbfillrect syscopyarea cfbimgblt
> [2692.737388]  sysfillrect sysimgblt snd_hda_intel fb_sys_fops cfbcopyare=
a snd_hda_codec ttm snd_hda_core snd_pcm_oss drm snd_mixer_oss agpgart snd_=
pcm igb snd_timer snd aesni_intel soundcore hwmon aes_x86_64 i2c_algo_bit a=
hci mpt3sas crypto_simd i2c_core libahci glue_helper mptsas raid_class liba=
ta mptscsih scsi_transport_sas mptbase scsi_mod wmi button unix
> [2692.737388] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 4.15.14 #1
> [2692.737388] Hardware name: Dell Inc. Precision T5610/0WN7Y6, BIOS A16 0=
2/05/2018
> [2692.737388] RIP: 0010:scsi_init_io+0x111/0x1a0 [scsi_mod]
> [2692.737388] RSP: 0018:ffff8806b7a03d78 EFLAGS: 00010046
> [2692.737388] RAX: 0000000000000000 RBX: ffff8806aa4a9c00 RCX: 0000000000=
000000
> [2692.737388] RDX: 0000000000000000 RSI: ffff8806aa4a9c00 RDI: ffff8806aa=
4a9d48
> [2692.737388] RBP: ffff8806aa4a9d48 R08: 0000000000000000 R09: ffff8806aa=
4a9d80
> [2692.737388] R10: ffff8806ab949088 R11: 0000000000000000 R12: ffff8806b2=
9bb000
> [2692.737388] R13: 0000000000000000 R14: ffff8806b29bb000 R15: ffff8806ac=
4f6950
> [2692.737388] FS:  0000000000000000(0000) GS:ffff8806b7a00000(0000) knlGS=
:0000000000000000
> [2692.737388] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [2692.737388] CR2: 00007f1359a8b756 CR3: 0000000001c09003 CR4: 0000000000=
1606e0
> [2692.737388] Call Trace:
> [2692.737388]  <IRQ>
> [2692.737388]  ? scsi_setup_cmnd+0xb3/0x140 [scsi_mod]
> [2692.737388]  ? scsi_prep_fn+0x53/0x130 [scsi_mod]
> [2692.737388]  ? blk_peek_request+0x136/0x220
> [2692.737388]  ? scsi_request_fn+0x2b/0x510 [scsi_mod]
> [2692.737388]  ? __blk_run_queue+0x34/0x50
> [2692.737388]  ? blk_run_queue+0x26/0x40
> [2692.737388]  ? scsi_run_queue+0x229/0x2b0 [scsi_mod]
> [2692.737388]  ? scsi_io_completion+0x3ce/0x5a0 [scsi_mod]
> [2692.737388]  ? blk_done_softirq+0x67/0x80
> [2692.737388]  ? __do_softirq+0xdb/0x1dd
> [2692.737388]  ? irq_exit+0xa3/0xb0
> [2692.737388]  ? do_IRQ+0x45/0xc0
> [2692.737388]  ? common_interrupt+0x77/0x77
> [2692.737388]  </IRQ>
> [2692.737388]  ? cpuidle_enter_state+0x124/0x200
> [2692.737388]  ? cpuidle_enter_state+0x119/0x200
> [2692.737388]  ? do_idle+0xdc/0x180
> [2692.737388]  ? cpu_startup_entry+0x14/0x20
> [2692.737388]  ? secondary_startup_64+0xa5/0xb0
> [2692.737388] Code: 8b 7b 30 e8 d2 6b 20 e1 49 8b 17 4c 89 ff 89 c6 89 44=
 24 04 e8 81 81 22 e1 85 c0 41 89 c4 74 55 41 bc 02 00 00 00 e9 39 ff ff ff=
 <0f> 0b 41 bc 01 00 00 00 e9 2c ff ff ff 48 8b 3d 6b dc 00 00 be
> [2692.737388] ---[ end trace 9801970f9b249e10 ]---
> [2692.737388] ------------[ cut here ]------------
> [2692.737388] kernel BUG at /usr/src/linux/dist/4.15.14-nobklcd/block/blk=
-core.c:3235!
> [2692.737388] invalid opcode: 0000 [#1] PREEMPT SMP
> [2692.737388] Modules linked in: dm_thin_pool dm_persistent_data dm_bio_p=
rison dm_bufio raid456 async_raid6_recov async_memcpy async_pq async_xor as=
ync_tx xor raid6_pq libcrc32c crc32c_generic md_mod dm_crypt algif_skcipher=
 af_alg dm_mod dax ext4 crc16 mbcache jbd2 af_packet iscsi_target_mod tcm_l=
oop vhost_scsi vhost target_core_file target_core_iblock target_core_pscsi =
target_core_mod nfsd exportfs dummy bridge stp llc ib_iser rdma_cm iw_cm ib=
_cm ib_core ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi netco=
nsole configfs sr_mod cdrom adt7475 hwmon_vid sd_mod sg coretemp x86_pkg_te=
mp_thermal kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_int=
el ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic nou=
veau video led_class drm_kms_helper cfbfillrect syscopyarea cfbimgblt
> [2692.737388]  sysfillrect sysimgblt snd_hda_intel fb_sys_fops cfbcopyare=
a snd_hda_codec ttm snd_hda_core snd_pcm_oss drm snd_mixer_oss agpgart snd_=
pcm igb snd_timer snd aesni_intel soundcore hwmon aes_x86_64 i2c_algo_bit a=
hci mpt3sas crypto_simd i2c_core libahci glue_helper mptsas raid_class liba=
ta mptscsih scsi_transport_sas mptbase scsi_mod wmi button unix
> [2692.737388] CPU: 8 PID: 0 Comm: swapper/8 Tainted: G        W        4.=
15.14 #1
> [2692.737388] Hardware name: Dell Inc. Precision T5610/0WN7Y6, BIOS A16 0=
2/05/2018
> [2692.737388] RIP: 0010:__blk_end_request_all+0x50/0x60
> [2692.737388] RSP: 0018:ffff8806b7a03df8 EFLAGS: 00010002
> [2692.737388] RAX: 0000000000000001 RBX: ffff8806ac4f6950 RCX: 0000000000=
000000
> [2692.737388] RDX: 0000000000000001 RSI: ffff8806ac36a480 RDI: 0000000000=
000000
> [2692.737388] RBP: 0000000000000001 R08: 0000000000000000 R09: ffff8806aa=
4a9d80
> [2692.737388] R10: ffff8806ab949088 R11: 0000000000000000 R12: ffff8806aa=
4a9c00
> [2692.737388] R13: ffff8806ac4f6950 R14: 0000000000000246 R15: ffff8806ac=
4f6950
> [2692.737388] FS:  0000000000000000(0000) GS:ffff8806b7a00000(0000) knlGS=
:0000000000000000
> [2692.737388] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [2692.737388] CR2: 00007f1359a8b756 CR3: 0000000001c09003 CR4: 0000000000=
1606e0
> [2692.737388] Call Trace:
> [2692.737388]  <IRQ>
> [2692.737388]  ? blk_peek_request+0x173/0x220
> [2692.737388]  ? scsi_request_fn+0x2b/0x510 [scsi_mod]
> [2692.737388]  ? __blk_run_queue+0x34/0x50
> [2692.737388]  ? blk_run_queue+0x26/0x40
> [2692.737388]  ? scsi_run_queue+0x229/0x2b0 [scsi_mod]
> [2692.737388]  ? scsi_io_completion+0x3ce/0x5a0 [scsi_mod]
> [2692.737388]  ? blk_done_softirq+0x67/0x80
> [2692.737388]  ? __do_softirq+0xdb/0x1dd
> [2692.737388]  ? irq_exit+0xa3/0xb0
> [2692.737388]  ? do_IRQ+0x45/0xc0
> [2692.737388]  ? common_interrupt+0x77/0x77
> [2692.737388]  </IRQ>
> [2692.737388]  ? cpuidle_enter_state+0x124/0x200
> [2692.737388]  ? cpuidle_enter_state+0x119/0x200
> [2692.737388]  ? do_idle+0xdc/0x180
> [2692.737388]  ? cpu_startup_entry+0x14/0x20
> [2692.737388]  ? secondary_startup_64+0xa5/0xb0
> [2692.737388] Code: ff ff ff 84 c0 75 24 c3 0f 0b 48 8b 87 40 01 00 00 31=
 c9 48 85 c0 74 df 8b 48 58 40 0f b6 f6 8b 57 58 e8 04 ff ff ff 84 c0 74 dc=
 <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 41 54 55 53 48
> [2692.737388] RIP: __blk_end_request_all+0x50/0x60 RSP: ffff8806b7a03df8
> [2692.737388] ---[ end trace 9801970f9b249e11 ]---
> [2692.737388] Kernel panic - not syncing: Fatal exception in interrupt
> [2692.737388] Kernel Offset: disabled
> [2692.737388] ---[ end Kernel panic - not syncing: Fatal exception in int=
errupt



--=20
Thanks,
//richard

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Richard Weinberger wrote:
> On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> > I reported this before but noone responded.
> 
> Because you're sending only to LKML.
> CC'ing storage folks.

Thank you.  I wasn't sure who I needed to send it to.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gU2F0LCAyMDE4LTAzLTMxIGF0IDE4OjEyIC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IFJpY2hhcmQgV2VpbmJlcmdlciB3cm90ZToNCj4gPiBPbiBTYXQsIE1hciAzMSwgMjAxOCBhdCAz
OjU5IEFNLCBXYWtrbyBXYXJuZXIgPHdha2tvQGFuaW14LmV1Lm9yZz4gd3JvdGU6DQo+ID4gPiBJ
IHJlcG9ydGVkIHRoaXMgYmVmb3JlIGJ1dCBub29uZSByZXNwb25kZWQuDQo+ID4gDQo+ID4gQmVj
YXVzZSB5b3UncmUgc2VuZGluZyBvbmx5IHRvIExLTUwuDQo+ID4gQ0MnaW5nIHN0b3JhZ2UgZm9s
a3MuDQo+IA0KPiBUaGFuayB5b3UuICBJIHdhc24ndCBzdXJlIHdobyBJIG5lZWRlZCB0byBzZW5k
IGl0IHRvLg0KDQpIZWxsbyBXYWtrbywNCg0KQ2FuIHlvdSBzaGFyZSB0aGUgb3V0cHV0IG9mIGxz
c2NzaT8gSSB3b3VsZCBsaWtlIHRvIGtub3cgd2hldGhlciBvciBub3QgeW91DQphcmUgdXNpbmcg
YSAoUylBVEEgQ0RST00uDQoNClRoYW5rcywNCg0KQmFydC4NCg0KDQoNCg0K

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Sat, 2018-03-31 at 18:12 -0400, Wakko Warner wrote:
> > Richard Weinberger wrote:
> > > On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> > > > I reported this before but noone responded.
> > > 
> > > Because you're sending only to LKML.
> > > CC'ing storage folks.
> > 
> > Thank you.  I wasn't sure who I needed to send it to.
> 
> Can you share the output of lsscsi? I would like to know whether or not you
> are using a (S)ATA CDROM.

>From the target:
[4:0:0:0]    cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr0 
[5:0:0:0]    cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr1 
[6:0:0:0]    cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr2 

>From the initiator:
[19:0:0:0]   cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr1
[19:0:0:1]   cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr2
[19:0:0:2]   cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr3


I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
>From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
/dev/sr1 and then do find -type f | xargs cat > /dev/null the target
crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
the initiator with out problems.  I'll test other kernels between 4.9 and
4.14.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gU3VuLCAyMDE4LTA0LTAxIGF0IDA3OjM3IC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IEJhcnQgVmFuIEFzc2NoZSB3cm90ZToNCj4gPiBPbiBTYXQsIDIwMTgtMDMtMzEgYXQgMTg6MTIg
LTA0MDAsIFdha2tvIFdhcm5lciB3cm90ZToNCj4gPiA+IFJpY2hhcmQgV2VpbmJlcmdlciB3cm90
ZToNCj4gPiA+ID4gT24gU2F0LCBNYXIgMzEsIDIwMTggYXQgMzo1OSBBTSwgV2Fra28gV2FybmVy
IDx3YWtrb0BhbmlteC5ldS5vcmc+IHdyb3RlOg0KPiA+ID4gPiA+IEkgcmVwb3J0ZWQgdGhpcyBi
ZWZvcmUgYnV0IG5vb25lIHJlc3BvbmRlZC4NCj4gPiA+ID4gDQo+ID4gPiA+IEJlY2F1c2UgeW91
J3JlIHNlbmRpbmcgb25seSB0byBMS01MLg0KPiA+ID4gPiBDQydpbmcgc3RvcmFnZSBmb2xrcy4N
Cj4gPiA+IA0KPiA+ID4gVGhhbmsgeW91LiAgSSB3YXNuJ3Qgc3VyZSB3aG8gSSBuZWVkZWQgdG8g
c2VuZCBpdCB0by4NCj4gPiANCj4gPiBDYW4geW91IHNoYXJlIHRoZSBvdXRwdXQgb2YgbHNzY3Np
PyBJIHdvdWxkIGxpa2UgdG8ga25vdyB3aGV0aGVyIG9yIG5vdCB5b3UNCj4gPiBhcmUgdXNpbmcg
YSAoUylBVEEgQ0RST00uDQo+IA0KPiBGcm9tIHRoZSB0YXJnZXQ6DQo+IFs0OjA6MDowXSAgICBj
ZC9kdmQgIEFUQVBJICAgIGlIQVMyMjQgICBCICAgICAgR0wwNSAgL2Rldi9zcjAgDQo+IFs1OjA6
MDowXSAgICBjZC9kdmQgIEFUQVBJICAgIGlIQVM0MjIgICA4ICAgICAgNEwxMSAgL2Rldi9zcjEg
DQo+IFs2OjA6MDowXSAgICBjZC9kdmQgIFBCRFMgICAgIERWRCstUlcgREgtMTZXMVMgMkQxNCAg
L2Rldi9zcjIgDQo+IA0KPiBGcm9tIHRoZSBpbml0aWF0b3I6DQo+IFsxOTowOjA6MF0gICBjZC9k
dmQgIEFUQVBJICAgIGlIQVMyMjQgICBCICAgICAgR0wwNSAgL2Rldi9zcjENCj4gWzE5OjA6MDox
XSAgIGNkL2R2ZCAgQVRBUEkgICAgaUhBUzQyMiAgIDggICAgICA0TDExICAvZGV2L3NyMg0KPiBb
MTk6MDowOjJdICAgY2QvZHZkICBQQkRTICAgICBEVkQrLVJXIERILTE2VzFTIDJEMTQgIC9kZXYv
c3IzDQo+IA0KPiBJIHRlc3RlZCA0LjE0LjMyIGxhc3QgbmlnaHQgd2l0aCB0aGUgc2FtZSBvb3Bz
LiAgNC45LjkxIHdvcmtzIGZpbmUuDQo+IEZyb20gdGhlIGluaXRpYXRvciwgaWYgSSBkbyBjYXQg
L2Rldi9zcjEgPiAvZGV2L251bGwgaXQgd29ya3MuICBJZiBJIG1vdW50DQo+IC9kZXYvc3IxIGFu
ZCB0aGVuIGRvIGZpbmQgLXR5cGUgZiB8IHhhcmdzIGNhdCA+IC9kZXYvbnVsbCB0aGUgdGFyZ2V0
DQo+IGNyYXNoZXMuICBJJ20gdXNpbmcgdGhlIGJ1aWx0aW4gaXNjc2kgdGFyZ2V0IHdpdGggcHNj
c2kuICBJIGNhbiBidXJuIGZyb20NCj4gdGhlIGluaXRpYXRvciB3aXRoIG91dCBwcm9ibGVtcy4g
IEknbGwgdGVzdCBvdGhlciBrZXJuZWxzIGJldHdlZW4gNC45IGFuZA0KPiA0LjE0Lg0KDQooK0xl
ZSBhbmQgQ2hyaXMpDQoNCkhlbGxvIFdha2tvLA0KDQpBbHRob3VnaCBJJ20gbm90IHN1cmUgdGhh
dCB3aGF0IEkgcmFuIGludG8gaXMgZXhhY3RseSB0aGUgc2FtZSBhcyB3aGF0IHlvdQ0KcmFuIGlu
dG8sIHRoZXJlIGlzIGRlZmluaXRlbHkgc29tZXRoaW5nIHdyb25nIHdpdGggd2hhdCBJIGVuY291
bnRlcmVkLiBXaGF0DQpJIHJhbiBpbnRvIHdpdGggTGludXMnIGxhdGVzdCBtYXN0ZXIgYnJhbmNo
IGluZGljYXRlcyB0d28gaXNzdWVzIC0gb25lIGluDQp0aGUgaVNDU0kgaW5pdGlhdG9yIGFuZCBv
bmUgaW4gdGhlIGJsb2NrIGxheWVyOg0KDQpzY3NpIDM6MDowOjE6IERpcmVjdC1BY2Nlc3MgICAg
IExJTy1PUkcgIEZJTEVJTyAgICAgICAgICAgNC4wICBQUTogMCBBTlNJOiA1DQpzZCAyOjA6MDox
OiBbc2RkXSBBdHRhY2hlZCBTQ1NJIGRpc2sNCnNkIDM6MDowOjE6IFdhcm5pbmchIFJlY2VpdmVk
IGFuIGluZGljYXRpb24gdGhhdCB0aGUgTFVOIGFzc2lnbm1lbnRzIG9uIHRoaXMNCnRhcmdldCBo
YXZlIGNoYW5nZWQuIFRoZSBMaW51eCBTQ1NJIGxheWVyIGRvZXMgbm90IGF1dG9tYXRpY2FsDQpz
ZCAzOjA6MDoxOiBBdHRhY2hlZCBzY3NpIGdlbmVyaWMgc2c4IHR5cGUgMA0Kc2QgMzowOjA6MTog
W3NkZl0gMTI4IDUxMi1ieXRlIGxvZ2ljYWwgYmxvY2tzOiAoNjUuNSBrQi82NC4wIEtpQikNCnNk
IDM6MDowOjE6IFtzZGZdIFdyaXRlIFByb3RlY3QgaXMgb2ZmDQpzZCAzOjA6MDoxOiBbc2RmXSBN
b2RlIFNlbnNlOiA0MyAwMCAwMCAwOA0Kc2QgMzowOjA6MTogW3NkZl0gV3JpdGUgY2FjaGU6IGRp
c2FibGVkLCByZWFkIGNhY2hlOiBlbmFibGVkLCBkb2Vzbid0DQpzdXBwb3J0IERQTyBvciBGVUEN
CmlTQ1NJL2lxbi4xOTkzLTA4Lm9yZy5kZWJpYW46MDE6M2I2OGIxYjNkMmViOiBVbnN1cHBvcnRl
ZCBTQ1NJIE9wY29kZSAweGEzLA0Kc2VuZGluZyBDSEVDS19DT05ESVRJT04uDQpzZCAzOjA6MDoy
OiBbc2RlXSBBdHRhY2hlZCBTQ1NJIGRpc2sNCnNkIDM6MDowOjE6IFtzZGZdIEF0dGFjaGVkIFND
U0kgZGlzaw0KDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PQ0KV0FSTklORzogSEFSRElSUS1zYWZlIC0+IEhBUkRJUlEtdW5zYWZlIGxvY2sgb3Jk
ZXIgZGV0ZWN0ZWQNCjQuMTYuMC1yYzctZGJnKyAjMyBOb3QgdGFpbnRlZA0KLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmt3b3JrZXIvNjoxSC8x
NTUgW0hDMFswXTpTQzBbMF06SEUwOlNFMV0gaXMgdHJ5aW5nIHRvIGFjcXVpcmU6DQogKCYoJnNl
c3Npb24tPmZyd2RfbG9jayktPnJsb2NrKXsrLi0ufSwgYXQ6IFs8MDAwMDAwMDA3ZWI2NzhlYz5d
DQppc2NzaV9laF9jbWRfdGltZWRfb3V0KzB4NmIvMHg1YTAgW2xpYmlzY3NpXQ0KDQphbmQgdGhp
cyB0YXNrIGlzIGFscmVhZHkgaG9sZGluZzoNCiAoJigmcS0+X19xdWV1ZV9sb2NrKS0+cmxvY2sp
ey0uLS59LCBhdDogWzwwMDAwMDAwMDNjNTg0MWVjPl0NCmJsa190aW1lb3V0X3dvcmsrMHg0NS8w
eDIyMA0Kd2hpY2ggd291bGQgY3JlYXRlIGEgbmV3IGxvY2sgZGVwZW5kZW5jeToNCiAoJigmcS0+
X19xdWV1ZV9sb2NrKS0+cmxvY2spey0uLS59IC0+ICgmKCZzZXNzaW9uLT5mcndkX2xvY2spLT5y
bG9jayl7Ky4tLn0NCg0KYnV0IHRoaXMgbmV3IGRlcGVuZGVuY3kgY29ubmVjdHMgYSBIQVJESVJR
LWlycS1zYWZlIGxvY2s6DQogKCYoJnEtPl9fcXVldWVfbG9jayktPnJsb2NrKXstLi0ufQ0KDQou
Li4gd2hpY2ggYmVjYW1lIEhBUkRJUlEtaXJxLXNhZmUgYXQ6DQogIF9yYXdfc3Bpbl9sb2NrX2ly
cXNhdmUrMHg0Ni8weDYwDQogIGF0YV9xY19zY2hlZHVsZV9laCsweGI5LzB4MTEwIFtsaWJhdGFd
DQogIGF0YV9zZmZfaHNtX21vdmUrMHgxMTQvMHhiMTAgW2xpYmF0YV0NCiAgX19hdGFfc2ZmX3Bv
cnRfaW50cisweGVjLzB4MWEwIFtsaWJhdGFdDQogIGF0YV9ibWRtYV9wb3J0X2ludHIrMHhlZi8w
eDE2MCBbbGliYXRhXQ0KICBhdGFfYm1kbWFfaW50ZXJydXB0KzB4MTYwLzB4MmUwIFtsaWJhdGFd
DQogIF9faGFuZGxlX2lycV9ldmVudF9wZXJjcHUrMHg3Mi8weDQ2MA0KICBoYW5kbGVfaXJxX2V2
ZW50X3BlcmNwdSsweDY2LzB4ZDANCiAgaGFuZGxlX2lycV9ldmVudCsweDU0LzB4OTANCiAgaGFu
ZGxlX2VkZ2VfaXJxKzB4ZTkvMHgyZDANCiAgaGFuZGxlX2lycSsweDE3Yi8weDIxMA0KICBkb19J
UlErMHg2Yy8weDE0MA0KICByZXRfZnJvbV9pbnRyKzB4MC8weDFlDQogIG5hdGl2ZV9zYWZlX2hh
bHQrMHgyLzB4MTANCiAgZGVmYXVsdF9pZGxlKzB4MWYvMHgyMDANCiAgZG9faWRsZSsweDFiYy8w
eDFmMA0KICBjcHVfc3RhcnR1cF9lbnRyeSsweDE5LzB4MjANCiAgc3RhcnRfa2VybmVsKzB4NDdm
LzB4NGExDQogIHNlY29uZGFyeV9zdGFydHVwXzY0KzB4YTUvMHhiMA0KDQp0byBhIEhBUkRJUlEt
aXJxLXVuc2FmZSBsb2NrOg0KICgmKCZzZXNzaW9uLT5mcndkX2xvY2spLT5ybG9jayl7Ky4tLn0N
Cg0KLi4uIHdoaWNoIGJlY2FtZSBIQVJESVJRLWlycS11bnNhZmUgYXQ6DQouLi4NCiAgX3Jhd19z
cGluX2xvY2tfYmgrMHgzNC8weDQwDQogIGlzY3NpX2Nvbm5fc2V0dXArMHgyMzkvMHgzMjAgW2xp
YmlzY3NpXQ0KICBpc2NzaV90Y3BfY29ubl9zZXR1cCsweDE0LzB4ODAgW2xpYmlzY3NpX3RjcF0N
CiAgaXNjc2lfc3dfdGNwX2Nvbm5fY3JlYXRlKzB4MWYvMHgyNGEgW2lzY3NpX3RjcF0NCiAgaXNj
c2lfaWZfcngrMHgxM2M5LzB4MjBkMCBbc2NzaV90cmFuc3BvcnRfaXNjc2ldDQogIG5ldGxpbmtf
dW5pY2FzdCsweDI5OS8weDMzMA0KICBuZXRsaW5rX3NlbmRtc2crMHg0MzUvMHg1ODANCiAgX19f
c3lzX3NlbmRtc2crMHg0NDcvMHg0ZDANCiAgX19zeXNfc2VuZG1zZysweGFkLzB4MTIwDQogIGRv
X3N5c2NhbGxfNjQrMHhmMy8weDJjMA0KICBlbnRyeV9TWVNDQUxMXzY0X2FmdGVyX2h3ZnJhbWUr
MHg0Mi8weGI3DQoNCm90aGVyIGluZm8gdGhhdCBtaWdodCBoZWxwIHVzIGRlYnVnIHRoaXM6DQoN
CiBQb3NzaWJsZSBpbnRlcnJ1cHQgdW5zYWZlIGxvY2tpbmcgc2NlbmFyaW86DQoNCiAgICAgICBD
UFUwICAgICAgICAgICAgICAgICAgICBDUFUxDQogICAgICAgLS0tLSAgICAgICAgICAgICAgICAg
ICAgLS0tLQ0KICBsb2NrKCYoJnNlc3Npb24tPmZyd2RfbG9jayktPnJsb2NrKTsNCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICBsb2NhbF9pcnFfZGlzYWJsZSgpOw0KICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIGxvY2soJigmcS0+X19xdWV1ZV9sb2NrKS0+cmxvY2spOw0KICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxvY2soJigmc2Vzc2lvbi0+ZnJ3ZF9sb2NrKS0+
cmxvY2spOw0KICA8SW50ZXJydXB0Pg0KICAgIGxvY2soJigmcS0+X19xdWV1ZV9sb2NrKS0+cmxv
Y2spOw0KDQogKioqIERFQURMT0NLICoqKg0KDQozIGxvY2tzIGhlbGQgYnkga3dvcmtlci82OjFI
LzE1NToNCiAjMDogICgod3FfY29tcGxldGlvbikia2Jsb2NrZCIpeysuKy59LCBhdDogWzwwMDAw
MDAwMDExNmZlZDg0Pl0NCnByb2Nlc3Nfb25lX3dvcmsrMHgzODcvMHhhNTANCiAjMTogICgod29y
a19jb21wbGV0aW9uKSgmcS0+dGltZW91dF93b3JrKSl7Ky4rLn0sIGF0OiBbPDAwMDAwMDAwMTE2
ZmVkODQ+XQ0KcHJvY2Vzc19vbmVfd29yaysweDM4Ny8weGE1MA0KICMyOiAgKCYoJnEtPl9fcXVl
dWVfbG9jayktPnJsb2NrKXstLi0ufSwgYXQ6IFs8MDAwMDAwMDAzYzU4NDFlYz5dDQpibGtfdGlt
ZW91dF93b3JrKzB4NDUvMHgyMjANCg0KdGhlIGRlcGVuZGVuY2llcyBiZXR3ZWVuIEhBUkRJUlEt
aXJxLXNhZmUgbG9jayBhbmQgdGhlIGhvbGRpbmcgbG9jazoNCi0+ICgmKCZxLT5fX3F1ZXVlX2xv
Y2spLT5ybG9jayl7LS4tLn0gb3BzOiA1NTg1IHsNCiAgIElOLUhBUkRJUlEtVyBhdDoNCiAgICAg
ICAgICAgICAgICAgICAgX3Jhd19zcGluX2xvY2tfaXJxc2F2ZSsweDQ2LzB4NjANCiAgICAgICAg
ICAgICAgICAgICAgYXRhX3FjX3NjaGVkdWxlX2VoKzB4YjkvMHgxMTAgW2xpYmF0YV0NCiAgICAg
ICAgICAgICAgICAgICAgYXRhX3NmZl9oc21fbW92ZSsweDExNC8weGIxMCBbbGliYXRhXQ0KICAg
ICAgICAgICAgICAgICAgICBfX2F0YV9zZmZfcG9ydF9pbnRyKzB4ZWMvMHgxYTAgW2xpYmF0YV0N
CiAgICAgICAgICAgICAgICAgICAgYXRhX2JtZG1hX3BvcnRfaW50cisweGVmLzB4MTYwIFtsaWJh
dGFdDQogICAgICAgICAgICAgICAgICAgIGF0YV9ibWRtYV9pbnRlcnJ1cHQrMHgxNjAvMHgyZTAg
W2xpYmF0YV0NCiAgICAgICAgICAgICAgICAgICAgX19oYW5kbGVfaXJxX2V2ZW50X3BlcmNwdSsw
eDcyLzB4NDYwDQogICAgICAgICAgICAgICAgICAgIGhhbmRsZV9pcnFfZXZlbnRfcGVyY3B1KzB4
NjYvMHhkMA0KICAgICAgICAgICAgICAgICAgICBoYW5kbGVfaXJxX2V2ZW50KzB4NTQvMHg5MA0K
ICAgICAgICAgICAgICAgICAgICBoYW5kbGVfZWRnZV9pcnErMHhlOS8weDJkMA0KICAgICAgICAg
ICAgICAgICAgICBoYW5kbGVfaXJxKzB4MTdiLzB4MjEwDQogICAgICAgICAgICAgICAgICAgIGRv
X0lSUSsweDZjLzB4MTQwDQogICAgICAgICAgICAgICAgICAgIHJldF9mcm9tX2ludHIrMHgwLzB4
MWUNCiAgICAgICAgICAgICAgICAgICAgbmF0aXZlX3NhZmVfaGFsdCsweDIvMHgxMA0KICAgICAg
ICAgICAgICAgICAgICBkZWZhdWx0X2lkbGUrMHgxZi8weDIwMA0KICAgICAgICAgICAgICAgICAg
ICBkb19pZGxlKzB4MWJjLzB4MWYwDQogICAgICAgICAgICAgICAgICAgIGNwdV9zdGFydHVwX2Vu
dHJ5KzB4MTkvMHgyMA0KICAgICAgICAgICAgICAgICAgICBzdGFydF9rZXJuZWwrMHg0N2YvMHg0
YTENCiAgICAgICAgICAgICAgICAgICAgc2Vjb25kYXJ5X3N0YXJ0dXBfNjQrMHhhNS8weGIwDQog
ICBJTi1TT0ZUSVJRLVcgYXQ6DQogICAgICAgICAgICAgICAgICAgIF9yYXdfc3Bpbl9sb2NrX2ly
cXNhdmUrMHg0Ni8weDYwDQogICAgICAgICAgICAgICAgICAgIHNjc2lfZW5kX3JlcXVlc3QrMHgx
OTkvMHgzMTAgW3Njc2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBzY3NpX2lvX2NvbXBsZXRp
b24rMHgzYmUvMHg5ODAgW3Njc2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBibGtfZG9uZV9z
b2Z0aXJxKzB4MTc3LzB4MWMwDQogICAgICAgICAgICAgICAgICAgIF9fZG9fc29mdGlycSsweDEx
Ny8weDVmNQ0KICAgICAgICAgICAgICAgICAgICBpcnFfZXhpdCsweGU4LzB4ZjANCiAgICAgICAg
ICAgICAgICAgICAgZG9fSVJRKzB4YjYvMHgxNDANCiAgICAgICAgICAgICAgICAgICAgcmV0X2Zy
b21faW50cisweDAvMHgxZQ0KICAgICAgICAgICAgICAgICAgICBuYXRpdmVfc2FmZV9oYWx0KzB4
Mi8weDEwDQogICAgICAgICAgICAgICAgICAgIGRlZmF1bHRfaWRsZSsweDFmLzB4MjAwDQogICAg
ICAgICAgICAgICAgICAgIGRvX2lkbGUrMHgxYmMvMHgxZjANCiAgICAgICAgICAgICAgICAgICAg
Y3B1X3N0YXJ0dXBfZW50cnkrMHgxOS8weDIwDQogICAgICAgICAgICAgICAgICAgIHN0YXJ0X2tl
cm5lbCsweDQ3Zi8weDRhMQ0KICAgICAgICAgICAgICAgICAgICBzZWNvbmRhcnlfc3RhcnR1cF82
NCsweGE1LzB4YjANCiAgIElOSVRJQUwgVVNFIGF0Og0KICAgICAgICAgICAgICAgICAgIF9yYXdf
c3Bpbl9sb2NrX2lycSsweDNiLzB4NTANCiAgICAgICAgICAgICAgICAgICBibGtjZ19pbml0X3F1
ZXVlKzB4OTcvMHgxYzANCiAgICAgICAgICAgICAgICAgICBibGtfYWxsb2NfcXVldWVfbm9kZSsw
eDQxZC8weDRiMA0KICAgICAgICAgICAgICAgICAgIGJsa19tcV9pbml0X3F1ZXVlKzB4MjQvMHg2
MA0KICAgICAgICAgICAgICAgICAgIHZpcnRibGtfcHJvYmUrMHg2MzMvMHhmZWYgW3ZpcnRpb19i
bGtdDQogICAgICAgICAgICAgICAgICAgdmlydGlvX2Rldl9wcm9iZSsweDI1OS8weDM4MCBbdmly
dGlvXQ0KICAgICAgICAgICAgICAgICAgIGRyaXZlcl9wcm9iZV9kZXZpY2UrMHg0NjkvMHg2ODAN
CiAgICAgICAgICAgICAgICAgICBfX2RyaXZlcl9hdHRhY2grMHhlZi8weDEyMA0KICAgICAgICAg
ICAgICAgICAgIGJ1c19mb3JfZWFjaF9kZXYrMHhlNC8weDEzMA0KICAgICAgICAgICAgICAgICAg
IGJ1c19hZGRfZHJpdmVyKzB4MjRjLzB4MzgwDQogICAgICAgICAgICAgICAgICAgZHJpdmVyX3Jl
Z2lzdGVyKzB4YzYvMHgxNzANCiAgICAgICAgICAgICAgICAgICBhdGFfZ2VuZXJpY19pbml0X29u
ZSsweDViLzB4MjYwIFthdGFfZ2VuZXJpY10NCiAgICAgICAgICAgICAgICAgICBkb19vbmVfaW5p
dGNhbGwrMHg3OS8weDFiNw0KICAgICAgICAgICAgICAgICAgIGRvX2luaXRfbW9kdWxlKzB4ZGUv
MHgzMmQNCiAgICAgICAgICAgICAgICAgICBsb2FkX21vZHVsZSsweDM5NjQvMHg0N2QwDQogICAg
ICAgICAgICAgICAgICAgU1lTQ19maW5pdF9tb2R1bGUrMHgxNzYvMHgxYTANCiAgICAgICAgICAg
ICAgICAgICBkb19zeXNjYWxsXzY0KzB4ZjMvMHgyYzANCiAgICAgICAgICAgICAgICAgICBlbnRy
eV9TWVNDQUxMXzY0X2FmdGVyX2h3ZnJhbWUrMHg0Mi8weGI3DQogfQ0KIC4uLiBrZXkgICAgICBh
dDogWzwwMDAwMDAwMGRhNTEwMTI1Pl0gX19rZXkuNTMzNzUrMHgwLzB4NDANCiAuLi4gYWNxdWly
ZWQgYXQ6DQogICBfcmF3X3NwaW5fbG9jaysweDJmLzB4NDANCiAgIGlzY3NpX2VoX2NtZF90aW1l
ZF9vdXQrMHg2Yi8weDVhMCBbbGliaXNjc2ldDQogICBzY3NpX3RpbWVzX291dCsweGNjLzB4M2Yw
IFtzY3NpX21vZF0NCiAgIGJsa19ycV90aW1lZF9vdXQrMHgyZi8weDcwDQogICBibGtfdGltZW91
dF93b3JrKzB4MWIwLzB4MjIwDQogICBwcm9jZXNzX29uZV93b3JrKzB4NDQ2LzB4YTUwDQogICB3
b3JrZXJfdGhyZWFkKzB4N2IvMHg2ZDANCiAgIGt0aHJlYWQrMHgxYjcvMHgxZTANCiAgIHJldF9m
cm9tX2ZvcmsrMHgyNC8weDMwDQoNCg0KdGhlIGRlcGVuZGVuY2llcyBiZXR3ZWVuIHRoZSBsb2Nr
IHRvIGJlIGFjcXVpcmVkDQogYW5kIEhBUkRJUlEtaXJxLXVuc2FmZSBsb2NrOg0KLT4gKCYoJnNl
c3Npb24tPmZyd2RfbG9jayktPnJsb2NrKXsrLi0ufSBvcHM6IDE5ODUgew0KICAgSEFSRElSUS1P
Ti1XIGF0Og0KICAgICAgICAgICAgICAgICAgICBfcmF3X3NwaW5fbG9ja19iaCsweDM0LzB4NDAN
CiAgICAgICAgICAgICAgICAgICAgaXNjc2lfY29ubl9zZXR1cCsweDIzOS8weDMyMCBbbGliaXNj
c2ldDQogICAgICAgICAgICAgICAgICAgIGlzY3NpX3RjcF9jb25uX3NldHVwKzB4MTQvMHg4MCBb
bGliaXNjc2lfdGNwXQ0KICAgICAgICAgICAgICAgICAgICBpc2NzaV9zd190Y3BfY29ubl9jcmVh
dGUrMHgxZi8weDI0YSBbaXNjc2lfdGNwXQ0KICAgICAgICAgICAgICAgICAgICBpc2NzaV9pZl9y
eCsweDEzYzkvMHgyMGQwIFtzY3NpX3RyYW5zcG9ydF9pc2NzaV0NCiAgICAgICAgICAgICAgICAg
ICAgbmV0bGlua191bmljYXN0KzB4Mjk5LzB4MzMwDQogICAgICAgICAgICAgICAgICAgIG5ldGxp
bmtfc2VuZG1zZysweDQzNS8weDU4MA0KICAgICAgICAgICAgICAgICAgICBfX19zeXNfc2VuZG1z
ZysweDQ0Ny8weDRkMA0KICAgICAgICAgICAgICAgICAgICBfX3N5c19zZW5kbXNnKzB4YWQvMHgx
MjANCiAgICAgICAgICAgICAgICAgICAgZG9fc3lzY2FsbF82NCsweGYzLzB4MmMwDQogICAgICAg
ICAgICAgICAgICAgIGVudHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdmcmFtZSsweDQyLzB4YjcNCiAg
IElOLVNPRlRJUlEtVyBhdDoNCiAgICAgICAgICAgICAgICAgICAgX3Jhd19zcGluX2xvY2tfYmgr
MHgzNC8weDQwDQogICAgICAgICAgICAgICAgICAgIGlzY3NpX3F1ZXVlY29tbWFuZCsweGVmLzB4
OTYwIFtsaWJpc2NzaV0NCiAgICAgICAgICAgICAgICAgICAgc2NzaV9kaXNwYXRjaF9jbWQrMHgx
YzcvMHg1NTAgW3Njc2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBzY3NpX3JlcXVlc3RfZm4r
MHg4MjMvMHhhZjAgW3Njc2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBfX2Jsa19ydW5fcXVl
dWUrMHhjNS8weDE2MA0KICAgICAgICAgICAgICAgICAgICBibGtfcnVuX3F1ZXVlKzB4NDgvMHg3
MA0KICAgICAgICAgICAgICAgICAgICBzY3NpX3J1bl9xdWV1ZSsweDQ3NS8weDVkMCBbc2NzaV9t
b2RdDQogICAgICAgICAgICAgICAgICAgIHNjc2lfZW5kX3JlcXVlc3QrMHgxY2QvMHgzMTAgW3Nj
c2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBzY3NpX2lvX2NvbXBsZXRpb24rMHgzYmUvMHg5
ODAgW3Njc2lfbW9kXQ0KICAgICAgICAgICAgICAgICAgICBibGtfZG9uZV9zb2Z0aXJxKzB4MTc3
LzB4MWMwDQogICAgICAgICAgICAgICAgICAgIF9fZG9fc29mdGlycSsweDExNy8weDVmNQ0KICAg
ICAgICAgICAgICAgICAgICBydW5fa3NvZnRpcnFkKzB4MmUvMHg1MA0KICAgICAgICAgICAgICAg
ICAgICBzbXBib290X3RocmVhZF9mbisweDMxNC8weDQxMA0KICAgICAgICAgICAgICAgICAgICBr
dGhyZWFkKzB4MWI3LzB4MWUwDQogICAgICAgICAgICAgICAgICAgIHJldF9mcm9tX2ZvcmsrMHgy
NC8weDMwDQogICBJTklUSUFMIFVTRSBhdDoNCiAgICAgICAgICAgICAgICAgICBfcmF3X3NwaW5f
bG9ja19iaCsweDM0LzB4NDANCiAgICAgICAgICAgICAgICAgICBpc2NzaV9jb25uX3NldHVwKzB4
MjM5LzB4MzIwIFtsaWJpc2NzaV0NCiAgICAgICAgICAgICAgICAgICBpc2NzaV90Y3BfY29ubl9z
ZXR1cCsweDE0LzB4ODAgW2xpYmlzY3NpX3RjcF0NCiAgICAgICAgICAgICAgICAgICBpc2NzaV9z
d190Y3BfY29ubl9jcmVhdGUrMHgxZi8weDI0YSBbaXNjc2lfdGNwXQ0KICAgICAgICAgICAgICAg
ICAgIGlzY3NpX2lmX3J4KzB4MTNjOS8weDIwZDAgW3Njc2lfdHJhbnNwb3J0X2lzY3NpXQ0KICAg
ICAgICAgICAgICAgICAgIG5ldGxpbmtfdW5pY2FzdCsweDI5OS8weDMzMA0KICAgICAgICAgICAg
ICAgICAgIG5ldGxpbmtfc2VuZG1zZysweDQzNS8weDU4MA0KICAgICAgICAgICAgICAgICAgIF9f
X3N5c19zZW5kbXNnKzB4NDQ3LzB4NGQwDQogICAgICAgICAgICAgICAgICAgX19zeXNfc2VuZG1z
ZysweGFkLzB4MTIwDQogICAgICAgICAgICAgICAgICAgZG9fc3lzY2FsbF82NCsweGYzLzB4MmMw
DQogICAgICAgICAgICAgICAgICAgZW50cnlfU1lTQ0FMTF82NF9hZnRlcl9od2ZyYW1lKzB4NDIv
MHhiNw0KIH0NCiAuLi4ga2V5ICAgICAgYXQ6IFs8MDAwMDAwMDBkZjhmNDdiNT5dIF9fa2V5LjY4
MjkwKzB4MC8weGZmZmZmZmZmZmZmZjYzMDANCltsaWJpc2NzaV0NCiAuLi4gYWNxdWlyZWQgYXQ6
DQogICBfcmF3X3NwaW5fbG9jaysweDJmLzB4NDANCiAgIGlzY3NpX2VoX2NtZF90aW1lZF9vdXQr
MHg2Yi8weDVhMCBbbGliaXNjc2ldDQogICBzY3NpX3RpbWVzX291dCsweGNjLzB4M2YwIFtzY3Np
X21vZF0NCiAgIGJsa19ycV90aW1lZF9vdXQrMHgyZi8weDcwDQogICBibGtfdGltZW91dF93b3Jr
KzB4MWIwLzB4MjIwDQogICBwcm9jZXNzX29uZV93b3JrKzB4NDQ2LzB4YTUwDQogICB3b3JrZXJf
dGhyZWFkKzB4N2IvMHg2ZDANCiAgIGt0aHJlYWQrMHgxYjcvMHgxZTANCiAgIHJldF9mcm9tX2Zv
cmsrMHgyNC8weDMwDQoNCg0Kc3RhY2sgYmFja3RyYWNlOg0KQ1BVOiA2IFBJRDogMTU1IENvbW06
IGt3b3JrZXIvNjoxSCBOb3QgdGFpbnRlZCA0LjE2LjAtcmM3LWRiZysgIzMNCkhhcmR3YXJlIG5h
bWU6IEJvY2hzIEJvY2hzLCBCSU9TIEJvY2hzIDAxLzAxLzIwMTENCldvcmtxdWV1ZToga2Jsb2Nr
ZCBibGtfdGltZW91dF93b3JrDQpDYWxsIFRyYWNlOg0KIGR1bXBfc3RhY2srMHg4NS8weGM1DQog
Y2hlY2tfdXNhZ2UrMHg2ZTcvMHg3MDANCiA/IGNoZWNrX3VzYWdlX2ZvcndhcmRzKzB4MjIwLzB4
MjIwDQogPyBmaW5kX25leHRfYW5kX2JpdCsweDUxLzB4ZTANCiA/IGNwdW1hc2tfbmV4dF9hbmQr
MHgyMC8weDMwDQogPyBmaW5kX2J1c2llc3RfZ3JvdXArMHhjOTQvMHgxMDEwDQogPyBjbGFzc19l
cXVhbCsweDExLzB4MjANCiA/IF9fYmZzKzB4NjIvMHgyZTANCiA/IGNsYXNzX2VxdWFsKzB4MTEv
MHgyMA0KID8gX19iZnMrMHhmYi8weDJlMA0KID8gX19sb2NrX2FjcXVpcmUrMHgxN2FhLzB4MWFm
MA0KIF9fbG9ja19hY3F1aXJlKzB4MTdhYS8weDFhZjANCiA/IG1hcmtfbG9jaysweGM3LzB4Nzcw
DQogPyBkZWJ1Z19jaGVja19ub19sb2Nrc19mcmVlZCsweDFiMC8weDFiMA0KID8gX19sb2NrX2Fj
cXVpcmUrMHg1ODMvMHgxYWYwDQogPyBtYXJrX2xvY2srMHhjNy8weDc3MA0KID8gbG9ja19waW5f
bG9jaysweDE2MC8weDE2MA0KID8gZGVidWdfY2hlY2tfbm9fbG9ja3NfZnJlZWQrMHgxYjAvMHgx
YjANCiA/IGxvY2tfYWNxdWlyZSsweGM5LzB4MjYwDQogbG9ja19hY3F1aXJlKzB4YzkvMHgyNjAN
CiA/IGlzY3NpX2VoX2NtZF90aW1lZF9vdXQrMHg2Yi8weDVhMCBbbGliaXNjc2ldDQogX3Jhd19z
cGluX2xvY2srMHgyZi8weDQwDQogPyBpc2NzaV9laF9jbWRfdGltZWRfb3V0KzB4NmIvMHg1YTAg
W2xpYmlzY3NpXQ0KIGlzY3NpX2VoX2NtZF90aW1lZF9vdXQrMHg2Yi8weDVhMCBbbGliaXNjc2ld
DQogc2NzaV90aW1lc19vdXQrMHhjYy8weDNmMCBbc2NzaV9tb2RdDQogYmxrX3JxX3RpbWVkX291
dCsweDJmLzB4NzANCiBibGtfdGltZW91dF93b3JrKzB4MWIwLzB4MjIwDQogcHJvY2Vzc19vbmVf
d29yaysweDQ0Ni8weGE1MA0KID8gcHdxX2RlY19ucl9pbl9mbGlnaHQrMHgxMDAvMHgxMDANCiA/
IHdvcmtlcl90aHJlYWQrMHgxNzcvMHg2ZDANCiB3b3JrZXJfdGhyZWFkKzB4N2IvMHg2ZDANCiA/
IHByb2Nlc3Nfb25lX3dvcmsrMHhhNTAvMHhhNTANCiBrdGhyZWFkKzB4MWI3LzB4MWUwDQogPyBr
dGhyZWFkX2NyZWF0ZV93b3JrZXJfb25fY3B1KzB4YzAvMHhjMA0KIHJldF9mcm9tX2ZvcmsrMHgy
NC8weDMwDQoNClsgLi4uIF0NCg0KLS0tLS0tLS0tLS0tWyBjdXQgaGVyZSBdLS0tLS0tLS0tLS0t
DQprZXJuZWwgQlVHIGF0IGJsb2NrL2Jsay1jb3JlLmM6MzI2NyENCmludmFsaWQgb3Bjb2RlOiAw
MDAwIFsjMV0gUFJFRU1QVCBTTVAgS0FTQU4NCk1vZHVsZXMgbGlua2VkIGluOiBzZF9tb2QgY3Jj
MzJjX2dlbmVyaWMgdGFyZ2V0X2NvcmVfcHNjc2kNCnRhcmdldF9jb3JlX2libG9jayB0YXJnZXRf
Y29yZV9maWxlIGlzY3NpX3RhcmdldF9tb2QgdGFyZ2V0X2NvcmVfbW9kIGJyZA0KaTJjX3BpaXg0
IHNnIHZpcnRpb19iYWxsb29uIGkyY19jb3JlIGFmX3BhY2tldCBidXR0b24gaWJfaXNlciByZG1h
X2NtIGl3X2NtDQppYl9jbSBpYl9jb3JlIGNvbmZpZ2ZzIGlzY3NpX3RjcCBsaWJpc2NzaV90Y3Ag
bGliaXNjc2kgc2NzaV90cmFuc3BvcnRfaXNjc2kNCmlwX3RhYmxlcyB4X3RhYmxlcyBhdXRvZnM0
IGhpZF9nZW5lcmljIHVzYmhpZCBoaWQgZXh0NCBjcmMxNiBtYmNhY2hlIGpiZDINCnNyX21vZCBj
ZHJvbSBhdGFfZ2VuZXJpYyBwYXRhX2FjcGkgdmlydGlvX2JsayB2aXJ0aW9fbmV0IGF0YV9waWl4
IHhoY2lfcGNpDQp4aGNpX2hjZCBsaWJhdGEgdmlydGlvX3BjaSB1c2Jjb3JlIHNjc2lfbW9kIHZp
cnRpb19yaW5nIGludGVsX2FncCB1c2JfY29tbW9uDQppbnRlbF9ndHQgdmlydGlvIGFncGdhcnQN
CkNQVTogMiBQSUQ6IDE1MSBDb21tOiBzY3NpX2VoXzEgVGFpbnRlZDogRyAgICAgICAgVyAgICAg
ICAgNC4xNi4wLXJjNy1kYmcrDQojMw0KSGFyZHdhcmUgbmFtZTogQm9jaHMgQm9jaHMsIEJJT1Mg
Qm9jaHMgMDEvMDEvMjAxMQ0KUklQOiAwMDEwOl9fYmxrX2VuZF9yZXF1ZXN0X2FsbCsweGRhLzB4
ZTANClJTUDogMDAxODpmZmZmODgwMDYxOTJmOTgwIEVGTEFHUzogMDAwMTAwMDINCnNyIDI6MDow
OjM6IHJlamVjdGluZyBJL08gdG8gb2ZmbGluZSBkZXZpY2UNCnNyIDM6MDowOjM6IHJlamVjdGlu
ZyBJL08gdG8gb2ZmbGluZSBkZXZpY2UNClJBWDogMDAwMDAwMDAwMDAwMDAwMSBSQlg6IGZmZmY4
ODAwNjgxOGU3ODAgUkNYOiBmZmZmZmZmZjgxNDA2NWE2DQpSRFg6IDAwMDAwMDAwMDAwMDAwMDEg
UlNJOiAwMDAwMDAwMDAwMDAwMDAxIFJESTogZmZmZjg4MDA2ODE4ZTgzOA0KUkJQOiAwMDAwMDAw
MDAwMDAwMDBhIFIwODogMDAwMDAwMDAwMDAwMDAwMCBSMDk6IDAwMDAwMDAwMDAwMDAwMTINClIx
MDogZmZmZjg4MDA2MTkyZjU4OCBSMTE6IDAwMDAwMDAwNWU0Nzg2YTMgUjEyOiAwMDAwMDAwMDAw
MDAwMDAwDQpSMTM6IDAwMDAwMDAwMDAwMDAwMDAgUjE0OiBmZmZmODgwMDYxMjgwMTYwIFIxNTog
MDAwMDAwMDAwMDAwMDAwMQ0KRlM6ICAwMDAwMDAwMDAwMDAwMDAwKDAwMDApIEdTOmZmZmY4ODAw
NmQyODAwMDAoMDAwMCkga25sR1M6MDAwMDAwMDAwMDAwMDAwMA0KQ1M6ICAwMDEwIERTOiAwMDAw
IEVTOiAwMDAwIENSMDogMDAwMDAwMDA4MDA1MDAzMw0KQ1IyOiAwMDAwN2YxZmNjNTNmMDEwIENS
MzogMDAwMDAwMDA2NjZmYzAwMCBDUjQ6IDAwMDAwMDAwMDAwMDA2ZTANCkNhbGwgVHJhY2U6DQog
YmxrX3BlZWtfcmVxdWVzdCsweDFmZi8weDVmMA0KIHNjc2lfcmVxdWVzdF9mbisweDQ4LzB4YWYw
IFtzY3NpX21vZF0NCiA/IGxvY2tfYWNxdWlyZSsweGM5LzB4MjYwDQogX19ibGtfcnVuX3F1ZXVl
KzB4YzUvMHgxNjANCiBibGtfcnVuX3F1ZXVlKzB4NDgvMHg3MA0KIHNjc2lfcnVuX3F1ZXVlKzB4
NDc1LzB4NWQwIFtzY3NpX21vZF0NCiA/IHNjc2lfaW9fY29tcGxldGlvbisweDY5ZS8weDk4MCBb
c2NzaV9tb2RdDQogPyBzZGV2X2V2dF9hbGxvYysweDgwLzB4ODAgW3Njc2lfbW9kXQ0KID8gYmxr
X3F1ZXVlX2VuZF90YWcrMHgxMGEvMHgyMTANCiA/IF9fbGlzdF9hZGRfdmFsaWQrMHgyOS8weGEw
DQogPyBkb19yYXdfc3Bpbl91bmxvY2srMHg5MS8weDEyMA0KIHNjc2lfaW9fY29tcGxldGlvbisw
eDZhNi8weDk4MCBbc2NzaV9tb2RdDQogPyBsb2NrX2Rvd25ncmFkZSsweDIwMC8weDJiMA0KID8g
c2NzaV9lbmRfcmVxdWVzdCsweDMxMC8weDMxMCBbc2NzaV9tb2RdDQogPyBzY3NpX2RldmljZV91
bmJ1c3krMHgzYi8weDYwIFtzY3NpX21vZF0NCiBzY3NpX2VoX2ZsdXNoX2RvbmVfcSsweDFhNi8w
eDIxMCBbc2NzaV9tb2RdDQogYXRhX3Njc2lfcG9ydF9lcnJvcl9oYW5kbGVyKzB4Nzk0LzB4YjAw
IFtsaWJhdGFdDQogYXRhX3Njc2lfZXJyb3IrMHgxNjgvMHgxYTAgW2xpYmF0YV0NCiA/IGF0YV9z
Y3NpX3BvcnRfZXJyb3JfaGFuZGxlcisweGIwMC8weGIwMCBbbGliYXRhXQ0KID8gX3Jhd19zcGlu
X3VubG9ja19pcnFyZXN0b3JlKzB4NTkvMHg3MA0KIHNjc2lfZXJyb3JfaGFuZGxlcisweDFiNS8w
eGE0MCBbc2NzaV9tb2RdDQogPyBzY3NpX2VoX2dldF9zZW5zZSsweDNiMC8weDNiMCBbc2NzaV9t
b2RdDQogPyBfX3NjaGVkX3RleHRfc3RhcnQrMHg4LzB4OA0KID8gX193YWtlX3VwX2NvbW1vbisw
eDljLzB4MjMwDQogPyBtYXJrX2hlbGRfbG9ja3MrMHgxYy8weDkwDQogPyBfcmF3X3NwaW5fdW5s
b2NrX2lycXJlc3RvcmUrMHg1OS8weDcwDQogPyBzY3NpX2VoX2dldF9zZW5zZSsweDNiMC8weDNi
MCBbc2NzaV9tb2RdDQoga3RocmVhZCsweDFiNy8weDFlMA0KID8ga3RocmVhZF9jcmVhdGVfd29y
a2VyX29uX2NwdSsweGMwLzB4YzANCiByZXRfZnJvbV9mb3JrKzB4MjQvMHgzMA0KQ29kZTogODUg
YzAgNzUgMDIgMGYgMGIgNDggODkgZGYgZTggYjMgOTkgZWIgZmYgNGMgOGIgMjMgZTkgNmUgZmYg
ZmYgZmYgMGYNCjBiIGViIDgyIDQ5IDhkIDdjIDI0IDIwIGU4IDlkIDk4IGViIGZmIDQ1IDhiIDZj
IDI0IDIwIGViIDhjIDwwZj4gMGIgMGYgMWYgNDANCjAwIDBmIDFmIDQ0IDAwIDAwIDQxIDU3IDQx
IDU2IDQxIDU1IDQxIDU0IDU1IDQ4IA0KUklQOiBfX2Jsa19lbmRfcmVxdWVzdF9hbGwrMHhkYS8w
eGUwIFJTUDogZmZmZjg4MDA2MTkyZjk4MA0KLS0tWyBlbmQgdHJhY2UgYjljMjQyOWUzMWFjZWRi
NCBdLS0t

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Sun, 2018-04-01 at 07:37 -0400, Wakko Warner wrote:
> > Bart Van Assche wrote:
> > > On Sat, 2018-03-31 at 18:12 -0400, Wakko Warner wrote:
> > > > Richard Weinberger wrote:
> > > > > On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> > > > > > I reported this before but noone responded.
> > > > > 
> > > > > Because you're sending only to LKML.
> > > > > CC'ing storage folks.
> > > > 
> > > > Thank you.  I wasn't sure who I needed to send it to.
> > > 
> > > Can you share the output of lsscsi? I would like to know whether or not you
> > > are using a (S)ATA CDROM.
> > 
> > From the target:
> > [4:0:0:0]    cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr0 
> > [5:0:0:0]    cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr1 
> > [6:0:0:0]    cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr2 
> > 
> > From the initiator:
> > [19:0:0:0]   cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr1
> > [19:0:0:1]   cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr2
> > [19:0:0:2]   cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr3
> > 
> > I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> > From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> > /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> > crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> > the initiator with out problems.  I'll test other kernels between 4.9 and
> > 4.14.
> 
> (+Lee and Chris)
> 
> Hello Wakko,
> 
> Although I'm not sure that what I ran into is exactly the same as what you
> ran into, there is definitely something wrong with what I encountered. What
> I ran into with Linus' latest master branch indicates two issues - one in
> the iSCSI initiator and one in the block layer:
> 
> scsi 3:0:0:1: Direct-Access     LIO-ORG  FILEIO           4.0  PQ: 0 ANSI: 5
> sd 2:0:0:1: [sdd] Attached SCSI disk
> sd 3:0:0:1: Warning! Received an indication that the LUN assignments on this
> target have changed. The Linux SCSI layer does not automatical
> sd 3:0:0:1: Attached scsi generic sg8 type 0
> sd 3:0:0:1: [sdf] 128 512-byte logical blocks: (65.5 kB/64.0 KiB)
> sd 3:0:0:1: [sdf] Write Protect is off
> sd 3:0:0:1: [sdf] Mode Sense: 43 00 00 08
> sd 3:0:0:1: [sdf] Write cache: disabled, read cache: enabled, doesn't
> support DPO or FUA
> iSCSI/iqn.1993-08.org.debian:01:3b68b1b3d2eb: Unsupported SCSI Opcode 0xa3,
> sending CHECK_CONDITION.
> sd 3:0:0:2: [sde] Attached SCSI disk
> sd 3:0:0:1: [sdf] Attached SCSI disk
> 
> =====================================================
> WARNING: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected
> 4.16.0-rc7-dbg+ #3 Not tainted
> -----------------------------------------------------
> kworker/6:1H/155 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
>  (&(&session->frwd_lock)->rlock){+.-.}, at: [<000000007eb678ec>]
> iscsi_eh_cmd_timed_out+0x6b/0x5a0 [libiscsi]

[trimmed]

I'm not sure.  Mine happens as 2 oopses.  Both have <IRQ> </IRQ> lines.
The files mine happen in are drivers/scsi/scsi_lib.c followed by
block/blk-core.c

The first one, the stack trace began with <IRQ> then scsi_setup_cmnd.  I
tested 4.10.x, 4.11.x 4.12.x 4.14.x 4.15.x where x is the latest patch
(except for 4.15).  ALL crash.  4.9.91 doesn't.  4.10 added dump_stack
__warn scsi_init_io after <IRQ> and before scsi_setup_cmnd.  Within seconds
of issueing the command to read files, it crashes.  On 4.15, if I just do a
sequential read from the raw device, it doesn't crash.

What do you enable in the kernel to get those locking messages?

> stack backtrace:
> CPU: 6 PID: 155 Comm: kworker/6:1H Not tainted 4.16.0-rc7-dbg+ #3
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> Workqueue: kblockd blk_timeout_work
> Call Trace:
>  dump_stack+0x85/0xc5
>  check_usage+0x6e7/0x700
>  ? check_usage_forwards+0x220/0x220
>  ? find_next_and_bit+0x51/0xe0
>  ? cpumask_next_and+0x20/0x30
>  ? find_busiest_group+0xc94/0x1010
>  ? class_equal+0x11/0x20
>  ? __bfs+0x62/0x2e0
>  ? class_equal+0x11/0x20
>  ? __bfs+0xfb/0x2e0
>  ? __lock_acquire+0x17aa/0x1af0
>  __lock_acquire+0x17aa/0x1af0
>  ? mark_lock+0xc7/0x770
>  ? debug_check_no_locks_freed+0x1b0/0x1b0
>  ? __lock_acquire+0x583/0x1af0
>  ? mark_lock+0xc7/0x770
>  ? lock_pin_lock+0x160/0x160
>  ? debug_check_no_locks_freed+0x1b0/0x1b0
>  ? lock_acquire+0xc9/0x260
>  lock_acquire+0xc9/0x260
>  ? iscsi_eh_cmd_timed_out+0x6b/0x5a0 [libiscsi]
>  _raw_spin_lock+0x2f/0x40
>  ? iscsi_eh_cmd_timed_out+0x6b/0x5a0 [libiscsi]
>  iscsi_eh_cmd_timed_out+0x6b/0x5a0 [libiscsi]
>  scsi_times_out+0xcc/0x3f0 [scsi_mod]
>  blk_rq_timed_out+0x2f/0x70
>  blk_timeout_work+0x1b0/0x220
>  process_one_work+0x446/0xa50
>  ? pwq_dec_nr_in_flight+0x100/0x100
>  ? worker_thread+0x177/0x6d0
>  worker_thread+0x7b/0x6d0
>  ? process_one_work+0xa50/0xa50
>  kthread+0x1b7/0x1e0
>  ? kthread_create_worker_on_cpu+0xc0/0xc0
>  ret_from_fork+0x24/0x30
> 
> [ ... ]
> 
> ------------[ cut here ]------------
> kernel BUG at block/blk-core.c:3267!
> invalid opcode: 0000 [#1] PREEMPT SMP KASAN
> Modules linked in: sd_mod crc32c_generic target_core_pscsi
> target_core_iblock target_core_file iscsi_target_mod target_core_mod brd
> i2c_piix4 sg virtio_balloon i2c_core af_packet button ib_iser rdma_cm iw_cm
> ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
> ip_tables x_tables autofs4 hid_generic usbhid hid ext4 crc16 mbcache jbd2
> sr_mod cdrom ata_generic pata_acpi virtio_blk virtio_net ata_piix xhci_pci
> xhci_hcd libata virtio_pci usbcore scsi_mod virtio_ring intel_agp usb_common
> intel_gtt virtio agpgart
> CPU: 2 PID: 151 Comm: scsi_eh_1 Tainted: G        W        4.16.0-rc7-dbg+
> #3
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> RIP: 0010:__blk_end_request_all+0xda/0xe0
> RSP: 0018:ffff88006192f980 EFLAGS: 00010002
> sr 2:0:0:3: rejecting I/O to offline device
> sr 3:0:0:3: rejecting I/O to offline device
> RAX: 0000000000000001 RBX: ffff88006818e780 RCX: ffffffff814065a6
> RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff88006818e838
> RBP: 000000000000000a R08: 0000000000000000 R09: 0000000000000012
> R10: ffff88006192f588 R11: 000000005e4786a3 R12: 0000000000000000
> R13: 0000000000000000 R14: ffff880061280160 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffff88006d280000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f1fcc53f010 CR3: 00000000666fc000 CR4: 00000000000006e0
> Call Trace:
>  blk_peek_request+0x1ff/0x5f0
>  scsi_request_fn+0x48/0xaf0 [scsi_mod]
>  ? lock_acquire+0xc9/0x260
>  __blk_run_queue+0xc5/0x160
>  blk_run_queue+0x48/0x70
>  scsi_run_queue+0x475/0x5d0 [scsi_mod]
>  ? scsi_io_completion+0x69e/0x980 [scsi_mod]
>  ? sdev_evt_alloc+0x80/0x80 [scsi_mod]
>  ? blk_queue_end_tag+0x10a/0x210
>  ? __list_add_valid+0x29/0xa0
>  ? do_raw_spin_unlock+0x91/0x120
>  scsi_io_completion+0x6a6/0x980 [scsi_mod]
>  ? lock_downgrade+0x200/0x2b0
>  ? scsi_end_request+0x310/0x310 [scsi_mod]
>  ? scsi_device_unbusy+0x3b/0x60 [scsi_mod]
>  scsi_eh_flush_done_q+0x1a6/0x210 [scsi_mod]
>  ata_scsi_port_error_handler+0x794/0xb00 [libata]
>  ata_scsi_error+0x168/0x1a0 [libata]
>  ? ata_scsi_port_error_handler+0xb00/0xb00 [libata]
>  ? _raw_spin_unlock_irqrestore+0x59/0x70
>  scsi_error_handler+0x1b5/0xa40 [scsi_mod]
>  ? scsi_eh_get_sense+0x3b0/0x3b0 [scsi_mod]
>  ? __sched_text_start+0x8/0x8
>  ? __wake_up_common+0x9c/0x230
>  ? mark_held_locks+0x1c/0x90
>  ? _raw_spin_unlock_irqrestore+0x59/0x70
>  ? scsi_eh_get_sense+0x3b0/0x3b0 [scsi_mod]
>  kthread+0x1b7/0x1e0
>  ? kthread_create_worker_on_cpu+0xc0/0xc0
>  ret_from_fork+0x24/0x30
> Code: 85 c0 75 02 0f 0b 48 89 df e8 b3 99 eb ff 4c 8b 23 e9 6e ff ff ff 0f
> 0b eb 82 49 8d 7c 24 20 e8 9d 98 eb ff 45 8b 6c 24 20 eb 8c <0f> 0b 0f 1f 40
> 00 0f 1f 44 00 00 41 57 41 56 41 55 41 54 55 48 
> RIP: __blk_end_request_all+0xda/0xe0 RSP: ffff88006192f980
> ---[ end trace b9c2429e31acedb4 ]---
-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

[-- Attachment #1: Type: text/plain, Size: 357 bytes --]

On Sun, 2018-04-01 at 12:24 -0400, Wakko Warner wrote:
> What do you enable in the kernel to get those locking messages?

Hello Wakko,

I have attached the script to this e-mail that I use to enable a bunch of
kernel debugging options. Please note that enabling these options,
especially lockdep and kasan, will slow down the kernel.

Bart.




[-- Attachment #2: enable-kernel-debugging-options --]
[-- Type: application/x-shellscript, Size: 1593 bytes --]

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Bart Van Assche wrote:
> > On Sat, 2018-03-31 at 18:12 -0400, Wakko Warner wrote:
> > > Richard Weinberger wrote:
> > > > On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> > > > > I reported this before but noone responded.
> > > > 
> > > > Because you're sending only to LKML.
> > > > CC'ing storage folks.
> > > 
> > > Thank you.  I wasn't sure who I needed to send it to.
> > 
> > Can you share the output of lsscsi? I would like to know whether or not you
> > are using a (S)ATA CDROM.
> 
> >From the target:
> [4:0:0:0]    cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr0 
> [5:0:0:0]    cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr1 
> [6:0:0:0]    cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr2 
> 
> >From the initiator:
> [19:0:0:0]   cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr1
> [19:0:0:1]   cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr2
> [19:0:0:2]   cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr3
> 
> 
> I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> >From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> the initiator with out problems.  I'll test other kernels between 4.9 and
> 4.14.

So I've tested 4.x.y where x one of 10 11 12 14 15 and y is the latest patch
(except for 4.15 which was 1 behind)
Each of these kernels crash within seconds or immediate of doing find -type
f | xargs cat > /dev/null from the initiator.

I did a diff between 4.9.91 and 4.10.17 on scsi_lib.c.  Here's the
difference around the line reported (in this case 1043).  I've added the
4.10.17 oops at the end:

@@ -1029,10 +1038,10 @@ int scsi_init_io(struct scsi_cmnd *cmd)
        struct scsi_device *sdev = cmd->device;
        struct request *rq = cmd->request;
        bool is_mq = (rq->mq_ctx != NULL);
-       int error;
+       int error = BLKPREP_KILL;
 
-       if (WARN_ON_ONCE(!rq->nr_phys_segments))
-               return -EINVAL;
+       if (WARN_ON_ONCE(!blk_rq_nr_phys_segments(rq)))
+               goto err_exit;
 
        error = scsi_init_sgtable(rq, &cmd->sdb);
        if (error)

Oops:
[ 158.157590] ------------[ cut here ]------------
[ 158.157601] WARNING: CPU: 0 PID: 0 at /usr/src/linux/dist/4.10.17-nobklcd/drivers/scsi/scsi_lib.c:1043 scsi_init_io+0x1d7/0x1e0 [scsi_mod]
[ 158.157603] Modules linked in: iscsi_target_mod tcm_loop af_packet vhost_scsi vhost target_core_file target_core_iblock target_core_pscsi target_core_mod nfsd exportfs dummy bridge stp llc ib_iser rdma_cm iw_cm ib_cm ib_core ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi netconsole configfs sr_mod cdrom sd_mod sg adt7475 hwmon_vid coretemp x86_pkg_temp_thermal kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic nouveau video led_class drm_kms_helper cfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea ttm drm agpgart snd_hda_intel snd_hda_codec snd_hda_core mptsas snd_pcm_oss snd_mixer_oss mptscsih mpt3sas snd_pcm mptbase snd_timer raid_class aesni_intel snd scsi_transport_sas
[ 158.157634]  igb soundcore aes_x86_64 crypto_simd ahci glue_helper libahci hwmon libata i2c_algo_bit i2c_core scsi_mod wmi hed button unix
[ 158.157642] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.10.17 #1
[ 158.157644] Hardware name: Dell Inc. Precision T5610/0WN7Y6, BIOS A16 02/05/2018
[ 158.157645] Call Trace:
[ 158.157647]  <IRQ>
[ 158.157651]  ? dump_stack+0x46/0x5a
[ 158.157653]  ? __warn+0xb4/0xd0
[ 158.157656]  ? scsi_init_io+0x1d7/0x1e0 [scsi_mod]
[ 158.157658]  ? scsi_setup_cmnd+0x4c/0x140 [scsi_mod]
[ 158.157661]  ? scsi_prep_fn+0xe3/0x170 [scsi_mod]
[ 158.157663]  ? swiotlb_unmap_sg_attrs+0x44/0x60
[ 158.157665]  ? blk_peek_request+0x130/0x200
[ 158.157668]  ? scsi_request_fn+0x2b/0x510 [scsi_mod]
[ 158.157670]  ? __blk_run_queue+0x2a/0x40
[ 158.157672]  ? blk_run_queue+0x1c/0x30
[ 158.157675]  ? scsi_run_queue+0x229/0x2b0 [scsi_mod]
[ 158.157677]  ? scsi_io_completion+0x3d6/0x5c0 [scsi_mod]
[ 158.157680]  ? blk_done_softirq+0x67/0x80
[ 158.157682]  ? __do_softirq+0xdb/0x200
[ 158.157683]  ? irq_exit+0xa3/0xb0
[ 158.157686]  ? do_IRQ+0x45/0xc0
[ 158.157689]  ? common_interrupt+0x7c/0x7c
[ 158.157690]  </IRQ>
[ 158.157693]  ? cpuidle_enter_state+0x144/0x1f0
[ 158.157694]  ? cpuidle_enter_state+0x139/0x1f0
[ 158.157696]  ? do_idle+0xd3/0x190
[ 158.157698]  ? cpu_startup_entry+0x14/0x20
[ 158.157700]  ? start_kernel+0x391/0x399
[ 158.157701]  ? start_cpu+0x14/0x14
[ 158.157703] ---[ end trace 8d60c2e92fac2697 ]---
[ 158.157711] ------------[ cut here ]------------
[ 158.157732] kernel BUG at /usr/src/linux/dist/4.10.17-nobklcd/block/blk-core.c:2916!
[ 158.157755] invalid opcode: 0000 [#1] PREEMPT SMP
[ 158.157770] Modules linked in: iscsi_target_mod tcm_loop af_packet vhost_scsi vhost target_core_file target_core_iblock target_core_pscsi target_core_mod nfsd exportfs dummy bridge stp llc ib_iser rdma_cm iw_cm ib_cm ib_core ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi netconsole configfs sr_mod cdrom sd_mod sg adt7475 hwmon_vid coretemp x86_pkg_temp_thermal kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic nouveau video led_class drm_kms_helper cfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea ttm drm agpgart snd_hda_intel snd_hda_codec snd_hda_core mptsas snd_pcm_oss snd_mixer_oss mptscsih mpt3sas snd_pcm mptbase snd_timer raid_class aesni_intel snd scsi_transport_sas
[ 158.157968]  igb soundcore aes_x86_64 crypto_simd ahci glue_helper libahci hwmon libata i2c_algo_bit i2c_core scsi_mod wmi hed button unix
[ 158.158005] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W       4.10.17 #1
[ 158.158024] Hardware name: Dell Inc. Precision T5610/0WN7Y6, BIOS A16 02/05/2018
[ 158.158045] task: ffffffff8180e4c0 task.stack: ffffffff81800000
[ 158.158063] RIP: 0010:__blk_end_request_all+0x2a/0x30
[ 158.158077] RSP: 0018:ffff8806b7803df0 EFLAGS: 00010002
[ 158.158093] RAX: 0000000000000001 RBX: ffff8806abfdb2f0 RCX: 0000000000000000
[ 158.158113] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff8806abfdb2f0
[ 158.158134] RBP: ffff8806accb28d0 R08: 0000000000000000 R09: 0000000000000000
[ 158.158153] R10: ffffffff81806a40 R11: 0000000000000000 R12: 00000000ffffff87
[ 158.158173] R13: 00000000fffffffb R14: 00000000fffffffb R15: 0000000000000000
[ 158.158193] FS:  0000000000000000(0000) GS:ffff8806b7800000(0000) knlGS:0000000000000000
[ 158.158215] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 158.158231] CR2: 00007ffdeb1091b8 CR3: 0000000001809000 CR4: 00000000001406f0
[ 158.158250] Call Trace:
[ 158.158258]  <IRQ>
[ 158.158265]  ? blk_peek_request+0x16b/0x200
[ 158.158279]  ? scsi_request_fn+0x2b/0x510 [scsi_mod]
[ 158.158294]  ? __blk_run_queue+0x2a/0x40
[ 158.158306]  ? blk_run_queue+0x1c/0x30
[ 158.158319]  ? scsi_run_queue+0x229/0x2b0 [scsi_mod]
[ 158.158334]  ? scsi_io_completion+0x3d6/0x5c0 [scsi_mod]
[ 158.158350]  ? blk_done_softirq+0x67/0x80
[ 158.158362]  ? __do_softirq+0xdb/0x200
[ 158.158374]  ? irq_exit+0xa3/0xb0
[ 158.158384]  ? do_IRQ+0x45/0xc0
[ 158.158394]  ? common_interrupt+0x7c/0x7c
[ 158.158407]  </IRQ>
[ 158.158415]  ? cpuidle_enter_state+0x144/0x1f0
[ 158.158429]  ? cpuidle_enter_state+0x139/0x1f0
[ 158.158443]  ? do_idle+0xd3/0x190
[ 158.158453]  ? cpu_startup_entry+0x14/0x20
[ 158.158466]  ? start_kernel+0x391/0x399
[ 158.158478]  ? start_cpu+0x14/0x14
[ 158.158488] Code: 00 48 8b 87 70 01 00 00 31 c9 48 85 c0 75 0d 8b 57 58 e8 1a ff ff ff 84 c0 75 10 c3 8b 48 58 8b 57 58 e8 0a ff ff ff 84 c0 74 f0 <0f> 0b 0f 1f 40 00 41 56 41 55 41 bd fb ff ff ff 41 54 41 bc 87 
[ 158.158550] RIP: __blk_end_request_all+0x2a/0x30 RSP: ffff8806b7803df0
[ 158.161579] ---[ end trace 8d60c2e92fac2698 ]---
[ 158.161579] Kernel panic - not syncing: Fatal exception in interrupt
[ 158.161579] Kernel Offset: disabled
[ 158.161579] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Wakko Warner wrote:
> > Bart Van Assche wrote:
> > > On Sat, 2018-03-31 at 18:12 -0400, Wakko Warner wrote:
> > > > Richard Weinberger wrote:
> > > > > On Sat, Mar 31, 2018 at 3:59 AM, Wakko Warner <wakko@animx.eu.org> wrote:
> > > > > > I reported this before but noone responded.
> > > > > 
> > > > > Because you're sending only to LKML.
> > > > > CC'ing storage folks.
> > > > 
> > > > Thank you.  I wasn't sure who I needed to send it to.
> > > 
> > > Can you share the output of lsscsi? I would like to know whether or not you
> > > are using a (S)ATA CDROM.
> > 
> > >From the target:
> > [4:0:0:0]    cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr0 
> > [5:0:0:0]    cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr1 
> > [6:0:0:0]    cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr2 
> > 
> > >From the initiator:
> > [19:0:0:0]   cd/dvd  ATAPI    iHAS224   B      GL05  /dev/sr1
> > [19:0:0:1]   cd/dvd  ATAPI    iHAS422   8      4L11  /dev/sr2
> > [19:0:0:2]   cd/dvd  PBDS     DVD+-RW DH-16W1S 2D14  /dev/sr3
> > 
> > 
> > I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> > >From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> > /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> > crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> > the initiator with out problems.  I'll test other kernels between 4.9 and
> > 4.14.
> 
> So I've tested 4.x.y where x one of 10 11 12 14 15 and y is the latest patch
> (except for 4.15 which was 1 behind)
> Each of these kernels crash within seconds or immediate of doing find -type
> f | xargs cat > /dev/null from the initiator.

I tried 4.10.0.  It doesn't completely lockup the system, but the device
that was used hangs.  So from the initiator, it's /dev/sr1 and from the
target it's /dev/sr0.  Attempting to read /dev/sr0 after the oops causes the
process to hang in D state.

Here's the oops.  There was also another line that was not seen in the newer
kernels.
[ 323.105044] ------------[ cut here ]------------
[ 323.105057] WARNING: CPU: 0 PID: 0 at /usr/src/linux/dist/4.10/drivers/scsi/scsi_lib.c:1043 scsi_init_io+0x143/0x1f0 [scsi_mod]
[ 323.105058] Modules linked in: iscsi_target_mod af_packet tcm_loop vhost_scsi vhost target_core_file target_core_iblock target_core_pscsi target_core_mod nfsd exportfs dummy bridge stp llc ib_iser rdma_cm iw_cm ib_cm ib_core ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi netconsole configfs sr_mod cdrom sd_mod sg adt7475 hwmon_vid coretemp x86_pkg_temp_thermal kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic nouveau video led_class drm_kms_helper cfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt fb_sys_fops cfbcopyarea ttm drm snd_hda_intel agpgart snd_hda_codec snd_hda_core snd_pcm_oss igb snd_mixer_oss aesni_intel snd_pcm aes_x86_64 hwmon snd_timer crypto_simd i2c_algo_bit mptsas snd glue_helper
[ 323.105089]  mpt3sas i2c_core mptscsih soundcore ahci mptbase raid_class libahci scsi_transport_sas libata scsi_mod button wmi hed unix
[ 323.105097] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.10.0 #1
[ 323.105098] Hardware name: Dell Inc. Precision T5610/0WN7Y6, BIOS A16 02/05/2018
[ 323.105100] Call Trace:
[ 323.105101]  <IRQ>
[ 323.105105]  ? dump_stack+0x46/0x5a
[ 323.105107]  ? __warn+0xb4/0xd0
[ 323.105110]  ? scsi_init_io+0x143/0x1f0 [scsi_mod]
[ 323.105113]  ? scsi_setup_cmnd+0x4c/0x140 [scsi_mod]
[ 323.105115]  ? scsi_prep_fn+0xe3/0x170 [scsi_mod]
[ 323.105118]  ? swiotlb_unmap_sg_attrs+0x44/0x60
[ 323.105119]  ? blk_peek_request+0x130/0x200
[ 323.105122]  ? scsi_request_fn+0x2b/0x510 [scsi_mod]
[ 323.105124]  ? __blk_run_queue+0x2a/0x40
[ 323.105126]  ? blk_run_queue+0x1c/0x30
[ 323.105129]  ? scsi_run_queue+0x229/0x2b0 [scsi_mod]
[ 323.105131]  ? scsi_io_completion+0x3d6/0x5c0 [scsi_mod]
[ 323.105133]  ? blk_done_softirq+0x67/0x80
[ 323.105135]  ? __do_softirq+0xdb/0x200
[ 323.105137]  ? irq_exit+0xa3/0xb0
[ 323.105139]  ? do_IRQ+0x45/0xc0
[ 323.105141]  ? common_interrupt+0x7c/0x7c
[ 323.105142]  </IRQ>
[ 323.105145]  ? cpuidle_enter_state+0x144/0x1f0
[ 323.105146]  ? cpuidle_enter_state+0x139/0x1f0
[ 323.105148]  ? do_idle+0xd3/0x190
[ 323.105150]  ? cpu_startup_entry+0x14/0x20
[ 323.105152]  ? start_kernel+0x391/0x399
[ 323.105154]  ? start_cpu+0x14/0x14
[ 323.105155] ---[ end trace f38cc734e4921bdc ]---
[ 323.105157] blk_peek_request: bad return=-22

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gU3VuLCAyMDE4LTA0LTAxIGF0IDE0OjI3IC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IFdha2tvIFdhcm5lciB3cm90ZToNCj4gPiBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+ID4gPiBJIHRl
c3RlZCA0LjE0LjMyIGxhc3QgbmlnaHQgd2l0aCB0aGUgc2FtZSBvb3BzLiAgNC45LjkxIHdvcmtz
IGZpbmUuDQo+ID4gPiBGcm9tIHRoZSBpbml0aWF0b3IsIGlmIEkgZG8gY2F0IC9kZXYvc3IxID4g
L2Rldi9udWxsIGl0IHdvcmtzLiAgSWYgSSBtb3VudA0KPiA+ID4gL2Rldi9zcjEgYW5kIHRoZW4g
ZG8gZmluZCAtdHlwZSBmIHwgeGFyZ3MgY2F0ID4gL2Rldi9udWxsIHRoZSB0YXJnZXQNCj4gPiA+
IGNyYXNoZXMuICBJJ20gdXNpbmcgdGhlIGJ1aWx0aW4gaXNjc2kgdGFyZ2V0IHdpdGggcHNjc2ku
ICBJIGNhbiBidXJuIGZyb20NCj4gPiA+IHRoZSBpbml0aWF0b3Igd2l0aCBvdXQgcHJvYmxlbXMu
ICBJJ2xsIHRlc3Qgb3RoZXIga2VybmVscyBiZXR3ZWVuIDQuOSBhbmQNCj4gPiA+IDQuMTQuDQo+
ID4gDQo+ID4gU28gSSd2ZSB0ZXN0ZWQgNC54Lnkgd2hlcmUgeCBvbmUgb2YgMTAgMTEgMTIgMTQg
MTUgYW5kIHkgaXMgdGhlIGxhdGVzdCBwYXRjaA0KPiA+IChleGNlcHQgZm9yIDQuMTUgd2hpY2gg
d2FzIDEgYmVoaW5kKQ0KPiA+IEVhY2ggb2YgdGhlc2Uga2VybmVscyBjcmFzaCB3aXRoaW4gc2Vj
b25kcyBvciBpbW1lZGlhdGUgb2YgZG9pbmcgZmluZCAtdHlwZQ0KPiA+IGYgfCB4YXJncyBjYXQg
PiAvZGV2L251bGwgZnJvbSB0aGUgaW5pdGlhdG9yLg0KPiANCj4gSSB0cmllZCA0LjEwLjAuICBJ
dCBkb2Vzbid0IGNvbXBsZXRlbHkgbG9ja3VwIHRoZSBzeXN0ZW0sIGJ1dCB0aGUgZGV2aWNlDQo+
IHRoYXQgd2FzIHVzZWQgaGFuZ3MuICBTbyBmcm9tIHRoZSBpbml0aWF0b3IsIGl0J3MgL2Rldi9z
cjEgYW5kIGZyb20gdGhlDQo+IHRhcmdldCBpdCdzIC9kZXYvc3IwLiAgQXR0ZW1wdGluZyB0byBy
ZWFkIC9kZXYvc3IwIGFmdGVyIHRoZSBvb3BzIGNhdXNlcyB0aGUNCj4gcHJvY2VzcyB0byBoYW5n
IGluIEQgc3RhdGUuDQoNCkhlbGxvIFdha2tvLA0KDQpUaGFuayB5b3UgZm9yIGhhdmluZyBuYXJy
b3dlZCBkb3duIHRoaXMgZnVydGhlci4gSSB0aGluayB0aGF0IHlvdSBlbmNvdW50ZXJlZA0KYSBy
ZWdyZXNzaW9uIGVpdGhlciBpbiB0aGUgYmxvY2sgbGF5ZXIgY29yZSBvciBpbiB0aGUgU0NTSSBj
b3JlLiBVbmZvcnR1bmF0ZWx5DQp0aGUgbnVtYmVyIG9mIGNoYW5nZXMgYmV0d2VlbiBrZXJuZWwg
dmVyc2lvbnMgdjQuOSBhbmQgdjQuMTAgaW4gdGhlc2UgdHdvDQpzdWJzeXN0ZW1zIGlzIGh1Z2Uu
IEkgc2VlIHR3byBwb3NzaWJsZSB3YXlzIGZvcndhcmQ6DQotIEVpdGhlciB0aGF0IHlvdSBwZXJm
b3JtIGEgYmlzZWN0IHRvIGlkZW50aWZ5IHRoZSBwYXRjaCB0aGF0IGludHJvZHVjZWQgdGhpcw0K
ICByZWdyZXNzaW9uLiBIb3dldmVyLCBJJ20gbm90IHN1cmUgd2hldGhlciB5b3UgYXJlIGZhbWls
aWFyIHdpdGggdGhlIGJpc2VjdA0KICBwcm9jZXNzLg0KLSBPciB0aGF0IHlvdSBpZGVudGlmeSB0
aGUgY29tbWFuZCB0aGF0IHRyaWdnZXJzIHRoaXMgY3Jhc2ggc3VjaCB0aGF0IG90aGVycw0KICBj
YW4gcmVwcm9kdWNlIHRoaXMgaXNzdWUgd2l0aG91dCBuZWVkaW5nIGFjY2VzcyB0byB5b3VyIHNl
dHVwLg0KDQpIb3cgYWJvdXQgcmVwcm9kdWNpbmcgdGhpcyBjcmFzaCB3aXRoIHRoZSBiZWxvdyBw
YXRjaCBhcHBsaWVkIG9uIHRvcCBvZg0Ka2VybmVsIHY0LjE1Lng/IFRoZSBhZGRpdGlvbmFsIG91
dHB1dCBzZW50IGJ5IHRoaXMgcGF0Y2ggdG8gdGhlIHN5c3RlbSBsb2cNCnNob3VsZCBhbGxvdyB1
cyB0byByZXByb2R1Y2UgdGhpcyBpc3N1ZSBieSBzdWJtaXR0aW5nIHRoZSBzYW1lIFNDU0kgY29t
bWFuZA0Kd2l0aCBzZ19yYXcuDQoNClRoYW5rcywNCg0KQmFydC4NCg0KDQpTdWJqZWN0OiBbUEFU
Q0hdIFJlcG9ydCBjb21tYW5kcyB3aXRoIG5vIHBoeXNpY2FsIHNlZ21lbnRzIGluIHRoZSBzeXN0
ZW0gbG9nDQoNCi0tLQ0KIGRyaXZlcnMvc2NzaS9zY3NpX2xpYi5jIHwgNCArKystDQogMSBmaWxl
IGNoYW5nZWQsIDMgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KDQpkaWZmIC0tZ2l0IGEv
ZHJpdmVycy9zY3NpL3Njc2lfbGliLmMgYi9kcml2ZXJzL3Njc2kvc2NzaV9saWIuYw0KaW5kZXgg
NmI2YTY3MDVmNmU1Li43NGEzOWRiNTdkNDkgMTAwNjQ0DQotLS0gYS9kcml2ZXJzL3Njc2kvc2Nz
aV9saWIuYw0KKysrIGIvZHJpdmVycy9zY3NpL3Njc2lfbGliLmMNCkBAIC0xMDkzLDggKzEwOTMs
MTAgQEAgaW50IHNjc2lfaW5pdF9pbyhzdHJ1Y3Qgc2NzaV9jbW5kICpjbWQpDQogCWJvb2wgaXNf
bXEgPSAocnEtPm1xX2N0eCAhPSBOVUxMKTsNCiAJaW50IGVycm9yID0gQkxLUFJFUF9LSUxMOw0K
IA0KLQlpZiAoV0FSTl9PTl9PTkNFKCFibGtfcnFfbnJfcGh5c19zZWdtZW50cyhycSkpKQ0KKwlp
ZiAoV0FSTl9PTl9PTkNFKCFibGtfcnFfbnJfcGh5c19zZWdtZW50cyhycSkpKSB7DQorCQlzY3Np
X3ByaW50X2NvbW1hbmQoY21kKTsNCiAJCWdvdG8gZXJyX2V4aXQ7DQorCX0NCiANCiAJZXJyb3Ig
PSBzY3NpX2luaXRfc2d0YWJsZShycSwgJmNtZC0+c2RiKTsNCiAJaWYgKGVycm9yKQ0K

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Sun, 2018-04-01 at 14:27 -0400, Wakko Warner wrote:
> > Wakko Warner wrote:
> > > Wakko Warner wrote:
> > > > I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> > > > From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> > > > /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> > > > crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> > > > the initiator with out problems.  I'll test other kernels between 4.9 and
> > > > 4.14.
> > > 
> > > So I've tested 4.x.y where x one of 10 11 12 14 15 and y is the latest patch
> > > (except for 4.15 which was 1 behind)
> > > Each of these kernels crash within seconds or immediate of doing find -type
> > > f | xargs cat > /dev/null from the initiator.
> > 
> > I tried 4.10.0.  It doesn't completely lockup the system, but the device
> > that was used hangs.  So from the initiator, it's /dev/sr1 and from the
> > target it's /dev/sr0.  Attempting to read /dev/sr0 after the oops causes the
> > process to hang in D state.
> 
> Hello Wakko,
> 
> Thank you for having narrowed down this further. I think that you encountered
> a regression either in the block layer core or in the SCSI core. Unfortunately
> the number of changes between kernel versions v4.9 and v4.10 in these two
> subsystems is huge. I see two possible ways forward:
> - Either that you perform a bisect to identify the patch that introduced this
>   regression. However, I'm not sure whether you are familiar with the bisect
>   process.
> - Or that you identify the command that triggers this crash such that others
>   can reproduce this issue without needing access to your setup.
> 
> How about reproducing this crash with the below patch applied on top of
> kernel v4.15.x? The additional output sent by this patch to the system log
> should allow us to reproduce this issue by submitting the same SCSI command
> with sg_raw.

Sorry for not getting back in touch.  My internet was down.  I haven't tried
the patch yet.  I'll try to get to that tomorrow.  The system with the issue
is busy and I can't reboot it right now.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Sun, 2018-04-01 at 14:27 -0400, Wakko Warner wrote:
> > Wakko Warner wrote:
> > > Wakko Warner wrote:
> > > > I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> > > > From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> > > > /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> > > > crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> > > > the initiator with out problems.  I'll test other kernels between 4.9 and
> > > > 4.14.
> > > 
> > > So I've tested 4.x.y where x one of 10 11 12 14 15 and y is the latest patch
> > > (except for 4.15 which was 1 behind)
> > > Each of these kernels crash within seconds or immediate of doing find -type
> > > f | xargs cat > /dev/null from the initiator.
> > 
> > I tried 4.10.0.  It doesn't completely lockup the system, but the device
> > that was used hangs.  So from the initiator, it's /dev/sr1 and from the
> > target it's /dev/sr0.  Attempting to read /dev/sr0 after the oops causes the
> > process to hang in D state.
> 
> Hello Wakko,
> 
> Thank you for having narrowed down this further. I think that you encountered
> a regression either in the block layer core or in the SCSI core. Unfortunately
> the number of changes between kernel versions v4.9 and v4.10 in these two
> subsystems is huge. I see two possible ways forward:
> - Either that you perform a bisect to identify the patch that introduced this
>   regression. However, I'm not sure whether you are familiar with the bisect
>   process.
> - Or that you identify the command that triggers this crash such that others
>   can reproduce this issue without needing access to your setup.
> 
> How about reproducing this crash with the below patch applied on top of
> kernel v4.15.x? The additional output sent by this patch to the system log
> should allow us to reproduce this issue by submitting the same SCSI command
> with sg_raw.

Ok, so I tried this, but scsi_print_command doesn't print anything.  I added
a check for !rq and the same thing that blk_rq_nr_phys_segments does in an
if statement above this thinking it might have crashed during WARN_ON_ONCE.
It still didn't print anything.  My printk shows this:
[  36.263193] sr 3:0:0:0: cmd->request->nr_phys_segments is 0

I also had scsi_print_command in the same if block which again didn't print
anything.  Is there some debug option I need to turn on to make it print?  I
tried looking through the code for this and following some of the function
calls but didn't see any config options.

> Subject: [PATCH] Report commands with no physical segments in the system log
> 
> ---
>  drivers/scsi/scsi_lib.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 6b6a6705f6e5..74a39db57d49 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -1093,8 +1093,10 @@ int scsi_init_io(struct scsi_cmnd *cmd)
>  	bool is_mq = (rq->mq_ctx != NULL);
>  	int error = BLKPREP_KILL;
>  
> -	if (WARN_ON_ONCE(!blk_rq_nr_phys_segments(rq)))
> +	if (WARN_ON_ONCE(!blk_rq_nr_phys_segments(rq))) {
> +		scsi_print_command(cmd);
>  		goto err_exit;
> +	}
>  
>  	error = scsi_init_sgtable(rq, &cmd->sdb);
>  	if (error)
-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Bart Van Assche wrote:
> > On Sun, 2018-04-01 at 14:27 -0400, Wakko Warner wrote:
> > > Wakko Warner wrote:
> > > > Wakko Warner wrote:
> > > > > I tested 4.14.32 last night with the same oops.  4.9.91 works fine.
> > > > > From the initiator, if I do cat /dev/sr1 > /dev/null it works.  If I mount
> > > > > /dev/sr1 and then do find -type f | xargs cat > /dev/null the target
> > > > > crashes.  I'm using the builtin iscsi target with pscsi.  I can burn from
> > > > > the initiator with out problems.  I'll test other kernels between 4.9 and
> > > > > 4.14.
> > > > 
> > > > So I've tested 4.x.y where x one of 10 11 12 14 15 and y is the latest patch
> > > > (except for 4.15 which was 1 behind)
> > > > Each of these kernels crash within seconds or immediate of doing find -type
> > > > f | xargs cat > /dev/null from the initiator.
> > > 
> > > I tried 4.10.0.  It doesn't completely lockup the system, but the device
> > > that was used hangs.  So from the initiator, it's /dev/sr1 and from the
> > > target it's /dev/sr0.  Attempting to read /dev/sr0 after the oops causes the
> > > process to hang in D state.
> > 
> > Hello Wakko,
> > 
> > Thank you for having narrowed down this further. I think that you encountered
> > a regression either in the block layer core or in the SCSI core. Unfortunately
> > the number of changes between kernel versions v4.9 and v4.10 in these two
> > subsystems is huge. I see two possible ways forward:
> > - Either that you perform a bisect to identify the patch that introduced this
> >   regression. However, I'm not sure whether you are familiar with the bisect
> >   process.
> > - Or that you identify the command that triggers this crash such that others
> >   can reproduce this issue without needing access to your setup.
> > 
> > How about reproducing this crash with the below patch applied on top of
> > kernel v4.15.x? The additional output sent by this patch to the system log
> > should allow us to reproduce this issue by submitting the same SCSI command
> > with sg_raw.
> 
> Ok, so I tried this, but scsi_print_command doesn't print anything.  I added
> a check for !rq and the same thing that blk_rq_nr_phys_segments does in an
> if statement above this thinking it might have crashed during WARN_ON_ONCE.
> It still didn't print anything.  My printk shows this:
> [  36.263193] sr 3:0:0:0: cmd->request->nr_phys_segments is 0
> 
> I also had scsi_print_command in the same if block which again didn't print
> anything.  Is there some debug option I need to turn on to make it print?  I
> tried looking through the code for this and following some of the function
> calls but didn't see any config options.

I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
I added a dev_printk in scsi_print_command where the 2 if statements return.
Logs:
[  29.866415] sr 3:0:0:0: cmd->cmnd is NULL

> > Subject: [PATCH] Report commands with no physical segments in the system log
> > 
> > ---
> >  drivers/scsi/scsi_lib.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> > index 6b6a6705f6e5..74a39db57d49 100644
> > --- a/drivers/scsi/scsi_lib.c
> > +++ b/drivers/scsi/scsi_lib.c
> > @@ -1093,8 +1093,10 @@ int scsi_init_io(struct scsi_cmnd *cmd)
> >  	bool is_mq = (rq->mq_ctx != NULL);
> >  	int error = BLKPREP_KILL;
> >  
> > -	if (WARN_ON_ONCE(!blk_rq_nr_phys_segments(rq)))
> > +	if (WARN_ON_ONCE(!blk_rq_nr_phys_segments(rq))) {
> > +		scsi_print_command(cmd);
> >  		goto err_exit;
> > +	}
> >  
> >  	error = scsi_init_sgtable(rq, &cmd->sdb);
> >  	if (error)
> -- 
>  Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
>  million bugs.
-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gVGh1LCAyMDE4LTA0LTA1IGF0IDIyOjA2IC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IEkga25vdyBub3cgd2h5IHNjc2lfcHJpbnRfY29tbWFuZCBpc24ndCBkb2luZyBhbnl0aGluZy4g
IGNtZC0+Y21uZCBpcyBudWxsLg0KPiBJIGFkZGVkIGEgZGV2X3ByaW50ayBpbiBzY3NpX3ByaW50
X2NvbW1hbmQgd2hlcmUgdGhlIDIgaWYgc3RhdGVtZW50cyByZXR1cm4uDQo+IExvZ3M6DQo+IFsg
IDI5Ljg2NjQxNV0gc3IgMzowOjA6MDogY21kLT5jbW5kIGlzIE5VTEwNCg0KVGhhdCdzIHNvbWV0
aGluZyB0aGF0IHNob3VsZCBuZXZlciBoYXBwZW4uIEFzIG9uZSBjYW4gc2VlIGluDQpzY3NpX3Nl
dHVwX3Njc2lfY21uZCgpIGFuZCBzY3NpX3NldHVwX2ZzX2NtbmQoKSBib3RoIGZ1bmN0aW9ucyBp
bml0aWFsaXplDQp0aGF0IHBvaW50ZXIuIFNpbmNlIEkgaGF2ZSBub3QgeWV0IGJlZW4gYWJsZSB0
byByZXByb2R1Y2UgbXlzZWxmIHdoYXQgeW91DQpyZXBvcnRlZCwgd291bGQgaXQgYmUgcG9zc2li
bGUgZm9yIHlvdSB0byBiaXNlY3QgdGhpcyBpc3N1ZT8gWW91IHdpbGwgbmVlZA0KdG8gZm9sbG93
IHNvbWV0aGluZyBsaWtlIHRoZSBmb2xsb3dpbmcgcHJvY2VkdXJlIChzZWUgYWxzbw0KaHR0cHM6
Ly9naXQtc2NtLmNvbS9kb2NzL2dpdC1iaXNlY3QpOg0KDQpnaXQgY2xvbmUgZ2l0Oi8vZ2l0Lmtl
cm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0L3RvcnZhbGRzL2xpbnV4LmdpdA0KZ2l0
IGJpc2VjdCBzdGFydA0KZ2l0IGJpc2VjdCBiYWQgdjQuMTANCmdpdCBiaXNlY3QgZ29vZCB2NC45
DQoNCmFuZCB0aGVuIGJ1aWxkIHRoZSBrZXJuZWwsIGluc3RhbGwgaXQsIGJvb3QgdGhlIGtlcm5l
bCBhbmQgdGVzdCBpdC4NCkRlcGVuZGluZyBvbiB0aGUgcmVzdWx0LCBydW4gZWl0aGVyIGdpdCBi
aXNlY3QgYmFkIG9yIGdpdCBiaXNlY3QgZ29vZCBhbmQNCmtlZXAgZ29pbmcgdW50aWwgZ2l0IGJp
c2VjdCBjb21lcyB0byBhIGNvbmNsdXNpb24uIFRoaXMgY2FuIHRha2UgYW4gaG91ciBvcg0KbW9y
ZS4NCg0KQmFydC4NCg0KDQoNCg==

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Thu, 2018-04-05 at 22:06 -0400, Wakko Warner wrote:
> > I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
> > I added a dev_printk in scsi_print_command where the 2 if statements return.
> > Logs:
> > [  29.866415] sr 3:0:0:0: cmd->cmnd is NULL
> 
> That's something that should never happen. As one can see in
> scsi_setup_scsi_cmnd() and scsi_setup_fs_cmnd() both functions initialize
> that pointer. Since I have not yet been able to reproduce myself what you
> reported, would it be possible for you to bisect this issue? You will need
> to follow something like the following procedure (see also
> https://git-scm.com/docs/git-bisect):

I don't know how relevent it is, but this machine boots nfs and exports it's
dvd drives over iscsi with the target modules.  My scsi_target.lio is at the
end.  I removed the iqn name.  The options are default except for a few. 
Non default options I tabbed over.
eth0 is the nfs/localnet nic and eth1 is the
nic that iscsi goes over.
eth0 is onboard pci 8086:1502 (subsystem 1028:05d3)
eth1 is pci 8086:107d (subsystem 8086:1084)
Both use the e1000e driver

The initiator is running 4.4.107.
When running on the initiator, /dev/sr1 is the target /dev/sr0.  Therefor
cat /dev/sr1 > /dev/null seems to work.
mount /dev/sr1 /cdrom works
find /cdrom -type f | xargs cat > /dev/null immediately crashes the target.
Burning to /dev/sr1 seems to work.

I have another nic that uses igb instead, I'll see if that makes a
difference.

> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> git bisect start
> git bisect bad v4.10
> git bisect good v4.9
> 
> and then build the kernel, install it, boot the kernel and test it.
> Depending on the result, run either git bisect bad or git bisect good and
> keep going until git bisect comes to a conclusion. This can take an hour or
> more.

I'll try this.

Here's my scsi_target.lio:
storage pscsi {
	disk dvd0 {
	path /dev/sr0 
attribute {
emulate_3pc yes 
emulate_caw yes 
emulate_dpo no 
emulate_fua_read no 
emulate_model_alias no 
emulate_rest_reord no 
emulate_tas yes 
emulate_tpu no 
emulate_tpws no 
emulate_ua_intlck_ctrl no 
emulate_write_cache no 
enforce_pr_isids yes 
fabric_max_sectors 8192 
is_nonrot yes 
max_unmap_block_desc_count 0 
max_unmap_lba_count 0 
max_write_same_len 65535 
queue_depth 128 
unmap_granularity 0 
unmap_granularity_alignment 0 
}
}
	disk dvd1 {
	path /dev/sr1 
attribute {
emulate_3pc yes 
emulate_caw yes 
emulate_dpo no 
emulate_fua_read no 
emulate_model_alias no 
emulate_rest_reord no 
emulate_tas yes 
emulate_tpu no 
emulate_tpws no 
emulate_ua_intlck_ctrl no 
emulate_write_cache no 
enforce_pr_isids yes 
fabric_max_sectors 8192 
is_nonrot yes 
max_unmap_block_desc_count 0 
max_unmap_lba_count 0 
max_write_same_len 65535 
queue_depth 128 
unmap_granularity 0 
unmap_granularity_alignment 0 
}
}
	disk dvd2 {
	path /dev/sr2 
attribute {
emulate_3pc yes 
emulate_caw yes 
emulate_dpo no 
emulate_fua_read no 
emulate_model_alias no 
emulate_rest_reord no 
emulate_tas yes 
emulate_tpu no 
emulate_tpws no 
emulate_ua_intlck_ctrl no 
emulate_write_cache no 
enforce_pr_isids yes 
fabric_max_sectors 8192 
is_nonrot yes 
max_unmap_block_desc_count 0 
max_unmap_lba_count 0 
max_write_same_len 65535 
queue_depth 128 
unmap_granularity 0 
unmap_granularity_alignment 0 
}
}
}
fabric iscsi {
discovery_auth {
enable no 
mutual_password "" 
mutual_userid "" 
password "" 
userid "" 
}
	target iqn.<myiqn>:dvd tpgt 1 {
enable yes 
attribute {
	authentication no 
cache_dynamic_acls yes 
default_cmdsn_depth 64 
default_erl 0 
demo_mode_discovery yes 
	demo_mode_write_protect no 
fabric_prot_type 0 
	generate_node_acls yes 
login_timeout 15 
netif_timeout 2 
prod_mode_write_protect no 
t10_pi 0 
tpg_enabled_sendtargets 1 
}
auth {
password "" 
password_mutual "" 
userid "" 
userid_mutual "" 
}
parameter {
AuthMethod "CHAP,None" 
DataDigest "CRC32C,None" 
DataPDUInOrder yes 
DataSequenceInOrder yes 
DefaultTime2Retain 20 
DefaultTime2Wait 2 
ErrorRecoveryLevel no 
FirstBurstLength 65536 
HeaderDigest "CRC32C,None" 
IFMarkInt Reject 
IFMarker no 
ImmediateData yes 
InitialR2T yes 
MaxBurstLength 262144 
MaxConnections 1 
MaxOutstandingR2T 1 
MaxRecvDataSegmentLength 8192 
MaxXmitDataSegmentLength 262144 
OFMarkInt Reject 
OFMarker no 
TargetAlias "LIO Target" 
}
	lun 0 backend pscsi:dvd0 
	lun 1 backend pscsi:dvd1 
	lun 2 backend pscsi:dvd2 
	portal 0.0.0.0:3260 
}
}


-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Thu, 2018-04-05 at 22:06 -0400, Wakko Warner wrote:
> > I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
> > I added a dev_printk in scsi_print_command where the 2 if statements return.
> > Logs:
> > [  29.866415] sr 3:0:0:0: cmd->cmnd is NULL
> 
> That's something that should never happen. As one can see in
> scsi_setup_scsi_cmnd() and scsi_setup_fs_cmnd() both functions initialize
> that pointer. Since I have not yet been able to reproduce myself what you
> reported, would it be possible for you to bisect this issue? You will need
> to follow something like the following procedure (see also
> https://git-scm.com/docs/git-bisect):
> 
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> git bisect start
> git bisect bad v4.10
> git bisect good v4.9
> 
> and then build the kernel, install it, boot the kernel and test it.
> Depending on the result, run either git bisect bad or git bisect good and
> keep going until git bisect comes to a conclusion. This can take an hour or
> more.

I have 1 question.  Should make clean be done between tests?  My box
compiles the whole kernel in 2 minutes.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gRnJpLCAyMDE4LTA0LTA2IGF0IDIxOjAzIC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IEJhcnQgVmFuIEFzc2NoZSB3cm90ZToNCj4gPiBPbiBUaHUsIDIwMTgtMDQtMDUgYXQgMjI6MDYg
LTA0MDAsIFdha2tvIFdhcm5lciB3cm90ZToNCj4gPiA+IEkga25vdyBub3cgd2h5IHNjc2lfcHJp
bnRfY29tbWFuZCBpc24ndCBkb2luZyBhbnl0aGluZy4gIGNtZC0+Y21uZCBpcyBudWxsLg0KPiA+
ID4gSSBhZGRlZCBhIGRldl9wcmludGsgaW4gc2NzaV9wcmludF9jb21tYW5kIHdoZXJlIHRoZSAy
IGlmIHN0YXRlbWVudHMgcmV0dXJuLg0KPiA+ID4gTG9nczoNCj4gPiA+IFsgIDI5Ljg2NjQxNV0g
c3IgMzowOjA6MDogY21kLT5jbW5kIGlzIE5VTEwNCj4gPiANCj4gPiBUaGF0J3Mgc29tZXRoaW5n
IHRoYXQgc2hvdWxkIG5ldmVyIGhhcHBlbi4gQXMgb25lIGNhbiBzZWUgaW4NCj4gPiBzY3NpX3Nl
dHVwX3Njc2lfY21uZCgpIGFuZCBzY3NpX3NldHVwX2ZzX2NtbmQoKSBib3RoIGZ1bmN0aW9ucyBp
bml0aWFsaXplDQo+ID4gdGhhdCBwb2ludGVyLiBTaW5jZSBJIGhhdmUgbm90IHlldCBiZWVuIGFi
bGUgdG8gcmVwcm9kdWNlIG15c2VsZiB3aGF0IHlvdQ0KPiA+IHJlcG9ydGVkLCB3b3VsZCBpdCBi
ZSBwb3NzaWJsZSBmb3IgeW91IHRvIGJpc2VjdCB0aGlzIGlzc3VlPyBZb3Ugd2lsbCBuZWVkDQo+
ID4gdG8gZm9sbG93IHNvbWV0aGluZyBsaWtlIHRoZSBmb2xsb3dpbmcgcHJvY2VkdXJlIChzZWUg
YWxzbw0KPiA+IGh0dHBzOi8vZ2l0LXNjbS5jb20vZG9jcy9naXQtYmlzZWN0KToNCj4gPiANCj4g
PiBnaXQgY2xvbmUgZ2l0Oi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0
L3RvcnZhbGRzL2xpbnV4LmdpdA0KPiA+IGdpdCBiaXNlY3Qgc3RhcnQNCj4gPiBnaXQgYmlzZWN0
IGJhZCB2NC4xMA0KPiA+IGdpdCBiaXNlY3QgZ29vZCB2NC45DQo+ID4gDQo+ID4gYW5kIHRoZW4g
YnVpbGQgdGhlIGtlcm5lbCwgaW5zdGFsbCBpdCwgYm9vdCB0aGUga2VybmVsIGFuZCB0ZXN0IGl0
Lg0KPiA+IERlcGVuZGluZyBvbiB0aGUgcmVzdWx0LCBydW4gZWl0aGVyIGdpdCBiaXNlY3QgYmFk
IG9yIGdpdCBiaXNlY3QgZ29vZCBhbmQNCj4gPiBrZWVwIGdvaW5nIHVudGlsIGdpdCBiaXNlY3Qg
Y29tZXMgdG8gYSBjb25jbHVzaW9uLiBUaGlzIGNhbiB0YWtlIGFuIGhvdXIgb3INCj4gPiBtb3Jl
Lg0KPiANCj4gSSBoYXZlIDEgcXVlc3Rpb24uICBTaG91bGQgbWFrZSBjbGVhbiBiZSBkb25lIGJl
dHdlZW4gdGVzdHM/ICBNeSBib3gNCj4gY29tcGlsZXMgdGhlIHdob2xlIGtlcm5lbCBpbiAyIG1p
bnV0ZXMuDQoNCklmIHlvdSB0cnVzdCB0aGF0IHRoZSBidWlsZCBzeXN0ZW0gd2lsbCBmaWd1cmUg
b3V0IGFsbCBkZXBlbmRlbmNpZXMgdGhlbg0KcnVubmluZyBtYWtlIGNsZWFuIGlzIG5vdCBuZWNl
c3NhcnkuIFBlcnNvbmFsbHkgSSBhbHdheXMgcnVuIG1ha2UgY2xlYW4NCmR1cmluZyBhIGJpc2Vj
dCBiZWZvcmUgcmVidWlsZGluZyB0aGUga2VybmVsIGJlY2F1c2UgaWYgYSBoZWFkZXIgZmlsZSBo
YXMNCmNoYW5nZWQgaW4gZS5nLiB0aGUgYmxvY2sgbGF5ZXIgYSBodWdlIG51bWJlciBvZiBmaWxl
cyBoYXMgdG8gYmUgcmVidWlsdA0KYW55d2F5Lg0KDQpCYXJ0Lg0KDQoNCg==

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Thu, 2018-04-05 at 22:06 -0400, Wakko Warner wrote:
> > I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
> > I added a dev_printk in scsi_print_command where the 2 if statements return.
> > Logs:
> > [  29.866415] sr 3:0:0:0: cmd->cmnd is NULL
> 
> That's something that should never happen. As one can see in
> scsi_setup_scsi_cmnd() and scsi_setup_fs_cmnd() both functions initialize
> that pointer. Since I have not yet been able to reproduce myself what you
> reported, would it be possible for you to bisect this issue? You will need
> to follow something like the following procedure (see also
> https://git-scm.com/docs/git-bisect):

After doing 3 successful compiles with good/bad, I got this error and was
not able to compile any more kernels:
  CC      scripts/mod/devicetable-offsets.s
scripts/mod/empty.c:1:0: error: code model kernel does not support PIC mode
 /* empty file to figure out endianness / word size */
 
scripts/mod/devicetable-offsets.c:1:0: error: code model kernel does not support PIC mode
 #include <linux/kbuild.h>
 
scripts/Makefile.build:153: recipe for target 'scripts/mod/devicetable-offsets.s' failed

I don't think it found the bad commit.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Bart Van Assche wrote:
> > On Thu, 2018-04-05 at 22:06 -0400, Wakko Warner wrote:
> > > I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
> > > I added a dev_printk in scsi_print_command where the 2 if statements return.
> > > Logs:
> > > [  29.866415] sr 3:0:0:0: cmd->cmnd is NULL
> > 
> > That's something that should never happen. As one can see in
> > scsi_setup_scsi_cmnd() and scsi_setup_fs_cmnd() both functions initialize
> > that pointer. Since I have not yet been able to reproduce myself what you
> > reported, would it be possible for you to bisect this issue? You will need
> > to follow something like the following procedure (see also
> > https://git-scm.com/docs/git-bisect):
> 
> After doing 3 successful compiles with good/bad, I got this error and was
> not able to compile any more kernels:
>   CC      scripts/mod/devicetable-offsets.s
> scripts/mod/empty.c:1:0: error: code model kernel does not support PIC mode
>  /* empty file to figure out endianness / word size */
>  
> scripts/mod/devicetable-offsets.c:1:0: error: code model kernel does not support PIC mode
>  #include <linux/kbuild.h>
>  
> scripts/Makefile.build:153: recipe for target 'scripts/mod/devicetable-offsets.s' failed
> 
> I don't think it found the bad commit.

I forgot to mention my gcc version.
gcc (Debian 6.2.1-7) 6.2.1 20161215

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gU2F0LCAyMDE4LTA0LTA3IGF0IDEyOjUzIC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IEJhcnQgVmFuIEFzc2NoZSB3cm90ZToNCj4gPiBPbiBUaHUsIDIwMTgtMDQtMDUgYXQgMjI6MDYg
LTA0MDAsIFdha2tvIFdhcm5lciB3cm90ZToNCj4gPiA+IEkga25vdyBub3cgd2h5IHNjc2lfcHJp
bnRfY29tbWFuZCBpc24ndCBkb2luZyBhbnl0aGluZy4gIGNtZC0+Y21uZCBpcyBudWxsLg0KPiA+
ID4gSSBhZGRlZCBhIGRldl9wcmludGsgaW4gc2NzaV9wcmludF9jb21tYW5kIHdoZXJlIHRoZSAy
IGlmIHN0YXRlbWVudHMgcmV0dXJuLg0KPiA+ID4gTG9nczoNCj4gPiA+IFsgIDI5Ljg2NjQxNV0g
c3IgMzowOjA6MDogY21kLT5jbW5kIGlzIE5VTEwNCj4gPiANCj4gPiBUaGF0J3Mgc29tZXRoaW5n
IHRoYXQgc2hvdWxkIG5ldmVyIGhhcHBlbi4gQXMgb25lIGNhbiBzZWUgaW4NCj4gPiBzY3NpX3Nl
dHVwX3Njc2lfY21uZCgpIGFuZCBzY3NpX3NldHVwX2ZzX2NtbmQoKSBib3RoIGZ1bmN0aW9ucyBp
bml0aWFsaXplDQo+ID4gdGhhdCBwb2ludGVyLiBTaW5jZSBJIGhhdmUgbm90IHlldCBiZWVuIGFi
bGUgdG8gcmVwcm9kdWNlIG15c2VsZiB3aGF0IHlvdQ0KPiA+IHJlcG9ydGVkLCB3b3VsZCBpdCBi
ZSBwb3NzaWJsZSBmb3IgeW91IHRvIGJpc2VjdCB0aGlzIGlzc3VlPyBZb3Ugd2lsbCBuZWVkDQo+
ID4gdG8gZm9sbG93IHNvbWV0aGluZyBsaWtlIHRoZSBmb2xsb3dpbmcgcHJvY2VkdXJlIChzZWUg
YWxzbw0KPiA+IGh0dHBzOi8vZ2l0LXNjbS5jb20vZG9jcy9naXQtYmlzZWN0KToNCj4gDQo+IEFm
dGVyIGRvaW5nIDMgc3VjY2Vzc2Z1bCBjb21waWxlcyB3aXRoIGdvb2QvYmFkLCBJIGdvdCB0aGlz
IGVycm9yIGFuZCB3YXMNCj4gbm90IGFibGUgdG8gY29tcGlsZSBhbnkgbW9yZSBrZXJuZWxzOg0K
PiAgIENDICAgICAgc2NyaXB0cy9tb2QvZGV2aWNldGFibGUtb2Zmc2V0cy5zDQo+IHNjcmlwdHMv
bW9kL2VtcHR5LmM6MTowOiBlcnJvcjogY29kZSBtb2RlbCBrZXJuZWwgZG9lcyBub3Qgc3VwcG9y
dCBQSUMgbW9kZQ0KPiAgLyogZW1wdHkgZmlsZSB0byBmaWd1cmUgb3V0IGVuZGlhbm5lc3MgLyB3
b3JkIHNpemUgKi8NCj4gIA0KPiBzY3JpcHRzL21vZC9kZXZpY2V0YWJsZS1vZmZzZXRzLmM6MTow
OiBlcnJvcjogY29kZSBtb2RlbCBrZXJuZWwgZG9lcyBub3Qgc3VwcG9ydCBQSUMgbW9kZQ0KPiAg
I2luY2x1ZGUgPGxpbnV4L2tidWlsZC5oPg0KPiAgDQo+IHNjcmlwdHMvTWFrZWZpbGUuYnVpbGQ6
MTUzOiByZWNpcGUgZm9yIHRhcmdldCAnc2NyaXB0cy9tb2QvZGV2aWNldGFibGUtb2Zmc2V0cy5z
JyBmYWlsZWQNCj4gDQo+IEkgZG9uJ3QgdGhpbmsgaXQgZm91bmQgdGhlIGJhZCBjb21taXQuDQoN
CkhhdmUgeW91IHRyaWVkIHRvIG1vZGlmeSB0aGUga2VybmVsIE1ha2VmaWxlIGFzIGluZGljYXRl
ZCBpbiB0aGUgZm9sbG93aW5nDQplLW1haWw/IFRoaXMgc2hvdWxkIG1ha2UgdGhlIGtlcm5lbCBi
dWlsZDoNCg0KaHR0cHM6Ly9saXN0cy51YnVudHUuY29tL2FyY2hpdmVzL2tlcm5lbC10ZWFtLzIw
MTYtTWF5LzA3NzE3OC5odG1sDQoNClRoYW5rcywNCg0KQmFydC4=

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Bart Van Assche wrote:
> On Sat, 2018-04-07 at 12:53 -0400, Wakko Warner wrote:
> > Bart Van Assche wrote:
> > > On Thu, 2018-04-05 at 22:06 -0400, Wakko Warner wrote:
> > > > I know now why scsi_print_command isn't doing anything.  cmd->cmnd is null.
> > > > I added a dev_printk in scsi_print_command where the 2 if statements return.
> > > > Logs:
> > > > [  29.866415] sr 3:0:0:0: cmd->cmnd is NULL
> > > 
> > > That's something that should never happen. As one can see in
> > > scsi_setup_scsi_cmnd() and scsi_setup_fs_cmnd() both functions initialize
> > > that pointer. Since I have not yet been able to reproduce myself what you
> > > reported, would it be possible for you to bisect this issue? You will need
> > > to follow something like the following procedure (see also
> > > https://git-scm.com/docs/git-bisect):
> > 
> > After doing 3 successful compiles with good/bad, I got this error and was
> > not able to compile any more kernels:
> >   CC      scripts/mod/devicetable-offsets.s
> > scripts/mod/empty.c:1:0: error: code model kernel does not support PIC mode
> >  /* empty file to figure out endianness / word size */
> >  
> > scripts/mod/devicetable-offsets.c:1:0: error: code model kernel does not support PIC mode
> >  #include <linux/kbuild.h>
> >  
> > scripts/Makefile.build:153: recipe for target 'scripts/mod/devicetable-offsets.s' failed
> > 
> > I don't think it found the bad commit.
> 
> Have you tried to modify the kernel Makefile as indicated in the following
> e-mail? This should make the kernel build:
> 
> https://lists.ubuntu.com/archives/kernel-team/2016-May/077178.html

Thanks.  That helped.

I finished with git bisect.  Here's the output:
84c8590646d5b35804bac60eb58b145839b5893e is the first bad commit
commit 84c8590646d5b35804bac60eb58b145839b5893e
Author: Ming Lei <tom.leiming@gmail.com>
Date:   Fri Nov 11 20:05:32 2016 +0800

    target: avoid accessing .bi_vcnt directly
    
    When the bio is full, bio_add_pc_page() will return zero,
    so use this information tell when the bio is full.
    
    Also replace access to .bi_vcnt for pr_debug() with bio_segments().
    
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Signed-off-by: Ming Lei <tom.leiming@gmail.com>
    Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
    Signed-off-by: Jens Axboe <axboe@fb.com>

:040000 040000 a3ebbb71c52ee4eb8c3be4d033b81179211bf704 de39a328dbd1b18519946b3ad46d9302886e0dd0 M      drivers

I did a diff between HEAD^ and HEAD and manually patched the file from
4.15.14.  It's not an exact revert.  I'm running it now and it's working.
I'll do a better test later on.  Here's the patch:

--- a/drivers/target/target_core_pscsi.c	2018-02-04 14:31:31.077316617 -0500
+++ b/drivers/target/target_core_pscsi.c	2018-04-08 11:43:49.588641374 -0400
@@ -915,7 +915,9 @@
 					bio, page, bytes, off);
 			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
 				bio_segments(bio), nr_vecs);
-			if (rc != bytes) {
+			if (rc != bytes)
+				goto fail;
+			if (bio->bi_vcnt > nr_vecs) {
 				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
 					" %d i: %d bio: %p, allocating another"
 					" bio\n", bio->bi_vcnt, i, bio);

I really appreciate your time and assistance with this.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Bart Van Assche wrote:
> > Have you tried to modify the kernel Makefile as indicated in the following
> > e-mail? This should make the kernel build:
> > 
> > https://lists.ubuntu.com/archives/kernel-team/2016-May/077178.html
> 
> Thanks.  That helped.
> 
> I finished with git bisect.  Here's the output:
> 84c8590646d5b35804bac60eb58b145839b5893e is the first bad commit
> commit 84c8590646d5b35804bac60eb58b145839b5893e
> Author: Ming Lei <tom.leiming@gmail.com>
> Date:   Fri Nov 11 20:05:32 2016 +0800
> 
>     target: avoid accessing .bi_vcnt directly
>     
>     When the bio is full, bio_add_pc_page() will return zero,
>     so use this information tell when the bio is full.
>     
>     Also replace access to .bi_vcnt for pr_debug() with bio_segments().
>     
>     Reviewed-by: Christoph Hellwig <hch@lst.de>
>     Signed-off-by: Ming Lei <tom.leiming@gmail.com>
>     Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
>     Signed-off-by: Jens Axboe <axboe@fb.com>
> 
> :040000 040000 a3ebbb71c52ee4eb8c3be4d033b81179211bf704 de39a328dbd1b18519946b3ad46d9302886e0dd0 M      drivers
> 
> I did a diff between HEAD^ and HEAD and manually patched the file from
> 4.15.14.  It's not an exact revert.  I'm running it now and it's working.
> I'll do a better test later on.  Here's the patch:
> 
> --- a/drivers/target/target_core_pscsi.c	2018-02-04 14:31:31.077316617 -0500
> +++ b/drivers/target/target_core_pscsi.c	2018-04-08 11:43:49.588641374 -0400
> @@ -915,7 +915,9 @@
>  					bio, page, bytes, off);
>  			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
>  				bio_segments(bio), nr_vecs);
> -			if (rc != bytes) {
> +			if (rc != bytes)
> +				goto fail;
> +			if (bio->bi_vcnt > nr_vecs) {
>  				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
>  					" %d i: %d bio: %p, allocating another"
>  					" bio\n", bio->bi_vcnt, i, bio);
> 
> I really appreciate your time and assistance with this.

One thing I noticed after doing this is errors in the kernel log on the
initiator:
[9072625.181744] sr 26:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
[9072625.181802] sr 26:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
[9072625.181835] sr 26:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
[9072625.181866] sr 26:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 0a 81 22 00 00 80 00
[9072625.181919] blk_update_request: I/O error, dev sr1, sector 2753672

When doing the exact same thing on the target, no mention.  My patch may not
be right, but it doesn't cause an oops.

I'm going to try 4.16.1 and see what happens.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Bart Van Assche]

T24gU3VuLCAyMDE4LTA0LTA4IGF0IDEyOjAyIC0wNDAwLCBXYWtrbyBXYXJuZXIgd3JvdGU6DQo+
IEkgZmluaXNoZWQgd2l0aCBnaXQgYmlzZWN0LiAgSGVyZSdzIHRoZSBvdXRwdXQ6DQo+IDg0Yzg1
OTA2NDZkNWIzNTgwNGJhYzYwZWI1OGIxNDU4MzliNTg5M2UgaXMgdGhlIGZpcnN0IGJhZCBjb21t
aXQNCj4gY29tbWl0IDg0Yzg1OTA2NDZkNWIzNTgwNGJhYzYwZWI1OGIxNDU4MzliNTg5M2UNCj4g
QXV0aG9yOiBNaW5nIExlaSA8dG9tLmxlaW1pbmdAZ21haWwuY29tPg0KPiBEYXRlOiAgIEZyaSBO
b3YgMTEgMjA6MDU6MzIgMjAxNiArMDgwMA0KPiANCj4gICAgIHRhcmdldDogYXZvaWQgYWNjZXNz
aW5nIC5iaV92Y250IGRpcmVjdGx5DQo+ICAgICANCj4gICAgIFdoZW4gdGhlIGJpbyBpcyBmdWxs
LCBiaW9fYWRkX3BjX3BhZ2UoKSB3aWxsIHJldHVybiB6ZXJvLA0KPiAgICAgc28gdXNlIHRoaXMg
aW5mb3JtYXRpb24gdGVsbCB3aGVuIHRoZSBiaW8gaXMgZnVsbC4NCj4gICAgIA0KPiAgICAgQWxz
byByZXBsYWNlIGFjY2VzcyB0byAuYmlfdmNudCBmb3IgcHJfZGVidWcoKSB3aXRoIGJpb19zZWdt
ZW50cygpLg0KPiAgICAgDQo+ICAgICBSZXZpZXdlZC1ieTogQ2hyaXN0b3BoIEhlbGx3aWcgPGhj
aEBsc3QuZGU+DQo+ICAgICBTaWduZWQtb2ZmLWJ5OiBNaW5nIExlaSA8dG9tLmxlaW1pbmdAZ21h
aWwuY29tPg0KPiAgICAgUmV2aWV3ZWQtYnk6IFNhZ2kgR3JpbWJlcmcgPHNhZ2lAZ3JpbWJlcmcu
bWU+DQo+ICAgICBTaWduZWQtb2ZmLWJ5OiBKZW5zIEF4Ym9lIDxheGJvZUBmYi5jb20+DQo+IA0K
PiA6MDQwMDAwIDA0MDAwMCBhM2ViYmI3MWM1MmVlNGViOGMzYmU0ZDAzM2I4MTE3OTIxMWJmNzA0
IGRlMzlhMzI4ZGJkMWIxODUxOTk0NmIzYWQ0NmQ5MzAyODg2ZTBkZDAgTSAgICAgIGRyaXZlcnMN
Cj4gDQo+IEkgZGlkIGEgZGlmZiBiZXR3ZWVuIEhFQUReIGFuZCBIRUFEIGFuZCBtYW51YWxseSBw
YXRjaGVkIHRoZSBmaWxlIGZyb20NCj4gNC4xNS4xNC4gIEl0J3Mgbm90IGFuIGV4YWN0IHJldmVy
dC4gIEknbSBydW5uaW5nIGl0IG5vdyBhbmQgaXQncyB3b3JraW5nLg0KPiBJJ2xsIGRvIGEgYmV0
dGVyIHRlc3QgbGF0ZXIgb24uICBIZXJlJ3MgdGhlIHBhdGNoOg0KPiANCj4gLS0tIGEvZHJpdmVy
cy90YXJnZXQvdGFyZ2V0X2NvcmVfcHNjc2kuYwkyMDE4LTAyLTA0IDE0OjMxOjMxLjA3NzMxNjYx
NyAtMDUwMA0KPiArKysgYi9kcml2ZXJzL3RhcmdldC90YXJnZXRfY29yZV9wc2NzaS5jCTIwMTgt
MDQtMDggMTE6NDM6NDkuNTg4NjQxMzc0IC0wNDAwDQo+IEBAIC05MTUsNyArOTE1LDkgQEANCj4g
IAkJCQkJYmlvLCBwYWdlLCBieXRlcywgb2ZmKTsNCj4gIAkJCXByX2RlYnVnKCJQU0NTSTogYmlv
LT5iaV92Y250OiAlZCBucl92ZWNzOiAlZFxuIiwNCj4gIAkJCQliaW9fc2VnbWVudHMoYmlvKSwg
bnJfdmVjcyk7DQo+IC0JCQlpZiAocmMgIT0gYnl0ZXMpIHsNCj4gKwkJCWlmIChyYyAhPSBieXRl
cykNCj4gKwkJCQlnb3RvIGZhaWw7DQo+ICsJCQlpZiAoYmlvLT5iaV92Y250ID4gbnJfdmVjcykg
ew0KPiAgCQkJCXByX2RlYnVnKCJQU0NTSTogUmVhY2hlZCBiaW8tPmJpX3ZjbnQgbWF4OiINCj4g
IAkJCQkJIiAlZCBpOiAlZCBiaW86ICVwLCBhbGxvY2F0aW5nIGFub3RoZXIiDQo+ICAJCQkJCSIg
YmlvXG4iLCBiaW8tPmJpX3ZjbnQsIGksIGJpbyk7DQoNCkhlbGxvIE1pbmcsDQoNCkNhbiB5b3Ug
aGF2ZSBhIGxvb2sgYXQgdGhpcz8gVGhlIHN0YXJ0IG9mIHRoaXMgZS1tYWlsIHRocmVhZCBpcyBh
dmFpbGFibGUgYXQNCmh0dHBzOi8vd3d3Lm1haWwtYXJjaGl2ZS5jb20vbGludXgtc2NzaUB2Z2Vy
Lmtlcm5lbC5vcmcvbXNnNzI1NzQuaHRtbC4NCg0KVGhhbmtzLA0KDQpCYXJ0Lg0KDQoNCg0K

Re: 4.15.14 crash with iscsi target and dvd

[Ming Lei]

On Mon, Apr 09, 2018 at 09:30:11PM +0000, Bart Van Assche wrote:
> On Sun, 2018-04-08 at 12:02 -0400, Wakko Warner wrote:
> > I finished with git bisect.  Here's the output:
> > 84c8590646d5b35804bac60eb58b145839b5893e is the first bad commit
> > commit 84c8590646d5b35804bac60eb58b145839b5893e
> > Author: Ming Lei <tom.leiming@gmail.com>
> > Date:   Fri Nov 11 20:05:32 2016 +0800
> > 
> >     target: avoid accessing .bi_vcnt directly
> >     
> >     When the bio is full, bio_add_pc_page() will return zero,
> >     so use this information tell when the bio is full.
> >     
> >     Also replace access to .bi_vcnt for pr_debug() with bio_segments().
> >     
> >     Reviewed-by: Christoph Hellwig <hch@lst.de>
> >     Signed-off-by: Ming Lei <tom.leiming@gmail.com>
> >     Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
> >     Signed-off-by: Jens Axboe <axboe@fb.com>
> > 
> > :040000 040000 a3ebbb71c52ee4eb8c3be4d033b81179211bf704 de39a328dbd1b18519946b3ad46d9302886e0dd0 M      drivers
> > 
> > I did a diff between HEAD^ and HEAD and manually patched the file from
> > 4.15.14.  It's not an exact revert.  I'm running it now and it's working.
> > I'll do a better test later on.  Here's the patch:
> > 
> > --- a/drivers/target/target_core_pscsi.c	2018-02-04 14:31:31.077316617 -0500
> > +++ b/drivers/target/target_core_pscsi.c	2018-04-08 11:43:49.588641374 -0400
> > @@ -915,7 +915,9 @@
> >  					bio, page, bytes, off);
> >  			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
> >  				bio_segments(bio), nr_vecs);
> > -			if (rc != bytes) {
> > +			if (rc != bytes)
> > +				goto fail;
> > +			if (bio->bi_vcnt > nr_vecs) {
> >  				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
> >  					" %d i: %d bio: %p, allocating another"
> >  					" bio\n", bio->bi_vcnt, i, bio);
> 
> Hello Ming,
> 
> Can you have a look at this? The start of this e-mail thread is available at
> https://www.mail-archive.com/linux-scsi@vger.kernel.org/msg72574.html.

Sure, thanks for your sharing.

Wakko, could you test the following patch and see if there is any
difference?

--
diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
index 0d99b242e82e..6147178f1f37 100644
--- a/drivers/target/target_core_pscsi.c
+++ b/drivers/target/target_core_pscsi.c
@@ -888,7 +888,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
 		if (len > 0 && data_len > 0) {
 			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
 			bytes = min(bytes, data_len);
-
+ new_bio:
 			if (!bio) {
 				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
 				nr_pages -= nr_vecs;
@@ -931,6 +931,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
 				 * be allocated with pscsi_get_bio() above.
 				 */
 				bio = NULL;
+				goto new_bio;
 			}
 
 			data_len -= bytes;

-- 
Ming

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Ming Lei wrote:
> On Mon, Apr 09, 2018 at 09:30:11PM +0000, Bart Van Assche wrote:
> > Hello Ming,
> > 
> > Can you have a look at this? The start of this e-mail thread is available at
> > https://www.mail-archive.com/linux-scsi@vger.kernel.org/msg72574.html.
> 
> Sure, thanks for your sharing.
> 
> Wakko, could you test the following patch and see if there is any
> difference?

Sure, one question, is this against 4.15 or does it matter.  Last I looked,
4.16 hasn't changed from 4.15 for that file.

> diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> index 0d99b242e82e..6147178f1f37 100644
> --- a/drivers/target/target_core_pscsi.c
> +++ b/drivers/target/target_core_pscsi.c
> @@ -888,7 +888,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
>  		if (len > 0 && data_len > 0) {
>  			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>  			bytes = min(bytes, data_len);
> -
> + new_bio:
>  			if (!bio) {
>  				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
>  				nr_pages -= nr_vecs;
> @@ -931,6 +931,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
>  				 * be allocated with pscsi_get_bio() above.
>  				 */
>  				bio = NULL;
> +				goto new_bio;
>  			}
>  
>  			data_len -= bytes;
> 
> -- 
> Ming
-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Ming Lei]

On Mon, Apr 09, 2018 at 07:43:01PM -0400, Wakko Warner wrote:
> Ming Lei wrote:
> > On Mon, Apr 09, 2018 at 09:30:11PM +0000, Bart Van Assche wrote:
> > > Hello Ming,
> > > 
> > > Can you have a look at this? The start of this e-mail thread is available at
> > > https://www.mail-archive.com/linux-scsi@vger.kernel.org/msg72574.html.
> > 
> > Sure, thanks for your sharing.
> > 
> > Wakko, could you test the following patch and see if there is any
> > difference?
> 
> Sure, one question, is this against 4.15 or does it matter.  Last I looked,
> 4.16 hasn't changed from 4.15 for that file.

It is two-line change, and I am sure it can be applied on 4.15 cleanly. 

Thanks,
Ming

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Ming Lei wrote:
> Sure, thanks for your sharing.
> 
> Wakko, could you test the following patch and see if there is any
> difference?
> 
> --
> diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> index 0d99b242e82e..6147178f1f37 100644
> --- a/drivers/target/target_core_pscsi.c
> +++ b/drivers/target/target_core_pscsi.c
> @@ -888,7 +888,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
>  		if (len > 0 && data_len > 0) {
>  			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
>  			bytes = min(bytes, data_len);
> -
> + new_bio:
>  			if (!bio) {
>  				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
>  				nr_pages -= nr_vecs;
> @@ -931,6 +931,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
>  				 * be allocated with pscsi_get_bio() above.
>  				 */
>  				bio = NULL;
> +				goto new_bio;
>  			}
>  
>  			data_len -= bytes;

Sorry for the delay.  I reverted my change, added this one.  I didn't
reboot, I just unloaded and loaded this one.
Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
target.

Doesn't crash, however on the initiator I see this:
[9273849.707777] ISO 9660 Extensions: RRIP_1991A
[9273863.359718] scsi_io_completion: 13 callbacks suppressed
[9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
[9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
[9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
[9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
[9273863.360116] blk_update_request: 13 callbacks suppressed
[9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
[9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
[9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
[9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
[9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
[9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912

To cause this, I mounted the dvd as seen in the first line and ran this
command: find /cdrom2 -type f | xargs -tn1 cat > /dev/null
I did some various tests.  Each test was done after umount and mount to
clear the cache.
cat <file> > /dev/null causes the message.
dd if=<file> of=/dev/null bs=2048 doesn't
	using bs=4096 doesn't
	using bs=64k doesn't
	using bs=128k does
cat uses a blocksize of 128k.

The following was done without being mounted.
ddrescue -f -f /dev/sr1 /dev/null 
	doesn't cause the message
dd if=/dev/sr1 of=/dev/null bs=128k
	doesn't cause the message
using bs=256k causes the message once:
[9275916.857409] sr 27:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
[9275916.857482] sr 27:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
[9275916.857520] sr 27:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
[9275916.857556] sr 27:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 00 00 00 00 00 80 00
[9275916.857614] blk_update_request: I/O error, dev sr1, sector 0

If I access the disc from the target natively either by mounting and
accessing files or working with the device directly (ie dd) no errors are
logged on the target.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Wakko Warner wrote:
> Ming Lei wrote:
> > Sure, thanks for your sharing.
> > 
> > Wakko, could you test the following patch and see if there is any
> > difference?
> > 
> > --
> > diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> > index 0d99b242e82e..6147178f1f37 100644
> > --- a/drivers/target/target_core_pscsi.c
> > +++ b/drivers/target/target_core_pscsi.c
> > @@ -888,7 +888,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> >  		if (len > 0 && data_len > 0) {
> >  			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> >  			bytes = min(bytes, data_len);
> > -
> > + new_bio:
> >  			if (!bio) {
> >  				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
> >  				nr_pages -= nr_vecs;
> > @@ -931,6 +931,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> >  				 * be allocated with pscsi_get_bio() above.
> >  				 */
> >  				bio = NULL;
> > +				goto new_bio;
> >  			}
> >  
> >  			data_len -= bytes;
> 
> Sorry for the delay.  I reverted my change, added this one.  I didn't
> reboot, I just unloaded and loaded this one.
> Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
> target.
> 
> Doesn't crash, however on the initiator I see this:
> [9273849.707777] ISO 9660 Extensions: RRIP_1991A
> [9273863.359718] scsi_io_completion: 13 callbacks suppressed
> [9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> [9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> [9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> [9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
> [9273863.360116] blk_update_request: 13 callbacks suppressed
> [9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
> [9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> [9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> [9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> [9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
> [9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912

Just FYI: The jobs that I do that uses the disc over iscsi didn't cause any
kernel messages on either system (except for the informational when the disc
was mounted)

I have a dumb question though.  Could the label be placed just after the
'if' statement instead of before it?  bio is set to null and the 'if'
statement checks if it's null, which it always would be after the goto.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Ming Lei]

On Tue, Apr 10, 2018 at 08:45:25PM -0400, Wakko Warner wrote:
> Ming Lei wrote:
> > Sure, thanks for your sharing.
> > 
> > Wakko, could you test the following patch and see if there is any
> > difference?
> > 
> > --
> > diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> > index 0d99b242e82e..6147178f1f37 100644
> > --- a/drivers/target/target_core_pscsi.c
> > +++ b/drivers/target/target_core_pscsi.c
> > @@ -888,7 +888,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> >  		if (len > 0 && data_len > 0) {
> >  			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
> >  			bytes = min(bytes, data_len);
> > -
> > + new_bio:
> >  			if (!bio) {
> >  				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
> >  				nr_pages -= nr_vecs;
> > @@ -931,6 +931,7 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> >  				 * be allocated with pscsi_get_bio() above.
> >  				 */
> >  				bio = NULL;
> > +				goto new_bio;
> >  			}
> >  
> >  			data_len -= bytes;
> 
> Sorry for the delay.  I reverted my change, added this one.  I didn't
> reboot, I just unloaded and loaded this one.
> Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
> target.
> 
> Doesn't crash, however on the initiator I see this:
> [9273849.707777] ISO 9660 Extensions: RRIP_1991A
> [9273863.359718] scsi_io_completion: 13 callbacks suppressed
> [9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> [9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> [9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> [9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
> [9273863.360116] blk_update_request: 13 callbacks suppressed
> [9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
> [9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> [9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> [9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> [9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
> [9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912
> 
> To cause this, I mounted the dvd as seen in the first line and ran this
> command: find /cdrom2 -type f | xargs -tn1 cat > /dev/null
> I did some various tests.  Each test was done after umount and mount to
> clear the cache.
> cat <file> > /dev/null causes the message.
> dd if=<file> of=/dev/null bs=2048 doesn't
> 	using bs=4096 doesn't
> 	using bs=64k doesn't
> 	using bs=128k does
> cat uses a blocksize of 128k.
> 
> The following was done without being mounted.
> ddrescue -f -f /dev/sr1 /dev/null 
> 	doesn't cause the message
> dd if=/dev/sr1 of=/dev/null bs=128k
> 	doesn't cause the message
> using bs=256k causes the message once:
> [9275916.857409] sr 27:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> [9275916.857482] sr 27:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
> [9275916.857520] sr 27:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
> [9275916.857556] sr 27:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 00 00 00 00 00 80 00
> [9275916.857614] blk_update_request: I/O error, dev sr1, sector 0
> 
> If I access the disc from the target natively either by mounting and
> accessing files or working with the device directly (ie dd) no errors are
> logged on the target.

OK, thanks for your test.

Could you test the following patch and see if there is still the failure
message?

diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
index 0d99b242e82e..6137287b52fb 100644
--- a/drivers/target/target_core_pscsi.c
+++ b/drivers/target/target_core_pscsi.c
@@ -913,9 +913,11 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
 
 			rc = bio_add_pc_page(pdv->pdv_sd->request_queue,
 					bio, page, bytes, off);
+			if (rc != bytes)
+				goto fail;
 			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
 				bio_segments(bio), nr_vecs);
-			if (rc != bytes) {
+			if (/*rc != bytes*/0) {
 				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
 					" %d i: %d bio: %p, allocating another"
 					" bio\n", bio->bi_vcnt, i, bio);

-- 
Ming

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Ming Lei wrote:
> On Tue, Apr 10, 2018 at 08:45:25PM -0400, Wakko Warner wrote:
> > Sorry for the delay.  I reverted my change, added this one.  I didn't
> > reboot, I just unloaded and loaded this one.
> > Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
> > target.
> > 
> > Doesn't crash, however on the initiator I see this:
> > [9273849.707777] ISO 9660 Extensions: RRIP_1991A
> > [9273863.359718] scsi_io_completion: 13 callbacks suppressed
> > [9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > [9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > [9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > [9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
> > [9273863.360116] blk_update_request: 13 callbacks suppressed
> > [9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
> > [9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > [9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > [9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > [9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
> > [9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912
> > 
> > To cause this, I mounted the dvd as seen in the first line and ran this
> > command: find /cdrom2 -type f | xargs -tn1 cat > /dev/null
> > I did some various tests.  Each test was done after umount and mount to
> > clear the cache.
> > cat <file> > /dev/null causes the message.
> > dd if=<file> of=/dev/null bs=2048 doesn't
> > 	using bs=4096 doesn't
> > 	using bs=64k doesn't
> > 	using bs=128k does
> > cat uses a blocksize of 128k.
> > 
> > The following was done without being mounted.
> > ddrescue -f -f /dev/sr1 /dev/null 
> > 	doesn't cause the message
> > dd if=/dev/sr1 of=/dev/null bs=128k
> > 	doesn't cause the message
> > using bs=256k causes the message once:
> > [9275916.857409] sr 27:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > [9275916.857482] sr 27:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
> > [9275916.857520] sr 27:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
> > [9275916.857556] sr 27:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 00 00 00 00 00 80 00
> > [9275916.857614] blk_update_request: I/O error, dev sr1, sector 0
> > 
> > If I access the disc from the target natively either by mounting and
> > accessing files or working with the device directly (ie dd) no errors are
> > logged on the target.
> 
> OK, thanks for your test.
> 
> Could you test the following patch and see if there is still the failure
> message?
> 
> diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> index 0d99b242e82e..6137287b52fb 100644
> --- a/drivers/target/target_core_pscsi.c
> +++ b/drivers/target/target_core_pscsi.c
> @@ -913,9 +913,11 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
>  
>  			rc = bio_add_pc_page(pdv->pdv_sd->request_queue,
>  					bio, page, bytes, off);
> +			if (rc != bytes)
> +				goto fail;
>  			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
>  				bio_segments(bio), nr_vecs);
> -			if (rc != bytes) {
> +			if (/*rc != bytes*/0) {
>  				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
>  					" %d i: %d bio: %p, allocating another"
>  					" bio\n", bio->bi_vcnt, i, bio);

Target doesn't crash but the errors on the initiator are still there.

Seems that if I do large transfers, I see this in the initiator's logs.
With the previous patch, I burned 3 dvds at the same time, compared the
files to the originals and I have a script that catalogs the files.  The
files consist of debian packages and source files.  The 3 operations did not
show any errors in the kernel log on either end.

I did this test:
initiator: dd if=/dev/sr1 bs=512k count=1024 | md5sum
target:    dd if=/dev/sr0 bs=512k count=1024 | md5sum

Result: the same.  It's OK even with the i/o errors shown on the initiator.

The above patch was added on top of the one you gave me before, but I don't
believe that that would be an issue.

...  Now if someone could help me with a kvm virtualization problem I'm
having with 4.16 that wasn't there with 4.15...

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: 4.15.14 crash with iscsi target and dvd

[Ming Lei]

On Thu, Apr 12, 2018 at 09:43:02PM -0400, Wakko Warner wrote:
> Ming Lei wrote:
> > On Tue, Apr 10, 2018 at 08:45:25PM -0400, Wakko Warner wrote:
> > > Sorry for the delay.  I reverted my change, added this one.  I didn't
> > > reboot, I just unloaded and loaded this one.
> > > Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
> > > target.
> > > 
> > > Doesn't crash, however on the initiator I see this:
> > > [9273849.707777] ISO 9660 Extensions: RRIP_1991A
> > > [9273863.359718] scsi_io_completion: 13 callbacks suppressed
> > > [9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > [9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > > [9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > > [9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
> > > [9273863.360116] blk_update_request: 13 callbacks suppressed
> > > [9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
> > > [9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > [9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > > [9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > > [9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
> > > [9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912
> > > 
> > > To cause this, I mounted the dvd as seen in the first line and ran this
> > > command: find /cdrom2 -type f | xargs -tn1 cat > /dev/null
> > > I did some various tests.  Each test was done after umount and mount to
> > > clear the cache.
> > > cat <file> > /dev/null causes the message.
> > > dd if=<file> of=/dev/null bs=2048 doesn't
> > > 	using bs=4096 doesn't
> > > 	using bs=64k doesn't
> > > 	using bs=128k does
> > > cat uses a blocksize of 128k.
> > > 
> > > The following was done without being mounted.
> > > ddrescue -f -f /dev/sr1 /dev/null 
> > > 	doesn't cause the message
> > > dd if=/dev/sr1 of=/dev/null bs=128k
> > > 	doesn't cause the message
> > > using bs=256k causes the message once:
> > > [9275916.857409] sr 27:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > [9275916.857482] sr 27:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
> > > [9275916.857520] sr 27:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
> > > [9275916.857556] sr 27:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 00 00 00 00 00 80 00
> > > [9275916.857614] blk_update_request: I/O error, dev sr1, sector 0
> > > 
> > > If I access the disc from the target natively either by mounting and
> > > accessing files or working with the device directly (ie dd) no errors are
> > > logged on the target.
> > 
> > OK, thanks for your test.
> > 
> > Could you test the following patch and see if there is still the failure
> > message?
> > 
> > diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> > index 0d99b242e82e..6137287b52fb 100644
> > --- a/drivers/target/target_core_pscsi.c
> > +++ b/drivers/target/target_core_pscsi.c
> > @@ -913,9 +913,11 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> >  
> >  			rc = bio_add_pc_page(pdv->pdv_sd->request_queue,
> >  					bio, page, bytes, off);
> > +			if (rc != bytes)
> > +				goto fail;
> >  			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
> >  				bio_segments(bio), nr_vecs);
> > -			if (rc != bytes) {
> > +			if (/*rc != bytes*/0) {
> >  				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
> >  					" %d i: %d bio: %p, allocating another"
> >  					" bio\n", bio->bi_vcnt, i, bio);
> 
> Target doesn't crash but the errors on the initiator are still there.

OK, then this error log isn't related with my commit, because the patch
I sent to you in last email is to revert my commit simply.

But the following patch is one correct fix for your crash.

https://marc.info/?l=linux-kernel&m=152331690727052&w=2


Thanks,
Ming

Re: 4.15.14 crash with iscsi target and dvd

[Wakko Warner]

Ming Lei wrote:
> On Thu, Apr 12, 2018 at 09:43:02PM -0400, Wakko Warner wrote:
> > Ming Lei wrote:
> > > On Tue, Apr 10, 2018 at 08:45:25PM -0400, Wakko Warner wrote:
> > > > Sorry for the delay.  I reverted my change, added this one.  I didn't
> > > > reboot, I just unloaded and loaded this one.
> > > > Note: /dev/sr1 as seen from the initiator is /dev/sr0 (physical disc) on the
> > > > target.
> > > > 
> > > > Doesn't crash, however on the initiator I see this:
> > > > [9273849.707777] ISO 9660 Extensions: RRIP_1991A
> > > > [9273863.359718] scsi_io_completion: 13 callbacks suppressed
> > > > [9273863.359788] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > > [9273863.359909] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > > > [9273863.359974] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > > > [9273863.360036] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f6 96 00 00 80 00
> > > > [9273863.360116] blk_update_request: 13 callbacks suppressed
> > > > [9273863.360177] blk_update_request: I/O error, dev sr1, sector 9165400
> > > > [9273875.864648] sr 26:0:0:0: [sr1] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > > [9273875.864738] sr 26:0:0:0: [sr1] tag#1 Sense Key : 0x2 [current] 
> > > > [9273875.864801] sr 26:0:0:0: [sr1] tag#1 ASC=0x8 ASCQ=0x0 
> > > > [9273875.864890] sr 26:0:0:0: [sr1] tag#1 CDB: opcode=0x28 28 00 00 22 f7 16 00 00 80 00
> > > > [9273875.864971] blk_update_request: I/O error, dev sr1, sector 9165912
> > > > 
> > > > To cause this, I mounted the dvd as seen in the first line and ran this
> > > > command: find /cdrom2 -type f | xargs -tn1 cat > /dev/null
> > > > I did some various tests.  Each test was done after umount and mount to
> > > > clear the cache.
> > > > cat <file> > /dev/null causes the message.
> > > > dd if=<file> of=/dev/null bs=2048 doesn't
> > > > 	using bs=4096 doesn't
> > > > 	using bs=64k doesn't
> > > > 	using bs=128k does
> > > > cat uses a blocksize of 128k.
> > > > 
> > > > The following was done without being mounted.
> > > > ddrescue -f -f /dev/sr1 /dev/null 
> > > > 	doesn't cause the message
> > > > dd if=/dev/sr1 of=/dev/null bs=128k
> > > > 	doesn't cause the message
> > > > using bs=256k causes the message once:
> > > > [9275916.857409] sr 27:0:0:0: [sr1] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
> > > > [9275916.857482] sr 27:0:0:0: [sr1] tag#0 Sense Key : 0x2 [current] 
> > > > [9275916.857520] sr 27:0:0:0: [sr1] tag#0 ASC=0x8 ASCQ=0x0 
> > > > [9275916.857556] sr 27:0:0:0: [sr1] tag#0 CDB: opcode=0x28 28 00 00 00 00 00 00 00 80 00
> > > > [9275916.857614] blk_update_request: I/O error, dev sr1, sector 0
> > > > 
> > > > If I access the disc from the target natively either by mounting and
> > > > accessing files or working with the device directly (ie dd) no errors are
> > > > logged on the target.
> > > 
> > > OK, thanks for your test.
> > > 
> > > Could you test the following patch and see if there is still the failure
> > > message?
> > > 
> > > diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
> > > index 0d99b242e82e..6137287b52fb 100644
> > > --- a/drivers/target/target_core_pscsi.c
> > > +++ b/drivers/target/target_core_pscsi.c
> > > @@ -913,9 +913,11 @@ pscsi_map_sg(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> > >  
> > >  			rc = bio_add_pc_page(pdv->pdv_sd->request_queue,
> > >  					bio, page, bytes, off);
> > > +			if (rc != bytes)
> > > +				goto fail;
> > >  			pr_debug("PSCSI: bio->bi_vcnt: %d nr_vecs: %d\n",
> > >  				bio_segments(bio), nr_vecs);
> > > -			if (rc != bytes) {
> > > +			if (/*rc != bytes*/0) {
> > >  				pr_debug("PSCSI: Reached bio->bi_vcnt max:"
> > >  					" %d i: %d bio: %p, allocating another"
> > >  					" bio\n", bio->bi_vcnt, i, bio);
> > 
> > Target doesn't crash but the errors on the initiator are still there.
> 
> OK, then this error log isn't related with my commit, because the patch
> I sent to you in last email is to revert my commit simply.
> 
> But the following patch is one correct fix for your crash.
> 
> https://marc.info/?l=linux-kernel&m=152331690727052&w=2

Ok, that'll be the one I used.  Do you know when it'll go upstream?

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.