Linux-NVME Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Tony Asleson <tasleson@redhat.com>
To: linux-nvme@lists.infradead.org
Subject: nvmet: Kernel oops when doing mkfs on nvme-tcp device
Date: Fri, 20 Mar 2020 16:36:42 -0500	[thread overview]
Message-ID: <17c25d8d-0dc2-9dc8-2b65-954034c23646@redhat.com> (raw)

Using two different VMs with nvme-tcp, I tried to create a fs and it
hung with kernel oops on client VM.

Both client/initiator and server/target are running Fedora 31,
5.5.8-200.fc31.x86_64

Reproduces every time.  I also tried creating a gpt partition table on
it first, which worked.  However, when I tried to create FS it oops'd
again and hanged mkfs command.

Thanks,
-Tony

# cat /etc/nvmet/config.json
{
  "hosts": [
    {
      "nqn":
"nqn.2014-08.org.nvmexpress:uuid:faadcb53-9214-4fbb-a746-7d94c023b53e"
    }
  ],
  "ports": [
    {
      "addr": {
        "adrfam": "ipv4",
        "traddr": "192.168.2.80",
        "treq": "not specified",
        "trsvcid": "8009",
        "trtype": "tcp"
      },
      "ana_groups": [
        {
          "ana": {
            "state": "optimized"
          },
          "grpid": 1
        }
      ],
      "param": {
        "inline_data_size": "16384"
      },
      "portid": 1,
      "referrals": [],
      "subsystems": []
    }
  ],
  "subsystems": [
    {
      "allowed_hosts": [

"nqn.2014-08.org.nvmexpress:uuid:faadcb53-9214-4fbb-a746-7d94c023b53e"
      ],
      "attr": {
        "allow_any_host": "0",
        "serial": "967af933980faedc",
        "version": "1.3"
      },
      "namespaces": [
        {
          "ana_grpid": 1,
          "device": {
            "nguid": "00000000-0000-0000-0000-000000000000",
            "path": "/dev/sdc",
            "uuid": "c2c140e4-aff2-4df4-8926-1686a485e99d"
          },
          "enable": 1,
          "nsid": 1
        }
      ],
      "nqn":
"nqn.2014-08.org.nvmexpress:NVMf:uuid:23d9d3e0-a83e-45d7-a902-3e80679385f8"
    }
  ]
}


# mkfs.ext4 /dev/nvme0n1

[  125.031254] nvme-fabrics ctl: Failed to read smart log (error -5)
[  125.031509] nvme nvme0: new ctrl: NQN
"nqn.2014-08.org.nvmexpress.discovery", addr 192.168.2.80:8009
[  125.034133] nvme nvme0: Removing ctrl: NQN
"nqn.2014-08.org.nvmexpress.discovery"
[  125.246827] nvme nvme0: creating 1 I/O queues.
[  125.249356] nvme nvme0: mapped 1/0/0 default/read/poll queues.
[  125.250788] nvme nvme0: new ctrl: NQN
"nqn.2014-08.org.nvmexpress:NVMf:uuid:23d9d3e0-a83e-45d7-a902-3e80679385f8",
addr 192.168.2.80:8009
[  125.257115] nvme0n1: detected capacity change from 0 to 34359738368
[  153.188620] blk_update_request: I/O error, dev nvme0c0n1, sector
67108736 op 0x9:(WRITE_ZEROES) flags 0x5000800 phys_seg 0 prio class 0
[  153.191395] BUG: kernel NULL pointer dereference, address:
0000000000000008
[  153.191440] #PF: supervisor read access in kernel mode
[  153.191468] #PF: error_code(0x0000) - not-present page
[  153.191495] PGD 0 P4D 0
[  153.191513] Oops: 0000 [#1] SMP PTI
[  153.191534] CPU: 0 PID: 237 Comm: kworker/0:1H Not tainted
5.5.9-200.fc31.x86_64 #1
[  153.191574] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[  153.191619] Workqueue: nvme_tcp_wq nvme_tcp_io_work [nvme_tcp]
[  153.191660] RIP: 0010:nvme_tcp_io_work+0x303/0x790 [nvme_tcp]
[  153.191692] Code: ff ff 41 8b 86 98 00 00 00 83 f8 02 0f 85 6d fd ff
ff 49 8b 46 28 48 89 04 24 49 8b 46 78 49 8b 56 68 41 8b 6e 34 41 2b 6e
38 <8b> 58 08 44 8b 60 0c 4c 8b 38 48 29 d3 49 01 d4 48 39 eb 48 0f 47
[  153.191783] RSP: 0018:ffffbc118020fde8 EFLAGS: 00010206
[  153.191810] RAX: 0000000000000000 RBX: 00000000579f5801 RCX:
0000000000000000
[  153.191863] RDX: 0000000000000000 RSI: 0000000000000011 RDI:
ffff9322579f5900
[  153.191907] RBP: 0000000000001000 R08: 0000000000001000 R09:
0000000000000000
[  153.191944] R10: 0000000000000009 R11: 0000000000000000 R12:
ffff932259d90ee0
[  153.191981] R13: 0000000000000048 R14: ffff9322579f58a0 R15:
0000000000000048
[  153.192018] FS:  0000000000000000(0000) GS:ffff93225bc00000(0000)
knlGS:0000000000000000
[  153.192059] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  153.192087] CR2: 0000000000000008 CR3: 0000000116ff2005 CR4:
00000000000606f0
[  153.192126] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  153.192163] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  153.192199] Call Trace:
[  153.192228]  process_one_work+0x1b5/0x360
[  153.192263]  worker_thread+0x50/0x3c0
[  153.192286]  kthread+0xf9/0x130
[  153.192311]  ? process_one_work+0x360/0x360
[  153.192334]  ? kthread_park+0x90/0x90
[  153.192361]  ret_from_fork+0x35/0x40
[  153.192390] Modules linked in: nvme_tcp nvme_fabrics nvme_core rfkill
snd_intel8x0 intel_rapl_msr intel_rapl_common intel_powerclamp
snd_ac97_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
snd_pcsp ac97_bus snd_pcm e1000 snd_timer intel_rapl_perf joydev snd
soundcore i2c_piix4 vboxguest ip_tables xfs libcrc32c vmwgfx
drm_kms_helper ttm drm crc32c_intel serio_raw ata_generic pata_acpi video
[  153.194139] CR2: 0000000000000008
[  153.194619] ---[ end trace 94d1bf7f7728b447 ]---
[  153.195102] RIP: 0010:nvme_tcp_io_work+0x303/0x790 [nvme_tcp]
[  153.195565] Code: ff ff 41 8b 86 98 00 00 00 83 f8 02 0f 85 6d fd ff
ff 49 8b 46 28 48 89 04 24 49 8b 46 78 49 8b 56 68 41 8b 6e 34 41 2b 6e
38 <8b> 58 08 44 8b 60 0c 4c 8b 38 48 29 d3 49 01 d4 48 39 eb 48 0f 47
[  153.197031] RSP: 0018:ffffbc118020fde8 EFLAGS: 00010206
[  153.197525] RAX: 0000000000000000 RBX: 00000000579f5801 RCX:
0000000000000000
[  153.198026] RDX: 0000000000000000 RSI: 0000000000000011 RDI:
ffff9322579f5900
[  153.198511] RBP: 0000000000001000 R08: 0000000000001000 R09:
0000000000000000
[  153.198997] R10: 0000000000000009 R11: 0000000000000000 R12:
ffff932259d90ee0
[  153.199828] R13: 0000000000000048 R14: ffff9322579f58a0 R15:
0000000000000048
[  153.200406] FS:  0000000000000000(0000) GS:ffff93225bc00000(0000)
knlGS:0000000000000000
[  153.200876] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  153.201347] CR2: 0000000000000008 CR3: 0000000116ff2005 CR4:
00000000000606f0
[  153.201816] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  153.202284] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[  183.652086] block nvme0n1: no usable path - requeuing I/O


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

             reply	other threads:[~2020-03-20 21:37 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-20 21:36 Tony Asleson [this message]
2020-03-20 22:23 ` nvmet: Kernel oops when doing mkfs on nvme-tcp device Chaitanya Kulkarni
2020-03-20 23:24   ` Tony Asleson
2020-03-20 22:54 ` Sagi Grimberg
2020-03-20 22:59   ` Sagi Grimberg
2020-03-20 23:46     ` Tony Asleson
2020-03-21  0:43       ` Chaitanya Kulkarni
2020-03-23 17:56         ` Tony Asleson
2020-03-23 19:42           ` Sagi Grimberg
2020-03-23 20:10             ` Tony Asleson
2020-03-20 23:43   ` Tony Asleson
2020-03-23  8:33     ` Sagi Grimberg
2020-03-23 15:57       ` Tony Asleson
2020-03-23 19:41         ` Sagi Grimberg
2020-03-23 20:46           ` Chaitanya Kulkarni
2020-03-23 20:52             ` Sagi Grimberg
2020-03-23 22:02               ` Sagi Grimberg
2020-03-23 23:37                 ` Tony Asleson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=17c25d8d-0dc2-9dc8-2b65-954034c23646@redhat.com \
    --to=tasleson@redhat.com \
    --cc=linux-nvme@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox