address control file not exists / kernel 4.4.32

[monsterzz]

I'm trying to run go-tcmu with custom handler on linux 4.4 and found that there is no address control file in configfs.

DEBU[0000] Got a TCMU mailbox, version: 2
           source="device.go:310"
DEBU[0000] mapsize: 1118208
           source="device.go:311"
DEBU[0000] mbFlags: 0
           source="device.go:312"
DEBU[0000] mbCmdrOffset: 128
           source="device.go:313"
DEBU[0000] mbCmdrSize: 65408
           source="device.go:314"
DEBU[0000] mbCmdHead: 0
           source="device.go:315"
DEBU[0000] mbCmdTail: 0
           source="device.go:316"
DEBU[0000] SCSI EVPD Inquiry 0x0
           source="cmd_handler.go:103"
DEBU[0000] SCSI EVPD Inquiry 0x83
           source="cmd_handler.go:103"
FATA[0000] Couldn't create tcmu in /dev/xxx: open /sys/kernel/config/target/loopback/naa.50000000098f6bcd/tpgt_1/address: no such file or directory

After ~30s kernel hangs:

Nov 23 02:22:25 pool03e kernel: [ 2224.774431] NMI watchdog: BUG: hard lockup - CPU#10 stuck for 10s! [swapper/10:0] at native_queued_spin_lock_slowpath+0x121/0x170
Nov 23 02:22:25 pool03e kernel: [ 2224.774490] Modules linked in: tcm_loop target_core_pscsi target_core_file target_core_iblock target_core_user target_core_mod uio ipt_MASQUERADE nf_nat_masquerade_ipv4 bridge iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype nf_conntrack_ipv6 nf_defrag_ipv6 xt_u32 xt_tcpudp xt_comment xt_multiport xt_conntrack nf_conntrack veth overlay quota_v2 quota_tree tcp_diag inet_diag unix_diag ip6table_filter ip6_tables iptable_filter ip_tables x_tables cls_cgroup sch_htb netconsole configfs dummy 8021q mrp garp stp llc intel_powerclamp coretemp kvm_intel gpio_ich joydev input_leds mac_hid kvm lpc_ich 8250_fintek tpm_infineon i5500_temp serio_raw shpchp irqbypass crc32_pclmul i7core_edac ioatdma edac_core mlx4_en mlx4_core vxlan udp_tunnel ip6_udp_tunnel tcp_htcp hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 igb raid0 i2c_algo_bit dca ahci multipath ptp psmouse libahci pps_core linear fjes [last unloaded: pmi_msghandler]
Nov 23 02:22:25 pool03e kernel: [ 2224.774543] CPU: 10 PID: 0 Comm: swapper/10 Not tainted 4.4.32-11 #1
Nov 23 02:22:25 pool03e kernel: [ 2224.774544] Hardware name: Supermicro X8DTU/X8DTU, BIOS 2.1c       08/03/2012
Nov 23 02:22:25 pool03e kernel: [ 2224.774545] task: ffff881808ede200 ti: ffff880c08d90000 task.ti: ffff880c08d90000
Nov 23 02:22:25 pool03e kernel: [ 2224.774547] RIP: 0010:[<ffffffff810f4a01>]  [<ffffffff810f4a01>] native_queued_spin_lock_slowpath+0x121/0x170
Nov 23 02:22:25 pool03e kernel: [ 2224.774550] RSP: 0018:ffff88180f303d20  EFLAGS: 00000046
Nov 23 02:22:25 pool03e kernel: [ 2224.774551] RAX: 0000000000000000 RBX: 0000000000000002 RCX: ffff88180f317380
Nov 23 02:22:25 pool03e kernel: [ 2224.774552] RDX: ffffffff81f50a78 RSI: 00000000002c0000 RDI: ffff881725e38500
Nov 23 02:22:25 pool03e kernel: [ 2224.774553] RBP: ffff88180f303d20 R08: 0000000000000101 R09: 0000000000000000
Nov 23 02:22:25 pool03e kernel: [ 2224.774555] R10: 0000000000000658 R11: 0000000000000005 R12: ffff881725e38500
Nov 23 02:22:25 pool03e kernel: [ 2224.774556] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff
Nov 23 02:22:25 pool03e kernel: [ 2224.774557] FS:  0000000000000000(0000) GS:ffff88180f300000(0000) knlGS:0000000000000000
Nov 23 02:22:25 pool03e kernel: [ 2224.774558] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Nov 23 02:22:25 pool03e kernel: [ 2224.774559] CR2: 00007f5bb9ffafb8 CR3: 0000000001e0b000 CR4: 00000000000006e0
Nov 23 02:22:25 pool03e kernel: [ 2224.774561] Stack:
Nov 23 02:22:25 pool03e kernel: [ 2224.774562]  ffff88180f303d38 ffffffff8180eeae ffff881725e383f0 ffff88180f303d88
Nov 23 02:22:25 pool03e kernel: [ 2224.774564]  ffffffffa045ac74 ffffffff82130ac2 ffff880b00000022 0000000000000286
Nov 23 02:22:25 pool03e kernel: [ 2224.774566]  ffff881725e38660 ffff88180f303dd8 0000000000000000 00000000000000ff
Nov 23 02:22:25 pool03e kernel: [ 2224.774568] Call Trace:
Nov 23 02:22:25 pool03e kernel: [ 2224.774569]  <IRQ> 
Nov 23 02:22:25 pool03e kernel: [ 2224.774576]  [<ffffffff8180eeae>] _raw_spin_lock_irqsave+0x3e/0x50
Nov 23 02:22:25 pool03e kernel: [ 2224.774596]  [<ffffffffa045ac74>] target_complete_cmd+0x44/0x1f0 [target_core_mod]
Nov 23 02:22:25 pool03e kernel: [ 2224.774599]  [<ffffffffa04aeba1>] tcmu_check_expired_cmd+0x41/0x60 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774601]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774605]  [<ffffffff813c800a>] idr_for_each+0x9a/0xe0
Nov 23 02:22:25 pool03e kernel: [ 2224.774607]  [<ffffffffa04aeb60>] ? tcmu_open+0x40/0x40 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774610]  [<ffffffffa04af2d8>] tcmu_device_timedout+0x58/0x80 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774612]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774615]  [<ffffffff8110e102>] call_timer_fn+0x42/0x160
Nov 23 02:22:25 pool03e kernel: [ 2224.774617]  [<ffffffff8110d90a>] ? cascade+0x4a/0x80
Nov 23 02:22:25 pool03e kernel: [ 2224.774618]  [<ffffffff8110f801>] run_timer_softirq+0x281/0x2d0
Nov 23 02:22:25 pool03e kernel: [ 2224.774621]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774624]  [<ffffffff810b2714>] __do_softirq+0xd4/0x2c0
Nov 23 02:22:25 pool03e kernel: [ 2224.774625]  [<ffffffff810b2b2b>] irq_exit+0x7b/0xa0
Nov 23 02:22:25 pool03e kernel: [ 2224.774629]  [<ffffffff81811b86>] smp_apic_timer_interrupt+0x46/0x60
Nov 23 02:22:25 pool03e kernel: [ 2224.774631]  [<ffffffff8180fe22>] apic_timer_interrupt+0x82/0x90

I've searched thru open-iscsi/tcmu-runner and Documentation/target/tcmu-design.txt and didn't found any references to this control file. Seems that they're using another kernel API to add new devices (UIO as far as I understand).

Am I running incompatible kernel version? Which minimum version do I need to get go-tcmu working?

Thanks,
Gleb.

[nak3]

FATA[0000] Couldn't create tcmu in /dev/xxx: open /sys/kernel/config/target/loopback/naa.50000000098f6bcd/tpgt_1/address: no such file or directory

It looks like address file was missed. /loopback/${WWN}/tpgt_1/address was introduced via this upstream patch tcm_loop: Show address of tpg in configfs, so kernel 4.4 might not include this.

You can check it with targetcli if your kernel version creates address file or not.

# targetcli /loopback create
Created target naa.500140518a1bcbc1.
# ls /sys/kernel/config/target/loopback/naa.500140518a1bcbc1/tpgt_1/address
/sys/kernel/config/target/loopback/naa.500140518a1bcbc1/tpgt_1/address