(The command that is blocked is an "ip netns add ..." command.)
This happens in the openstack CI on ubuntu kernel 4.4.0-93-generic.
On another box (not openstack CI), this issue seems correlated with a lot of "unregister_netdevice: waiting for lo to become free. Usage count = X" (with varying values for X: 1, 3, 6).
networking-bagpipe fullstack job hits the following kernel issue when a new tests are added that use more netns's:
Sep 01 14:48:40 ubuntu- xenial- rax-dfw- 10739585 kernel: INFO: "task ip:1358 blocked for more than 120 seconds. xenial- rax-dfw- 10739585 kernel: Tainted: G OE 4.4.0-93-generic #116-Ubuntu xenial- rax-dfw- 10739585 kernel: "echo 0 > /proc/sys/ kernel/ hung_task_ timeout_ secs" disables this message. xenial- rax-dfw- 10739585 kernel: ip D ffff880166acfdc8 0 1358 1356 0x00000000 xenial- rax-dfw- 10739585 kernel: ffff880166acfdc8 ffff880166acfd98 ffff880205a88000 ffff8800eb29d940 xenial- rax-dfw- 10739585 kernel: ffff880166ad0000 ffffffff81ef78a4 ffff8800eb29d940 00000000ffffffff xenial- rax-dfw- 10739585 kernel: ffffffff81ef78a8 ffff880166acfde0 ffffffff8183f0d5 ffffffff81ef78a0 xenial- rax-dfw- 10739585 kernel: Call Trace: xenial- rax-dfw- 10739585 kernel: [<ffffffff8183f 0d5>] schedule+0x35/0x80 xenial- rax-dfw- 10739585 kernel: [<ffffffff8183f 37e>] schedule_ preempt_ disabled+ 0xe/0x10 xenial- rax-dfw- 10739585 kernel: [<ffffffff81840 fb9>] __mutex_ lock_slowpath+ 0xb9/0x130 xenial- rax-dfw- 10739585 kernel: [<ffffffff81841 04f>] mutex_lock+ 0x1f/0x30 xenial- rax-dfw- 10739585 kernel: [<ffffffff8172d a4e>] copy_net_ ns+0x6e/ 0x120 xenial- rax-dfw- 10739585 kernel: [<ffffffff810a1 74b>] create_ new_namespaces+ 0x11b/0x1d0 xenial- rax-dfw- 10739585 kernel: [<ffffffff810a1 98a>] unshare_ nsproxy_ namespaces+ 0x5a/0xb0 xenial- rax-dfw- 10739585 kernel: [<ffffffff81080 b41>] SyS_unshare+ 0x1f1/0x3a0 xenial- rax-dfw- 10739585 kernel: [<ffffffff81843 1f2>] entry_SYSCALL_ 64_fastpath+ 0x16/0x71
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
Sep 01 14:48:40 ubuntu-
( http:// logs.openstack. org/66/ 500066/ 1/check/ gate-networking -bagpipe- dsvm-fullstack- ubuntu- xenial- nv/99f751d/ logs/syslog. txt.gz )
(The command that is blocked is an "ip netns add ..." command.)
This happens in the openstack CI on ubuntu kernel 4.4.0-93-generic.
On another box (not openstack CI), this issue seems correlated with a lot of "unregister_ netdevice: waiting for lo to become free. Usage count = X" (with varying values for X: 1, 3, 6).