Segmentation fault while running infer_simple.py #607
Description
When I run the following command -
gdb --args python2 tools/infer_simple.py --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml --output-dir /tmp/detectron-visualizations --image-ext jpg --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl demo
I get a Segmentation fault output backtrace as follows:-
INFO infer_simple.py: 133: Processing demo/33823288584_1d21cf0a26_k.jpg -> /tmp/detectron-visualizations/33823288584_1d21cf0a26_k.jpg.pdf
[New Thread 0x7fffc4237700 (LWP 22606)]
[New Thread 0x7fffc6a38700 (LWP 22607)]
[New Thread 0x7fffcda3a700 (LWP 22608)]
[New Thread 0x7fffc9239700 (LWP 22609)]
[New Thread 0x7fff77d05700 (LWP 22610)]
[New Thread 0x7fff77504700 (LWP 22611)]
[New Thread 0x7fff76d03700 (LWP 22612)]
[New Thread 0x7fff76502700 (LWP 22613)]
[New Thread 0x7fff75d01700 (LWP 22614)]
[New Thread 0x7fff75500700 (LWP 22615)]
[New Thread 0x7fff74cff700 (LWP 22616)]
[New Thread 0x7fff744fe700 (LWP 22617)]
[New Thread 0x7fff73cfd700 (LWP 22618)]
[New Thread 0x7fff734fc700 (LWP 22619)]
[New Thread 0x7fff72cfb700 (LWP 22620)]
[New Thread 0x7fff724fa700 (LWP 22621)]
[New Thread 0x7fff71cf9700 (LWP 22622)]
[New Thread 0x7fff714f8700 (LWP 22623)]
[New Thread 0x7fff70cf7700 (LWP 22624)]
I0809 14:08:29.244617 22577 net_async_base.cc:435] Using specified CPU pool size: 4; NUMA node id: -1
I0809 14:08:29.244650 22577 net_async_base.cc:440] Created new CPU pool, size: 4; NUMA node id: -1
[New Thread 0x7fff55fff700 (LWP 22625)]
[New Thread 0x7fff557fe700 (LWP 22626)]
[New Thread 0x7fff54ffd700 (LWP 22627)]
[New Thread 0x7fff4dfff700 (LWP 22628)]
Thread 44 "CaffeTaskThread" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff55fff700 (LWP 22625)]
0x00007ffff7874532 in __GI___libc_free (mem=0x28b5320) at malloc.c:2967
2967 malloc.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7874532 in __GI___libc_free (mem=0x28b5320) at malloc.c:2967
#1 0x00007fffeec63099 in ?? ()
from /home/siddharthnavin/.local/lib/python2.7/site-packages/numpy/core/multiarray.so
#2 0x00007fffeec631a7 in ?? ()
from /home/siddharthnavin/.local/lib/python2.7/site-packages/numpy/core/multiarray.so
#3 0x00000000004ba22c in frame_dealloc.lto_priv.285 (f=
Frame 0x7ffebf36d330, for file /home/siddharthnavin/Detectron/detectron/ops/generate_proposals.py, line 104, in forward (shift_y=<numpy.ndarray at remote 0x7fffd2270f80>, shifts=<numpy.ndarray at remote 0x7fffd22708f0>, num_images=1, A=3, K=1000, all_anchors=<numpy.ndarray at remote 0x7fffd22703f0>, rois=<numpy.ndarray at remote 0x7fffd2ae89e0>, roi_probs=<numpy.ndarray at remote 0x7fffd2ae8940>, im_i=0, im_i_boxes=<numpy.ndarray at remote 0x7fffd2270300>, im_i_probs=<numpy.ndarray at remote 0x7fffd2ae8990>, batch_inds=<numpy.ndarray at remote 0x7fffd2ae8a30>, im_i_rois=<numpy.ndarray at remote 0x7fffd2ae88f0>)) at ../Objects/frameobject.c:458
#4 PyEval_EvalCodeEx () at ../Python/ceval.c:3593
#5 0x00000000004d54b9 in function_call.lto_priv () at ../Objects/funcobject.c:523
#6 0x00000000004eebee in PyObject_Call (kw=0x0,
arg=(<GenerateProposalsOp(_feat_stride=<float at remote 0x2892c90>, _num_anchors=3, _train=False, _anchors=<numpy.ndarray at remote 0x7fffd22e7f80>) at remote 0x7fffd23c9650>, [<caffe2.python.caffe2_pybind11_state_gpu.TensorCPU at remote 0x7fffd22bd0a0>, <caffe2.python.caffe2_pybind11_state_gpu.TensorCPU at remote 0x7fffd22bd148>, <caffe2.python.caffe2_pybind11_state_gpu.TensorCPU at remote 0x7fffd22bd180>], [<caffe2.python.caffe2_pybind11_state_gpu.TensorCPU at remote 0x7fffd22bd1b8>, <caffe2.python.caffe2_pybind11_state_gpu.TensorCPU at remote 0x7fffd22bd1f0>]),
func=<function at remote 0x7fffd2e97410>) at ../Objects/abstract.c:2546
#7 instancemethod_call.lto_priv () at ../Objects/classobject.c:2602
#8 0x00000000004a577e in PyObject_Call () at ../Objects/abstract.c:2546
#9 0x00000000004c5e10 in PyEval_CallObjectWithKeywords () at ../Python/ceval.c:4219
#10 0x00007fffb9571313 in pybind11::object pybind11::detail::object_apipybind11::handle::operator()<(pybind11::return_value_policy)1, std::vector<pybind11::object, std::allocatorpybind11::object >&, std::vector<pybind11::object, std::allocatorpybind11::object >&>(std::vector<pybind11::object, std::allocatorpybind11::object >&, std::vector<pybind11::object, std::allocatorpybind11::object >&) const ()
from /usr/local/lib/python2.7/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.so
#11 0x00007fffb9574a5e in caffe2::python::PythonOpBase<caffe2::CPUContext, false>::RunOnDevice() ()
from /usr/local/lib/python2.7/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.so
#12 0x00007fffb957189b in caffe2::Operatorcaffe2::CPUContext::Run(int) ()
from /usr/local/lib/python2.7/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.so
#13 0x00007fffb9587feb in caffe2::GPUFallbackOp<caffe2::python::PythonOp<caffe2::CPUContext, false>, caffe2::SkipIndices<> >::RunOnDevice() () from /usr/local/lib/python2.7/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.so
#14 0x00007fffb9585f1d in caffe2::Operatorcaffe2::CUDAContext::RunAsync(int) ()
from /usr/local/lib/python2.7/dist-packages/caffe2/python/caffe2_pybind11_state_gpu.so
#15 0x00007fffb87c76c8 in caffe2::AsyncNetBase::run(int, int) () from /usr/local/lib/libcaffe2.so
#16 0x00007fffb87db057 in caffe2::AsyncSchedulingNet::schedule(int, bool)::{lambda()#1}::operator()() const ()
from /usr/local/lib/libcaffe2.so
#17 0x00007fffb87cd619 in caffe2::TaskThreadPool::main_loop(unsigned long) () from /usr/local/lib/libcaffe2.so
#18 0x00007ffff161f57f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x00007ffff7bc16ba in start_thread (arg=0x7fff55fff700) at pthread_create.c:333
#20 0x00007ffff78f741d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109