Skip to content

Crab popularity collector returns meaningless error message if MV refresh fails #83

@nikmagini

Description

@nikmagini

When an MV refresh procedure fails, the CRAB popdb collector script popdbRefresh.py returns an obscure error message [*].

The reason is that multiprocessing.Pool used by popdbRefresh.py to run the refresh procedures in parallel requires that all returns - including errors - are pickleable.

The Oracle exceptions in the _refresh_SingleMV functions should be properly caught and handled to avoid being propagated upwards and triggering the UnpickingError from multiprocessing.Pool

[*]

INFO:data popularity:got connection for _refresh_SingleMV table MV_block_stat0, mode F
INFO:data popularity:got connection for _refresh_SingleMV table MV_DS_Files, mode F
INFO:data popularity:got connection for _refresh_SingleMV table MV_USER_USERID, mode F
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.6/threading.py", line 484, in run
self.__target(_self.__args, *_self.__kwargs)
File "/usr/lib64/python2.6/multiprocessing/pool.py", line 259, in _handle_results
task = get()

UnpicklingError: NEWOBJ class argument has NULL tp_new

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions