support tensor parallel #2673

Job	Run time
build-and-test-cpp-x86_64 (ubuntu-20.04, mkl)	5m 46s
build-and-test-cpp-x86_64 (ubuntu-20.04, dnnl)	5m 13s
build-and-push-docker-images	28m 7s
build-and-test-cpp-x86_64 (macos-11, mkl)	10m 20s
build-and-test-cpp-aarch64 (openblas)	4m 18s
build-and-test-cpp-x86_64 (macos-11, dnnl)	14m 14s
build-python-wheels (ubuntu-20.04, auto64)	33m 20s
check-python-style	10s
build-python-wheels (macos-11, auto64)	41m 10s
build-python-wheels (windows-2019, auto64)	41m 29s
build-python-wheels (ubuntu-20.04, aarch64)	1h 13m 44s
build-python-wheels (macos-11, arm64)	13m 39s
build-and-deploy-docs	38s
test-python-wheels (ubuntu-20.04)	10m 24s
test-python-wheels (macos-11)	3m 52s
test-python-wheels (windows-2019)	4m 28s
publish-python-wheels-on-pypi	0s
	4h 50m 52s

Provide feedback