Support DESK

Follow

H6.2 - matchIT Hub for Spark - Installation

Previous Article matchIT Hub Index Next Article 

To install matchIT Hub for Spark to your home directory simply download the tar and extract:

cd ~
tar -zxvf mHubSpark-1.8-linux.tar.gz

Installation Contents

The tar will extract to a folder called mHubSpark. This folder contains the following:

lib Folder containing matchIT Hub native shared object library.
libstdc++ Folder containing C++ standard libraries required by matchIT Hub.
samples Folder containing sample applications.
HubSpark.jar Jar containing the matchIT Hub for Spark package.
matchITHub.jar Jar containing the matchIT Hub Java interface.
readme Text file containing overview of product.
release_notes Text file containing release notes.

Native Libraries

The native library files should be placed in folder, e.g. /usr/local/lib64, that can be synced with all the processing nodes. Execute the following steps:

cd mHubSpark/lib
chmod 755 ./libmatchITHub.so
sudo cp libmatchITHub.so /usr/local/lib64
cd ../libstdc++
chmod 755 ./lib*.so
sudo cp lib*.so /usr/local/lib64

Ensure that the files in /usr/local/lib64 are synced with all processing nodes in the cluster and that spark-env.sh contains these lines:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64
SPARK_YARN_USER_ENV="LD_LIBRARY_PATH=$LD_LIBRARY_PATH"

Previous Article matchIT Hub Index Next Article 
Was this article helpful?
0 out of 0 found this helpful

0 Comments

Please sign in to leave a comment.