Lodestar Big Data Connekt Toolkit is a software suite that integrates
processing in Apache Hadoop with operations in Operational Data
Store (ODS). Our toolkit has been designed to work seamlessly
with Hadoop and a variety of ODS. The toolkit allows for fast,
seamless bi-directional data transfer between ODS and the Hadoop
Operational Data Lake (HODL). They are easy to use with existing
skill sets, greatly simplifying development of Big Data solutions.
Connekt delivers high speed connectivity, performance, and security
for Big Data applications.
Lodestar Big Data Connekt Toolkit overview
Big Data allows the modern enterprise to process large amounts
of structured and unstructured data. Insights gleaned from these
solutions allow them to disrupt competitors, get actionable
intel on individual customers, and exploit new analyses. Hadoop
Operational Data Lakes which are created by merging unstructured
and structured data are then used for exploratory analysis of
raw data.
Lodestar Connekt provides
bi-directional data flow and allows for the integration of results
with ODS for real-time queries, advanced analytics, and complex
data management.
The components of the Connekt Toolkit are:
-
SQL Connector for Hadoop Distributed File System (HDFS)
-
Loader for Hadoop
-
Advanced Analytics for Hadoop
-
XQuery for Hadoop
-
Data Integrator
Conneckt SQL Connector for Hadoop Distributed File System (HDFS)
Connekt SQL Connector for HDFS allows you to query data in the
HODL (Hadoop Operational Data Lake) using SQL. The data is accessed
via external tables. Data is loaded by selecting data from the
HODL and ODS and inserting it into an external Hadoop RDBMS
table. Lodestar Connekt offers out of the box flexibility in
terms of dealing with data types and
Hive partitioning.
Connekt Loader for Hadoop
Connekt Loader is a high performance load tool for fast movement
of data between ODS and HODL. When pumping data out of HODL,
Connekt Loader for Hadoop sorts, partitions, and converts data
into ODS types on Hadoop before loading into ODS. Transforming
the data into ODS types reduces database CPU usage during the
load, minimizing impact on database applications. This feature
makes this connector useful for continuous and frequent loads.
Connekt Loader for Hadoop intelligently distributes data across
Hadoop nodes while loading data in parallel. This minimizes
the effects of data skew, a common concern in parallel applications.
Connekt Loader for Hadoop can load a wide variety of input formats:
text files, Hive tables, log files (parse and load), NoSQL,
and more. Through Hive it can also load from input formats (e.g.:
Parquet, JSON files) and input sources (e.g.: HBase) accessible
to Hive. In addition, Connekt Loader for Hadoop can read proprietary
data formats through custom input format implementations provided
by the user.
Lodestar R-LYTICS for Hadoop
Lodestar R-LYTICS for Hadoop runs R code in Hadoop for scalable
analytics. The connector hides complexities of Hadoop-based
computing from the R user, executing R code in parallel in Hadoop
from stand-alone desktop applications developed in any IDE the
R user chooses.
R-LYTICS for Hadoop enables faster insights with the rich collection
of scalable, high performance, parallel implementations of common
statistical and predictive techniques in Hadoop, without requiring
data movement to any other platform. It supports rapid development
with R-style debugging capabilities of parallel R code on user
desktops, simulating parallelism under the covers. The connector
enables analysts to combine datafrom several environments –
client desktop, HDFS, Hive, HODL and in-memory R data structures
– all in the context of a single analytic task execution, greatly
simplifying data assembly and preparation.
LODESTAR CONNEKT Enterprise
Lodestar Connekt Enterprise has a comprehensive set of knowledge
modules for Hadoop. The modules allow complex bidirectional
data transformation operations executed through a familiar graphical
interface, greatly simplifying running jobs in Hadoop. Data
movement from one source to another can be graphically defined
(e.g.: from ODS to HODL, HODL to Hive, etc.). Knowledge modules
for Connekt Loader for Hadoop and Connekt SQL Connector provide
high speed data movement. Connekt Enterprise eliminates the
need to write complex code for Hadoop applications.
