Skip to main content

Installation

Starlake CLI

Prerequisites

Make sure you have Java 11+ installed on your machine.

You can check your Java version by typing java -version in a terminal.

If you don't have Java 11+ installed, you can download it from Oracle JDK or OpenJDK

Install Starlake

To install starlake, you need to download the setup script from github. The script will in turn download required dependencies and copy them to the bin subdirectory.

sh <(curl https://raw.githubusercontent.com/starlake-ai/starlake/master/distrib/setup.sh)

The following folders should now have been created and contain Starlake dependencies.


starlake
└── bin
├── deps
├── sl
└── spark

note

Any extra library you may need (Oracle client for example) need to be copied in the bin/deps folder.

Starlake is now installed with all its dependencies. You can run the CLI by typing starlake.

This will display the commands supported by the CLI.

Starlake Version 1.2.0
Usage:
starlake [command]
Available commands =>
lineage
bootstrap
bq2yml or bq-info
compare
cnxload
esload
extract-data
extract-schema
import
infer-schema
kafkaload
load
metrics
parquet2csv
transform
watch
xls2yml
yml2ddl
table-dependencies
yml2xls

That's it! We now need to bootstrap a new project.

Graph Visualization

Starlake provides features to visualize the lineage of a table, the relationship between tables, and table level and row level acess policies.

To use these features, you need to install the GraphViz on top of which the starlake graph generator is built.

sudo [apt|yum] install graphviz

VS Code extension

Starlake comes with a vs-code plugin that allows you to interact with the Starlake CLI. You can install it from the vs-code marketplace.