Lantern CLI
Installation
Run with Docker
You can use the Docker image to quickly run Lantern CLI with docker.
Run CPU image
docker run --rm lanterndata/lantern-cli -h
Run GPU image
nvidia-docker run --rm lanterndata/lantern-cli:gpu -h
Using volumes
When running create-embeddings
command pass -v lantern-models:/models
to Docker and --data-path /models
to CLI, so the downloaded models will persist in a Docker volume.
Example:
docker run -v lantern-models:/models --rm --network host lanterndata/lantern-cli create-embeddings --model 'BAAI/bge-large-en' --uri 'postgresql://[postgres@host.docker.internal](mailto:postgres@host.docker.internal):5432/postgres' --table "wiki" --column "content" --out-column "content_embedding" --pk "id" --batch-size 40 --data-path /models
In case of start-daemon
command, just pass -v lantern-daemon-data:/var/lib/lantern-daemon
to Docker
Install ONNX Runtime
ONNX Runtime is required for Lantern CLI Embeddings. This is pre-packaged in the Docker image. It needs to be installed separately when installing from binaries. For Cargo, it is required except for Linux.
-
Get ONNX Runtime library
Get ONNX Runtime library from Github release page
bashCopyPLATFORM=linux ARCH=x64 ONNX_VERSION=1.16.1 wget "https://github.com/microsoft/onnxruntime/releases/download/v${ONNX_VERSION}/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}.tgz"
Note: If you have GPU, download the GPU version of onnxruntime library.
-
Extract archive
Extract the downloaded archive in
/usr/local/lib
bashCopymkdir -p /usr/local/lib tar xzf "onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}.tar" -C /usr/local/lib
-
Setup environment
Export path to ONNX library
bashCopyexport ORT_STRATEGY=system export ORT_DYLIB_PATH=/usr/local/lib/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}/lib/libonnxruntime.so
Note: On MacOS the library will have
.dylib
extension instead of.os
You can export this variables from your shell profile or add them told
search path like this:bashCopyecho "/usr/local/lib/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}/lib/libonnxruntime.so" > /etc/ld.so.conf.d/onnx.conf ldconfig
Install From Binaries
Prerequisities
- ONNX Runtime - see previous section for instructions
Supported platforms
- Linux (x86_64)
Use our releases from GitHub
You can download prebuilt binaries from our releases page
Install With Cargo
Prerequisites
- Rust >=1.70.0
- Cargo
- ONNX Runtime - see earlier section for instructions
Installation
cargo install --git https://github.com/lanterndata/lantern_extras.git
Verify installation
lantern-cli -h