Migrate from Pinecone to Lantern Cloud

This guide assumes that you are using Pinecone, and that you want to use Lantern Cloud instead.

Steps

1. Create a Lantern Cloud database

   Sign up for Lantern Cloud and create a database. Obtain a database URL. We'll call this LANTERN_DATABASE_URL.

2. Install the lantern-pinecone client

   The most straightforward way to migrate from Pinecone to Lantern Cloud is by using the lantern-pinecone Python client. Even if you don't want to use lantern-pinecone as your primary data client, you can use it to migrate the data and then interact with your data using another data client.

   bash

   Copy

   pip install lantern-pinecone

3. Initialize client and migrate

   Next, we will initialize the Lantern client.

   python

   Copy

   import lantern_pinecone
   
   LANTERN_DATABASE_URL = "<your_lantern_database_url>"
   lantern_pinecone.init(LANTERN_DATABASE_URL)

   For optimal performance, provide a list of pinecone_ids to the client. Otherwise, the client will use a workaround to query the Pinecone API to retrieve the IDs. For more details, see [1] [2].

   In the example below, we assume that the Pinecone has vectors with sequential ids from 0 to 1000.

   python

   Copy

   pinecone_ids = list(map(lambda x: str(x), range(1000)))
   
   index = lantern_pinecone.create_from_pinecone(
        api_key=<your_pinecone_api_key>,
        environment=<your_pinecone_environment>,
        index_name=<pinecone_index_name>,
        pinecone_ids=pinecone_ids,
   )

   See the documentation for more details on the lantern-pinecone client.

4. Final steps

   After this step the data will be copied to your database under a table with the same name as your <pinecone_index_name> with HNSW index on embedding column and GIN index on the metadata column

   You can view index stats using

   python

   Copy

   index.describe_index_stats()

Support

To read more about the lantern-pinecone client, check out the Github repo.

Reach out to support@lantern.dev for any questions or assistance with migrations. We're happy to help.