Vertex AI

You need to enable the Vertex AI API and set up the Google Application Default Credentials.

Chat model

We recommend configuring Claude 3.5 Sonnet as your chat model.

YAML

config.yaml

models:
  - name: Claude 3.5 Sonnet
    provider: vertexai
    model: claude-3-5-sonnet-20240620
    env:
      projectId: <PROJECT_ID>
      region: us-east5

Autocomplete model

We recommend configuring Codestral or code-gecko as your autocomplete model.

YAML

config.yaml

models:
  - name: Codestral (Vertex AI)
    provider: vertexai
    model: codestral
    roles:
      - autocomplete
    env:
      projectId: <PROJECT_ID>
      region: us-central1
      

Embeddings model

We recommend configuring text-embedding-004 as your embeddings model.

YAML

config.yaml

models:
  - name: Text Embedding-004
    provider: vertexai
    model: text-embedding-004
    env:
      projectId: <PROJECT_ID>
      region: us-central1
    roles:
      - embed

Reranking model

Vertex AI currently does not offer any reranking models.

Click here to see a list of reranking model providers.

Express mode

You can use VertexAI in express mode by only providing an API Key. Only some Gemini models are supported in express mode for now.