Chat Role

Chat model role

A “chat model” is an LLM that is trained to respond in a conversational format. Because they should be able to answer general questions and generate complex code, the best chat models are typically large, often 405B+ parameters.

In Continue, these models are used for normal Chat. The selected chat model will also be used for Edit and Apply if no edit or apply models are specified, respectively.

Recommended Chat models

Best overall experience

For the best overall Chat experience, you will want to use a 400B+ parameter model or one of the frontier models.

Claude Opus 4.6 and Claude Sonnet 4 from Anthropic

Our current top recommendations are Claude Opus 4.6 and Claude Sonnet 4 from Anthropic.

YAML

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Claude Opus 4.6
    provider: anthropic
    model: claude-opus-4-6
    apiKey: <YOUR_ANTHROPIC_API_KEY>

Gemma from Google DeepMind

If you prefer to use an open-weight model, then the Gemma family of Models from Google DeepMind is a good choice. You will need to decide if you use it through a SaaS model provider, e.g. Together, or self-host it, e.g. Ollama.

YAML

Ollama

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: "Gemma 4"
    provider: "ollama"
    model: "gemma4"

Together

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: "Gemma 3 27B"
    provider: "together"
    model: "google/gemma-2-27b-it"
    apiKey: <YOUR_TOGETHER_API_KEY>

GPT-5.1 from OpenAI

If you prefer to use a model from OpenAI, then we recommend GPT-5.1.

YAML

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: GPT-5.1
    provider: openai
    model: gpt-5.1
    apiKey: <YOUR_OPENAI_API_KEY>

Grok-4 from xAI

If you prefer to use a model from xAI, then we recommend Grok-4.

YAML

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Grok-4.1
    provider: xAI
    model: grok-4-1-fast-non-reasoning
    apiKey: <YOUR_XAI_API_KEY>

Gemini 3.1 Pro from Google

If you prefer to use a model from Google, then we recommend Gemini 3.1 Pro.

YAML

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Gemini 3.1 Pro
    provider: gemini
    model: gemini-3.1-pro-preview
    apiKey: <YOUR_GEMINI_API_KEY>

Local, Offline Experience

For the best local, offline Chat experience, you will want to use a model that is large but fast enough on your machine.

Llama 3.1 8B

If your local machine can run an 8B parameter model, then we recommend running Llama 3.1 8B on your machine (e.g. using Ollama or LM Studio).

YAML

Ollama

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Llama 3.1 8B
    provider: ollama
    model: llama3.1:8b

LM Studio

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Llama 3.1 8B
    provider: lmstudio
    model: llama3.1:8b

Msty

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Llama 3.1 8B
    provider: msty
    model: llama3.1:8b

DeepSeek Coder 2 16B

If your local machine can run a 16B parameter model, then we recommend running DeepSeek Coder 2 16B (e.g. using Ollama or LM Studio).

YAML

Ollama

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: DeepSeek Coder 2 16B
    provider: ollama
    model: deepseek-coder-v2:16b

LM Studio

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: DeepSeek Coder 2 16B
    provider: lmstudio
    model: deepseek-coder-v2:16b

Msty

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: DeepSeek Coder 2 16B
    provider: msty
    model: deepseek-coder-v2:16b

Other experiences

There are many more models and providers you can use with Chat beyond those mentioned above. Read more here