The model you use for for Chat mode will be

  • used with Edit mode by default but can be switched
  • always used with Agent mode if the model supports tool calling

Our strong recommendation is to use Claude Sonnet 4 from Anthropic.

Its strong tool calling and reasoning capabilities make it the best model for Agent mode.

  1. Get your API key from Anthropic
  2. Add Claude Sonnet 4 to your assistant on Continue Hub
  3. Add ANTHROPIC_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

​
Other hosted models

These models have varying tool calling and reasoning capabilities.

Gemini 2.5 Pro from Google

  1. Get your API key from Google AI Studio
  2. Add Gemini 2.5 Pro to your assistant on Continue Hub
  3. Add GEMINI_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

o3 from OpenAI

  1. Get your API key from OpenAI
  2. Add o3 to your assistant on Continue Hub
  3. Add OPENAI_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

Grok 4 from xAI

  1. Get your API key from xAI
  2. Add Grok 4 to your assistant on Continue Hub
  3. Add XAI_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

Devstral Medium from Mistral AI

  1. Get your API key from Mistral AI
  2. Add Devstral Medium to your assistant on Continue Hub
  3. Add MISTRAL_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

Kimi K2 from Moonshot AI

  1. Get your API key from TogetherAI
  2. Add Kimi K2 to your assistant on Continue Hub
  3. Add TOGETHER_API_KEY as a User Secret on Continue Hub here
  4. Click Reload config in the assistant selector in the Continue IDE extension

​
Local models

These models can be run on your computer if you have enough VRAM.

Their limited tool calling and reasoning capabilities will make it challenging to use Agent mode.

Qwen2.5-Coder 7B from Qwen

  1. Add Qwen2.5-Coder 7B to your assistant on Continue Hub
  2. Run the model with Ollama
  3. Click Reload config in the assistant selector in the Continue IDE extension

Gemma 3 4B from Google

  1. Add Gemma 3 4B to your assistant on Continue Hub
  2. Run the model with Ollama
  3. Click Reload config in the assistant selector in the Continue IDE extension

Llama 3.1 8B from Meta

  1. Add Llama 3.1 8B to your assistant on Continue Hub
  2. Run the model with Ollama
  3. Click Reload config in the assistant selector in the Continue IDE extension

For more detailed setup instructions, see here