Check if your chosen model is still supported by referring to the model documentation. If a model has been deprecated, you may encounter a 404 error when attempting to use it.

Groq provides the fastest available inference for open-source language models, including the entire Llama 3.1 family.

  1. Obtain an API key here
  2. Update your Continue config file like this:
config.yaml
models:
  - name: Llama 3.3 70b Versatile
    provider: groq
    model: llama-3.3-70b-versatile
    apiKey: <YOUR_GROQ_API_KEY>
    roles:
      - chat