LM Studio is an application for Mac, Windows, and Linux that makes it easy to locally run open-source models and comes with a great UI. To get started with LM Studio, download from the website, use the UI to download a model, and then start the local inference server. Continue can then be configured to use the LMStudio LLM class:

config.yaml
models:
  - name: LM Studio
    provider: lmstudio
    model: llama2-7b

​
Embeddings model

LMStudio supports embeddings endpoints, and comes with the nomic-ai/nomic-embed-text-v1.5-GGUF model (as of Nov 2024, check your models)

config.yaml
models:
  - name: Nomic Embed Text
    provider: lmstudio
    model: nomic-ai/nomic-embed-text-v1.5-GGUF
    roles:
      - embed

​
Setting up a remote instance

To configure a remote instance of LM Studio, add the "apiBase" property to your model in config.json:

config.yaml
models:
  - name: Nomic Embed Text
    provider: lmstudio
    model: nomic-ai/nomic-embed-text-v1.5-GGUF
    roles:
      - embed

This apiBase will now be used instead of the default http://localhost:1234/v1.

View the source