Ollamex v0.2.0 released, now supports the /embeddings endpoint

14 January, 2024 • 2 min read

elixir software open source API LLM embeddings

Ollamex v0.2.0 has been released, now with support for the /embeddings API endpoint of ollama version 0.1.20 for generating embeddings using self-hosted LLMs (Large Language Models).

The /chat, /generate and /embeddings endpoints all return different data structures, and the latter in particular simply returns a list of floats without any other information, but it works. I expect that the ollama.ai team will in the future change this, and this will make another update of Ollamex necessary. The original release post explains the timeout problem that shaped the /chat and /generate functions.

There are now two new functions, Ollamex.embeddings/2 and Ollamex.embeddings_with_timeout/3. The former still takes an %Ollamex.PromptRequest{} struct and an %Ollamex.API{} struct as arguments, and the latter also takes an optional timeout value in milliseconds, though this is for consistency with the other functions, since the /embeddings endpoint doesn’t suffer from the same problem of endless \n and \t repetitions.

Example

  iex> p = %Ollamex.PromptRequest{model: "llama2", prompt: "Explain the main features and benefits of the Elixir programming language in a single, concise paragraph."}
  %Ollamex.PromptRequest{
    model: "llama2",
    prompt: "Explain the main features and benefits of the Elixir programming language in a single, concise paragraph.",
    raw: false,
    format: nil,
    stream: true,
    options: nil,
    images: []
  }
  iex> Ollamex.embeddings(p, api)
  %Ollamex.LLMResponse{
    context: nil,
    created_at: nil,
    done: nil,
    eval_count: nil,
    eval_duration: nil,
    model: "llama2",
    prompt_eval_count: nil,
    prompt_eval_duration: nil,
    response: nil,
    total_duration: nil,
    message: nil,
    embedding: [-1.6268974542617798, -1.4279855489730835, -0.46105068922042847,
    0.7557640671730042, -0.17748284339904785, ...],
    errors: nil
  }