Ruby Llm

The Ruby AI framework

Build AI features the Ruby way: models, tools, agents, evaluations, and Rails conventions, designed together, with one consistent API.

chat = RubyLLM.chat
chat.ask "What's the best way to learn Ruby?"
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.with_model("claude-sonnet-5").ask "Now say it as a haiku"

RubyLLM.paint("a sunset over mountains in watercolor").save("sunset.png")

https://github.com/user-attachments/assets/519fcb20-ab8c-4af3-9185-0953b3ba7b81

[!NOTE] Using RubyLLM? Share your story! Takes 5 minutes.

#23 providers. One API.

Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint.

Browse models and pricing · Connect a provider

#A complete AI framework for Ruby

More than a client for model APIs. RubyLLM gives your application the conventions, the runtime, and every piece in between, designed to work together. Everything below ships in RubyLLM 2.1.

#Conversations

• Chat: Conversational AI with RubyLLM.chat
• Streaming: Real-time responses with blocks
• Attachments: Ask about images, videos, audio, PDFs, and other documents with with:
• Structured output: Define a Ruby schema and read the result with response.parsed
• Extended thinking: Control, view, and persist model deliberation
• Citations: Normalized source citations from documents, search, and grounding
• Prompt caching: Turn on the provider's cache with with_caching and cache_until_here
• Compaction: Let providers condense long conversations with with_compaction
• Token counting: Count a request before you send it with count_tokens
• Fallbacks and cancellation: Retry on backup models with with_fallbacks, stop a run with cancel

#Tools and agents

• Tools: Let AI call your Ruby methods
• Tool approval: Park a run until a human approves with requires_approval
• Provider tools: Web search, code execution, and MCP connectors with with_provider_tools
• MCP: Connect agents to your systems with server classes in app/mcp, per-user inputs, tool filtering, and OAuth with RubyLLM::MCP
• Agents: Reusable assistants with RubyLLM::Agent
• The agentic loop: Drive it yourself with ask_later, step, and complete?
• Prompt templates: ERB prompts in app/prompts, rendered with RubyLLM.render_prompt
• Workflows: Correlate multi-agent runs in your telemetry with RubyLLM.workflow
• Evaluations: Reusable datasets, model criteria, and Ruby assertions with RubyLLM::Evaluation
• Judgments: Probabilities, choices, and scores with RubyLLM::Judge and RubyLLM.judge

#Individual operations

• Image generation: Create images with RubyLLM.paint
• Video generation: Create videos with RubyLLM.animate
• Audio: Transcribe speech with RubyLLM.transcribe and generate it with RubyLLM.speak
• OCR: Turn documents into markdown with RubyLLM.ocr
• Embeddings: Generate embeddings with RubyLLM.embed
• Reranking: Order retrieval candidates by relevance with RubyLLM.rerank
• Moderation: Content flags, categories, and scores with RubyLLM.moderate
• Files: Upload once and reuse across chats with RubyLLM.upload
• Batches: Provider-side batch processing with provider-specific discounts via RubyLLM.batch

#Rails and production

• Rails: Active Record persistence, Active Storage attachments, Hotwire streaming, and generators
• Cost tracking: A per-attempt usage ledger behind chat.tokens and chat.cost
• Observability: Instrumentation events and OpenTelemetry tracing
• Async: Fiber-based concurrency
• Model registry: Browse capabilities, limits, and pricing across providers
• Providers: OpenAI, Azure, xAI, Anthropic, Gemini, VertexAI, Bedrock, Cohere, DeepSeek, Hetzner, Mistral, Ollama, Ollama Cloud, OpenRouter, Perplexity, GPUStack, ElevenLabs, Deepgram, TypeSafe, and any OpenAI-compatible API

#Coming in 2.2

These are on main and documented under rubyllm.com/next:

• Code execution: Let the model run Ruby, Python, JavaScript, and shell in a sandbox on your servers with with_code_execution
• Skills: Teach the model your workflows with folders of instructions and scripts in the Agent Skills format with with_skills
• File stores: Keep attachments and long tool results out of the conversation with with_file_store
• Message queueing: Add messages while a chat runs with queue
• More providers: Meta, MiniMax, Moonshot AI, and Z.ai

#Evaluate your agents

Keep questions and expected answers in a dataset. RubyLLM checks correctness by default. Run the same evaluation from Ruby, RSpec, Minitest, or Rake.

app/agents/support_agent.rb

class SupportAgent < RubyLLM::Agent
  instructions "Accept returns of unopened items within 30 days."
end

app/evals/support_evaluation.rb

class SupportEvaluation < RubyLLM::Evaluation
  def perform(input)
    SupportAgent.new.ask(input)
  end
end

app/evals/support_evaluation.yml

cases:
  - name: within_return_window
    inputs: Can I return an unopened item after 14 days?
    expected_output: Yes, unopened items can be returned within 30 days.
  - name: past_return_window
    inputs: Can I return an unopened item after 45 days?
    expected_output: No, the 30-day return window has passed.

Run the evaluation:

report = SupportEvaluation.run
report.passed?
report.cost.total

The model evaluator checks meaning, so answers can use different wording. Declare your own criteria, add Ruby assertions, or choose a configured evaluator agent or judgment model. Read the evaluation guides.

#Feels at home in Rails

Save conversations with Active Record and stream replies with Hotwire. The generators give you a working chat UI. Watch the two-minute demo.

https://github.com/user-attachments/assets/65422091-9338-47da-a303-92b918bd1345

# Install Rails Integration
bin/rails generate ruby_llm:install
bin/rails db:migrate
bin/rails ruby_llm:load_models

# Add Chat UI (optional)
bin/rails generate ruby_llm:chat_ui

class Chat < ApplicationRecord
  acts_as_chat
end

chat = Chat.create! model: "gpt-6-luna"
chat.ask "What's in this file?", with: "report.pdf"

Start your Rails server and visit http://localhost:3000/chats to try the chat interface. See the Rails guide for persistence, streaming, and background jobs.

#Start with one line. Add files, tools, and agents

These examples use 2.1.1. Follow Getting Started to install it and configure the providers you want to try. For 1.x, use the 1.x docs.

# Just ask
RubyLLM.chat.ask "What's the best way to learn Ruby?"

# Ask about files with a model that supports their input types
chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "What's happening in this video?", with: "video.mp4"
chat.ask "Describe this meeting", with: "meeting.wav"
chat.ask "Summarize this document", with: "contract.pdf"
chat.ask "Explain this code", with: "app.rb"

# Multiple files at once
chat.ask "Analyze these files", with: ["diagram.png", "report.pdf", "notes.txt"]

# Stream responses
chat.ask "Tell me a story about Ruby" do |chunk|
  print chunk.content
end

# Generate images
image = RubyLLM.paint "a sunset over mountains in watercolor style"
image.save "sunset.png"

# Generate videos
video = RubyLLM.animate "a paper boat sailing down a rainy gutter"
video.save "paper_boat.mp4"

# Create embeddings
embedding = RubyLLM.embed "Ruby is elegant and expressive"
embedding.vector

# Rank search results
documents = ["Reset your password in Settings.", "Invoices arrive by email."]
ranked = RubyLLM.rerank("How do I reset my password?", documents, model: "rerank-v3.5")
ranked.results.first.document

# Transcribe audio to text
transcript = RubyLLM.transcribe "meeting.wav"
puts transcript.text

# Turn text into speech
speech = RubyLLM.speak "Hello, welcome to RubyLLM!"
speech.save "welcome.mp3"

# Extract document text as markdown
document = RubyLLM.ocr "contract.pdf"
puts document.markdown

# Check whether a moderation model flags content
RubyLLM.moderate("Some user-generated content").flagged?

# Ask a typed question about your data
class Urgency < RubyLLM::Judge
  probability :urgent, "Does this need attention today?"
end

Urgency.judge("Please refund my duplicate charge today.").urgent.probability

# Let AI use your code
class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    JSON.parse(Faraday.get(url).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"

# Define an agent with instructions + tools
class WeatherAssistant < RubyLLM::Agent
  model "gpt-6-luna"
  instructions "Be concise and always use tools for weather."
  tools Weather
end

WeatherAssistant.new.ask "What's the weather in Berlin?"

# Get structured output
class ProductSchema < Schematist::Schema
  string :name
  number :price
  array :features do
    string
  end
end

response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "product.txt"
response.parsed

#Installation

Install RubyLLM 2.1:

bundle add ruby_llm --version 2.1.1

Configure a provider in your script, or in config/initializers/ruby_llm.rb in Rails:

require 'ruby_llm'

RubyLLM.configure do |config|
  config.openai_api_key = ENV.fetch('OPENAI_API_KEY')
end

Configure the other providers used by the examples as needed: Gemini for files, xAI for video, Mistral for OCR, and Cohere for reranking. Getting Started shows each setup beside its example. Upgrading an existing app? Follow the upgrade guide.

#AI coding assistants

Give your coding assistant the RubyLLM API and documentation that match your application. From your application directory, install the skill packaged with your gem:

gem exec skill add "$(bundle show ruby_llm)" --skill rubyllm --agents coding

--agents coding installs it for the coding assistants your project uses, not into your app's app/skills. In Claude Code, you can also install it as a plugin:

/plugin marketplace add crmne/ruby_llm
/plugin install rubyllm@ruby_llm

See AI Coding Assistants for setup and updates.

#Documentation

Guides · API reference · Models · Upgrading · 1.x docs

#Contributing

See CONTRIBUTING.md.

#License

Released under the MIT License.