Why Rails Is a Great Choice for Building AI Agent Backends

Why Rails Is a Great Choice for Building AI Agent Backends

While most AI Agent tutorials default to Python + FastAPI, Ruby on Rails offers several unique advantages—mature ActiveJob, powerful ActionCable WebSocket support, and a rich ecosystem of authentication, authorization, and ORM infrastructure.

By combining Rails with Python’s LangChain​ (via a sidecar process or the langchainrbgem) or directly calling Anthropic / OpenAI APIs, Rails can fully serve as the orchestration and service layer of an Agent system.

We adopt the ReAct (Reason + Act)​ pattern for the Agent’s main loop. Rails handles request ingestion and state management, while LangChain manages LLM inference and tool routing.

Design Principle: Every Think → Act → Observecycle is recorded as an AgentStep. This makes debugging, auditing, and resuming from breakpoints extremely straightforward.

Environment Setup & Dependencies 

Core Gems

# Gemfile

# AI & LangChain
gem 'langchainrb', '~> 0.19'
gem 'anthropic'           # Claude SDK
gem 'ruby-openai'         # OpenAI fallback

# Vector storage
gem 'pgvector'            # PostgreSQL vector extension
gem 'neighbor'            # Rails vector similarity search

# Background jobs
gem 'sidekiq', '~> 7.0'

# Real-time communication
gem 'redis'
gem 'actioncable'

Database Migrations

# db/migrate/create_agent_sessions.rb

class CreateAgentSessions < ActiveRecord::Migration[7.1]
  def change
    create_table :agent_sessions do |t|
      t.references :user, null: false, foreign_key: true
      t.string     :status, default: 'pending'
      t.string     :goal, null: false
      t.jsonb      :context, default: {}
      t.timestamps
    end

    create_table :agent_steps do |t|
      t.references :agent_session, null: false, foreign_key: true
      t.integer    :step_index
      t.string     :thought      # LLM reasoning
      t.string     :action       # Tool name
      t.jsonb      :action_input
      t.text       :observation  # Tool output
      t.timestamps
    end

    add_column :agent_sessions, :messages, :jsonb, default: []
    enable_extension 'vector'
    add_column :agent_sessions, :embedding, :vector, limit: 1536
  end
end

Agent Core Executor

The heart of the Agent is an AgentExecutorservice object implementing the ReAct loop:

LLM → Parse Intent → Execute Tool → Feed Result Back → Repeat

# app/services/agent_executor.rb

class AgentExecutor
  MAX_ITERATIONS = 10

  def initialize(session:, tools:, llm: nil)
    @session = session
    @tools   = tools.index_by(&:name)
    @llm     = llm || default_llm
    @memory  = ConversationMemory.new(session)
  end

  def run(goal)
    @session.update!(status: 'running', goal: goal)
    messages = @memory.load

    MAX_ITERATIONS.times do |i|
      response = @llm.chat(
        messages: messages + [{ role: 'user', content: goal }],
        tools: tool_definitions
      )

      case response.stop_reason
      when 'end_turn'
        return finalize(response.content)

      when 'tool_use'
        tool_use = response.content.find { |b| b.type == 'tool_use' }
        result   = execute_tool(tool_use, step_index: i)

        messages << { role: 'assistant', content: response.content }
        messages << {
          role: 'user',
          content: [{
            type: 'tool_result',
            tool_use_id: tool_use.id,
            content: result
          }]
        }

        broadcast_step(step_index: i, tool: tool_use.name, result: result)
      end
    end

    finalize("Max iterations reached")
  end

  private

  def execute_tool(tool_use, step_index:)
    tool = @tools[tool_use.name]
    result = tool.call(tool_use.input)

    @session.agent_steps.create!(
      step_index: step_index,
      action: tool_use.name,
      action_input: tool_use.input,
      observation: result
    )
    result
  end

  def broadcast_step(**payload)
    AgentChannel.broadcast_to(@session, payload)
  end

  def finalize(answer)
    @session.update!(status: 'completed')
    @memory.save
    answer
  end

  def default_llm
    Langchain::LLM::Anthropic.new(
      api_key: ENV['ANTHROPIC_API_KEY'],
      default_options: { model: 'claude-opus-4-5' }
    )
  end
end

Running Agents as Background Jobs

# app/jobs/agent_job.rb

class AgentJob < ApplicationJob
  queue_as :agents
  sidekiq_options retry: 2, dead: false

  def perform(session_id)
    session  = AgentSession.find(session_id)
    tools    = [WebSearchTool.new, CodeRunnerTool.new, DatabaseTool.new]
    executor = AgentExecutor.new(session: session, tools: tools)
    executor.run(session.goal)
  rescue => e
    session&.update!(status: 'failed', context: { error: e.message })
    raise
  end
end

Extensible Tool System

Each tool declares a name, description​ (critical for LLM understanding), and input schema.

# app/tools/base_tool.rb

module Tools
  class BaseTool
    attr_reader :name, :description

    def call(input)
      raise NotImplementedError
    end

    def to_claude_definition
      {
        name: name,
        description: description,
        input_schema: input_schema
      }
    end
  end
end

# app/tools/web_search_tool.rb

class WebSearchTool < Tools::BaseTool
  def name        = 'web_search'
  def description = 'Search the web in real time using Tavily. Input a query and receive relevant summaries.'

  def input_schema
    {
      type: 'object',
      properties: {
        query: { type: 'string', description: 'Search query' }
      },
      required: ['query']
    }
  end

  def call(input)
    client  = Tavily::Client.new(api_key: ENV['TAVILY_API_KEY'])
    results = client.search(query: input['query'], max_results: 5)
    results.map { |r| "**#{r[:title]}**: #{r[:content]}" }.join("\n\n")
  rescue => e
    "Search failed: #{e.message}"
  end
end

Agent Memory & Session State Management 

Memory is layered into three levels:

# app/services/conversation_memory.rb

class ConversationMemory
  def initialize(session)
    @session = session
  end

  def load
    messages = @session.messages || []
    return messages if within_token_limit?(messages)
    summarize_and_trim(messages)
  end

  def save
    @session.update!(messages: @session.reload.messages)
    embed_and_store
  end

  private

  def within_token_limit?(messages)
    messages.sum { |m| m[:content].to_s.length } < 80_000
  end

  def summarize_and_trim(messages)
    recent   = messages.last(10)
    old_text = messages[0..-11].map { |m| m[:content] }.join("\n")
    summary  = LLM.summarize(old_text)
    [{ role: 'system', content: "History summary: #{summary}" }] + recent
  end
end

ActionCable: Streaming Agent Steps in Real Time

# app/channels/agent_channel.rb

class AgentChannel < ApplicationCable::Channel
  def subscribed
    session = AgentSession.find_by(id: params[:session_id])
    if session&.user == current_user
      stream_for session
    else
      reject
    end
  end
end

From the executor:

AgentChannel.broadcast_to(session, {
  step: 3,
  tool: 'web_search',
  status: 'running',
  result: '...'
})

Frontend clients subscribe using @rails/actioncableor StimulusJS, rendering each step in real time so users can see the Agent “thinking” and “acting.”

Production Deployment Considerations

# config/sidekiq.yml

:concurrency: 10
:queues:
  - [agents, 3]
  - [default, 2]
  - [mailers, 1]

:limits:
  agents: 5

Additional production concerns:

Rails + LangChain​ provides a mature, scalable foundation for AI Agent systems. From ActiveJob-based async orchestration​ to ActionCable-powered real-time streaming, every part of the Rails ecosystem aligns naturally with Agent architecture.

Core Design Principles 

These three pillars form the foundation of a maintainable, extensible Agent backend.