Building a Laravel Claude Chatbot: Streaming Responses and Conversation Memory

If you’ve shipped a Laravel Claude streaming chatbot and watched it work perfectly on your machine, then stall in production, you already know that streaming and memory are really two separate engineering problems wearing one UI. A blocking call to the Claude API on a long response can take ten or twenty seconds to resolve. Users don’t wait quietly. They refresh, they assume it’s broken, they leave. Solve the transport problem and you’re left with a second one: Claude has no memory of its own. Every request is stateless. If you want a chatbot that remembers what a user said three turns ago, you have to build that yourself.

This guide builds both pieces as one coherent system: a streaming transport layer using Server-Sent Events, and a database-backed conversation memory layer with three pruning strategies you can choose between depending on the shape of your product. Read the complete Claude API integration guide first if you haven’t wired up basic request/response handling with Anthropic yet, this article assumes that foundation and builds the production layer on top of it. For the broader integration landscape across providers, the Module 2 hub is the place to start.

Laravel now ships a first-party laravel/ai SDK, and it’s worth acknowledging directly: for most integrations, reach for it first. We’re not using it here, deliberately. The SDK abstracts the streaming transport in a way that hides the line-buffer parsing this guide exists to explain. If you want to understand what’s actually happening between Anthropic’s SSE stream and your frontend, and you should, working at the HTTP client level gives you that visibility. Once you understand it, decide for yourself whether the abstraction is worth the trade-off for your team.

Why Streaming Is Not Optional for AI Chat

Without streaming, an eight-second Claude response is eight seconds of blank screen. It doesn’t matter that your servers are healthy and your queries are fast. The user sees nothing and assumes the worst. Streaming pushes tokens as they’re generated, so the first word appears in under a second on most responses, and a mid-stream failure still leaves the user with something rather than nothing.

There’s a secondary benefit most tutorials skip: streaming makes backpressure visible. When tokens stall, you know something’s wrong at the API level immediately. A blocking call just times out with no context at all.

Architecture Overview

The request lifecycle has seven steps, and each component has exactly one job:

1. A validated POST hits the streaming controller.
2. The user’s message is persisted to chat_messages immediately.
3. ConversationMemoryManager retrieves and prunes the conversation history.
4. ClaudeService opens a streaming connection to the Anthropic API.
5. Token deltas are pushed to the client via SSE as they arrive.
6. The complete assistant message, with token counts, is persisted once the stream closes.
7. The frontend consumer appends tokens to the UI in real time.

No orchestration logic leaks into the controller. It doesn’t know what SSE is or how Claude formats a payload. That belongs to the service layer, and keeping it there is what makes this testable.

Database Schema and Models

We use the more complete migration here: a compound index on (conversation_id, id), a token_estimate column so pruning doesn’t need a secondary API call, and a user_id foreign key for auditability.

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        Schema::create('chat_messages', function (Blueprint $table) {
            $table->id();
            $table->string('conversation_id', 64)->index();
            $table->enum('role', ['user', 'assistant', 'system']);
            $table->longText('content');
            $table->unsignedInteger('token_estimate')->nullable();
            $table->foreignId('user_id')->nullable()->constrained()->nullOnDelete();
            $table->timestamps();

            $table->index(['conversation_id', 'id']);
        });
    }

    public function down(): void
    {
        Schema::dropIfExists('chat_messages');
    }
};

Note longText, not text. A text column caps at 65 KB, and a complex Claude response will exceed that. Every history query filters by conversation and orders by insertion sequence, so the compound index isn’t optional, it’s what keeps a fifty-message conversation from doing a table scan.

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Builder;

class ChatMessage extends Model
{
    protected $fillable = [
        'conversation_id',
        'role',
        'content',
        'token_estimate',
        'user_id',
    ];

    protected $casts = [
        'token_estimate' => 'integer',
    ];

    public function scopeForConversation(Builder $query, string $conversationId): Builder
    {
        return $query->where('conversation_id', $conversationId);
    }

    public function scopeExcludingSystem(Builder $query): Builder
    {
        return $query->where('role', '!=', 'system');
    }
}

The Service Layer

ConversationMemoryManager

This class has one job. It retrieves and prunes. It doesn’t call the API and it doesn’t persist anything.

<?php

namespace App\Services\Chat;

use App\Models\ChatMessage;
use Illuminate\Support\Collection;

class ConversationMemoryManager
{
    private const CHARS_PER_TOKEN = 3.5;

    public function __construct(
        private readonly int $maxTokenBudget = 80_000,
        private readonly int $fetchLimit     = 50,
    ) {}

    public function getHistory(string $conversationId): Collection
    {
        $messages = ChatMessage::forConversation($conversationId)
            ->excludingSystem()
            ->orderBy('id')
            ->limit($this->fetchLimit)
            ->get(['role', 'content', 'token_estimate']);

        return $this->pruneToTokenBudget($messages);
    }

    private function pruneToTokenBudget(Collection $messages): Collection
    {
        $budget = $this->maxTokenBudget;
        $kept   = collect();

        foreach ($messages->reverse() as $message) {
            $tokens = $message->token_estimate ?? $this->estimateTokens($message->content);

            if ($budget - $tokens < 0) {
                break;
            }

            $budget -= $tokens;
            $kept->prepend($message);
        }

        return $kept;
    }

    public function estimateTokens(string $text): int
    {
        return (int) ceil(mb_strlen($text) / self::CHARS_PER_TOKEN);
    }
}

Walking backwards and prepending preserves recency correctly. The fetchLimit is a hard cap before token counting even starts, a safety net for conversations that have run very long.

[Efficiency Gain] claude-sonnet-5 supports a 200,000-token context window. The 80,000-token budget above is not a technical limit, it’s a cost decision. Sending 180,000 tokens of history on every turn is roughly double the cost of 80,000, for context the model may not need. Set the budget to what the conversation actually requires.

ClaudeService

This is the merged class. stream() is primary, positioned first, and uses the line-buffer parsing pattern that’s the most genuinely useful thing in this guide. chat() handles the non-streaming path and returns a ChatResult value object.

Here’s the failure that trips up almost every first attempt at this: the Anthropic streaming API returns SSE-formatted text (data: {...}\n\n). If you iterate the response body with a plain foreach, you get arbitrary binary chunks, not lines. A chunk might contain half a line, two full lines, and a partial third. You cannot json_decode a raw chunk. You need a line buffer that accumulates bytes and only processes complete lines, retaining whatever’s incomplete for the next read.

<?php

namespace App\Services\Chat;

use Illuminate\Http\Client\PendingRequest;
use Illuminate\Http\Client\RequestException;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Http;
use RuntimeException;

class ClaudeService
{
    private PendingRequest $client;

    public function __construct()
    {
        $this->client = Http::baseUrl('https://api.anthropic.com/v1')
            ->withHeaders([
                'x-api-key'         => config('services.anthropic.key'),
                'anthropic-version' => config('services.anthropic.version'),
                'Content-Type'      => 'application/json',
            ])
            ->timeout(120)
            ->retry(3, 2000, function (\Exception $e) {
                if ($e instanceof RequestException) {
                    return in_array($e->response->status(), [429, 529], true);
                }
                return false;
            });
    }

    public function stream(array $messages, callable $onChunk, string $systemPrompt = ''): ChatResult
    {
        $payload = [
            'model'      => config('services.anthropic.model'),
            'max_tokens' => 1024,
            'messages'   => $messages,
            'stream'     => true,
        ];

        if ($systemPrompt !== '') {
            $payload['system'] = $systemPrompt;
        }

        $response = $this->client
            ->withOptions(['stream' => true])
            ->post('/messages', $payload);

        if ($response->failed()) {
            throw new RuntimeException('Claude API request failed: ' . $response->status());
        }

        $body        = $response->toPsrResponse()->getBody();
        $buffer      = '';
        $fullText    = '';
        $inputTokens = 0;
        $outputTokens = 0;

        while (!$body->eof()) {
            $buffer .= $body->read(1024);
            $lines   = explode("\n", $buffer);
            $buffer  = array_pop($lines);

            foreach ($lines as $line) {
                $line = trim($line);

                if (!str_starts_with($line, 'data: ')) {
                    continue;
                }

                $decoded = json_decode(substr($line, 6), true);

                if (json_last_error() !== JSON_ERROR_NONE) {
                    continue;
                }

                match ($decoded['type'] ?? null) {
                    'message_start' => $inputTokens = $decoded['message']['usage']['input_tokens'] ?? 0,
                    'content_block_delta' => (function () use ($decoded, $onChunk, &$fullText) {
                        $text = $decoded['delta']['text'] ?? '';
                        $fullText .= $text;
                        $onChunk($text);
                    })(),
                    'message_delta' => $outputTokens = $decoded['usage']['output_tokens'] ?? $outputTokens,
                    'error' => throw new RuntimeException(
                        'Claude stream error: ' . ($decoded['error']['message'] ?? 'unknown')
                    ),
                    default => null,
                };
            }
        }

        return new ChatResult($fullText, $inputTokens, $outputTokens);
    }

    public function chat(Collection $history, string $systemPrompt = ''): ChatResult
    {
        $payload = [
            'model'      => config('services.anthropic.model'),
            'max_tokens' => 1024,
            'messages'   => $this->formatMessages($history),
        ];

        if ($systemPrompt !== '') {
            $payload['system'] = $systemPrompt;
        }

        $response = $this->client->post('/messages', $payload);

        if ($response->failed()) {
            $error = $response->json('error.message', 'Unknown API error');
            throw new RuntimeException("Claude API error (HTTP {$response->status()}): {$error}");
        }

        $json = $response->json();

        return new ChatResult(
            text:         $json['content'][0]['text'] ?? '',
            inputTokens:  $json['usage']['input_tokens'] ?? 0,
            outputTokens: $json['usage']['output_tokens'] ?? 0,
        );
    }

    public function formatMessages(Collection $history): array
    {
        return $history
            ->filter(fn ($m) => in_array($m->role, ['user', 'assistant'], true))
            ->map(fn ($m) => [
                'role'    => $m->role,
                'content' => [['type' => 'text', 'text' => $m->content]],
            ])
            ->values()
            ->toArray();
    }
}

<?php

namespace App\Services\Chat;

readonly class ChatResult
{
    public function __construct(
        public string $text,
        public int    $inputTokens,
        public int    $outputTokens,
    ) {}

    public function totalTokens(): int
    {
        return $this->inputTokens + $this->outputTokens;
    }
}

Two things worth being precise about. First, the retry() middleware operates on the initial connection attempt, not on failures mid-stream, a 429 that arrives after you’ve already started reading chunks is not something retry() will catch, it surfaces as a type: error event inside the stream instead, which is why the error match arm above throws explicitly. Second, capturing input_tokens from message_start and output_tokens from message_delta means your streaming path logs real usage data, not an estimate. Skipping this is a common gap: teams ship streaming, then wonder six weeks later why their token dashboards only reflect non-streaming traffic.

[Production Pitfall] timeout(120) is not optional. P95 Claude latency on complex prompts routinely exceeds 60 seconds under load. Without an explicit timeout, Guzzle falls back to PHP’s default socket timeout, which terminates a long response mid-stream without throwing an exception your catch block will see. Monitor your actual P95 before tuning this down.

If you’re extending this with per-user token budgets and tiered rate limiting at the middleware layer, which you should for anything multi-tenant, the Laravel AI Middleware: Token Tracking & Rate Limiting guide covers that pattern and slots cleanly on top of this service.

Registering Services

<?php

namespace App\Providers;

use App\Services\Chat\ClaudeService;
use App\Services\Chat\ConversationMemoryManager;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function register(): void
    {
        $this->app->singleton(ClaudeService::class);

        $this->app->singleton(ConversationMemoryManager::class, fn () => new ConversationMemoryManager(
            maxTokenBudget: 80_000,
            fetchLimit: 50,
        ));
    }
}

Both are singletons because neither carries per-request state. There’s no reason to build a new HTTP client on every call.

The SSE Transport Layer

<?php

namespace App\Http\Controllers;

use App\Models\ChatMessage;
use App\Services\Chat\ClaudeService;
use App\Services\Chat\ConversationMemoryManager;
use Illuminate\Http\Request;

class ChatStreamController extends Controller
{
    public function __construct(
        private readonly ClaudeService $claude,
        private readonly ConversationMemoryManager $memory,
    ) {}

    public function stream(Request $request)
    {
        $validated = $request->validate([
            'conversation_id' => ['required', 'uuid'],
            'message'         => ['required', 'string', 'max:4000'],
        ]);

        $conversationId = $validated['conversation_id'];

        ChatMessage::create([
            'conversation_id' => $conversationId,
            'role'            => 'user',
            'content'         => $validated['message'],
            'token_estimate'  => $this->memory->estimateTokens($validated['message']),
            'user_id'         => auth()->id(),
        ]);

        $history        = $this->memory->getHistory($conversationId);
        $claudeMessages = $this->claude->formatMessages($history);

        return response()->stream(function () use ($claudeMessages, $conversationId) {
            try {
                $result = $this->claude->stream($claudeMessages, function (string $chunk) {
                    echo 'data: ' . json_encode(['text' => $chunk]) . "\n\n";
                    ob_flush();
                    flush();
                });
            } catch (\RuntimeException $e) {
                report($e);
                echo "event: error\n";
                echo 'data: ' . json_encode(['message' => 'Stream interrupted. Please retry.']) . "\n\n";
                ob_flush();
                flush();
                return;
            }

            if ($result->text !== '') {
                ChatMessage::create([
                    'conversation_id' => $conversationId,
                    'role'            => 'assistant',
                    'content'         => $result->text,
                    'token_estimate'  => $result->outputTokens,
                ]);
            }

            echo "event: end\ndata: done\n\n";
            ob_flush();
            flush();
        }, 200, [
            'Content-Type'      => 'text/event-stream',
            'Cache-Control'     => 'no-cache',
            'Connection'        => 'keep-alive',
            'X-Accel-Buffering' => 'no',
        ]);
    }
}

[Production Pitfall] X-Accel-Buffering: no. If you’re behind Nginx in production, and you probably are, Nginx buffers proxy responses by default. Without this header, tokens accumulate in Nginx’s buffer and the client receives them in large delayed batches, which defeats the entire point of streaming. This is the single most common reason streaming works locally and fails in production. If tokens still arrive in bursts after adding this header, check output_buffering in php.ini next.

// routes/api.php

use App\Http\Controllers\ChatStreamController;
use Illuminate\Support\Facades\Route;

Route::middleware(['auth:sanctum', 'throttle:60,1'])->group(function () {
    Route::post('/chat/stream', [ChatStreamController::class, 'stream']);
});

On the frontend, we use fetch() with a ReadableStream reader rather than EventSource. EventSource only supports GET requests, so it can’t carry a CSRF token or a JSON body. For anything past a throwaway prototype, that rules it out.

async function sendMessage(conversationId, message) {
    const response = await fetch('/chat/stream', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'X-CSRF-TOKEN': document.querySelector('meta[name="csrf-token"]').content,
        },
        body: JSON.stringify({ conversation_id: conversationId, message }),
    });

    const reader  = response.body.getReader();
    const decoder = new TextDecoder();
    const output  = document.querySelector('#output');
    let buffer    = '';

    while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        buffer += decoder.decode(value, { stream: true });
        const lines = buffer.split('\n');
        buffer = lines.pop();

        for (const line of lines) {
            if (line.startsWith('data: ')) {
                const json = line.slice(6).trim();
                if (json === 'done') continue;
                try {
                    const parsed = JSON.parse(json);
                    if (parsed.text) output.textContent += parsed.text;
                    if (parsed.message) console.error('Stream error:', parsed.message);
                } catch (_) {}
            }
            if (line.startsWith('event: end')) {
                reader.cancel();
                return;
            }
        }
    }
}

Conversation Memory: Three Strategies

The token-aware pruning built into ConversationMemoryManager is the right default, but it helps to understand what it’s a trade-off against.

Strategy 1: Fixed message window. Keep the last N messages, discard the rest.

public function getHistoryWindowed(string $conversationId, int $limit = 20): Collection
{
    return ChatMessage::forConversation($conversationId)
        ->excludingSystem()
        ->orderBy('id', 'desc')
        ->limit($limit)
        ->get()
        ->reverse()
        ->values();
}

Predictable, and cheap. The risk is abrupt context loss: a preference the user set 25 messages ago vanishes at message 26 with no warning. Fine for transactional bots. Not fine for a long-lived assistant relationship.

Strategy 2: Token-aware pruning. Walk backwards through history, accumulate token counts, stop at budget. Always preserves the most recent context and adapts to conversation density automatically, a thread of one-liners keeps far more turns than one full of dense technical explanations.

Strategy 3: Summarisation with a queued job. When a conversation exceeds budget, summarise the older portion instead of discarding it, and store the summary as a compact system message.

<?php

namespace App\Jobs;

use App\Models\ChatMessage;
use App\Services\Chat\ClaudeService;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Collection;

class SummariseConversationHistory implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(
        private readonly string $conversationId,
        private readonly Collection $messagesToSummarise,
    ) {}

    public function handle(ClaudeService $claude): void
    {
        $historyText = $this->messagesToSummarise
            ->map(fn ($m) => strtoupper($m->role) . ': ' . $m->content)
            ->implode("\n");

        $result = $claude->chat(collect([
            (object) [
                'role'    => 'user',
                'content' => 'Summarise the following conversation concisely. '
                    . 'Focus on facts, decisions, and user preferences established. '
                    . "Omit pleasantries.\n\n" . $historyText,
            ],
        ]));

        ChatMessage::create([
            'conversation_id' => $this->conversationId,
            'role'            => 'system',
            'content'         => '[CONVERSATION SUMMARY]: ' . $result->text,
            'token_estimate'  => (int) ceil(mb_strlen($result->text) / 3.5),
        ]);

        ChatMessage::forConversation($this->conversationId)
            ->whereIn('id', $this->messagesToSummarise->pluck('id'))
            ->delete();
    }
}

[Architect’s Note] Summarisation should run as a queued job, not synchronously inside the request lifecycle. Dispatch it when a conversation’s estimated token count crosses a threshold. The current request uses existing history; the next request benefits from the compacted summary, with no latency added to the user waiting on a response. Running this on Laravel Horizon gives you the visibility to confirm these jobs aren’t backing up under load. For the broader contract and governance layer this pattern sits underneath, see Production-Grade AI Architecture in Laravel.

One structural note: storing summaries as system-role rows means your excludingSystem() scope filters them out of retrieval by default. Either add a dedicated summary role to the enum and include it explicitly, or store summaries in a separate conversation_summaries table and prepend them before sending to Claude. The separate table scales cleaner and keeps your message schema free of role-type ambiguity.

Redis Caching for Hot Conversations

For high-traffic applications, hitting the database on every message to load history adds latency you don’t need to pay.

<?php

namespace App\Services\Chat;

use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Cache;

class CachedConversationMemoryManager extends ConversationMemoryManager
{
    private const CACHE_TTL_SECONDS = 300;

    public function getHistory(string $conversationId): Collection
    {
        return Cache::remember(
            key: "chat_history:{$conversationId}",
            ttl: self::CACHE_TTL_SECONDS,
            callback: fn () => parent::getHistory($conversationId),
        );
    }

    public function invalidate(string $conversationId): void
    {
        Cache::forget("chat_history:{$conversationId}");
    }
}

Call invalidate() after every new message is persisted. The next load hits the database once and serves from cache for the rest of that active session.

[Edge Case Alert] Cache the output of pruneToTokenBudget(), not the raw unpruned history. If you cache the full history and your token budget changes between requests, you’ll serve stale, oversized histories straight from cache with no pruning applied. Cache the result, not the input.

Failure Modes Reference Table

Failure	Root Cause	Fix
Tokens arrive in bursts, not individually	Nginx buffering	Add X-Accel-Buffering: no
Stream works locally, fails silently in production	PHP output buffering enabled	Confirm ob_flush() + flush(); check output_buffering in php.ini
JSON decode fails on chunks	Raw chunk iteration without a line buffer	Use the read() + eof() loop from ClaudeService::stream()
429 errors under load	No retry logic, or retry applied mid-stream where it can’t help	Use Http::retry() scoped to 429/529 on the initial connection; handle mid-stream errors via the error SSE event
Lost assistant messages	Persisting before the stream closes	Always persist inside the stream callback, after the loop completes
CSRF token rejection	Using EventSource (GET only)	Switch to fetch() with a ReadableStream reader
Streaming token counts missing from dashboards	Not capturing message_start / message_delta usage events	Extract input_tokens and output_tokens inside the stream loop, as shown above

Testing

You can’t assert on Claude’s exact output, it’s probabilistic. What you test is the architecture around it.

<?php

namespace Tests\Unit;

use App\Services\Chat\ConversationMemoryManager;
use PHPUnit\Framework\TestCase;

class ConversationMemoryManagerTest extends TestCase
{
    public function test_prunes_to_token_budget(): void
    {
        $manager = new ConversationMemoryManager(maxTokenBudget: 100, fetchLimit: 50);

        $messages = collect(range(1, 20))->map(fn ($i) => (object) [
            'role'           => $i % 2 === 0 ? 'assistant' : 'user',
            'content'        => str_repeat('a', 35),
            'token_estimate' => 10,
        ]);

        $history = $manager->getHistory('fake-id');

        $this->assertLessThanOrEqual(10, $history->count());
    }
}

<?php

namespace Tests\Feature;

use Illuminate\Support\Facades\Http;
use Tests\TestCase;

class ChatControllerTest extends TestCase
{
    public function test_returns_conversation_id_and_reply(): void
    {
        Http::fake([
            'https://api.anthropic.com/*' => Http::response([
                'content' => [['type' => 'text', 'text' => 'Hello! How can I help?']],
                'usage'   => ['input_tokens' => 42, 'output_tokens' => 12],
            ], 200),
        ]);

        $response = $this->postJson('/api/chat', ['message' => 'Hello there.']);

        $response->assertStatus(200)
            ->assertJsonStructure(['conversation_id', 'reply']);
    }
}

For the streaming path, swap ClaudeService for an anonymous class rather than faking HTTP, it’s faster to write and doesn’t require simulating SSE framing:

$this->instance(ClaudeService::class, new class {
    public function stream(array $messages, callable $onChunk, string $systemPrompt = ''): ChatResult
    {
        $onChunk('Hello ');
        $onChunk('from fake Claude.');
        return new ChatResult('Hello from fake Claude.', 20, 5);
    }

    public function formatMessages($messages): array
    {
        return $messages->toArray();
    }
});

This is exactly why the service layer exists. A controller calling Http::post() directly is untestable without HTTP faking. A controller depending on ClaudeService is testable with a five-line anonymous class.

When Streaming Is the Wrong Choice

Streaming is a UX decision for interactive, human-facing responses. It isn’t a default you apply everywhere.

Don’t stream batch jobs. If you’re processing a thousand documents overnight, nobody’s watching a cursor blink. Run those as queued jobs with a plain chat() call and move on, it’s simpler and there’s no open-connection complexity to manage at scale.

Don’t stream very short responses. If your average reply is two sentences, the time-to-first-token advantage is negligible, and you’ve added frontend and backend complexity for nothing.

Don’t stream if you need to validate the full response before displaying anything. Streaming means showing text before you’ve seen all of it. If downstream logic needs to parse a JSON tool call result first, buffer internally and display as a unit once it’s complete.

Apply streaming where it demonstrably improves the experience. Default to simpler async patterns everywhere else.

---

Frequently Asked Questions