README

One expressive, testable interface for Claude, OpenAI and Gemini in Laravel. Streaming, tool calling, Blade-based prompt templates, response caching, retries with cross-provider fallback, and a dollar cost attached to every call.

use Touseef\LaravelLlm\Facades\Llm;

$answer = Llm::ask('Explain prompt caching in one sentence.');

That call streams through the default driver, applies your retry policy, prices the response, records it for the request, and fires an event you can log against — none of which you had to wire up.

Why this exists

Laravel has a first-class HTTP client, queue, cache and mailer, but reaching an LLM still means gluing a raw provider SDK to your app and re-solving the same four problems in every project: swapping providers, retrying transient failures, keeping prompts readable, and knowing what a request actually cost. The PHP/Laravel ecosystem has far fewer of these than Python does.

Llm is a thin, opinionated layer over the anthropic-ai/sdk, openai-php/client and google-gemini-php/client packages. The provider SDKs do the transport; this package owns the parts that are the same whichever model you talk to.

• One interface, three providers — write against Llm, switch LLM_DRIVER between anthropic, openai and gemini, or pin a driver per call with ->using().
• Retries and fallback — transient failures (429, 5xx, connection drops) retry with exponential backoff; deterministic failures (a 400, a bad key) fail fast. Exhaust one provider and it falls through to the next.
• Prompts as Blade views — loops, conditionals, partials, escaping and version control, instead of string concatenation.
• Response caching — content-addressed on everything that can change the answer, so a prompt edit misses cleanly.
• Cost tracking — every response carries costUsd; an optional migration logs one row per call; a middleware stamps the spend onto response headers.
• Testable by design — Llm::fake() swaps in an in-memory driver with assertSent helpers; no HTTP, no keys, no flake.

Requirements

The provider SDKs are suggested, not required — install only the driver you use. Reaching for a driver whose SDK is missing throws a clear MissingDependencyException telling you what to install.

Installation

composer require touseef/laravel-llm

The service provider auto-registers. Publish the config if you want to edit it:

php artisan vendor:publish --tag=llm-config

Set your keys in .env:

LLM_DRIVER=anthropic
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
GEMINI_API_KEY=AIza...

Architecture

flowchart TD
    A["Llm facade"] --> B["LlmManager<br/>resolves drivers, holds Fake"]
    B --> C["PendingRequest<br/>fluent builder"]
    C -->|cache hit| D["ResponseCache"]
    C -->|per attempt| E["retry + fallback loop"]
    E --> F["Driver contract"]
    F --> G["AnthropicDriver"]
    F --> H["OpenAiDriver"]
    F --> N["GeminiDriver"]
    F --> I["FakeDriver (tests)"]
    C --> J["CostCalculator → costUsd"]
    C --> K["UsageRecorder + events"]
    K --> L["RecordUsageLog → llm_usage_logs"]
    K --> M["TrackLlmCost middleware → X-Llm-* headers"]

PendingRequest owns everything provider-independent — caching, retries, fallback, cost, events — so each Driver only has to translate one normalised request into its own wire format and back. Adding a provider is one class implementing five methods.

Usage

A single call

use Touseef\LaravelLlm\Facades\Llm;

$response = Llm::chat('Summarise this ticket in one line.')
    ->system('You are a concise support assistant.')
    ->model('claude-opus-4-8')
    ->maxTokens(256)
    ->send();

$response->text;         // the answer
$response->usage;        // Usage: input/output/cache tokens
$response->costUsd;      // e.g. 0.0021  (null if the model is unpriced)
$response->latencyMs;    // wall-clock for the call

->text() is shorthand for ->send()->text.

Choosing a provider

Llm::using('openai')->model('gpt-4o-mini')->user('Ping')->text();
Llm::using('gemini')->model('gemini-2.5-flash')->user('Ping')->text();

The same ChatRequest reaches all three. Gemini's wire format differs the most — the assistant role is called model, the system prompt is a separate systemInstruction rather than a turn, tool parameters are typed objects instead of raw JSON Schema, and tool results come back as a user turn of functionResponse parts. All of that is absorbed by the driver; none of it reaches your code.

Streaming

$stream = Llm::chat('Write a haiku about pgvector.')->stream();

foreach ($stream as $delta) {
    echo $delta;                 // tokens as they arrive
}

$final = $stream->getReturn();   // assembled ChatResponse

Streaming deliberately skips the fallback chain and the response cache — bytes are already on the wire, so silently restarting on another provider would corrupt the output.

Retries and fallback

Llm::using('anthropic')
    ->fallback('openai', 'gpt-4o-mini')   // try OpenAI if Claude keeps failing
    ->retry(3, 250)                       // 3 attempts, 250ms base, exponential backoff
    ->user('Draft a release note.')
    ->send();

A default fallback chain can live in config/llm.php:

'fallbacks' => ['openai' => 'gpt-4o-mini'],

Only transient failures retry. A 400 or an unknown-model error fails immediately — retrying it just burns latency.

Prompt templates (Blade)

Prompts are the part of an LLM app that changes most often and that non-engineers want to read. Keep them as views under resources/views/prompts:

{{-- resources/views/prompts/triage.blade.php --}}
@system
You triage support tickets. Respond with a JSON object: {"priority": "...", "team": "..."}.
Valid priorities: {{ implode(', ', $priorities) }}.
@endsystem

@user
Ticket from {{ $customer }}:

{{ $body }}
@enduser

Llm::prompt('triage', [
    'priorities' => ['low', 'normal', 'high', 'urgent'],
    'customer'   => $ticket->customer_name,
    'body'       => $ticket->body,
])->send();

@system / @user split the template into the two turns the API takes; a template with neither is sent as a single user message. Because it is Blade, interpolated user input is escaped — a customer cannot inject a fake @system block.

Tool calling

Define a tool once; it renders for both providers. Give it a handler and let ->run() drive the loop:

use Touseef\LaravelLlm\Tools\Tool;

$weather = Tool::make(
    name: 'get_weather',
    description: 'Current weather for a city.',
    parameters: [
        'type' => 'object',
        'properties' => ['city' => ['type' => 'string']],
        'required' => ['city'],
    ],
    handler: fn (array $args) => Weather::for($args['city'])->toArray(),
);

$response = Llm::chat('What should I wear in Lisbon today?')
    ->tool($weather)
    ->run();   // calls the tool, feeds the result back, returns the final answer

->run($maxIterations) caps the loop so a confused model cannot spin forever. A handler that throws is reported back to the model as a tool error rather than crashing your request.

Response caching

Llm::chat('Explain HNSW indexing.')->cache(600)->send();  // cache this call for 10 min

Or turn it on globally (LLM_CACHE_ENABLED=true) and opt a call out with ->cache(null). The key hashes the driver, model, messages, tools and sampling parameters, so any change misses cleanly. A cache hit reports costUsd = 0.0 and cached = true.

Anthropic prompt caching

For a large, stable system prompt sent on every request, mark it cacheable provider-side:

Llm::chat($question)->system($bigManual)->cacheSystemPrompt()->send();

Cost tracking

Every response is priced from the rate table in config/llm.php. To persist it, publish and run the migration:

php artisan vendor:publish --tag=llm-migrations
php artisan migrate

Then enable logging (LLM_USAGE_LOGGING=true). One row per call lands in llm_usage_logs with tokens, cost, latency, cache-hit flag, prompt_key, the authenticated user id, and your metadata.

To surface spend on the response, add the middleware:

// bootstrap/app.php (Laravel 11+)
->withMiddleware(function (Middleware $middleware) {
    $middleware->append(\Touseef\LaravelLlm\Http\Middleware\TrackLlmCost::class);
})

Responses then carry X-Llm-Calls, X-Llm-Tokens and X-Llm-Cost-Usd. Handy for answering "which endpoint is burning the budget" before the monthly bill does.

Events

LlmResponseReceived fires after every successful call (cache hits included); LlmRequestFailed fires per failed attempt. Listen to build budgets, alerts or dashboards without touching call sites.

Testing

use Touseef\LaravelLlm\Facades\Llm;

it('summarises a ticket', function () {
    $fake = Llm::fake('Priority: high, Team: billing');

    $summary = app(TicketSummariser::class)->handle($ticket);

    expect($summary)->toContain('billing');

    $fake->assertSentCount(1)
        ->assertSent(fn ($request) => str_contains($request->system, 'concise'));
});

Queue several answers (Llm::fake(['first', 'second'])), return canned ChatResponse objects, throw exceptions to exercise error paths, or pass a closure for dynamic replies. No HTTP, no API keys.

Configuration

Everything lives in config/llm.php and is env-driven: default driver, fallback chain, retry policy, per-driver model and token defaults, thinking/effort, cache store and TTL, usage logging, and the model pricing table. See the published config — every option is commented.

Known limitations

• OpenAI streaming reports zero usage. Chat-completion streams omit token counts unless stream_options requests them, and support varies by model, so the streamed ChatResponse reports empty usage rather than a guess. Non-streaming OpenAI calls are unaffected.
• OpenAI and Gemini pricing ship empty. Anthropic list prices are included; add your own rates for the other two in config/llm.php so the package never reports a cost it can't stand behind.
• Cost figures are list prices, not your negotiated rate — confirm against billing before invoicing on them.
• ->effort() is Anthropic-only. It maps to a documented parameter there and has no equivalent elsewhere, so the other drivers ignore it rather than approximating one.
• ->thinking() on Gemini needs a budget. Gemini takes a thinking budget as a token count whose valid range differs per model, so the parameter is only sent when you set GEMINI_THINKING_BUDGET. Without it the call still works, using the model's own default.
• Gemini tool-call ids are synthetic. Gemini matches a function response to its call by name and only sometimes returns an id, so the driver generates one and strips it again on the way back. Don't persist a Gemini ToolCall::$id and expect it to mean anything to the API.

Contributing

composer test, composer lint, composer stan all run in CI on PHP 8.2/8.3 × Laravel 10/11. See CONTRIBUTING.md. Design rationale is in DECISIONS.md.

License

MIT — see LICENSE.