README

AI-powered LLMs.txt generation module for Magento 2 and Mage-OS stores. Automatically creates optimized llms.txt files using OpenAI to help AI systems understand your store content.

What is LLMs.txt?

LLMs.txt is a proposed standard (similar to robots.txt) that helps AI systems like ChatGPT, Claude, and others better understand your website by providing a concise, structured summary of your most important content.

Learn more: llmstxt.org

Features

• 🤖 AI-Powered Generation - Uses OpenAI to intelligently curate your store content
• ⚡ One-Click Generation - Click a button, review, and publish
• ✏️ Fully Editable - Edit AI-generated content or write your own
• 🏪 Multi-Store Support - Different content per store view
• 📊 Token Counter - Ensures content stays under recommended limits
• 🎯 Standards Compliant - Follows llmstxt.org specification

Installation

composer require mage-os/module-llm-txt
bin/magento module:enable MageOS_LlmTxt
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento cache:flush

Configuration

1. Navigate to: Stores → Configuration → AI → LLMs.txt

2. General Settings

   • Enable the module
   • Enter your OpenAI API key (get one here)
   • Select AI model (GPT-4o Mini recommended for speed and cost)

3. AI-Powered Generation

   • Set site name
   • Add site description
   • Specify up to 10 categories that best represent your store
   • Specify up to 10 products that best represent your store
   • Specify up to 10 CMS pages that best represent your store
   • Click "Generate with AI"

4. Review & Edit

   • AI generates curated content from your store data
   • Review the generated markdown
   • Edit as needed
   • Save configuration

5. Verify

   • Visit https://yourdomain.com/llms.txt
   • Content is served as plain text

How It Works

When you click "Generate with AI":

1. Data Collection - Gathers top categories, sample products, and key CMS pages
2. AI Analysis - Sends data to OpenAI with optimized prompt
3. Content Generation - AI creates concise, well-structured llms.txt
4. Review - You can edit the generated content before publishing

Example Output

# Your Store Name

> Your one-stop shop for quality products and exceptional service

## Shop by Category
- [Electronics](https://example.com/electronics.html): Explore cutting-edge tech and gadgets
- [Clothing](https://example.com/clothing.html): Fashion for every style and occasion

## Featured Products
- [Premium Headphones](https://example.com/headphones.html): Studio-quality sound
- [Organic Cotton T-Shirt](https://example.com/t-shirt.html): Sustainable fashion

## Customer Resources
- [About Us](https://example.com/about): Our story and mission
- [Contact](https://example.com/contact): Get in touch with our team

Manual Override

Don't want to use AI? No problem:

1. Enable "Use Manual Content" checkbox
2. Write your own llms.txt in markdown format
3. Save

Requirements

• PHP 8.1, 8.2, or 8.3
• Magento 2.4.x / Mage-OS 1.x
• OpenAI API key (for AI generation feature)
• Guzzle HTTP client (included with Magento)

Cost

OpenAI API usage is minimal:

• ~$0.001 per generation with GPT-5 Mini
• ~$0.010 per generation with GPT-5.4
• ~$0.001 per generation with GPT-4o Mini
• ~$0.005 per generation with GPT-4o

You only pay when clicking "Generate with AI".

Technical Details

Architecture

• Custom Router - Intercepts /llms.txt requests at sort order 22, only when module is enabled
• Custom Page Layout - Minimal layout with empty root container; renders plain text without Magento header/footer
• Store Data Collector - Collects configured categories, products, and CMS pages with meta descriptions using store emulation
• Prompt Builder - Formats collected store data into a structured OpenAI prompt with llms.txt specification constraints
• LLMs.txt Generator - Orchestrates generation: collect → build prompt → call API → append additional content
• OpenAI Client - Sends requests to https://api.openai.com/v1/responses via Guzzle

Testing

Unit tests included:

vendor/bin/phpunit app/code/MageOS/LlmTxt/Test/Unit

Coverage:

• Config model
• Generator model
• StoreDataCollector
• OpenAI Client
• Router
• Block rendering

License

MIT License - see LICENSE file for details

Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes with tests
4. Submit a pull request

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

Related Resources

• LLMs.txt Official Site
• LLMs.txt Directory
• OpenAI Platform
• Mage-OS