charm / parsing
A PEG grammar parser, for parsing most context free grammar such as programming languages, database queries and mathematical expressions.
Requires
- charm/map: ^1.0.2
- charm/options: ^1.0
- charm/util-phpencode: ^1.0
- charm/vector: ^1.0
- psr/log: >=1.0
Requires (Dev)
- nikic/php-parser: ^5
- phpunit/phpunit: ^9
Suggests
- nikic/php-parser: ^5 - required by Charm\Parsing\Lexers\PhpLexer (PHP token-mode lexer)
Provides
None
Conflicts
None
Replaces
None
README
A PEG grammar library for PHP: describe a language as rules with optional inline PHP, and parse it with an interpreter or compile the same grammar into a standalone PHP class with no runtime dependency. Both engines produce the same values and the same error positions; a conformance suite runs every behaviour through both.
Install
composer require charm/parsing
PHP 8.1 or newer.
Sixty seconds
calc.cpeg:
Calc = _ Expr $
Expr = Expr "*" _ Expr <? return $_0 * $_2; ?>
| Expr "+" _ Expr <? return $_0 + $_2; ?>
| "(" _ Expr ")" _ <? return $_1; ?>
| Number
Number = /[0-9]+/ _ <? return (int) $_0; ?>
_ = /\s*/ %silent
The first rule is where parsing starts. Expr starts with itself, which a PEG parser normally cannot do;
here the alternatives listed first bind tighter (* before +). _ is whitespace and %silent keeps it out
of $_0, $_1, ... in snippets. Calc has no snippet: _ and $ have no value, so Expr is the only element that
can produce one and the sequence returns it as it is. Interpreted:
require 'vendor/autoload.php';
$grammar = file_get_contents('calc.cpeg');
echo Charm\cpeg_parse($grammar, '2 + 3 * (4 + 1)'), "\n";
try {
Charm\cpeg_parse($grammar, '2 + * 3');
} catch (RuntimeException $e) {
echo $e->getMessage(), "\n";
}
17
Failed parsing "Expr" on parsed source:1:4, unexpected "*"
The same grammar compiled into a class with no dependency on this library:
vendor/bin/charm-compile calc.cpeg -c CalcParser -o CalcParser.php
require 'CalcParser.php'; // nothing else to load
$parser = new CalcParser();
echo $parser->parse('2 + 3 * (4 + 1)'), "\n";
try {
$parser->parse('2 + * 3');
} catch (RuntimeException $e) {
echo $e->getMessage(), "\n";
}
17
Parse error at line 1, column 5: expected "(", found '*'
Without snippets the parse returns the parse tree: Expr = Expr "+" Expr | /[0-9]+/ on 1+2+3 gives
[["1","+","2"],"+","3"].
Which engine
Interpreter (cpeg_parse(), Charm\Parser) | Compiled (charm-compile) | |
|---|---|---|
| Setup | none | one command, a generated file to keep in step with the grammar |
| Speed | the baseline | about 6 times faster on a realistic grammar |
| Needs at run time | this library | nothing (string grammars) |
| Errors | Error\ParseError, Error\State\CommittedError | a generated {Class}ParseError with sourceLine, sourceColumn, expected, found, committed |
Token mode (@lexer) | no | yes |
$context in snippets | yes | yes |
Develop with the interpreter, compile when parse time matters or when you ship a parser without the library. REFERENCE.md has the numbers and what slows a compiled parser down.
What it does
- Rules,
|ordered choice,?*+, groups,&/!lookahead, regexes, case-insensitive keywords through"select"i(or(?i:...)in a regex) - Left recursion with precedence and associativity: alternative order,
%prec,%assoc, an@prectable - Values you control: sequences return lists (or the value itself when only one element has one),
name:captures,...spread,%silentrules, inline PHP snippets with$_0,$name,$state,$MISS - Commit
@for errors that say what was expected where - Template rules (
List<T, Sep>), a token mode that parses a lexer's token stream - Generated parsers throw their own
{Class}ParseError; positions are the furthest terminal that failed, the same in both engines
Documentation
- Tutorial: parsing SQL: build a
SELECTparser step by step; the finished grammar isexample-grammars/sql.cpeg - REFERENCE.md: every operator, pragma, snippet variable, API call and CLI flag, with the value it yields
- docs/index.md: the map of the chapters (getting started, grammar syntax, snippets, precedence, token mode, lexing, compiled parsers and the CLI, errors)
- CHANGELOG.md
example-grammars/: JSON, JavaScript expressions, ES3, ECMAScript regular expressions, yacc
Run the tests with php run-tests.php (or composer test).
Licence
GPL-3.0-only, with an additional permission (a Bison-style output exception). In plain words: the library and
the compiler stay GPL, but a parser that charm-compile generates from your grammar is yours: it is not
covered by the GPL and you may distribute it under any terms. See LICENCE.md and the
reference.