Parsing

Three independent modules, reached through Operator.Parsing. Each is pure Luau with no Roblox API calls, so they run outside Studio. The client and the server run exactly this code, which is how the console validates inline without a round trip.

local Operator = require(game:GetService("ServerStorage").Operator)
local Tokenizer, Parser, Binder =
	Operator.Parsing.Tokenizer, Operator.Parsing.Parser, Operator.Parsing.Binder

Tokenizer

local Tokenizer = Operator.Parsing.Tokenizer

Tokenizer.tokenize

Tokenizer.tokenize(source: string): TokenizeResult

local result = Tokenizer.tokenize([[kick bob "being rude"]])

for _, token in result.tokens do
	print(token.value, token.raw, token.start, token.finish, token.quoted, token.terminated)
end

Tokenizer.MaxInputLength

Tokenizer.MaxInputLength: number  --> 2000

Input longer than this is refused outright rather than parsed. That is far above any real command, while still bounding the work a hostile client can cause.

Token

Field	Meaning
value	unescaped and unquoted
raw	the exact source slice
start / finish	1-based source positions, which the console uses to underline
quoted	whether a quoted section was used
terminated	whether its quote was closed

Tokenizer.Diagnostics

UnterminatedQuote · TrailingEscape · InputTooLong

Parser

local Parser = Operator.Parsing.Parser

Parser.parse

Parser.parse(tokens: { Token }): ParsedCommand

Splits tokens into positional words and named flags.

local parsed = Parser.parse(Tokenizer.tokenize([[ban bob --reason="spam bot" --silent]]).tokens)

print(parsed.words[1].value)          --> ban
print(parsed.flags.reason.value)      --> spam bot
print(parsed.flags.silent.value)      --> nil, a presence flag
print(parsed.flagOrder)               --> { "reason", "silent" }

Parser.Diagnostics

MalformedFlag · DuplicateFlag

Binder

local Binder = Operator.Parsing.Binder

Binder.bind

Binder.bind(options: BindOptions): BindResult

Binds words to named arguments. The caller strips the command path from words first, so the binder knows nothing about the registry.

local result = Binder.bind({
	words = words,
	flags = parsed.flags,
	flagOrder = parsed.flagOrder,
	argSpecs = command.argSpecs,
	flagSpecs = command.flagSpecs,
	inputLength = #source,
})

A finished command already carries matching argSpecs and flagSpecs, built once at :run(), so the dispatcher does not rebuild them per call.

Binder.validateSpecs

Binder.validateSpecs(argSpecs: { ArgSpec }): (boolean, string?)

Deliberately separate from bind. The command builder calls it at :run() so a malformed spec fails at load rather than on first use.

ArgSpec

Field	Effect
name	the key the value is bound to
optional	absence is not an error; must not precede a required argument
variadic	collects every remaining word into a list; must be last
rest	joins every remaining word into one string; must be last

Binder.Diagnostics

MissingArgument · TooManyArguments · UnknownFlag · MissingFlagValue · UnexpectedFlagValue

Diagnostics

Nothing in the parsing layer throws on bad user input. Every problem is reported as a diagnostic carrying a kind, a message, and a source range, so the client can render inline validation while the server rejects any input that produced one.

Module	Diagnostics
Tokenizer.Diagnostics	UnterminatedQuote, TrailingEscape, InputTooLong
Parser.Diagnostics	MalformedFlag, DuplicateFlag
Binder.Diagnostics	MissingArgument, TooManyArguments, UnknownFlag, MissingFlagValue, UnexpectedFlagValue

See Command Syntax for the rules these enforce.