Compiled Programs

Experimental: The .mqc format and its CLI may change. Keep the query source and recompile .mqc files after upgrading mq.

mq compile saves a query as Tarn VM bytecode in a .mqc file. Pass the .mqc file in place of the query to run it. Modules, including HTTP imports, are resolved at compile time and stored in the file, so running it needs no module files or network access.

Repeated simple literal values within the same bytecode chunk share one entry, which can make .mqc files smaller when a query repeats those values.

mq compile -f query.mq            # writes query.mqc
mq compile '.h | to_text()' -o h.mqc
mq query.mqc README.md

As when running a query, mq compile takes a query string, or a query file with -f. A query file compiles to the same name with a .mqc extension; use -o to choose another path. A query string needs -o. The program runs only when its path ends in .mqc, and -f does not accept it.

The usual input and output options apply:

mq query.mqc -F json docs/*.md
cat notes.md | mq query.mqc -U

What is fixed at compile time

These options shape the compiled program, so pass them after mq compile (for example mq compile -I csv -f query.mq). mq compile rejects other options:

Module-level let values are computed once, when the program is compiled.

What is read at run time

• Values from --arg, --argjson, --rawfile, --slurpfile, and __FILE__, outside modules. A module can’t read them. A name that matches a builtin function, such as --args len 5, is rejected: the program resolved that name to the builtin when it was compiled.
• Permissions such as --allow-read and --allow-net. A compiled program gets no permission that the running mq does not grant. Permissions passed to mq compile apply only while module-level let values are computed, and are not saved.
• Environment variables, such as $HOME or s"${$HOME}" (with --allow-env). mq compile rejects an environment read in a module-level let, because the value would be saved in the file.

Compatibility

A .mqc file runs only on the same mq version that compiled it. Recompile the source after upgrading mq. The file keeps the original query and the sources of your own modules, so runtime errors point at the original code. Standard modules such as csv are not stored; their sources come from mq itself.

mq checks the file’s version, section versions and flags, structure, and checksum, then verifies its bytecode before running it. Files larger than 256 MiB, sections with more than one million collection entries, and bytecode that exceeds the verifier’s work limit are rejected. The checksum detects corruption. It does not show who created the file.

Embedding

Rust applications can use the same format through mq-lang with the mqc feature:

#![allow(unused)]
fn main() {
let mut engine = mq_lang::DefaultEngine::default();
engine.load_builtin_module();
let mqc = engine.precompile("upcase()", &[])?;
std::fs::write("upcase.mqc", mqc.as_bytes())?;

// Checks the file's structure, checksum, and mq version.
let mqc = mq_lang::Mqc::try_from(std::fs::read("upcase.mqc")?)?;
let program = engine.load(&mqc)?;
let input = mq_lang::parse_text_input("hello")?;
let output = engine.eval_compiled(&program, input.into_iter())?;
}