Modules and Imports

mq provides several ways to organize and reuse code: module, import, and include.

Module

Defines a module to group related functions and prevent naming conflicts using the syntax module name: ... end.

module module_name:
  def function1(): ...
  def function2(): ...
end

Functions within a module can be accessed using qualified access syntax:

module_name::function1()

Examples

# Define a math module
module math:
  def add(a, b): a + b;
  def sub(a, b): a - b;
  def mul(a, b): a * b;
end

# Use functions from the module
| math::add(5, 3)
# => 8

Scope

A module, inline or imported, sees only:

• Its own functions and let/var values.
• Built-in functions.
• Modules it imports itself, and inline modules defined earlier (by qualified name).

It does not see the query’s own let, var, or def bindings, modules imported outside it, or runtime values such as __FILE__ and --arg. Referring to them is an error.

module csv_tools:
  import "csv"
  def rows(text): csv::csv_parse(text, true);
end
| csv_tools::rows("a,b\n1,2")
# => [{"a": "1", "b": "2"}]

Module-level let values are computed once per run.

A module may bind each name only once across its let, var and def declarations. Declaring module m again extends the existing module with the new members. A member name already bound by an earlier declaration is a compile error:

module m: def a(): 1; def a(): 2; end                  # error: module binds "a" more than once
module m: def a(): 1; end | module m: def b(): 2; end  # ok: m has a and b
module m: def a(): 1; end | module m: def a(): 2; end  # error: module binds "a" more than once

Import

Loads a module from an external file using the syntax import "module_path". The imported module is available with its defined name and can be accessed using qualified access syntax.

The import directive searches for .mq files in the following locations:

• $HOME/.mq - User’s home directory mq folder
• $ORIGIN/../lib/mq - Library directory relative to the source file
• $ORIGIN/../lib - Parent lib directory relative to the source file
• $ORIGIN - Current directory relative to the source file

import "module_name"

Examples

math.mq:

def add(a, b): a + b;
def sub(a, b): a - b;

main.mq:

# Import the math module
import "math"

# Use functions with qualified access
| math::add(10, 5)  # Returns 15
| math::sub(10, 5)  # Returns 5

Import Aliases

Use import "module_path" as alias to bind the module under a different name, useful for shortening long module paths or avoiding naming conflicts. Only the alias is bound; the module’s original name is not available.

import "math" as m

| m::add(10, 5)  # Returns 15
| m::sub(10, 5)  # Returns 5

Include

Loads functions from an external file directly into the current namespace using the syntax include "module_name". Unlike import, functions are available without a namespace prefix.

The include directive searches for .mq files in the same locations as import.

include "module_name"

Examples

math.mq:

def add(a, b): a + b;
def sub(a, b): a - b;

main.mq:

# Include math functions
include "math"

# Functions are available directly
| add(2, 3)  # Returns 5
| sub(10, 4) # Returns 6

Circular dependencies

A module that imports or includes itself, directly or through other modules, is an error:

Circular module dependency: a -> b -> a

Loading the same module from several places is fine as long as it does not depend back on itself.

Built-in modules

mq ships several built-in modules for parsing common structured data formats. They are available via import without any additional installation.

These modules are also used automatically when you process a file whose extension matches (see CLI auto-parsing).

Example

import "json"
| json::json_parse()

JSON Pointer

The json module also supports JSON Pointer (RFC 6901) references, built on get_path/set_path/del_path.

import "json"
| let doc = {"a/b": [10, 20]}
| json::json_pointer_get(doc, "/a~1b/1")
# => 20

JSON Patch

The json module can compare two values and apply the result as a JSON Patch (RFC 6902). Values from YAML, TOML and front matter work the same way.

import "json"
| let a = {"name": "mq", "tags": ["md"]}
| let b = {"title": "mq", "tags": ["md", "cli"]}
| json::json_diff(a, b)
# => [{"op": "add", "path": "/tags/1", "value": "cli"}, {"op": "move", "from": "/name", "path": "/title"}]

Dicts are compared by key and arrays by index, so an element inserted at the front of an array shows up as a run of replace operations.

JSON Merge Patch and 3-way merge

import "json"
| json::json_merge_patch({"a": 1, "b": {"c": 2, "d": 3}}, {"a": None, "b": {"c": 9}})
# => {"b": {"c": 9, "d": 3}}

import "json"
| json::merge3({"a": 1, "b": 1}, {"a": 2, "b": 1}, {"a": 1, "b": 3})
# => {"value": {"a": 2, "b": 3}, "conflicts": []}

merge3 only merges nested dicts key-by-key; arrays and scalars are compared as a whole leaf, so a change to any element of an array both sides touched is one conflict on the whole array. A key added independently by both sides with different values is also a conflict, even though neither base had it.

Combine with paths to list a pointer for every leaf: paths(doc) | map(json::json_pointer_from_path).

Markdown Builder (md)

The md module provides functions for constructing markdown nodes from scratch, rather than filtering or transforming existing ones. Each function returns a markdown value that can be combined with others using md::doc(), which merges nodes into a single markdown value. md::doc() accepts either a variable number of arguments or a single array, and flattens nested arrays automatically, so the result of map() (or any function returning a plain array of nodes) can be spliced in directly as children, and None entries (e.g. from conditional branches) are dropped.

Note: This module is under development. APIs and behavior may change without notice.

import "md"
| md::doc(
    md::h("My Project", 1),
    md::text("Run `cargo install mq`."),
    md::code("cargo install mq", "bash"),
    map(["fast", "composable", "jq-like"], fn(x): md::list(x);),
  )

Since the current value (self) is automatically passed when a call is missing an argument, builder calls also read naturally in pipeline position:

import "md"
| "My Project" | md::h(1)    # equivalent to md::h("My Project", 1)
# => "# My Project"

Lists and tables are built the same way:

import "md"
| md::doc(
    # List
    md::list("Plain item"),
    md::list("Nested item", 1),
    md::list("Ordered item", 0, true),
    md::list("Checked item", 0, false, true),
    # Table
    md::table_row(["Name", "Age"]),
    md::table_align(["left", "right"]),
    md::table_row(["Alice", "30"]),
  )

Inline and heading helpers cover the rest of the common formatting cases:

import "md"
| md::doc(
    md::h2("Notes"),          # md::h1 .. md::h6
    md::highlight("key"),     # ==key==
    md::comment("todo"),      # <!-- todo -->
    md::wikilink("Page", "alias"),  # [[Page|alias]]
    md::embed("image.png"),   # ![[image.png]]
    md::escape("a*b_c"),      # a\*b\_c
    md::hard_break("a\nb"),   # single newlines become hard line breaks
  )

Reference indexing and resolution

md::reference_index(md_nodes) and md::resolve_references(md_nodes, index = None) index and resolve reference-style links ([text][ident], including the shortcut form [ident]), reference images (![alt][ident]), and footnotes ([^ident]) against their [ident]: url / [^ident]: ... definitions. Both need every document node at once: pass -A on the CLI, or pipe through nodes in an inline query/script.

reference_index groups every link and footnote definition by identifier:

import "md"
| nodes
| md::reference_index()
# => {"links": {"ident": [{ident, label, url, title, location}, ...]}, "footnotes": {"ident": [{ident, location}, ...]}}

Each identifier maps to an array of every definition found for it, so a duplicate definition (more than one [ident]: line for the same ident) shows up as an array with more than one entry.

resolve_references walks every reference node and looks it up in the index, returning one dict per reference with its type, ident, label, location, whether it resolved to a definition, whether that definition is duplicate, the definition picked (CommonMark’s first-match-wins, or None if unresolved), and every candidate definitions:

import "md"
| nodes
| md::resolve_references()
| filter(fn(r): !r[:resolved];)
# => reference nodes with no matching definition (for building linkcheck-style diagnostics)

index defaults to md::reference_index(md_nodes), resolving references against their own document. Pass an explicit index built from a different set of nodes to resolve references that got separated from their definitions, for example when a document is split or a section is moved into another file, which is exactly the case where a reference legitimately comes back unresolved.

Property-Based Test Generators (gen)

The gen module is bundled with mq-test (it is not available to the plain mq command) and provides generator combinators for mq-test’s # @property(count, generators) annotation (see the mq-test crate’s README for the annotation itself). A generator is a plain function fn(seed): value; built on top of the seeded rand/rand_int/ random_string builtins, so the same seed always produces the same value. Combinators build bigger generators out of smaller ones, deriving an independent sub-seed for each part so nested generators don’t correlate with one another.

import "gen"
| (gen::int(1, 10))(42)

The Markdown generators produce source text (a string), so a failing document shrinks by truncation. Parse it inside the property with to_markdown(source).

gen::tuple is what # @property(...) calls under the hood to turn an array of per-parameter generators into one argument list per iteration; a failing iteration’s reported index doubles as the seed to reproduce it with, by calling gen::tuple(generators)(seed) again with the same generators array.

Shrinking (gen::shrink_value, gen::shrink)

When a # @property(...) case fails, mq-test automatically searches for a smaller, simpler failing example before reporting it. The failure message shows both the shrunk arguments and the original ones, so the report stays easy to reason about even when the raw seed produced a large or convoluted value:

import "gen"
| gen::shrink([10], fn(a): a[0] > 3;)
# => [4]

Both are also usable directly (not just through # @property(...)) by hand-written property tests.

Small-scope tests (enumerate)

# @small_scope(max_size, enumerators, options) runs a test on every input of size <= max_size, smallest first, instead of on random inputs. The first failure is a smallest counterexample, so no shrinking is needed. A clean run means no counterexample was found within the listed scope; it says nothing about larger inputs. The enumerate module is bundled with mq-test (not available to the plain mq command) and imported automatically. An enumerator is a dict whose tier(k) returns every value of size exactly k.

# @small_scope(3, [enumerate::markdown({"kinds": ["heading", "paragraph"], "min_blocks": 1})])
def test_first_block(src):
  assert_true(is_markdown(to_markdown(src)[0]))
end

Every enumerated input is checked; build inputs that satisfy a precondition with the enumerator options (here min_blocks). Enumeration stops with an error when max_cases (default 100000) is exceeded. See the mq-test crate’s README for the options and the full semantics.

HTTP Imports

When mq is built with the http-import feature, import and include accept HTTP/HTTPS URLs in addition to local file names.

Security note: HTTP imports are disabled by default and must be enabled with --allow-http-import. Once enabled, only URLs under github.com/harehare (resolved to raw.githubusercontent.com/harehare) are allowed; importing from any other domain also requires --allow-http-import=DOMAIN. This is a separate permission from --allow-net, which only gates the http() builtin, not module resolution.

Plain URL

import "https://example.com/mymod.mq"

GitHub shorthand

The scheme can be omitted for GitHub repositories. mq automatically maps the path to raw.githubusercontent.com.

github.com/{owner}/{path}[@{version}]

Example:

mq --allow-http-import 'import "github.com/harehare/kdl.mq" | kdl::kdl_parse("title \"Hello, World!\"")'

Caching

Fetched modules are cached in {system_cache_dir}/mq/ as {md5(url)}.mq files.

• Versioned URLs (e.g. @v0.1.0): cached indefinitely: the tag content is immutable.
• Mutable refs (HEAD, main, master, or no version): cached on first fetch. Pass --refresh-modules on the command line to discard the cache and re-fetch.

Lock file (mq.lock)

• First use of a URL (fetched or served from the cache): recorded in mq.lock.
• Later use, content unchanged: succeeds silently.
• Later use, content changed: fails with an error explaining the mismatch. Re-run with --refresh-modules to accept the new content and update mq.lock (for a versioned/tagged URL, whose cache --refresh-modules doesn’t touch, use --clear-cache instead).
• --refresh-modules / --clear-cache also drop the corresponding entries from mq.lock (mutable-ref entries only, and all entries respectively), matching their disk-cache behavior.
• Pass --no-lockfile to disable the check entirely (no file is read or written).
• Pass --lockfile <path> to use a different location instead of ./mq.lock. Missing parent directories are created automatically. Mutually exclusive with --no-lockfile.

Commit mq.lock alongside scripts that use HTTP imports so CI and teammates fetch the exact content you locked, the same way package-lock.json/deno.lock work.

CLI options

Examples:

# Enable HTTP imports, restricted to the built-in default domain
mq --allow-http-import 'self' file.md

# Force re-fetch of any HEAD/branch modules, accepting new content into mq.lock
mq --allow-http-import --refresh-modules 'self' file.md

# Only allow imports from example.com (in addition to the built-in default)
mq --allow-http-import=example.com 'self' file.md

# Allow multiple domains
mq --allow-http-import=example.com,raw.githubusercontent.com 'self' file.md

# Use a different lock file location
mq --allow-http-import --lockfile config/mq.lock 'self' file.md

# Skip the mq.lock check entirely
mq --no-lockfile 'self' file.md

# CI: fail if a script tries to import a URL not already recorded in the committed mq.lock
mq --allow-http-import --frozen 'self' file.md

Network and File-Write Capabilities

http(method, url) / http(method, url, body) / http(method, url, headers) / http(method, url, body, headers), write_file(path, content) and mkdir(path) are disabled by default and must be explicitly enabled with --allow-net / --allow-write. Calling them without the corresponding flag raises a runtime error explaining how to opt in.

method is a string or symbol ("post" or :post) and accepts any HTTP method (get, post, put, delete, patch, head, and so on). The optional body argument is sent as the request body regardless of method. The optional headers argument is a dict of string to string (e.g. {"Content-Type": "application/json"}) applied to the request.

http_all([...requests]) issues a batch of HTTP requests and returns an array of response bodies in the same order. Each request is a dict with a required url and optional method (defaults to get), body, and headers keys, e.g.:

mq --allow-net 'http_all([{"url": "https://example.com/a"}, {"url": "https://example.com/b", "method": "post", "body": "{}", "headers": {"Content-Type": "application/json"}}])'

Requests in the batch run concurrently when the runtime supports it, so fanning out to multiple endpoints is faster than calling http() in a loop. The same URL/domain policy applies as for http(): HTTPS only, and --allow-net (with or without a domain allowlist) is required.

Security note: http only accepts https:// URLs and is routed through the same SSRF-hardened client used for HTTP imports (no automatic redirects), and DNS results are filtered to publicly routable addresses, so a loopback/private/link-local address can’t be reached even with --allow-net set.

# Blocked by default
mq 'http(:get, "https://example.com")'

# Enabled explicitly
mq --allow-net 'http(:get, "https://example.com")'
mq --allow-net 'http(:delete, "https://example.com/resource/1")'
mq --allow-net 'http(:post, "https://example.com", "{}", {"Content-Type": "application/json"})'
mq --allow-net 'http(:post, "https://example.com", "{}", {"Content-Type": "application/json"})'
mq --allow-write 'write_file("out.md", "# Hello")'

--allow-net also accepts a domain allowlist, restricting http/http_* calls to just those domains (and any path under them) instead of granting unrestricted network access:

# Restrict to just example.com
mq --allow-net=example.com 'http(:get, "https://example.com")'

# Repeat the flag, or comma-separate, to allow more than one domain
mq --allow-net=example.com,api.example.org 'http(:get, "https://api.example.org")'

Comparison