Optional integrations

The standard engine works with local Parquet and needs no platform services. Rust applications can add the integrations below when their deployment needs them.

Contracts compiled elsewhere

An application that compiles contracts itself hands peQL the result with Engine::register_compiled(document, &compiled, &functions): compiled is parcel’s compiled form (Compilation::to_bytes), and peQL registers it as given, without compiling it. The application vouches for the bytes; checking who produced them, for example by a signature, is the application’s. A contract compiled by another version of parcel is refused.

Custom functions

Parcel supports WebAssembly functions used by contract expressions. Register a module and its function manifest for an owner before compiling contracts that use it:

peql function register module.wasm --manifest function.yaml --owner acme
peql function list

The file names above refer to your compiled module and parcel function manifest. peQL verifies the module through parcel-runtime and stores it for that owner. Contracts record the function versions and hashes they use.

Function authoring, the WebAssembly interface and manifest fields belong to parcel. See its function guide.

Other data sources

Rust applications can bind a DataFusion TableProvider to a registered contract with Engine::bind_table. This lets the application supply data while retaining the contract query path.

ObjectStoreParquet serves bindings from a prefix of an object store (s3://bucket/prefix/) the way the default resolver serves local directories: streamed listing tables, the same write path, and manifests beside the data. Install it with Engine::with_bindings. Pass the store in: ObjectStoreParquet::new("s3://bucket/prefix/", store) takes any object_store store, configured by the application; the s3 feature adds the S3 store.

The optional lance feature adds a Lance table provider on Unix. LanceTableProvider::open_uri opens a dataset by path or object-store URI. Building this feature requires protoc.

Flight SQL

The flight feature adds peql::flight::FlightSql, a Flight SQL service over tonic on a listener the application supplies (serve_unix, serve_tcp, or serve for any connection stream). GetFlightInfo and CreatePreparedStatement run Engine::check, so a refusal returns before any scan; DoGet runs Engine::query and puts {"envelope", "signature"} JSON in the first message’s app metadata; DoPut bulk ingest runs Engine::write for the contract’s owner.

Each request names its caller in the x-peql-caller header (a Caller as base64 JSON; caller_header builds it). The header is trusted, so only whoever authenticated the caller may reach the listener. FlightSql::for_caller fixes one caller instead.

A read leaves out rows that break a row rule and reports them per rule in each contract’s excluded in the envelope. A request whose x-peql-on-violation header (ON_VIOLATION_HEADER) is refuse is refused instead, at GetFlightInfo, when the data holds any such row; the header’s default is exclude. An ingest refuses rows that break a row rule unless its on_violation option (ON_VIOLATION_OPTION) is warn. exclude is refused for an ingest, because DoPut cannot return the set-aside rows; write through Engine::write or the parts API to set rows aside. See Rows that break a contract.

Result signing

Engine::with_signer signs every query’s envelope with an EnvelopeSigner; the result is QueryResult::signature. If signing fails, the query fails. SocketSigner reaches a signer over a Unix socket, TCP or any connected stream: one connection per envelope, the envelope as one JSON line out, and {"jws": ...} or {"error": ...} as one line back.