cirql Language Specification

Version: 0.1.0-draft Status: Pre-implementation Draft Date: 2026-03-14

1. Abstract

cirql is a domain-specific language (DSL) and runtime for composing JSON data pipelines. It combines jq-style data transformation with pluggable source stages — including GraphQL queries, REST/HTTP endpoints, local files, and stdin — enabling users to chain data sources, post-process results, propagate variables between stages, and dynamically reshape data, all from an interactive REPL or CLI tool called cq.

The pipeline engine operates on JSON. Source stages (query, http, file, stdin) produce JSON; transform stages (map, filter, reduce, sort, flatMap, limit, uniq) reshape it. Any source that produces JSON can feed into the same pipeline.

The cirql service at api.cirql.io provides a hosted execution environment for cirql pipelines, enabling team collaboration, saved pipelines, and managed endpoint connections.

2. Feasibility Assessment

2.1 Technical Feasibility

Rating: High

Each component of cirql maps to well-understood engineering problems:

2.2 Language Design Feasibility

The pipeline operator | is syntactically unambiguous because every stage begins with a distinguishing token that enables simple prefix-based dispatch. In practice this is either a keyword (query, http, file, stdin, map, filter, etc.) or a leading { for inline query blocks. The parser can cleanly separate:

• Source stages: start with query, http, file, or stdin, or with { for inline query stages
• Transform stages: start with map, filter, reduce, sort, flatMap, limit, uniq
• Variable references: start with $identifier syntax

The main design risk is type inference across stages: ensuring that field references in map/filter expressions match the shape of the previous stage’s output. This can be addressed with runtime checking initially, with optional static analysis later.

2.3 Implementation Estimate

A minimal viable cq REPL with http/file/stdin sources, basic map/filter/reduce, and JSON output is achievable in ~3,000-5,000 lines of Go. The full system with GraphQL support, optimization, and multi-source pipelines is ~10,000-15,000 lines.

3. Competitive Landscape

3.1 Direct Competitors

No direct competitor exists that combines all of: pluggable JSON sources (GraphQL, REST, files, stdin), jq-style pipeline composition, variable propagation between stages, and a dedicated REPL. This is cirql’s primary opportunity.

Notably, Hasura’s own RFC #10058 explicitly requests query chaining — the ability to use output from one query as input to the next. This validates the demand for exactly what cirql provides, but as a client-side tool that works against any endpoint.

3.2 Adjacent Tools by Category

GraphQL Federation / Gateway Platforms

GraphQL CLI Tools

JSON Processing Tools

GraphQL Query Composition Libraries

Data Pipeline Tools

3.3 Gap Analysis

No existing tool combines all of cirql’s capabilities:

3.4 Competitive Positioning

cirql occupies a unique position at the intersection of:

                    CLI / Scriptable
                         │
                    ┌────┼────┐
                    │  cirql  │
                    └────┼────┘
                         │
  Any JSON source ───────┼────── Pipeline composition
  (GraphQL, REST,        │
   files, stdin)    Data transforms

The closest workflow today requires: curl → jq → manual variable extraction → xargs → another curl → more jq. cirql collapses this entire pattern into a single composable expression that works with any JSON-producing source.

3.5 Competitive Risks

• Nearest threat: jq gains built-in HTTP fetch, or someone builds a pipeline wrapper around curl+jq.
• Biggest moat: The unified DSL — no one else treats “any JSON source + jq-style transforms + variable propagation” as a first-class composable language.

4. Utility Assessment

4.1 Highest-Value Use Cases

1. API exploration and prototyping — Developers exploring unfamiliar APIs (REST or GraphQL) can iteratively fetch, inspect, and refine — faster than any GUI tool.

2. Data extraction pipelines — Pull data from APIs or files, transform it, and output JSON/CSV for analysis. Replaces ad-hoc scripts.

3. REST API scripting — Pipe REST endpoints through transforms without writing bash glue. Replace curl | jq | xargs curl with a single cirql pipeline.

4. Cross-source queries — Fetch users from a REST API, then query each user’s details via GraphQL, then aggregate. Multi-hop, multi-source patterns are natural in cirql.

5. CI/CD and automation — Scriptable API queries for deployment checks, data validation, monitoring dashboards.

6. API testing — Chain sources to test workflows (create user → fetch user → update user → verify).

7. Microservice orchestration from the CLI — When you have multiple services (GraphQL, REST, or both) and need to compose calls across them ad-hoc, without deploying a gateway.

4.2 Target Audience

4.3 Why Now

• JSON is the universal data interchange format; a pipeline language for JSON is broadly useful beyond any single API paradigm
• GraphQL and REST APIs are mainstream — developers constantly glue curl and jq together
• No CLI tool has captured the “pipeline language for JSON data” space
• Developer tooling is shifting back toward CLI-first workflows (Warp, Fig, atuin)
• AI-assisted coding means more developers working in terminals

5. Language Specification

5.1 Grammar (EBNF)

pipeline      = stage { "|" stage } ;
stage         = query_stage | http_stage | file_stage | stdin_stage
              | map_stage | filter_stage | reduce_stage
              | sort_stage | flatmap_stage | limit_stage | uniq_stage ;

query_stage   = [ "query" ] query_body [ "(" var_bindings ")" ] ;
query_body    = "{" selection_set "}" ;
selection_set = field { field } ;
field         = name [ "(" arguments ")" ] [ "{" selection_set "}" ] ;
arguments     = argument { "," argument } ;
argument      = name ":" value ;
value         = variable | string | number | boolean | null
              | "[" [ value { "," value } ] "]"
              | "{" [ name ":" value { "," name ":" value } ] "}" ;
variable      = "$" name ;
header_value  = string | variable ;

http_stage    = "http" ( string | variable ) [ http_options ] ;
http_options  = "(" http_opt { "," http_opt } ")" ;
http_opt      = "method" ":" string
              | "headers" ":" "{" [ name ":" header_value { "," name ":" header_value } ] "}"
              | "body" ":" ( string | variable ) ;

file_stage    = "file" string ;

stdin_stage   = "stdin" ;

var_bindings  = var_binding { "," var_binding } ;
var_binding   = "$" name "=" expr ;

map_stage     = "map" "{" mapping { "," mapping } "}" ;
mapping       = name ":" expr ;

filter_stage  = "filter" expr ;
reduce_stage  = "reduce" reduce_op [ "(" expr ")" ] ;
reduce_op     = "count" | "sum" | "min" | "max" | "avg" | "first" | "last"
              | "group_by" | "collect" ;
              (* group_by(expr) returns an Object keyed by the expr value,
                 each value being a List of matching objects.
                 collect(expr) returns a List of the expr value from each object.
                 first/last return the first/last object in the result set. *)

sort_stage    = "sort" expr [ "asc" | "desc" ] ;
flatmap_stage = "flatMap" "{" mapping { "," mapping } "}" ;
limit_stage   = "limit" integer ;
uniq_stage    = "uniq" [ expr ] ;

expr          = or_expr ;
or_expr       = and_expr { "||" and_expr } ;
and_expr      = cmp_expr { "&&" cmp_expr } ;
cmp_expr      = add_expr { ( "==" | "!=" | ">" | "<" | ">=" | "<=" ) add_expr } ;
add_expr      = mul_expr { ( "+" | "-" ) mul_expr } ;
mul_expr      = unary_expr { ( "*" | "/" | "%" ) unary_expr } ;
unary_expr    = "!" unary_expr | primary ;
primary       = field_access | variable | literal | func_call | "(" expr ")" ;
field_access  = "." name { "." name | "[" "]" } ;
literal       = string | number | boolean | null ;
func_call     = name "(" [ expr { "," expr } ] ")" ;

name          = letter { letter | digit | "_" } ;
string        = '"' { char } '"' ;
number        = [ "-" ] digit { digit } [ "." digit { digit } ] ;
boolean       = "true" | "false" ;
null          = "null" ;
integer       = digit { digit } ;

5.2 Type System

cirql operates on a dynamic type system with the following value types:

Type coercion rules:

• Int promotes to Float in mixed arithmetic
• Null propagates through field access (.foo on null yields null)
• Comparison operators work across numeric types
• String concatenation via + when either operand is a String

5.3 Pipeline Semantics

A pipeline is an ordered sequence of stages. Data flows left to right through the | operator.

Data model: The fundamental data unit flowing between stages is a result set — a List of Object values. Each stage receives a result set and produces a result set.

Source stages (produce a result set from external data):

Transform stages (reshape a result set):

Source normalization: If the JSON response is a single object (not an array), it is wrapped in a single-element list to produce a valid result set. If the response is an array of primitives, each element is wrapped as {"value": <element>}.

flatMap semantics: flatMap works like map but when any mapping expression evaluates to a List, the result is expanded into multiple output objects (one per list element). This is the primary mechanism for navigating nested JSON structures and GraphQL connection edges. For example, if .variants.edges is a list of 3 items, flatMap { name: .title, qty: .variants.edges[].node.qty } produces 3 output objects, one per edge.

5.4 Variable Propagation

Variables are extracted from the previous stage’s result set and injected into subsequent source stages.

http "https://api.example.com/users"
| map { login: .username }
| query { user(login: $login) { bio followers { totalCount } } }

http "https://api.example.com/orgs"
| map { url: .repos_url }
| http $url
| flatMap { name: .name, stars: .stargazers_count }

Propagation rules:

1. $var references are resolved against each object in the current result set
2. If the result set has multiple objects, the source stage executes once per object (fan-out) or batches where possible
3. The optimizer MAY fuse fan-out queries into a single batched query when the target field accepts list types
4. Variables can interpolate into http URLs (http $url) and query arguments (query { user(login: $login) { ... } })

5.5 Built-in Functions

5.6 Output Formats

The cq CLI supports multiple output formats:

5.7 REPL Features

• Multi-line input: Queries spanning multiple lines (brace-matching)
• Pipeline continuation: | at end of line continues the pipeline
• History: Persistent command history across sessions
• Autocomplete: Schema-aware field and type completion via introspection
• Inline help: .help, .schema, .endpoints, .history dot-commands
• Variables: .set endpoint https://... for session configuration
• Pipeline replay: !! reruns last pipeline, !n reruns nth pipeline

5.8 Error Handling

6. Architecture

6.1 System Components

┌─────────────────────────────────────────────────────┐
│                     cq CLI / REPL                   │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  Readline │  │  History │  │  Schema Cache    │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
└────────────────────────┬────────────────────────────┘
                         │ input text
                         ▼
┌─────────────────────────────────────────────────────┐
│                      Parser                         │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  Lexer   │  │  Grammar │  │  AST Builder     │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
└────────────────────────┬────────────────────────────┘
                         │ Pipeline AST
                         ▼
┌─────────────────────────────────────────────────────┐
│                 Planner / Optimizer                  │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  Batcher │  │  Fuser   │  │  Dependency Anlz │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
└────────────────────────┬────────────────────────────┘
                         │ Execution Plan
                         ▼
┌─────────────────────────────────────────────────────┐
│                  Execution Engine                    │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  GraphQL │  │   HTTP   │  │  File / Stdin     │  │
│  │  Client  │  │  Client  │  │  Reader           │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
│  ┌──────────┐  ┌──────────────────────────────────┐ │
│  │  Batch   │  │  Fan-out Controller              │ │
│  │  Runner  │  │                                  │ │
│  └──────────┘  └──────────────────────────────────┘ │
└────────────────────────┬────────────────────────────┘
                         │ raw results
                         ▼
┌─────────────────────────────────────────────────────┐
│                  Post-Processor                     │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  Expr    │  │  Map /   │  │  Reduce /         │  │
│  │  Eval    │  │  Filter  │  │  Aggregate        │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
└────────────────────────┬────────────────────────────┘
                         │ processed results
                         ▼
┌─────────────────────────────────────────────────────┐
│                 Output Formatter                    │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │  JSON    │  │  Table   │  │  CSV / TSV        │  │
│  └──────────┘  └──────────┘  └──────────────────┘  │
└────────────────────────┬────────────────────────────┘
                         │ formatted output
                         ▼
                      stdout

6.2 API Service (api.cirql.io)

The hosted service provides:

Request format:

{
  "pipeline": "http \"https://api.example.com/users\" | filter .active == true | reduce count",
  "endpoint": "https://api.example.com/graphql",
  "variables": {},
  "format": "json"
}

The endpoint field is required only for pipelines containing query stages. Pipelines using http, file, or stdin sources do not require it.

6.3 Core Types (Go)

package cirql

// Value represents any cirql runtime value.
type Value = any

// Object is a key-value map.
type Object map[string]Value

// ResultSet is the data flowing between pipeline stages.
type ResultSet []Object

// Stage is a single step in a pipeline.
type Stage interface {
    Execute(input ResultSet) (ResultSet, error)
    String() string
}

// Pipeline is an ordered sequence of stages.
type Pipeline struct {
    Stages []Stage
    Vars   map[string]Value
}

// Argument is a named value in a GraphQL field call.
type Argument struct {
    Name  string
    Value Value
}

// Mapping is a key-expression pair used in map/flatMap stages.
type Mapping struct {
    Key  string
    Expr Expr
}

// QueryStage executes a GraphQL query.
type QueryStage struct {
    Query     string
    Arguments []Argument
    Endpoint  string
}

// HttpStage fetches JSON from an HTTP endpoint.
type HttpStage struct {
    URL     Expr              // string literal or $variable
    Method  string            // default "GET"
    Headers map[string]string
    Body    Expr              // optional, for POST/PUT
}

// FileStage reads JSON from a local file.
type FileStage struct {
    Path string
}

// StdinStage reads JSON from standard input.
type StdinStage struct{}

// MapStage transforms each object in the result set.
type MapStage struct {
    Mappings []Mapping
}

// FlatMapStage transforms and flattens — list-valued expressions
// expand into multiple output objects.
type FlatMapStage struct {
    Mappings []Mapping
}

// FilterStage selects objects matching a condition.
type FilterStage struct {
    Condition Expr
}

// ReduceStage aggregates a result set.
type ReduceStage struct {
    Op   ReduceOp
    Expr Expr // optional: field to aggregate
}

// SortStage reorders the result set.
type SortStage struct {
    Expr Expr
    Desc bool
}

// LimitStage truncates the result set to at most N objects.
type LimitStage struct {
    N int
}

// UniqStage deduplicates the result set.
// If Expr is non-nil, uniqueness is determined by that expression;
// otherwise full object equality is used.
type UniqStage struct {
    Expr Expr // optional
}

// Expr represents an expression in the cirql AST.
type Expr interface {
    Eval(obj Object) (Value, error)
}

// FieldAccess represents .field.subfield or .field[].subfield
type FieldAccess struct {
    // Each segment is either a field name (string) or array
    // iteration marker.
    Path []PathSegment
}

// PathSegment is either a named field or an array iterator [].
type PathSegment struct {
    Name     string // field name, empty if IsIter
    IsIter   bool   // true for []
}

// VarRef represents a $variable reference in an expression.
type VarRef struct {
    Name string
}

// UnaryExpr represents !expr.
type UnaryExpr struct {
    Op      Operator
    Operand Expr
}

// BinaryExpr represents left op right.
type BinaryExpr struct {
    Left  Expr
    Op    Operator
    Right Expr
}

// FuncCall represents name(args...).
type FuncCall struct {
    Name string
    Args []Expr
}

// Literal represents a constant value.
type Literal struct {
    Value Value
}

7. Implementation Milestones

Milestone 1: Parser + AST

• Lexer for cirql tokens
• Recursive descent parser for pipeline grammar
• AST node types for all stage kinds
• Unit tests for parsing

Milestone 2: Expression Evaluator

• Field access (.id, .user.name)
• Arithmetic operators (+, -, *, /, %)
• Comparison operators (==, !=, >, <, >=, <=)
• Logical operators (&&, ||, !)
• Built-in function dispatch
• Unit tests for evaluation

Milestone 3: Core Pipeline Runtime

• Sequential stage execution
• map, filter, reduce stage implementations
• Variable propagation between stages
• JSON output formatting
• Integration tests

Milestone 4: Source Stages

• HTTP client integration (http stage) with method/header/body support
• File reader (file stage) with JSON parsing
• Stdin reader (stdin stage) with streaming JSON support
• GraphQL client integration (query stage)
• Source normalization (wrap single objects, handle primitive arrays)
• Variable binding and interpolation for source URLs
• Error mapping from all source types
• Fan-out execution for list results

Milestone 5: CLI / REPL (cq)

• Line reader with history
• Multi-line input support
• Pipeline parsing and execution
• Pretty-printed JSON output
• Endpoint configuration (.set endpoint <url>)

Milestone 6: Schema Introspection

• __schema query execution
• Type-aware autocomplete
• Field validation before execution
• .schema dot-command for exploration

Milestone 7: Optimizer

• Query fusion (merge sequential queries to same endpoint)
• Batch detection (list-accepting arguments)
• Deduplication of identical sub-queries
• Parallel fan-out execution

Milestone 8: API Service

• HTTP server for pipeline execution
• Pipeline storage (save/load)
• Endpoint management
• Authentication and rate limiting

Milestone 9: Advanced Features

• Dynamic query generation from runtime data
• Additional source stages (SQL, gRPC, WebSocket)
• Result caching with TTL
• Table and CSV output formats
• Pipeline composition (named pipelines as stages)

8. Configuration

8.1 CLI Configuration (~/.config/cq/config.toml)

[defaults]
format = "pretty"         # pretty | compact | table | csv | tsv
color = true

[endpoints.github]
url = "https://api.github.com/graphql"
headers = { Authorization = "Bearer ${GITHUB_TOKEN}" }

[endpoints.shopify]
url = "https://mystore.myshopify.com/admin/api/2025-01/graphql.json"
headers = { X-Shopify-Access-Token = "${SHOPIFY_TOKEN}" }

[http.defaults]
timeout = "30s"
headers = { User-Agent = "cq/0.1" }

[history]
max_entries = 10000
file = "~/.config/cq/history"

[cache]
enabled = true
ttl = "5m"
max_size = "100MB"

8.2 Environment Variables

9. Example Pipelines

REST API: GitHub contributors (no GraphQL)

http "https://api.github.com/repos/golang/go/contributors"
| filter .contributions > 100
| map { login: .login, commits: .contributions }
| sort .commits desc
| limit 10

Local file: Analyze package.json dependencies

file "package.json"
| map { deps: keys(.dependencies) }
| flatMap { dep: .deps }
| sort .dep asc

stdin: Count log levels from piped input

cat app.log | jq -R 'fromjson?' | cq '
  stdin
  | map { level: .level }
  | reduce group_by(.level)
'

Mixed sources: Enrich REST data with GraphQL

http "https://api.example.com/users"
| map { login: .username }
| query { user(login: $login) { bio followers { totalCount } } }
| map { login: $login, bio: .user.bio, followers: .user.followers.totalCount }
| sort .followers desc

GitHub: Top contributors by stars (GraphQL)

query {
  organization(login: "golang") {
    repositories(first: 50, orderBy: {field: STARGAZERS, direction: DESC}) {
      nodes { name stargazerCount }
    }
  }
}
| map { name: .organization.repositories.nodes[].name,
        stars: .organization.repositories.nodes[].stargazerCount }
| filter .stars > 1000
| sort .stars desc
| limit 10

Shopify: Low-stock products

query {
  products(first: 100) {
    edges {
      node { title variants(first: 10) { edges { node { inventoryQuantity } } } }
    }
  }
}
| flatMap { title: .node.title, qty: .node.variants.edges[].node.inventoryQuantity }
| filter .qty < 5
| sort .qty asc

Multi-hop: Users and their open issues

query { users(first: 20) { nodes { login id } } }
| map { login: .nodes[].login, uid: .nodes[].id }
| query { user(login: $login) { issues(states: OPEN, first: 5) { totalCount } } }
| map { user: $login, open_issues: .user.issues.totalCount }
| filter .open_issues > 0
| sort .open_issues desc

10. speckit Metadata

speckit:
  name: cirql
  version: 0.1.0-draft
  type: language-and-tool
  components:
    - name: cq
      type: cli
      language: go
      description: "cirql REPL and CLI tool"
    - name: cirql-core
      type: library
      language: go
      description: "Parser, AST, evaluator, source stages, and pipeline runtime"
    - name: api.cirql.io
      type: service
      language: go
      description: "Hosted cirql pipeline execution service"
  dependencies:
    go: ">=1.22"
    modules:
      - github.com/machinebox/graphql
      - github.com/alecthomas/participle/v2
      - github.com/chzyer/readline
      - github.com/fatih/color
  repository: https://github.com/nicerobot/cirql
  website: https://cirql.io
  license: MIT