CommunityEnterprise

Differences from jq

While kJQ is based on jq, there are several small yet important differences designed to optimize kJQ for Kafka data processing workloads.

Function and transform names

kJQ uses kebab-case naming conventions instead of jq's mixed naming styles:

kJQ	jq
to-double	tonumber
to-long	tonumber
to-string	tostring
to-uuid	(not available)
upper-case	ascii_upcase
lower-case	ascii_downcase
from-date	fromdateiso8601
parse-json	fromjson
is-empty	isempty

Extended data types

kJQ supports additional data types beyond standard JSON, particularly useful for Kafka and AVRO+Protobuf data:

Tagged literals

kJQ introduces tagged literal syntax for rich data types:

#dt "2025-01-01T10:30:00Z"                    # Date literal
#uuid "550e8400-e29b-41d4-a716-446655440000"  # UUID literal

Standard jq only supports basic JSON literals and requires parsing functions for dates and UUIDs.

Native data types

• date - First-class date support with ISO 8601 parsing
• double - Explicit double-precision numbers (distinct from general numbers)
• uuid - Native UUID type with validation
• keyword - Clojure-style keywords (e.g., :topic)

Time and duration operations

kJQ provides built-in temporal operations optimized for stream processing:

Current time access

now                   # Current timestamp
now - pt1h            # One hour ago  
now + pt30m           # 30 minutes from now

ISO 8601 Duration syntax

pt5m                  # 5 minutes
pt1h                  # 1 hour  
pt2d                  # 2 days
pt1w                  # 1 week

Standard jq requires external date parsing and manual timestamp arithmetic for similar operations.

Kafka-specific features

Record metadata access

kJQ provides direct access to enriched Kafka record metadata:

.size                # Total serialized record size
.key-size            # Key size in bytes
.value-size          # Value size in bytes
.partition           # Record topic partition
.topic               # Record topic name
.offset              # Record offset
.serdes              # SerDes metadata
.registry            # Schema registry metadata
.key                 # Record key
.value               # Record value
.headers             # Record headers

Mathematical operations

Enhanced arithmetic

kJQ includes additional mathematical operators:

kJQ	jq	Purpose
quot	/	Integer division (quotient only)
rem	%	Remainder operation
mod	%	Modulo operation

Both rem and mod are available alongside the standard % operator, providing clarity for different mathematical contexts.

Missing jq features

Several advanced jq features are not available in kJQ, keeping the language focused on stream processing:

• path()
• paths
• leaf_paths
• range()
• until()
• while()
• with_entries()
• from_entries()
• to_entries()
• truncate_stream()
• fromstream()
• tostream()
• = (assignment)
• |= (update)
• +=, -=, etc.
• as $var
• def (function definitions)

Collection functions

kJQ introduces new collection testing functions not found in standard jq:

• within(values...) - Test if current value exists within the provided list
• inside(string) - Test if current value is contained within the given string

.value.country_code | within("US", "CA", "UK")
.value.error_code | inside("TIMEOUT_ERROR_CODE_001")

Simplified syntax

kJQ removes some of jq's more complex syntax patterns:

No variable assignment

Unlike jq, kJQ doesn't support variable binding with as or function definitions with def. This keeps expressions focused on data transformation rather than computation.

No update operations

kJQ is designed for filtering and selection rather than data modification, so update operators (|=, +=) are not available.

Streamlined function set

kJQ provides a curated set of functions optimized for Kafka message processing, removing many of jq's more specialized or rarely-used functions to maintain simplicity and performance.

Array and object construction

Construction of more complex data types like array and objects such as [1, 2, 3] or {"foo": "bar"} is not available. This keeps expressions focused on data transformation rather than computation.