Skip to main content
CommunityTeamEnterprise

Differences from jq

While kJQ is based on jq, there are several small yet important differences designed to optimize kJQ for Kafka data processing workloads.

Function and transform names​

kJQ uses kebab-case naming conventions instead of jq's mixed naming styles:

kJQjq
to-doubletonumber
to-longtonumber
to-stringtostring
to-uuid(not available)
upper-caseascii_upcase
lower-caseascii_downcase
from-datefromdateiso8601
parse-jsonfromjson
is-emptyisempty

Extended data types​

kJQ supports additional data types beyond standard JSON, particularly useful for Kafka and AVRO+Protobuf data:

Tagged literals​

kJQ introduces tagged literal syntax for rich data types:

#dt "2025-01-01T10:30:00Z"                    # Date literal
#uuid "550e8400-e29b-41d4-a716-446655440000"  # UUID literal

Standard jq only supports basic JSON literals and requires parsing functions for dates and UUIDs.

Native data types​

  • date - First-class date support with ISO 8601 parsing
  • double - Explicit double-precision numbers (distinct from general numbers)
  • uuid - Native UUID type with validation
  • keyword - Clojure-style keywords (e.g., :topic)

Time and duration operations​

kJQ provides built-in temporal operations optimized for stream processing:

Current time access​

now                   # Current timestamp
now - pt1h            # One hour ago  
now + pt30m           # 30 minutes from now

ISO 8601 Duration syntax​

pt5m                  # 5 minutes
pt1h                  # 1 hour  
pt2d                  # 2 days
pt1w                  # 1 week

Standard jq requires external date parsing and manual timestamp arithmetic for similar operations.

Kafka-specific features​

Record metadata access​

kJQ provides direct access to enriched Kafka record metadata:

.size                # Total serialized record size
.key-size            # Key size in bytes
.value-size          # Value size in bytes
.partition           # Record topic partition
.topic               # Record topic name
.offset              # Record offset
.serdes              # SerDes metadata
.registry            # Schema registry metadata
.key                 # Record key
.value               # Record value
.headers             # Record headers

Mathematical operations​

Enhanced arithmetic​

kJQ includes additional mathematical operators:

kJQjqPurpose
quot/Integer division (quotient only)
rem%Remainder operation
mod%Modulo operation

Both rem and mod are available alongside the standard % operator, providing clarity for different mathematical contexts.

Missing jq features​

Several advanced jq features are not available in kJQ, keeping the language focused on stream processing:

  • path()
  • paths
  • leaf_paths
  • range()
  • until()
  • while()
  • with_entries()
  • from_entries()
  • to_entries()
  • truncate_stream()
  • fromstream()
  • tostream()
  • = (assignment)
  • |= (update)
  • +=, -=, etc.
  • as $var
  • def (function definitions)

Collection functions​

kJQ introduces new collection testing functions not found in standard jq:

  • within(values...) - Test if current value exists within the provided list
  • inside(string) - Test if current value is contained within the given string
.value.country_code | within("US", "CA", "UK")
.value.error_code | inside("TIMEOUT_ERROR_CODE_001")

Simplified syntax​

kJQ removes some of jq's more complex syntax patterns:

No variable assignment​

Unlike jq, kJQ doesn't support variable binding with as or function definitions with def. This keeps expressions focused on data transformation rather than computation.

No update operations​

kJQ is designed for filtering and selection rather than data modification, so update operators (|=, +=) are not available.

Streamlined function set​

kJQ provides a curated set of functions optimized for Kafka message processing, removing many of jq's more specialized or rarely-used functions to maintain simplicity and performance.

Array and object construction​

Construction of more complex data types like array and objects such as [1, 2, 3] or {"foo": "bar"} is not available. This keeps expressions focused on data transformation rather than computation.