API Reference

Metadata that all persisted resources must have.

The kaspr version. Defaults to the latest.

The number of desired instances.

The container image to use. Defaults to the corresponding image of the version configuration.

Bootstrap server names to connect to. This should be given as a comma separated list of <hostname>:<port> pairs.

TLS configuration. Provide an empty entry {} to enable TLS or exclude the field to disable TLS.

Authentication configuration for Kafka.

Compute Resources required by each instance on the application.

Disk storage required by each instance of the application.

Python package management configuration for automatic package installation via init containers.

The default replication factor for topics created by the application.

Default number of partitions for topics created by the application.

This setting controls whether or not creation of topics is allowed.

This setting controls whether or not the message scheduler is enabled.

This setting controls whether or not scheduler debug statistics are printed to log.

Default number of partitions for internal scheduler related topics. Defaults to general topic partition configuration if not set.

How often we save checkpoint to storage (and to changelog topic)

Number of days the dispatcher will look back from current date to seek starting a point for dispatching messages when an checkpoint is not found. This is mostly used during initial app deployment.

How often we checkpoint the dispacher's location in the timetable (in seconds).

How often we checkpoint the janitor's location in the timetable (in seconds).

How often the janitor attempts to clean.

Number of seconds the janitor trails current from the highwater timetable location.

This is the maximum write buffer size. It represents the amount of data to build up in memory before converting to a sorted on-disk file. The default is 64 MB.

Maximum number of write buffers (memtables) that can be built in memory at the same time.

Target size for files at level-1 in the LSM tree. Used to determine the size of the SST (Sorted String Table) files that RocksDB generates during compactions. The default is 64 MB.

Size for caching uncompressed data. Defauls to about 30% of application's total memory budget

Size for caching compressed data. Defaults to 254MB.

A Bloom filter in RocksDB is used to quickly check whether a key might be in an SST (Sorted String Table) file without actually reading the file, which can significantly improve read performance.

If set to true, index and filter blocks will be stored in block cache, together with all other data blocks.

Base HTTP path for serving web requests.

Port number between 1024 and 65535 to use for the web server.

Metadata that all persisted resources must have. An agent must declare a label kaspr.io/app with the value of the app name it will be run on.

Specification of the desired behavior of the KasprAgent.

A short description of the agent for documentation purpose.

Input configuration for the agent.

Output configuration for the agent. Processed events are sent out to the optional output.

Sequence of operations that handle input events. Processors are optional and can be used to modify or act on the input events before they are sent to the output.

If true, the agent will create the input topic/channel if it does not exist.

Buffering configuration. This allows the agent to collect multiple events before processing them.

Input topic configuration; mutually exclusive with channel.

Input channel configuration; mutually exclusive with topic.

Topic names to subscribe to. This can be a single topic or a comma-separated list of topics. Required, but mutually exclusive with pattern.

Regex pattern of topic names to subscribe to. Required, but mutually exclusive with name.

Serializer for the key portion of the kafka message. Must be one of raw, json, pickle, or binary.

Serializer for the value portion of the kafka message. Must be one of raw, json, pickle, or binary.

Name of the channel to subscribe to

Maximum number of events to buffer before processing them.

Time window for buffering events. Events will be processed after this duration, even if the max limit has not been reached. Examples: 5s, 1m, 1h.

List of topics to publish to.

Topic name

Wait for broker acknoledgement before considering the message sent.

Serializer for the key portion of the kafka message. Must be one of raw, json, pickle, or binary.

Serializer for the value portion of the kafka message. Must be one of raw, json, pickle, or binary.

Custom code to be used to select the key of the message. The code must define a function that takes a single argument and returns a value.

Custom code to be used to select the value of the message. The code must define a function that takes a single argument and returns a value.

Custom code to be used to select the partition number of the message. The code must define a function that takes a single argument and returns an integer.

Custom code to be used to select the headers of the message. The code must define a function that takes a single argument and returns a key and value map.

Custom code to be used to select the messages to be sent. The code must define a function that takes a single argument and returns a boolean value. Predicates that return true will be sent to the output topic, while those that return false will be discarded.

Name of the function that is invoked. If code defines multiple functions, the first function is used as the entrypoint. To avoid ambiguity, it is recommended to explicilty specify the entrypoint.

Python code to be executed. The code must define a valid Python function to use as an entrypoint.

Sequence of operations to run.

Initialization code to run.

Chain of operations to run on input events.

A unique name for the operation. This is used to identify the operation in the pipeline.

List of tables which are made available in the operation's function.

Takes an input value and returns an output value which is passed down to the next operation or the agent's output .

Takes an input value and determines if the event is eligible to be passed down to the next operation or the agent's output. If the filter function returns true, the event is passed down, otherwise it is discarded.

The name of the KasprTable resource.

The name of the parameter in which the table is made available in the operation's function.

Metadata that all persisted resources must have. A join must declare a label kaspr.io/app with the value of the app name it belongs to.

Specification of the desired behavior of the KasprJoin.

A short description of the join for documentation purposes.

Name of the left-side KasprTable resource. This is the "driving" table whose changes trigger foreign key extraction.

Name of the right-side KasprTable resource (the "lookup" table). Must belong to the same KasprApp.

Python function that extracts the join key from left-table values. The returned value is used to look up the corresponding record in the right table.

Join semantics. inner skips emission when the right side has no match. left emits with right=null when no match exists. Allowed values: inner, left.

Name of the output channel for joined results. KasprAgents reference this via input.channel.name.

Status of the referenced KasprApp. Contains name (string) and status (AppFound or AppNotFound).

Status of the referenced left KasprTable. Contains name (string) and status (TableFound or TableNotFound).

Status of the referenced right KasprTable. Contains name (string) and status (TableFound or TableNotFound).

Name of the ConfigMap containing the serialized join spec.

Hash of the current join configuration.

Metadata that all persisted resources must have.

Specification of the desired table configuration.

The name of the table.

A short description of the table.

If true, the table is global. Global tables have a full copy of the data on every node.

Python code that returns a default value when a key is not found in the table.

Serializer for the table key. Must be raw or json.

Serializer for the table value. Must be raw or json.

Number of partitions in the table's changelog topic.

Additional Kafka topic configurations applied when creating the table's changelog topic (e.g., retention.ms, cleanup.policy).

Advanced table configuration options.

Window configuration for the table. Enables tumbling or hopping windowed aggregation.

Tumbling window configuration. Windows are fixed-size, non-overlapping time intervals.

Hopping window configuration. Windows are fixed-size but advance by a step interval, so they may overlap.

How events are assigned to windows. Must be stream (event timestamp), now (wall-clock time), or custom (user-defined).

Python function that extracts a timestamp from the event for window assignment. Only used when relativeTo is custom.

The window duration (e.g., 30s, 1m, 1h, 1d).

How long to retain data for each window after it closes (e.g., 30s, 1m, 1h, 1d).

The window duration (e.g., 30s, 1m, 1h, 1d).

The interval at which new windows are created (e.g., 30s, 1m, 1h, 1d).

How long to retain data for each window after it closes (e.g., 30s, 1m, 1h, 1d).

Name of the web view interface

A short description of the web view for documentation purpose.

Describes the request endpoint listener.

Describes the web view's response output.

HTTP method to use for the request. Must be one of GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS

Web path to listen on.

Response content type. Must be one of text/html, application/json, text/plain, application/octet-stream

HTTP status code to return on success.

Headers as key and value pairs to return in the response

Custom function to transform the response body on success or error.

Custom function to determine the status code that is returned in response.

Custom function to determine the headers returned in response.

Function to transform response on success.

Function to transform response on error.

The type of authentication to use. Must be one of scram-sha-256, scram-sha-512, or plain

Username used for authentication.

Details of the kubernetes secret where the authentication password is stored.

The name of the Kubernetes secret resource containting the password.

The name of the key in the Secret under which the password is stored.

Storage type, must be ephemeral or persistent-claim

The storage class to use for dynamic volume allocation.

Specifies if the persistent volume claim has to be deleted when the app is undeployed.

List of Python packages to install (e.g., ["requests", "pandas>=2.0.0"]).

Base URL of the Python Package Index (e.g., https://pypi.org/simple).

Additional package index URLs to search.

Hosts to trust for SSL verification (skip certificate checks).

Credentials for private PyPI registry authentication.

Cache configuration for storing installed packages across restarts.

Installation behavior policy (retries, timeout, failure handling).

Resource requirements for the init container that installs packages.

Cache backend type. Must be pvc or gcs. When set to pvc, uses a PersistentVolumeClaim (requires RWX storage class). When set to gcs, uses Google Cloud Storage.

Enable PVC-based caching. When false (default), uses ephemeral emptyDir storage per pod. Only applies when type is pvc.

The storage class to use for the packages cache PVC.

Size of the packages cache PVC (e.g., 256Mi, 1Gi).

Access mode for the cache PVC. Only ReadWriteMany is currently supported for shared package cache.

Whether to delete the PVC when the app is undeployed.

GCS cache configuration. Required when type is gcs.

GCS bucket name for storing cached package archives.

Key prefix for cached archives in the bucket.

Maximum archive size to upload to GCS. Archives exceeding this size are skipped.

Reference to a Secret containing a GCS service account key JSON file.

Name of the Secret containing the service account key.

Key within the Secret containing the JSON file.

Number of retry attempts for package installation.

Timeout in seconds for package installation. Minimum value is 60.

Behavior on installation failure. block prevents the pod from starting. allow lets the pod start without the packages.

Reference to a Secret containing PyPI registry credentials.

Name of the Secret containing credentials.

Key in the Secret for the username.

Key in the Secret for the password.