API Reference

Metadata that all persisted resources must have.

The kaspr version. Defaults to the latest.

The number of desired instances.

The container image to use. Defaults to the corresponding image of the version configuration.

Bootstrap server names to connect to. This should be given as a comma separated list of <hostname>:<port> pairs.

TLS configuration. Provide an empty entry {} to enable TLS or exclude the field to disable TLS.

Authentication configuration for Kafka.

Compute Resources required by each instance on the application.

Disk storage required by each instance of the application.

The default replication factor for topics created by the application.

Default number of partitions for topics created by the application.

This setting controls whether or not creation of topics is allowed.

This setting controls whether or not the message scheduler is enabled.

This setting controls whether or not scheduler debug statistics are printed to log.

Default number of partitions for internal scheduler related topics. Defaults to general topic partition configuration if not set.

How often we save checkpoint to storage (and to changelog topic)

Number of days the dispatcher will look back from current date to seek starting a point for dispatching messages when an checkpoint is not found. This is mostly used during initial app deployment.

How often we checkpoint the dispacher's location in the timetable (in seconds).

How often we checkpoint the janitor's location in the timetable (in seconds).

How often the janitor attempts to clean.

Number of seconds the janitor trails current from the highwater timetable location.

This is the maximum write buffer size. It represents the amount of data to build up in memory before converting to a sorted on-disk file. The default is 64 MB.

Maximum number of write buffers (memtables) that can be built in memory at the same time.

Target size for files at level-1 in the LSM tree. Used to determine the size of the SST (Sorted String Table) files that RocksDB generates during compactions. The default is 64 MB.

Size for caching uncompressed data. Defauls to about 30% of application's total memory budget

Size for caching compressed data. Defaults to 254MB.

A Bloom filter in RocksDB is used to quickly check whether a key might be in an SST (Sorted String Table) file without actually reading the file, which can significantly improve read performance.

If set to true, index and filter blocks will be stored in block cache, together with all other data blocks.

Base HTTP path for serving web requests.

Port number between 1024 and 65535 to use for the web server.

Metadata that all persisted resources must have. An agent must declare a label kaspr.io/app with the value of the app name it will be run on.

Specification of the desired behavior of the KasprAgent.

A short description of the agent for documentation purpose.

Input configuration for the agent.

Output configuration for the agent. Processed events are sent out to the optional output.

Sequence of operations that handle input events. Processors are optional and can be used to modify or act on the input events before they are sent to the output.

Input topic configuration; mutually exclusive with channel.

Input channel configuration; mutually exclusive with topic.

Topic names to subscribe to. This can be a single topic or a comma-separated list of topics. Required, but mutually exclusive with pattern.

Regex pattern of topic names to subscribe to. Required, but mutually exclusive with name.

Serializer for the key portion of the kafka message. Must be one of raw, json, pickle, or binary.

Serializer for the value portion of the kafka message. Must be one of raw, json, pickle, or binary.

Name of the channel to subscribe to

List of topics to publish to.

Topic name

Wait for broker acknoledgement before considering the message sent.

Serializer for the key portion of the kafka message. Must be one of raw, json, pickle, or binary.

Serializer for the value portion of the kafka message. Must be one of raw, json, pickle, or binary.

Custom code to be used to select the key of the message. The code must define a function that takes a single argument and returns a value.

Custom code to be used to select the value of the message. The code must define a function that takes a single argument and returns a value.

Custom code to be used to select the partition number of the message. The code must define a function that takes a single argument and returns an integer.

Custom code to be used to select the headers of the message. The code must define a function that takes a single argument and returns a key and value map.

Custom code to be used to select the messages to be sent. The code must define a function that takes a single argument and returns a boolean value. Predicates that return true will be sent to the output topic, while those that return false will be discarded.

Name of the function that is invoked. If code defines multiple functions, the first function is used as the entrypoint. To avoid ambiguity, it is recommended to explicilty specify the entrypoint.

Python code to be executed. The code must define a valid Python function to use as an entrypoint.

Sequence of operations to run.

Initialization code to run.

Chain of operations to run on input events.

A unique name for the operation. This is used to identify the operation in the pipeline.

List of tables which are made available in the operation's function.

Takes an input value and returns an output value which is passed down to the next operation or the agent's output .

Takes an input value and determines if the event is eligible to be passed down to the next operation or the agent's output. If the filter function returns true, the event is passed down, otherwise it is discarded.

The name of the KasprTable resource.

The name of the parameter in which the table is made available in the operation's function.

Name of the web view interface

A short description of the web view for documentation purpose.

Describes the request endpoint listener.

Describes the web view's response output.

HTTP method to use for the request. Must be one of GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS

Web path to listen on.

Describes the request endpoint listener.

Response content type. Must be one of text/html, application/json, text/plain, application/octet-stream

HTTP status code to return on success.

Headers as key and value pairs to return in the response

Custom function to transform the response body on success or error.

Custom function to determine the status code that is returned in response.

Custom function to determine the headers returned in response.

Function to transform response on success.

Function to transform response on error.

The type of authentication to use. Must be one of scram-sha-256, scram-sha-512, or plain

Username used for authentication.

Details of the kubernetes secret where the authentication password is stored.

The name of the Kubernetes secret resource containting the password.

The name of the key in the Secret under which the password is stored.

Storage type, must be ephemeral or persistent-claim

The storage class to use for dynamic volume allocation.

Specifies if the persistent volume claim has to be deleted when the app is undeployed.