creating-timeplus-apps

name	creating-timeplus-apps
description	Use when creating, packaging, or installing a Timeplus app (.tpapp) — converting existing SQL resources and dashboards into an installable app package, writing manifests, applying template variables, or debugging install failures

Creating Timeplus Apps

Overview

A Timeplus app (.tpapp) is a zip archive that bundles a streaming data pipeline — DDL resources (streams, views, materialized views) plus dashboards — into a single installable unit. The installer provisions everything in order and rolls back on failure.

Directory Structure

my-app/
├── manifest.yaml          # required
├── ddl/
│   ├── 001_first.sql      # executed in filename order
│   ├── 002_second.sql
│   └── ...
└── dashboards/
    └── main.json          # array of panel objects

Package it:

cd my-app && zip -r ../my-app.tpapp manifest.yaml ddl/ dashboards/

Install via API:

curl -X POST http://localhost:8000/default/api/v1beta2/apps/install \
  -F "file=@my-app.tpapp"

Override config: values at install time with config[<key>]=<value> form fields (multipart) — the neutron handler parses any form field matching config[*] into the rendered config map:

curl -X POST http://localhost:8000/default/api/v1beta2/apps/install \
  -F "file=@my-app.tpapp" \
  -F "config[strategy]=sign" \
  -F "config[num_stocks]=5"

For JSON-body installs (URL fetch), use {"url": "...", "config": {"strategy": "sign"}}.

manifest.yaml

package_format_version: 1          # must be 1
id: io.example.my-app              # reverse-domain, unique
name: My App
version: 1.0.0
author: Acme
description: What this app does.
icon: "data:image/png;base64,..."  # optional — base64 data URI; frontend shows default when absent
categories:                        # optional — free-form tags for discovery/filtering
  - security
  - observability
db_name: my_app                    # ^[a-z][a-z0-9_]{0,31}$, used as-is

config:                            # optional — user-supplied parameters
  - key: websocket_url
    type: string
    required: true
    description: WebSocket feed URL
  - key: api_key
    type: string
    required: true
    secret: true                   # mask value in UI; stored as IsSecret
    description: API key
  - key: timeout
    type: integer
    required: false
    default: "30"
    description: Connection timeout in seconds
  - key: tls_enabled
    type: bool
    required: false
    default: "false"
    description: Enable TLS
  - key: topics
    type: list
    required: true
    description: Kafka topics (JSON array of strings, e.g. '["a","b"]')
  - key: broker_type
    type: choice
    required: true
    description: Message broker
    options:
      - kafka
      - pulsar
      - redpanda
  - key: features
    type: multi_choice
    required: false
    default: '["metrics"]'
    description: Features to enable
    options:
      - metrics
      - tracing
      - alerting

python_packages:                   # optional — installed before any DDL runs
  - json5>=0.9.6
  - websocket-client>=1.4.0

resources:                         # executed in listed order
  - file: ddl/001_source.sql
    type: external_stream
    name: raw_feed
  - file: ddl/002_events.sql
    type: stream
    name: events
  - file: ddl/003_mv.sql
    type: materialized_view
    name: mv_events

dashboards:
  - file: dashboards/main.json
    name: My Dashboard
    description: Real-time view

Resource types

type	DDL verb	idempotent form	rolled back with
stream	CREATE STREAM	CREATE STREAM IF NOT EXISTS	DROP STREAM
external_stream	CREATE EXTERNAL STREAM	CREATE EXTERNAL STREAM IF NOT EXISTS	DROP STREAM
mutable_stream	CREATE MUTABLE STREAM	CREATE MUTABLE STREAM IF NOT EXISTS	DROP STREAM
materialized_view	CREATE MATERIALIZED VIEW	CREATE MATERIALIZED VIEW IF NOT EXISTS	DROP VIEW
view	CREATE VIEW	CREATE VIEW IF NOT EXISTS	DROP VIEW
external_table	CREATE TABLE	CREATE TABLE IF NOT EXISTS	DROP TABLE
udf	CREATE FUNCTION	CREATE OR REPLACE FUNCTION (see below)	DROP FUNCTION
task	CREATE TASK	CREATE TASK IF NOT EXISTS	DROP TASK
alert	CREATE ALERT	CREATE ALERT IF NOT EXISTS	DROP ALERT
input	CREATE INPUT	CREATE INPUT IF NOT EXISTS	DROP INPUT
dictionary	CREATE DICTIONARY	CREATE DICTIONARY IF NOT EXISTS	DROP DICTIONARY
format_schema	CREATE FORMAT SCHEMA	CREATE FORMAT SCHEMA IF NOT EXISTS	DROP FORMAT SCHEMA
named_collection	CREATE NAMED COLLECTION	CREATE NAMED COLLECTION IF NOT EXISTS	DROP NAMED COLLECTION

DDL Template Variables

DDL files are rendered with Go text/template using {{ }} delimiters.

Expression	Expands to
{{ .DB }}	The resolved database name (value of db_name)
{{ .Config.key_name }}	Value of config key (after defaults applied)

Always use dot notation for config values — {{ .Config.my_key }}, never {{ index .Config "my_key" }}.

-- ddl/002_events.sql
CREATE STREAM IF NOT EXISTS {{ .DB }}.events (
  id       string,
  payload  string
)
TTL to_datetime(_tp_time) + INTERVAL 24 HOUR;

-- ddl/001_source.sql
CREATE EXTERNAL STREAM IF NOT EXISTS {{ .DB }}.raw_feed (msg string)
SETTINGS url='{{ .Config.websocket_url }}', type='websocket';

Idempotency: every CREATE must be re-runnable

App upgrades re-run every DDL file against the existing database — the installer does not drop resources first. A CREATE that fails on the second run breaks upgrade. Every DDL file in apps/*/ddl/ must use one of the following forms:

Resource	Required form	Why
Everything except udf	CREATE … IF NOT EXISTS <name>	Re-running is a no-op. Existing rows, downstream consumers, and the resource UUID are preserved — critical for view and materialized_view, which are referenced by name from other resources and from dashboards.
udf	CREATE OR REPLACE FUNCTION <name>	UDFs are global (no database qualifier — see udf) and their body is typically what changes in an upgrade. OR REPLACE hot-swaps the implementation; IF NOT EXISTS would silently keep the old code on upgrade.

Do not use CREATE OR REPLACE VIEW. A view is a dependency for downstream MVs, queries, and dashboards. OR REPLACE would drop and recreate it, breaking anything referencing the view during the brief recreation window and on schema changes. Use CREATE VIEW IF NOT EXISTS — if the view definition needs to change, bump the app version and treat it as a migration (drop + recreate explicitly, or version the view name).

Forms that do NOT accept IF NOT EXISTS:

• SYSTEM INSTALL PYTHON PACKAGE — parser rejects IF NOT EXISTS. Don't put this in DDL; declare packages in manifest.yaml under python_packages (see Python packages).
• CREATE FUNCTION IF NOT EXISTS db.fn and CREATE FORMAT SCHEMA IF NOT EXISTS db.fs — both reject the db. prefix. UDFs and format schemas live in a global namespace; never qualify them with {{ .DB }}..

Dashboard Template Variables

Dashboard JSON is rendered with [[ ]] delimiters (to avoid collision with the frontend's {{filter_*}} runtime variables).

{
  "viz_content": "SELECT * FROM [[ .DB ]].events WHERE _tp_time > now() - {{filter_time_range}}"
}

Expression	Expands to
[[ .DB ]]	Database name
[[ .Config.key ]]	Config value
{{filter_*}}	Left as-is — resolved by the frontend at query time

Template processing runs before JSON parsing. This means template expressions inside JSON string values may contain unescaped " characters — the file does not need to be valid JSON before substitution.

Template Functions (Sprig)

Both DDL ({{ }}) and dashboard ([[ ]]) templates have the full Sprig function library available — the same library used by Helm. Use these to manipulate config values at install time.

Working with list config values

Config keys of type list are stored as a JSON array string (e.g. ["BTC-USD","ETH-USD","SOL-USD"]). Use fromJson to parse them before passing to other functions.

Render as comma-separated string (e.g. for dashboard selector inlineValues):

"inlineValues": "[[ join "," (fromJson .Config.product_ids) ]]"

→ "inlineValues": "BTC-USD,ETH-USD,SOL-USD"

Embed directly as JSON array (e.g. in a DDL Python string):

product_ids = '{{ .Config.product_ids }}'

→ product_ids = '["BTC-USD","ETH-USD","SOL-USD"]'

Get the first element (e.g. for a selector defaultValue):

"defaultValue": "[[ index (fromJson .Config.product_ids) 0 ]]"

→ "defaultValue": "BTC-USD"

Commonly used functions

Function	Example	Result
join sep list	join "," (fromJson .Config.topics)	a,b,c
fromJson s	fromJson .Config.product_ids	parsed slice
default val s	default "30" .Config.timeout	config value or fallback
upper s	upper .Config.env	PRODUCTION
lower s	lower .Config.env	production
trim s	trim .Config.url	strips whitespace
replace old new s	replace "-" "_" .Config.id	BTC_USD
splitList sep s	splitList "," .Config.tags	["a","b","c"]
first list	first (fromJson .Config.ids)	first element
last list	last (fromJson .Config.ids)	last element
len list	len (fromJson .Config.ids)	count

Full function reference: https://masterminds.github.io/sprig/

Dashboard JSON Reference

For the full dashboard panel specification — all chart types, viz_config fields, control panels, position grid, update modes, and working examples — see:

skill/references/dashboard-spec.md

This covers:

• Panel structure (id, title, position, viz_type, viz_content, viz_config)
• 12-column position grid and common width/height values
• Template variables ([[ .DB ]] vs {{filter_*}})
• Control panels: selector (dropdown) and text_input
• Chart types: line, area, bar, column, singleValue, table, ohlc, geo, md, grammar (3.2+ — generic Vistral-grammar-driven viz: scatter, layered marks, band-axis bars, stacked area, custom transforms, etc.). For the underlying VistralSpec grammar (marks, transforms, scales, encode channels), see the Vistral skill: vistral/agentskill/SKILL.md.
• All viz_config.config fields per chart type with defaults
• updateMode ("all" / "key" / "time") — when to use each
• Default color palette
• Common mistakes

Timeplus SQL Reference

For writing correct Timeplus streaming SQL in DDL files, refer to the Timeplus SQL skill: https://github.com/timeplus-io/AgentSkills/tree/main/timeplus-sql-guide

This covers streaming query syntax, window functions, tumble/hop aggregations, _tp_time semantics, and other Timeplus-specific SQL features used in streams, views, and materialized views.

File Ordering and Dependencies

Name DDL files with a numeric prefix so they execute in dependency order:

001_source_stream.sql            ← external streams / sources
002_target_stream.sql            ← destination streams
003_mv_extract.sql               ← materialized views (depend on streams)
004_v_aggregated.sql             ← views (depend on streams/MVs)

Config Types

Seven types are supported. Omitting type defaults to string.

Type	Stored as	Valid example	Notes
string	plain string	"localhost:9092"	Default type
integer	decimal string	"30", "-5"	Must be a whole number
float	decimal string	"3.14", "30"	Decimal or whole
bool	"true" or "false"	"true"	No other values accepted