Concepts
Overview
The four primitives that make up a Thyme feature pipeline.
Thyme is built around four primitives. You use all four when defining a feature pipeline.
| Primitive | What it is | Defined with |
|---|---|---|
| Dataset | A named, schema'd stream of events | @dataset |
| Pipeline | A windowed aggregation over a dataset | @pipeline + @inputs |
| Featureset | A collection of model-ready features | @featureset + @extractor |
| Source | A connector that feeds raw events into a dataset | @source |
How they connect
Source
Source
Postgres · Kafka · Iceberg
Dataset
Raw Dataset
@dataset
Pipeline
Pipeline
groupby · aggregate
Dataset
Aggregated Dataset
windowed stats
Featureset
Featureset
@extractor · model features
A source ingests raw events into a dataset. A pipeline aggregates that dataset into another dataset. An extractor reads the aggregated dataset and composes the final features that your model queries.
Learn more
- Datasets - schema, keys, timestamps
- Pipelines - windowed aggregations, groupby
- Featuresets - features, extractors, IDs
- Sources - connectors, CDC modes, polling