Model datasets

Datasets are static reference files you publish with your model. Use them for lookup tables, mappings, and other supporting data that your model run views need.

Datasets are different from inputs: inputs feed model runs, while datasets support how results are displayed and explored in model run views.

When to use datasets

Use a dataset when the data:

  • Changes infrequently
  • Is shared across many runs
  • Supports analysis in model run views
  • Doesn’t need to trigger model runs

If the data is part of each run’s forecast logic, configure it as an input instead.

Configure datasets

Add datasets in the datasets array in your .pd4castrrc.json file. Each dataset points to a local file in your project.

FieldTypeRequiredDescription
keystringYesUnique identifier for the dataset.
filestringYesPath to the local dataset file, relative to your project root.
fileFormatstringNoFile format: json, csv, or parquet. If omitted, format is inferred from the file extension.
sourcestringNoOptional dataset source ID. If omitted, the default shared source is used.

Example configuration

This example publishes three datasets with different file formats.

{
  "datasets": [
    {
      "key": "duid_info",
      "file": "datasets/duid_info.parquet"
    },
    {
      "key": "regional_boundaries",
      "file": "datasets/regional_boundaries.csv"
    },
    {
      "key": "price_overrides",
      "file": "datasets/price_overrides.json"
    }
  ]
}

Publish and update datasets

When you run pd4castr publish, the CLI uploads every dataset listed in your configuration.

To update dataset contents, edit the local file and publish again.

File format rules

Supported formats are:

  • csv
  • parquet
  • json

If you omit fileFormat, the CLI infers it from the file extension. If it can’t infer a supported format, publish fails and prompts you to set fileFormat explicitly.

Next steps