Dataset Config¶

pydantic model dsgrid.config.dataset_config.DatasetConfigModel[source]¶

Represents dataset configurations.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

digraph "Entity Relationship Diagram created by erdantic" {
graph [fontcolor=gray66,
fontname="Times New Roman,Times,Liberation Serif,serif",
fontsize=9,
nodesep=0.5,
rankdir=LR,
ranksep=1.5
];
node [fontname="Times New Roman,Times,Liberation Serif,serif",
fontsize=14,
label="\N",
shape=plain
];
edge [dir=both];
"dsgrid.config.dataset_config.DatasetConfigModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>DatasetConfigModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>dataset_id</td><td port="dataset_id">str</td></tr><tr><td>data_layout</td><td port="data_layout">UserDataLayout | None</td></tr><tr><td>registry_data_layout</td><td port="registry_data_layout">RegistryDataLayout | None</td></tr><tr><td>dataset_type</td><td port="dataset_type">InputDatasetType</td></tr><tr><td>dataset_qualifier_metadata</td><td port="dataset_qualifier_metadata">Union[QuantityModel, GrowthRateModel]</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>sector_description</td><td port="sector_description">str | None</td></tr><tr><td>data_source</td><td port="data_source">str | None</td></tr><tr><td>data_source_date</td><td port="data_source_date">str | None</td></tr><tr><td>data_source_version</td><td port="data_source_version">str | None</td></tr><tr><td>data_source_authors</td><td port="data_source_authors">list[str] | None</td></tr><tr><td>data_source_doi_url</td><td port="data_source_doi_url">str | None</td></tr><tr><td>origin_creator</td><td port="origin_creator">str | None</td></tr><tr><td>origin_organization</td><td port="origin_organization">str | None</td></tr><tr><td>origin_contributors</td><td port="origin_contributors">list[str] | None</td></tr><tr><td>origin_project</td><td port="origin_project">str | None</td></tr><tr><td>user_defined_metadata</td><td port="user_defined_metadata">dict[str, Any]</td></tr><tr><td>tags</td><td port="tags">list[str] | None</td></tr><tr><td>data_classification</td><td port="data_classification">DataClassificationType</td></tr><tr><td>enable_unit_conversion</td><td port="enable_unit_conversion">bool</td></tr><tr><td>use_project_geography_time_zone</td><td port="use_project_geography_time_zone">bool</td></tr><tr><td>dimensions</td><td port="dimensions">list[Union[DimensionModel, DateTimeDimensionModel, AnnualTimeDimensionModel, RepresentativePeriodTimeDimensionModel, DatetimeExternalTimeZoneDimensionModel, IndexTimeDimensionModel, NoOpTimeDimensionModel]]</td></tr><tr><td>dimension_references</td><td port="dimension_references">list[DimensionReferenceModel]</td></tr><tr><td>trivial_dimensions</td><td port="trivial_dimensions">list[DimensionType]</td></tr></table>>,
tooltip="dsgrid.config.dataset_config.DatasetConfigModel

Represents dataset configurations.
"];
"dsgrid.config.dataset_config.GrowthRateModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>GrowthRateModel</b></td></tr><tr><td>dataset_qualifier_type</td><td port="dataset_qualifier_type">Literal[DatasetQualifierType.GROWTH_RATE]</td></tr><tr><td>growth_rate_type</td><td port="growth_rate_type">GrowthRateType</td></tr></table>>,
tooltip="dsgrid.config.dataset_config.GrowthRateModel

Base data model for all dsgrid data models
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dataset_qualifier_metadata:e -> "dsgrid.config.dataset_config.GrowthRateModel":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dataset_config.QuantityModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>QuantityModel</b></td></tr><tr><td>dataset_qualifier_type</td><td port="dataset_qualifier_type">Literal[DatasetQualifierType.QUANTITY]</td></tr></table>>,
tooltip="dsgrid.config.dataset_config.QuantityModel

Base data model for all dsgrid data models
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dataset_qualifier_metadata:e -> "dsgrid.config.dataset_config.QuantityModel":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dataset_config.RegistryDataLayout" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>RegistryDataLayout</b></td></tr><tr><td>table_format</td><td port="table_format">TableFormat</td></tr><tr><td>value_format</td><td port="value_format">ValueFormat</td></tr><tr><td>pivoted_dimension_type</td><td port="pivoted_dimension_type">DimensionType | None</td></tr></table>>,
tooltip="dsgrid.config.dataset_config.RegistryDataLayout

Data layout stored in the dsgrid registry (without file paths).
"];
"dsgrid.config.dataset_config.DatasetConfigModel":registry_data_layout:e -> "dsgrid.config.dataset_config.RegistryDataLayout":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dataset_config.UserDataLayout" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>UserDataLayout</b></td></tr><tr><td>data_file</td><td port="data_file">FileSchema</td></tr><tr><td>lookup_data_file</td><td port="lookup_data_file">FileSchema | None</td></tr><tr><td>missing_associations</td><td port="missing_associations">list[str]</td></tr><tr><td>table_format</td><td port="table_format">TableFormat</td></tr><tr><td>value_format</td><td port="value_format">ValueFormat</td></tr><tr><td>pivoted_dimension_type</td><td port="pivoted_dimension_type">DimensionType | None</td></tr></table>>,
tooltip="dsgrid.config.dataset_config.UserDataLayout

User-defined data layout for dataset registration.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":data_layout:e -> "dsgrid.config.dataset_config.UserDataLayout":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.AnnualTimeDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>AnnualTimeDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr><tr><td>measurement_type</td><td port="measurement_type">MeasurementType</td></tr><tr><td>ranges</td><td port="ranges">list[AnnualRangeModel]</td></tr><tr><td>include_leap_day</td><td port="include_leap_day">bool</td></tr></table>>,
tooltip="dsgrid.config.dimensions.AnnualTimeDimensionModel

Defines an annual time dimension where timestamps are years.
Each \
value associated with a year represents the MEASUREMENT_TYPE over the entire year.
i.e., MEASUREMENT_TYPE = total means the \
value is the total over the year, not over the range frequency.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.AnnualTimeDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DateTimeDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>DateTimeDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr><tr><td>time_zone_format</td><td port="time_zone_format">Union[AlignedTimeSingleTimeZone, LocalTimeMultipleTimeZones]</td></tr><tr><td>measurement_type</td><td port="measurement_type">MeasurementType</td></tr><tr><td>ranges</td><td port="ranges">list[TimeRangeModel]</td></tr><tr><td>time_interval_type</td><td port="time_interval_type">TimeIntervalType</td></tr><tr><td>time_column</td><td port="time_column">str</td></tr><tr><td>localize_to_time_zone</td><td port="localize_to_time_zone">bool</td></tr></table>>,
tooltip="dsgrid.config.dimensions.DateTimeDimensionModel

Defines a time dimension where timestamps translate to datetime objects.&#\
xA;"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.DateTimeDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>DatetimeExternalTimeZoneDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr><tr><td>time_zone_format</td><td port="time_zone_format">Union[AlignedTimeSingleTimeZone, LocalTimeMultipleTimeZones]</td></tr><tr><td>measurement_type</td><td port="measurement_type">MeasurementType</td></tr><tr><td>ranges</td><td port="ranges">list[TimeRangeModel]</td></tr><tr><td>time_interval_type</td><td port="time_interval_type">TimeIntervalType</td></tr></table>>,
tooltip="dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel

Defines a time dimension where timestamps are tz-naive \
and require localizing to a time zone
using a time zone column.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>DimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>filename</td><td port="filename">str | None</td></tr><tr><td>file_hash</td><td port="file_hash">str | None</td></tr><tr><td>records</td><td port="records">list</td></tr></table>>,
tooltip="dsgrid.config.dimensions.DimensionModel

Defines a non-time dimension
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.DimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DimensionReferenceModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>DimensionReferenceModel</b></td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str</td></tr><tr><td>version</td><td port="version">str</td></tr></table>>,
tooltip="dsgrid.config.dimensions.DimensionReferenceModel

Reference to a dimension stored in the registry
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimension_references:e -> "dsgrid.config.dimensions.DimensionReferenceModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.IndexTimeDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>IndexTimeDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr><tr><td>measurement_type</td><td port="measurement_type">MeasurementType</td></tr><tr><td>ranges</td><td port="ranges">list[IndexRangeModel]</td></tr><tr><td>time_interval_type</td><td port="time_interval_type">TimeIntervalType</td></tr></table>>,
tooltip="dsgrid.config.dimensions.IndexTimeDimensionModel

Defines a time dimension where timestamps are indices and requires converting \
to datetime.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.IndexTimeDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.NoOpTimeDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>NoOpTimeDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr></table>>,
tooltip="dsgrid.config.dimensions.NoOpTimeDimensionModel

Defines a NoOp time dimension.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.NoOpTimeDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.RepresentativePeriodTimeDimensionModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>RepresentativePeriodTimeDimensionModel</b></td></tr><tr><td>id</td><td port="id">int | None</td></tr><tr><td>version</td><td port="version">str | None</td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType</td></tr><tr><td>dimension_id</td><td port="dimension_id">str | None</td></tr><tr><td>module</td><td port="module">str</td></tr><tr><td>class_name</td><td port="class_name">str</td></tr><tr><td>cls</td><td port="cls">Any</td></tr><tr><td>description</td><td port="description">str | None</td></tr><tr><td>time_type</td><td port="time_type">TimeDimensionType</td></tr><tr><td>measurement_type</td><td port="measurement_type">MeasurementType</td></tr><tr><td>format</td><td port="format">RepresentativePeriodFormat</td></tr><tr><td>ranges</td><td port="ranges">list[MonthRangeModel]</td></tr><tr><td>time_interval_type</td><td port="time_interval_type">TimeIntervalType</td></tr></table>>,
tooltip="dsgrid.config.dimensions.RepresentativePeriodTimeDimensionModel

Defines a representative time dimension.
"];
"dsgrid.config.dataset_config.DatasetConfigModel":dimensions:e -> "dsgrid.config.dimensions.RepresentativePeriodTimeDimensionModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.file_schema.FileSchema" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>FileSchema</b></td></tr><tr><td>path</td><td port="path">str | None</td></tr><tr><td>columns</td><td port="columns">list[Column]</td></tr><tr><td>ignore_columns</td><td port="ignore_columns">list[str]</td></tr></table>>,
tooltip="dsgrid.config.file_schema.FileSchema

Defines the format of a data file (CSV, JSON, Parquet).
"];
"dsgrid.config.dataset_config.UserDataLayout":data_file:e -> "dsgrid.config.file_schema.FileSchema":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dataset_config.UserDataLayout":lookup_data_file:e -> "dsgrid.config.file_schema.FileSchema":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.AlignedTimeSingleTimeZone" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>AlignedTimeSingleTimeZone</b></td></tr><tr><td>format_type</td><td port="format_type">Literal[TimeZoneFormat.ALIGNED_IN_ABSOLUTE_TIME]</td></tr><tr><td>time_zone</td><td port="time_zone">TimeZone</td></tr></table>>,
tooltip="dsgrid.config.dimensions.AlignedTimeSingleTimeZone

For each geography, data has the same set of timestamps in absolute \
time.
Timestamps in the data must be tz-aware.

E.g., data in CA and NY both start in 2018-01-01 00:00 EST.
"];
"dsgrid.config.dimensions.AnnualRangeModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>AnnualRangeModel</b></td></tr><tr><td>start</td><td port="start">str</td></tr><tr><td>end</td><td port="end">str</td></tr><tr><td>str_format</td><td port="str_format">str</td></tr><tr><td>frequency</td><td port="frequency">int</td></tr></table>>,
tooltip="dsgrid.config.dimensions.AnnualRangeModel

Defines a continuous range of annual time.
"];
"dsgrid.config.dimensions.AnnualTimeDimensionModel":ranges:e -> "dsgrid.config.dimensions.AnnualRangeModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DateTimeDimensionModel":time_zone_format:e -> "dsgrid.config.dimensions.AlignedTimeSingleTimeZone":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.LocalTimeMultipleTimeZones" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>LocalTimeMultipleTimeZones</b></td></tr><tr><td>format_type</td><td port="format_type">Literal[TimeZoneFormat.ALIGNED_IN_CLOCK_TIME]</td></tr><tr><td>time_zones</td><td port="time_zones">list[TimeZone]</td></tr></table>>,
tooltip="dsgrid.config.dimensions.LocalTimeMultipleTimeZones

For each geography, data has the same set of timestamps when interpreted \
as local clock time by adjusting
for the time zone of each geography.
Timestamps in the data must be tz-aware.

\
E.g., data in CA may start in 2018-01-01 00:00 PST while data in NY may start in 2018-01-01 00:00 EST.
They are aligned in clock \
time but not in absolute time.
"];
"dsgrid.config.dimensions.DateTimeDimensionModel":time_zone_format:e -> "dsgrid.config.dimensions.LocalTimeMultipleTimeZones":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.TimeRangeModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>TimeRangeModel</b></td></tr><tr><td>start</td><td port="start">str</td></tr><tr><td>end</td><td port="end">str</td></tr><tr><td>str_format</td><td port="str_format">str</td></tr><tr><td>frequency</td><td port="frequency">timedelta</td></tr></table>>,
tooltip="dsgrid.config.dimensions.TimeRangeModel

Defines a continuous range of time.
"];
"dsgrid.config.dimensions.DateTimeDimensionModel":ranges:e -> "dsgrid.config.dimensions.TimeRangeModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel":time_zone_format:e -> "dsgrid.config.dimensions.AlignedTimeSingleTimeZone":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel":time_zone_format:e -> "dsgrid.config.dimensions.LocalTimeMultipleTimeZones":_root:w [arrowhead=noneteetee,
arrowtail=nonenone];
"dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel":ranges:e -> "dsgrid.config.dimensions.TimeRangeModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.IndexRangeModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>IndexRangeModel</b></td></tr><tr><td>start</td><td port="start">int</td></tr><tr><td>end</td><td port="end">int</td></tr><tr><td>starting_timestamp</td><td port="starting_timestamp">str</td></tr><tr><td>str_format</td><td port="str_format">str</td></tr><tr><td>frequency</td><td port="frequency">timedelta</td></tr></table>>,
tooltip="dsgrid.config.dimensions.IndexRangeModel

Defines a continuous range of indices.
"];
"dsgrid.config.dimensions.IndexTimeDimensionModel":ranges:e -> "dsgrid.config.dimensions.IndexRangeModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.dimensions.MonthRangeModel" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>MonthRangeModel</b></td></tr><tr><td>start</td><td port="start">int</td></tr><tr><td>end</td><td port="end">int</td></tr></table>>,
tooltip="dsgrid.config.dimensions.MonthRangeModel

Defines a continuous range of time.
"];
"dsgrid.config.dimensions.RepresentativePeriodTimeDimensionModel":ranges:e -> "dsgrid.config.dimensions.MonthRangeModel":_root:w [arrowhead=crownone,
arrowtail=nonenone];
"dsgrid.config.file_schema.Column" [label=<<table border="0" cellborder="1" cellspacing="0"><tr><td port="_root" colspan="2"><b>Column</b></td></tr><tr><td>name</td><td port="name">str</td></tr><tr><td>dimension_type</td><td port="dimension_type">DimensionType | None</td></tr><tr><td>data_type</td><td port="data_type">str | None</td></tr></table>>,
tooltip="dsgrid.config.file_schema.Column

Base data model for all dsgrid data models
"];
"dsgrid.config.file_schema.FileSchema":columns:e -> "dsgrid.config.file_schema.Column":_root:w [arrowhead=crownone,
arrowtail=nonenone];
}

Fields:

data_classification (dsgrid.config.dataset_config.DataClassificationType)
data_layout (dsgrid.config.dataset_config.UserDataLayout | None)
data_source (str | None)
data_source_authors (list[str] | None)
data_source_date (str | None)
data_source_doi_url (str | None)
data_source_version (str | None)
dataset_id (str)
dataset_qualifier_metadata (dsgrid.config.dataset_config.QuantityModel | dsgrid.config.dataset_config.GrowthRateModel)
dataset_type (dsgrid.config.dataset_config.InputDatasetType)
description (str | None)
dimension_references (list[dsgrid.config.dimensions.DimensionReferenceModel])
dimensions (list[dsgrid.config.dimensions.DimensionModel | dsgrid.config.dimensions.DateTimeDimensionModel | dsgrid.config.dimensions.AnnualTimeDimensionModel | dsgrid.config.dimensions.RepresentativePeriodTimeDimensionModel | dsgrid.config.dimensions.DatetimeExternalTimeZoneDimensionModel | dsgrid.config.dimensions.IndexTimeDimensionModel | dsgrid.config.dimensions.NoOpTimeDimensionModel])
enable_unit_conversion (bool)
origin_contributors (list[str] | None)
origin_creator (str | None)
origin_organization (str | None)
origin_project (str | None)
registry_data_layout (dsgrid.config.dataset_config.RegistryDataLayout | None)
sector_description (str | None)
tags (list[str] | None)
trivial_dimensions (list[dsgrid.dimension.base_models.DimensionType])
use_project_geography_time_zone (bool)
user_defined_metadata (dict[str, Any])

Validators:

check_dataset_id » dataset_id
check_files » dimensions
check_layout_fields » all fields
check_names » dimensions
check_time_not_trivial » trivial_dimensions
check_time_zone » all fields

field data_classification: DataClassificationType [Required]¶

Data security classification (e.g., low, moderate).

Validated by:

check_layout_fields
check_time_zone

field data_layout: UserDataLayout | None = None¶

Defines the data layout (table format, value format, and file paths) for dataset registration.

Validated by:

check_layout_fields
check_time_zone

field data_source: str | None = None¶

Original data source name, e.g. ‘ComStock’, ‘EIA 861’.

Validated by:

check_layout_fields
check_time_zone

field data_source_authors: list[str] | None = None¶

List of authors for the original data source.

Validated by:

check_layout_fields
check_time_zone

field data_source_date: str | None = None¶

Date or year the original source data were published, e.g., ‘2021’ for ‘EIA AEO 2021’.

Validated by:

check_layout_fields
check_time_zone

field data_source_doi_url: str | None = None¶

Original data source doi or other url

Validated by:

check_layout_fields
check_time_zone

field data_source_version: str | None = None¶

Source data version, if applicable. For example, could specify preliminary versus final data.

Validated by:

check_layout_fields
check_time_zone

field dataset_id: str [Required]¶

Unique dataset identifier.

Validated by:

check_dataset_id
check_layout_fields
check_time_zone

field dataset_qualifier_metadata: QuantityModel | GrowthRateModel = QuantityModel(dataset_qualifier_type=<DatasetQualifierType.QUANTITY: 'quantity'>)¶

Additional metadata to include related to the dataset_qualifier

Validated by:

check_layout_fields
check_time_zone

field dataset_type: InputDatasetType = InputDatasetType.UNSPECIFIED¶

Input dataset type.

Validated by:

check_layout_fields
check_time_zone

field description: str | None = None¶

A detailed description of the dataset

Validated by:

check_layout_fields
check_time_zone

field dimension_references: list[DimensionReferenceModel] = []¶

List of registered dimension references that make up the dimensions of dataset.

Validated by:

check_layout_fields
check_time_zone

field dimensions: Annotated[list[DimensionModel | DateTimeDimensionModel | AnnualTimeDimensionModel | RepresentativePeriodTimeDimensionModel | DatetimeExternalTimeZoneDimensionModel | IndexTimeDimensionModel | NoOpTimeDimensionModel], BeforeValidator(func=handle_dimension_union, json_schema_input_type=PydanticUndefined)] = []¶

List of dimensions that make up the dimensions of dataset. They will be automatically registered during dataset registration and then converted to dimension_references.

Constraints:

func = <function handle_dimension_union at 0x7fe6e0d4ba60>
json_schema_input_type = PydanticUndefined

Validated by:

check_files
check_layout_fields
check_names
check_time_zone

field enable_unit_conversion: bool = True¶

If the dataset uses its dimension mapping for the metric dimension to also perform unit conversion, then this value should be false.

Validated by:

check_layout_fields
check_time_zone

field origin_contributors: list[str] | None = None¶

List of contributors to the compilation of this dataset for dsgrid, e.g., [‘Harry Potter’, ‘Ronald Weasley’]

Validated by:

check_layout_fields
check_time_zone

field origin_creator: str | None = None¶

First and last name of the person who formatted this dataset for dsgrid

Validated by:

check_layout_fields
check_time_zone

field origin_organization: str | None = None¶

Organization name of the origin_creator, e.g., ‘NREL’

Validated by:

check_layout_fields
check_time_zone

field origin_project: str | None = None¶

Name of the project for/from which this dataset was compiled, e.g., ‘IEF’, ‘Building Standard Scenarios’.

Validated by:

check_layout_fields
check_time_zone

field registry_data_layout: RegistryDataLayout | None = None¶

Defines the dataset’s data layout once stored in the registry.

Validated by:

check_layout_fields
check_time_zone

field sector_description: str | None = None¶

Sectoral description (e.g., residential, commercial, industrial, transportation, electricity)

Validated by:

check_layout_fields
check_time_zone

field tags: list[str] | None = None¶

List of data tags

Validated by:

check_layout_fields
check_time_zone

field trivial_dimensions: list[DimensionType] = []¶

List of trivial dimensions (i.e., 1-element dimensions) that do not exist in the load_data_lookup. List the dimensions by dimension type. Trivial dimensions are 1-element dimensions that are not present in the parquet data columns. Instead they are added by dsgrid as an alias column.

Validated by:

check_layout_fields
check_time_not_trivial
check_time_zone

field use_project_geography_time_zone: bool = False¶

If true, time zones will be applied from the project’s geography dimension. If false, the dataset’s geography dimension records must provide a time zone column.

Validated by:

check_layout_fields
check_time_zone

field user_defined_metadata: dict[str, Any] = {}¶

Additional user defined metadata fields

Validated by:

check_layout_fields
check_time_zone

validator check_dataset_id » dataset_id[source]¶: Check dataset ID validity

validator check_files » dimensions[source]¶: Validate dimension files are unique across all dimensions

validator check_layout_fields » all fields[source]¶: Ensure data_layout and registry_data_layout are mutually exclusive.

validator check_names » dimensions[source]¶: Validate dimension names are unique across all dimensions.

validator check_time_not_trivial » trivial_dimensions[source]¶

validator check_time_zone » all fields[source]¶: Validate whether required time zone information is present.