Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
dsgrid 0.3.3 documentation
dsgrid 0.3.3 documentation
  • Getting Started
    • Installation
    • Data Users
    • Dataset Submitters
    • Dataset Mappers
    • Project Coordinators
  • Published Data
    • Published Datasets
    • Working with Published Data
      • Query a Dataset with Python
      • Visualize Data with Tableau
  • User Guide
    • dsgrid Fundamentals
    • Tutorials
      • Create and Submit a Dataset
      • Map a Dataset to a Project’s Dimensions
      • Create a Project
      • Query a Project
      • Create a Derived Dataset
    • How-To Guides
      • How to Browse the Registry
      • How to Create Dataset Dimensions
      • How to Handle Missing Dimension Associations
      • How to Create Project Base Dimensions
      • How to Create Project Supplemental Dimensions
      • How to Filter a Query
      • How to Run dsgrid on Kestrel
      • How to Start a Spark Cluster on Kestrel
    • Dataset Registration
      • Dimension Concepts
      • Dataset Concepts
      • Data File Formats
      • Registration Checks
    • Dataset Mapping
      • Dimension Mapping Concepts
      • Dimension Mappings
      • Mapping Types
      • Mapping Workflows
    • Dataset Submittal
      • Submission Process
      • Submission Checks
    • Project Creation
      • Project Concepts
      • Base Dimensions
      • Supplemental Dimensions
      • Dataset Requirements
    • Project Queries
      • Query Concepts
      • Query Concepts
      • Aggregations
      • Filters
      • Output Formats
    • Project Derived Datasets
      • Derived Dataset Concepts
      • Derived Concepts
      • Growth Rates
      • Residuals
      • Workflows
    • Apache Spark
      • Apache Spark Overview
      • When to Use Spark
      • Kestrel Setup
      • Spark Cluster
  • Software Reference
    • Software Architecture
    • Data Models
      • Dimensions
      • Dimension Record Classes
      • Dataset Config
      • Dimension Mappings
      • Project Config
      • Enums
    • CLI Fundamentals
    • CLI Reference
    • Python API
  • Publications
  • Citation and Data Attribution
  • Contact
Back to top
View this page

Apache Spark¶

Guide for working with large-scale datasets using Apache Spark.

Core Concepts¶

  • Apache Spark Overview
  • When to Use Spark
  • Kestrel Setup
  • Spark Cluster

How-Tos¶

  • How to Run dsgrid on Kestrel

  • How to Start a Spark Cluster on Kestrel

Next
Apache Spark Overview
Previous
Workflows
Copyright © Copyright 2026, The Alliance for Sustainable Energy, LLC
Made with Sphinx and @pradyunsg's Furo
On this page
  • Apache Spark
    • Core Concepts
    • How-Tos