Ordeq

spark_explain.py

Ordeq

Getting started
Getting started
- Quickstart
- Introduction
- Concepts
  Concepts
  - IO
  - Nodes
  - Catalogs
  - Hooks
Guides & examples
Guides & examples
- Creating a IO class
- Testing nodes
- Examples
  Examples
  - Benchmark runtimes
  - Lazy IO
API
API
- ordeq
  ordeq
  - types.py
  - framework
    framework
    
    _registry.py
    
    catalog.py
    
    graph.py
    
    hook.py
    
    io.py
    
    nodes.py
    
    pipeline.py
    
    runner.py
- ordeq_args
  ordeq_args
  - command_line_arg.py
  - environment_variable.py
- ordeq_boto3
  ordeq_boto3
  - s3_object.py
- ordeq_cli_runner
  ordeq_cli_runner
  - runner.py
- ordeq_common
  ordeq_common
  - hooks
    hooks
    
    spy.py
  - io
    io
    
    bytes_buffer.py
    
    dataclass.py
    
    iterate.py
    
    match.py
    
    printer.py
    
    static.py
    
    string_buffer.py
- ordeq_faiss
  ordeq_faiss
  - index.py
- ordeq_files
  ordeq_files
  - bytes.py
  - csv.py
  - glob.py
  - json.py
  - pickle.py
  - text.py
  - yaml.py
- ordeq_huggingface
  ordeq_huggingface
  - dataset.py
- ordeq_ibis
  ordeq_ibis
  - io
    io
    
    parquet.py
- ordeq_joblib
  ordeq_joblib
  - joblib.py
- ordeq_matplotlib
  ordeq_matplotlib
  - figure.py
- ordeq_numpy
  ordeq_numpy
  - binary_array.py
  - text_array.py
- ordeq_pandas
  ordeq_pandas
- ordeq_polars
  ordeq_polars
  - eager
    eager
    
    csv.py
    
    excel.py
    
    parquet.py
  - lazy
    lazy
    
    csv.py
    
    parquet.py
- ordeq_project
  ordeq_project
  - gather.py
- ordeq_pydantic
  ordeq_pydantic
  - model.py
- ordeq_pymupdf
  ordeq_pymupdf
  - pdf_file.py
- ordeq_requests
  ordeq_requests
  - io
    io
    
    response.py
- ordeq_sentence_transformers
  ordeq_sentence_transformers
  - sentence_transformer.py
- ordeq_spark
  ordeq_spark
  - utils.py
  - hooks
    hooks
    
    spark_explain.py spark_explain.py
    Table of contents
    
    spark_explain
    
    SparkExplainHook
    
    spark_job_group.py
  - io
    io
    
    jdbc.py
    
    session.py
    
    static.py
    
    types.py
    
    files
    files
    
    csv.py
    
    json.py
    
    utils.py
    
    tables
    tables
    
    global_temp_view.py
    
    hive.py
    
    iceberg.py
    
    table.py
    
    temp_view.py
- ordeq_viz
  ordeq_viz
Contributing
Contributing

spark_explain.py

`SparkExplainHook`

Bases: OutputHook[DataFrame]

Hook to print the Spark execution plan before saving a DataFrame.