Skip to content

Ordeq

spark_explain.py

Ordeq

Getting started
Getting started
- Quickstart
- Introduction
- Concepts
  Concepts
  - IO
  - Nodes
  - Catalogs
  - Hooks
Guides & examples
Guides & examples
- Starting a new project
- Running & visualising
- Nodes
  Nodes
  - Node parameters
  - Testing nodes
- IO
  IO
- Hooks
  Hooks
  - Benchmark runtimes
- Adopting Ordeq
  Adopting Ordeq
  - Coming from Kedro
- Integrations
  Integrations
  - Docker
  - Marimo
  - Streamlit
Reference
Reference
- Packages
- API
  API
  - ordeq
    ordeq
    
    ordeq
    
    _catalog.py
    
    _graph.py
    
    _hook.py
    
    _io.py
    
    _nodes.py
    
    _resolve.py
    
    _runner.py
    
    types.py
  - ordeq_altair
    ordeq_altair
    
    ordeq_altair
    
    chart.py
  - ordeq_args
    ordeq_args
    
    ordeq_args
    
    command_line_arg.py
    
    environment_variable.py
  - ordeq_boto3
    ordeq_boto3
    
    ordeq_boto3
    
    s3_object.py
  - ordeq_common
    ordeq_common
    
    ordeq_common
    
    hooks
    hooks
    
    logger.py
    
    spy.py
    
    io
    io
    
    bytes_buffer.py
    
    dataclass.py
    
    iterate.py
    
    literal.py
    
    match.py
    
    printer.py
    
    string_buffer.py
  - ordeq_duckdb
    ordeq_duckdb
    
    ordeq_duckdb
    
    connection.py
    
    csv.py
    
    table.py
    
    view.py
  - ordeq_faiss
    ordeq_faiss
    
    ordeq_faiss
    
    index.py
  - ordeq_files
    ordeq_files
    
    ordeq_files
    
    bytes.py
    
    csv.py
    
    glob.py
    
    json.py
    
    pickle.py
    
    text.py
  - ordeq_huggingface
    ordeq_huggingface
    
    ordeq_huggingface
    
    dataset.py
    
    disk_dataset.py
  - ordeq_ibis
    ordeq_ibis
    
    ordeq_ibis
    
    io
    io
    
    parquet.py
  - ordeq_joblib
    ordeq_joblib
    
    ordeq_joblib
    
    joblib.py
  - ordeq_manifest
    ordeq_manifest
    
    ordeq_manifest
    
    manifest.py
    
    models.py
  - ordeq_matplotlib
    ordeq_matplotlib
    
    ordeq_matplotlib
    
    figure.py
  - ordeq_networkx
    ordeq_networkx
    
    ordeq_networkx
    
    networkx_gml.py
    
    networkx_graphml.py
    
    networkx_json.py
  - ordeq_numpy
    ordeq_numpy
    
    ordeq_numpy
    
    binary_array.py
    
    text_array.py
  - ordeq_pandas
    ordeq_pandas
    
    ordeq_pandas
    
    csv.py
    
    excel.py
    
    parquet.py
  - ordeq_polars
    ordeq_polars
    
    ordeq_polars
    
    eager
    eager
    
    csv.py
    
    excel.py
    
    parquet.py
    
    lazy
    lazy
    
    csv.py
    
    parquet.py
  - ordeq_pydantic
    ordeq_pydantic
    
    ordeq_pydantic
    
    json.py
    
    model.py
  - ordeq_pymupdf
    ordeq_pymupdf
    
    ordeq_pymupdf
    
    pdf_file.py
  - ordeq_pyproject
    ordeq_pyproject
    
    ordeq_pyproject
    
    pyproject.py
  - ordeq_requests
    ordeq_requests
    
    ordeq_requests
    
    io
    io
    
    response.py
  - ordeq_sentence_transformers
    ordeq_sentence_transformers
    
    ordeq_sentence_transformers
    
    sentence_transformer.py
  - ordeq_spark
    ordeq_spark
    
    ordeq_spark
    
    utils.py
    
    hooks
    hooks
    
    spark_explain.py spark_explain.py
    Table of contents
    
    spark_explain
    
    SparkExplainHook
    
    spark_job_group.py
    
    io
    io
    
    dataframe.py
    
    jdbc.py
    
    session.py
    
    types.py
    
    files
    files
    
    csv.py
    
    json.py
    
    utils.py
    
    tables
    tables
    
    global_temp_view.py
    
    hive.py
    
    iceberg.py
    
    table.py
    
    temp_view.py
  - ordeq_test_utils
    ordeq_test_utils
    
    ordeq_test_utils
    
    snapshot.py
  - ordeq_toml
    ordeq_toml
    
    ordeq_toml
    
    toml.py
  - ordeq_viz
    ordeq_viz
    
    ordeq_viz
    
    api.py
    
    graph.py
    
    to_kedro_viz.py
    
    to_mermaid.py
  - ordeq_yaml
    ordeq_yaml
    
    ordeq_yaml
    
    yaml.py
Contributing
Contributing

spark_explain.py

`SparkExplainHook`

Bases: OutputHook[DataFrame]

Hook to print the Spark execution plan before saving a DataFrame.