ordeq_bigquery

BigQueryJSON dataclass

Bases: IO[list[dict[str, Any]]]

IO for loading data using a user provided query and saving data from JSON-compatible data representation.

Parameters:

Name	Type	Description	Default
table_id	str | Table	BigQuery table identifier	required
client	Client	BigQuery client	required
query	str | None	SQL query to load data, e.g. SELECT * FROM my_table	None

load(**query_options)

Loads query results from BigQuery.

Parameters:

Name	Type	Description	Default
**query_options	Any	Additional options for the query.	{}

Returns:

Type	Description
list[dict[str, Any]]	List of rows as dictionaries. Raises typeerror with query is not
list[dict[str, Any]]	provided.

Raises:

Type	Description
TypeError	if query is not provided, i.e. is None

Example:

>>> from google.cloud import bigquery
>>> from ordeq_bigquery import BigQueryJSON
>>>
>>> client = bigquery.Client()  # doctest: +SKIP
>>> inp = BigQueryJSON(
...     query="SELECT * FROM my_table",
...     table_id="project.dataset.table",
...     client=client,
... )  # doctest: +SKIP
>>> rows = inp.load()  # doctest: +SKIP

save(data, **save_options)

Saves JSON rows to BigQuery.

Parameters:

Name	Type	Description	Default
data	list[dict[str, Any]]	Dictionary or list of dictionaries to insert.	required
**save_options	Any	Additional options for saving.	{}

Raises:

Type	Description
IOException	If insertion fails.

Example:

>>> from google.cloud import bigquery
>>> from ordeq_bigquery import BigQueryJSON
>>>
>>> client = bigquery.Client()  # doctest: +SKIP
>>> out = BigQueryJSON(
...     table_id="project.dataset.table", client=client
... )  # doctest: +SKIP
>>> out.save([
...     {"col1": "val1"},
...     {"col1": "val2"},
... ])  # doctest: +SKIP

BigQueryPandas dataclass

Bases: IO[DataFrame]

BigQueryTable extension for loading and saving Pandas DataFrames.

Example:

>>> from ordeq_bigquery import BigQueryPandas
>>> from google.cloud import bigquery
>>>
>>> client = bigquery.Client()  # doctest: +SKIP
>>> table = BigQueryPandas(
...     table_id="project.dataset.table", client=client
... )  # doctest: +SKIP
>>> df = table.load()  # doctest: +SKIP
>>> table.save(df)  # doctest: +SKIP

load(**load_options)

Loads the BigQuery table as a Pandas DataFrame.

Parameters:

Name	Type	Description	Default
**load_options	Any	Additional options passed to to_dataframe.	{}

Returns:

Type	Description
DataFrame	A Pandas DataFrame containing the table data.

Example:

>>> from ordeq_bigquery import BigQueryPandas
>>> from google.cloud import bigquery
>>>
>>> client = bigquery.Client()  # doctest: +SKIP
>>> table = BigQueryPandas(
...     table_id="project.dataset.table", client=client
... )  # doctest: +SKIP
>>> df = table.load()  # doctest: +SKIP

save(df, **save_options)

Saves a Pandas DataFrame to the BigQuery table.

Parameters:

Name	Type	Description	Default
df	DataFrame	The Pandas DataFrame to save.	required
**save_options	Any	Options passed to load_table_from_dataframe.	{}

Example:

>>> import pandas as pd
>>> from ordeq_bigquery import BigQueryPandas
>>>
>>> from google.cloud import bigquery
>>> client = bigquery.Client()  # doctest: +SKIP
>>> table = BigQueryPandas(
...     table_id="project.dataset.table", client=client
... )  # doctest: +SKIP
>>> df = pd.DataFrame({"a": [1, 2], "b": [3, 4]})
>>> table.save(df)  # doctest: +SKIP