API Reference

The polars_bigquery package provides an I/O connector for reading Google BigQuery tables and queries into Polars DataFrame and LazyFrame objects.

See Row predicate pushdown for details on which Polars expressions and data types are pushed down to BigQuery when using Client.scan_table.

polars_bigquery.Client

Client(*, quota_project_id: str, credentials_provider: CredentialProviderGCP | None = None, user_agent: str | None = None)

Client for reading data from Google BigQuery into Polars.

Wraps an arrow_bigquery.Client and a REST session to keep connections open and reuse credentials across multiple operations. Can be used as a context manager to automatically close the underlying REST session on exit.

Examples:

import polars as pl
import polars_bigquery

with polars_bigquery.Client(quota_project_id="my-project") as client:
    lf = client.scan_table("bigquery-public-data.usa_names.usa_1910_2013")
    df = (
        lf.filter((pl.col("state") == "WA") & (pl.col("year") >= 2000))
        .select(["name", "year", "number"])
        .collect()
    )

Initialize a BigQuery client.

Parameters:

Name	Type	Description	Default
`quota_project_id`	`str`	Google Cloud project ID used for quota and billing of BigQuery API requests and query jobs.	required
`credentials_provider`	`CredentialProviderGCP \| None`	Polars GCP credential provider used to obtain OAuth2 bearer tokens. If `None`, a default `polars.CredentialProviderGCP(quota_project_id=quota_project_id)` instance is created using Application Default Credentials (ADC).	`None`
`user_agent`	`str \| None`	Optional custom user-agent suffix appended to the client's default `polars-bigquery/<version>` user-agent header.	`None`

credentials_provider `property`

credentials_provider: CredentialProviderGCP

The polars.CredentialProviderGCP used to authenticate requests.

quota_project_id `property`

quota_project_id: str

The Google Cloud project ID used for quota and billing.

close

close() -> None

Close underlying REST session resources.

read_table

read_table(table: Any, *, maintain_order: bool = False) -> pl.DataFrame

Eagerly read a BigQuery table into a Polars DataFrame.

Reads the entire table directly via the BigQuery Storage Read API. For projection and filter pushdown, prefer Client.scan_table.

Parameters:

Name	Type	Description	Default
`table`	`Any`	The BigQuery table to read. Accepts a `"project.dataset.table"` string, an `arrow_bigquery.BigQueryTableId`, or any object with `project` (or `project_id`), `dataset_id`, and `table_id` attributes (such as `google.cloud.bigquery.TableReference`).	required
`maintain_order`	`bool`	Whether to preserve table row order by reading from a single Storage Read API stream. Defaults to `False`, which enables parallel multi-stream reads for higher throughput.	`False`

Returns:

Type	Description
`DataFrame`	A Polars `DataFrame` containing the table data.

Examples:

df = client.read_table("bigquery-public-data.utility_us.country_code_iso")

read_query

read_query(query: str, *, arrow_buffer_compression: str = 'lz4frame', dry_run: bool = False, maintain_order: bool = True, use_query_cache: bool | None = None) -> pl.DataFrame

Execute a GoogleSQL query and read the result into a Polars DataFrame.

Executes the query via the BigQuery jobs.query API with JOB_CREATION_OPTIONAL and streams the Arrow results via the BigQuery Storage Read API.

The whole result is read into memory. To process larger results lazily, see Scanning large query results.

Parameters:

Name	Type	Description	Default
`query`	`str`	GoogleSQL query string to execute in BigQuery.	required
`arrow_buffer_compression`	`str`	Compression codec for Arrow record batch buffers (`"lz4frame"`, `"zstd"`, or `"unspecified"`). Defaults to `"lz4frame"`.	`'lz4frame'`
`dry_run`	`bool`	If `True`, validates the query without executing it and returns a 1-row `DataFrame` containing job statistics and schema metadata (`bigquerySchema`, `projectId`, `jobId`, `location`, `totalBytesProcessed`, `cacheHit`, `statementType`, `creationTime`). Defaults to `False`.	`False`
`maintain_order`	`bool`	Whether to preserve the row order produced by an `ORDER BY` clause by reading from a single Storage Read API stream. Defaults to `True`. Set to `False` to switch to parallel multi-stream reads for large result sets when row order does not matter.	`True`
`use_query_cache`	`bool \| None`	Whether to look for the result in the query cache. If `None`, the BigQuery API default (`True`) is used.	`None`

Returns:

Type	Description
`DataFrame`	A Polars `DataFrame` containing the query results (or dry-run metadata when `dry_run=True`).

Raises:

Type	Description
`BigQueryError`	If the BigQuery query fails.

Examples:

df = client.read_query(
    """
    SELECT name, SUM(number) AS total_born
    FROM `bigquery-public-data.usa_names.usa_1910_2013`
    GROUP BY name
    ORDER BY total_born DESC
    LIMIT 100
    """
)

scan_table

scan_table(table: Any, *, schema: Schema | Mapping[str, DataType | DataTypeClass] | None = None) -> pl.LazyFrame

Lazily scan a BigQuery table into a Polars LazyFrame.

Fetches table schema and partitioning metadata via the BigQuery REST API when constructing the LazyFrame (unless schema is provided), and defers reading rows from the BigQuery Storage Read API until query execution (.collect() or .collect_batches()).

Supports both projection pushdown (only reading referenced columns) and row predicate pushdown (translating .filter() expressions into BigQuery Storage Read API row_restriction SQL clauses). See the Row predicate pushdown page for the complete list of supported Polars expressions and data types.

For ingestion-time partitioned tables, _PARTITIONDATE (pl.Date) and _PARTITIONTIME (pl.Datetime("us", "utc")) pseudo-columns are automatically added to the LazyFrame schema when schema is not provided, so partition pruning filters can be pushed down.

Parameters:

Name	Type	Description	Default
`table`	`Any`	The BigQuery table to scan. Accepts a `"project.dataset.table"` string, an `arrow_bigquery.BigQueryTableId`, or any object with `project` (or `project_id`), `dataset_id`, and `table_id` attributes (such as `google.cloud.bigquery.TableReference`).	required
`schema`	`Schema \| Mapping[str, DataType \| DataTypeClass] \| None`	Optional Polars schema (`polars.Schema` or a mapping of column names to Polars data types) for the table. When provided, `scan_table` avoids calling the BigQuery `tables.get` REST API to fetch table metadata. For ingestion-time partitioned tables, include `"_PARTITIONDATE": pl.Date` and/or `"_PARTITIONTIME": pl.Datetime("us", "utc")` in `schema` if you want to filter or select those pseudo-columns.	`None`

Returns:

Type	Description
`LazyFrame`	A Polars `LazyFrame` backed by the BigQuery Storage Read API.

Raises:

Type	Description
`BigQueryError`	If `schema` is `None` and fetching the table metadata from the BigQuery REST API fails.

Examples:

import polars as pl

lf = client.scan_table("bigquery-public-data.usa_names.usa_1910_2013")
df = (
    lf.filter(pl.col("name").str.starts_with("A") & (pl.col("year") >= 2010))
    .select(["name", "state", "year", "number"])
    .collect()
)

polars_bigquery.exceptions.BigQueryError

Bases: Exception

Error raised when a BigQuery REST API request or query job fails or times out.

API Reference

polars_bigquery.Client

credentials_provider property

quota_project_id property

close

read_table

read_query

scan_table

polars_bigquery.exceptions.BigQueryError

credentials_provider `property`

quota_project_id `property`