package sql

import "github.com/apache/beam/sdks/v2/go/pkg/beam/transforms/sql"

Package sql contains SQL transform APIs, allowing SQL queries to be used in Beam Go pipelines.

NOTE: This feature only works when an expansion service/handler is registered for SQL transform. The APIs are subject to change without backward compatibility guarantees.

Index ¶

• func Transform(s beam.Scope, query string, opts ...Option) beam.PCollection
• type Option
• func Dialect(dialect string) Option
• func ExpansionAddr(addr string) Option
• func Input(name string, in beam.PCollection) Option
• func OutputType(t reflect.Type, components ...typex.FullType) Option

Functions ¶

func Transform ¶

func Transform(s beam.Scope, query string, opts ...Option) beam.PCollection

Transform creates a SQL-based transform over zero or more PCollections and/or named data sources.

PCollection inputs can be added using the sql.Input option. SQL queries can also refer to external tables that can be resolved by the expansion service.

The output PCollection type must be specified by the sql.OutputType option.

Example:

in := beam.Create(s, 1, 2, 3)
out := sql.Transform(s, "SELECT COUNT(*) FROM t",
    sql.Input("t", in),
    sql.OutputType(reflect.TypeOf(int64(0))))
// `out` is a PCollection<int64> with a single element 3.

If an expansion service address is not provided as an option, one will be automatically started for the transform.

Types ¶

type Option ¶

type Option func(sqlx.Options)

Option is the base type of all the SQL transform options.

func Dialect ¶

func Dialect(dialect string) Option

Dialect specifies the SQL dialect, e.g. use 'zetasql' for ZetaSQL.

func ExpansionAddr ¶

func ExpansionAddr(addr string) Option

ExpansionAddr is the URL of the expansion service to use.

func Input ¶

func Input(name string, in beam.PCollection) Option

Input adds a named PCollection input to the transform.

func OutputType ¶

func OutputType(t reflect.Type, components ...typex.FullType) Option

OutputType specifies the output PCollection type of the transform. It must match the SQL output schema.

There is currently no default output type, so users must set this option. In the future, Row, once implemented, may become the default output type.

Source Files ¶

sql.go

Directories ¶