package topdown

import "github.com/open-policy-agent/opa/topdown"

Package topdown provides query evaluation support.

The topdown implementation is a (slightly modified) version of the standard "top-down evaluation" algorithm used in query languages such as Datalog. The main difference in the implementation is in the handling of Rego references.

At a high level, the topdown implementation consists of (1) term evaluation and (2) expression evaluation. Term evaluation involves handling expression terms in isolation whereas expression evaluation involves evaluating the terms in relation to each other.

During the term evaluation phase, the topdown implementation will evaluate references found in the expression to produce bindings for:

  1. Variables appearing in references
  2. References to Virtual Documents
  3. Comprehensions

Once terms have been evaluated in isolation, the overall expression can be evaluated. If the expression is simply a term (e.g., string, reference, variable, etc.) then it is compared against the boolean "false" value. If the value IS NOT "false", evaluation continues. On the other hand, if the expression is defined by a built-in operator (e.g., =, !=, min, max, etc.) then the appropriate built-in function is invoked to determine if evaluation continues and bind variables accordingly.

Index

Examples

Constants

const (

	// InternalErr represents an unknown evaluation error.
	InternalErr string = "eval_internal_error"

	// ConflictErr indicates a conflict was encountered during evaluation. For
	// instance, a conflict occurs if a rule produces multiple, differing values
	// for the same key in an object. Conflict errors indicate the policy does
	// not account for the data loaded into the policy engine.
	ConflictErr string = "eval_conflict_error"

	// TypeErr indicates evaluation stopped because an expression was applied to
	// a value of an inappropriate type.
	TypeErr string = "eval_type_error"
)

Functions

func Continue

func Continue(t *Topdown, key, value ast.Value, iter Iterator) error

Continue binds key to value in t and calls the iterator. This is a helper function for simple cases where a single value (e.g., a variable) needs to be bound to a value in order for the evaluation the proceed.

func Eval

func Eval(t *Topdown, iter Iterator) error

Eval evaluates the query in t and calls iter once for each set of bindings that satisfy all of the expressions in the query.

Example

Code:play 

package main

import (
	"bytes"
	"context"
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
	"github.com/open-policy-agent/opa/topdown"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	compiler := ast.NewCompiler()

	// Define a dummy query and some data that the query will execute against.
	query, err := compiler.QueryCompiler().Compile(ast.MustParseBody(`data.a[_] = x; x >= 2`))
	if err != nil {
		// Handle error.
	}

	var data map[string]interface{}

	// OPA uses Go's standard JSON library but assumes that numbers have been
	// decoded as json.Number instead of float64. You MUST decode with UseNumber
	// enabled.
	decoder := json.NewDecoder(bytes.NewBufferString(`{"a": [1,2,3,4]}`))
	decoder.UseNumber()

	if err := decoder.Decode(&data); err != nil {
		// Handle error.
	}

	// Instantiate the policy engine's storage layer.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	// Create a new transaction. Transactions allow the policy engine to
	// evaluate the query over a consistent snapshot fo the storage layer.
	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Prepare the evaluation parameters. Evaluation executes against the policy
	// engine's storage. In this case, we seed the storage with a single array
	// of number. Other parameters such as the input, tracing configuration,
	// etc. can be set on the Topdown object.
	t := topdown.New(ctx, query, compiler, store, txn)

	result := []interface{}{}

	// Execute the query and provide a callbakc function to accumulate the results.
	err = topdown.Eval(t, func(t *topdown.Topdown) error {

		// Each variable in the query will have an associated "binding".
		x := t.Binding(ast.Var("x"))

		// Alternatively, you can get a mapping of all bound variables.
		x = t.Vars()[ast.Var("x")]

		// The bindings are ast.Value types so we will convert to a native Go value here.
		v, err := topdown.ValueToInterface(x, t)
		if err != nil {
			return err
		}

		result = append(result, v)
		return nil
	})

	// Inspect the query result.
	fmt.Println("result:", result)
	fmt.Println("err:", err)

}

Output:

result: [2 3 4]
err: <nil>

func IsError

func IsError(err error) bool

IsError returns true if the err is an Error.

func MakeInput

func MakeInput(pairs [][2]*ast.Term) (ast.Value, error)

MakeInput converts the slice of key/value pairs into a single input value. The keys define an object hierarchy.

func PlugExpr

func PlugExpr(expr *ast.Expr, binding Binding) *ast.Expr

PlugExpr returns a copy of expr with bound terms substituted for the binding.

func PlugHead

func PlugHead(head *ast.Head, binding Binding) *ast.Head

PlugHead returns a copy of head with bound terms substituted for the binding.

func PlugTerm

func PlugTerm(term *ast.Term, binding Binding) *ast.Term

PlugTerm returns a copy of term with bound terms substituted for the binding.

func PlugValue

func PlugValue(v ast.Value, binding func(ast.Value) ast.Value) ast.Value

PlugValue returns a copy of v with bound terms substituted for the binding.

func PrettyTrace

func PrettyTrace(w io.Writer, trace []*Event)

PrettyTrace pretty prints the trace to the writer.

func RegisterBuiltinFunc

func RegisterBuiltinFunc(name ast.Var, fun BuiltinFunc)

RegisterBuiltinFunc adds a new built-in function to the evaluation engine.

func RegisterFunctionalBuiltin1

func RegisterFunctionalBuiltin1(name ast.Var, fun FunctionalBuiltin1)

RegisterFunctionalBuiltin1 adds a new built-in function to the evaluation engine.

Example

Code:play 

package main

import (
	"context"
	"fmt"

	"strings"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/topdown"
	"github.com/open-policy-agent/opa/topdown/builtins"
)

func main() {

	// Rego includes a number of built-in functions ("built-ins") for performing
	// standard operations like string manipulation, regular expression
	// matching, and computing aggregates.
	//
	// This test shows how to add a new built-in to Rego and OPA.

	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	// The ast package contains a registry that enumerates the built-ins
	// included in Rego. When adding a new built-in, you must update the
	// registry to include your built-in. Otherwise, the compiler will complain
	// when it encounters your built-in.
	builtin := &ast.Builtin{
		Name:      ast.Var("selective_upper"),
		NumArgs:   2,
		TargetPos: []int{1},
	}

	ast.RegisterBuiltin(builtin)

	// This is the implementation of the built-in that will be called during
	// query evaluation.
	builtinImpl := func(a ast.Value) (ast.Value, error) {

		str, err := builtins.StringOperand(a, 1)

		if err != nil {
			return nil, err
		}

		if str.Equal(ast.String("magic")) {
			// topdown.BuiltinEmpty indicates to the evaluation engine that the
			// expression is false/not defined.
			return nil, topdown.BuiltinEmpty{}
		}

		return ast.String(strings.ToUpper(string(str))), nil
	}

	// See documentation for registering functions that take different numbers
	// of arguments.
	topdown.RegisterFunctionalBuiltin1(builtin.Name, builtinImpl)

	// At this point, the new built-in has been registered and can be used in
	// queries. Our custom built-in converts strings to upper case but is not
	// defined for the input "magic".
	compiler := ast.NewCompiler()
	query, err := compiler.QueryCompiler().Compile(ast.MustParseBody(`selective_upper("custom", x); not selective_upper("magic", "MAGIC")`))
	if err != nil {
		// Handle error.
	}

	// Evaluate the query.
	t := topdown.New(ctx, query, compiler, nil, nil)

	topdown.Eval(t, func(t *topdown.Topdown) error {
		fmt.Println("x:", t.Binding(ast.Var("x")))
		return nil
	})

	// If you are adding new built-in functions to upstream OPA, you must also
	// update the [Language
	// Reference](http://www.openpolicyagent.org/documentation/references/language/)
	// and [How Do I Write
	// Policies](http://www.openpolicyagent.org/documentation/how-do-i-write-policies/)
	// documents. In addition, you must add tests for your new built-in. See the
	// existing integration tests in the topdown package.

}

Output:

x: "CUSTOM"

func RegisterFunctionalBuiltin2

func RegisterFunctionalBuiltin2(name ast.Var, fun FunctionalBuiltin2)

RegisterFunctionalBuiltin2 adds a new built-in function to the evaluation engine.

func RegisterFunctionalBuiltin3

func RegisterFunctionalBuiltin3(name ast.Var, fun FunctionalBuiltin3)

RegisterFunctionalBuiltin3 adds a new built-in function to the evaluation engine.

func RegisterFunctionalBuiltinVoid2

func RegisterFunctionalBuiltinVoid2(name ast.Var, fun FunctionalBuiltinVoid2)

RegisterFunctionalBuiltinVoid2 adds a new built-in function to the evaluation engine.

func ResetQueryIDs

func ResetQueryIDs()

ResetQueryIDs resets the query ID generator. This is only for test purposes.

func ResolveRefs

func ResolveRefs(v ast.Value, t *Topdown) (ast.Value, error)

ResolveRefs returns the value obtained by resolving references to base documents.

func ResolveRefsTerm

func ResolveRefsTerm(term *ast.Term, t *Topdown) (*ast.Term, error)

ResolveRefsTerm returns a copy of term obtained by resolving references to base documents.

func ValueToInterface

func ValueToInterface(v ast.Value, resolver Resolver) (interface{}, error)

ValueToInterface returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage. Composite AST values such as objects and arrays are converted recursively.

Types

type Binding

type Binding func(ast.Value) ast.Value

Binding defines the interface used to apply term bindings to terms, expressions, etc.

type BufferTracer

type BufferTracer []*Event

BufferTracer implements the Tracer interface by simply buffering all events received.

func NewBufferTracer

func NewBufferTracer() *BufferTracer

NewBufferTracer returns a new BufferTracer.

func (*BufferTracer) Enabled

func (b *BufferTracer) Enabled() bool

Enabled always returns true.

func (*BufferTracer) Trace

func (b *BufferTracer) Trace(t *Topdown, evt *Event)

Trace adds the event to the buffer.

type BuiltinEmpty

type BuiltinEmpty struct{}

BuiltinEmpty is used to signal that the built-in function evaluated, but the result is undefined so evaluation should not continue.

func (BuiltinEmpty) Error

func (BuiltinEmpty) Error() string

type BuiltinFunc

type BuiltinFunc func(t *Topdown, expr *ast.Expr, iter Iterator) (err error)

BuiltinFunc defines the interface that the evaluation engine uses to invoke built-in functions (built-ins). In most cases, custom built-ins can be implemented using the FunctionalBuiltin interfaces (which provide less control but are much simpler).

Users can implement their own built-ins and register them with OPA.

Built-ins are given the current evaluation context t with the expression expr to be evaluated. Built-ins can assume that the expression has been plugged with bindings from the current context however references to base documents will not have been resolved. If the built-in determines that the expression has evaluated successfully it should bind any output variables and invoke the iterator with the context produced by binding the output variables. Built-ins must be sure to unbind the outputs after the iterator returns.

type Error

type Error struct {
	Code     string        `json:"code"`
	Message  string        `json:"message"`
	Location *ast.Location `json:"location,omitempty"`
}

Error is the error type returned by the Eval and Query functions when an evaluation error occurs.

func (*Error) Error

func (e *Error) Error() string

type Event

type Event struct {
	Op       Op            // Identifies type of event.
	Node     interface{}   // Contains AST node relevant to the event.
	QueryID  uint64        // Identifies the query this event belongs to.
	ParentID uint64        // Identifies the parent query this event belongs to.
	Locals   *ast.ValueMap // Contains local variable bindings from the query context.
}

Event contains state associated with a tracing event.

func (*Event) Equal

func (evt *Event) Equal(other *Event) bool

Equal returns true if this event is equal to the other event.

func (*Event) HasBody

func (evt *Event) HasBody() bool

HasBody returns true if the Event contains an ast.Body.

func (*Event) HasExpr

func (evt *Event) HasExpr() bool

HasExpr returns true if the Event contains an ast.Expr.

func (*Event) HasRule

func (evt *Event) HasRule() bool

HasRule returns true if the Event contains an ast.Rule.

func (*Event) String

func (evt *Event) String() string

type FunctionalBuiltin1

type FunctionalBuiltin1 func(op1 ast.Value) (output ast.Value, err error)

FunctionalBuiltin1 defines an interface for simple functional built-ins.

Implement this interface if your built-in function takes one input and produces one output.

If an error occurs, the functional built-in should return a descriptive message. The message should not be prefixed with the built-in name as the framework takes care of this.

type FunctionalBuiltin2

type FunctionalBuiltin2 func(op1, op2 ast.Value) (output ast.Value, err error)

FunctionalBuiltin2 defines an interface for simple functional built-ins.

Implement this interface if your built-in function takes two inputs and produces one output.

If an error occurs, the functional built-in should return a descriptive message. The message should not be prefixed with the built-in name as the framework takes care of this.

type FunctionalBuiltin3

type FunctionalBuiltin3 func(op1, op2, op3 ast.Value) (output ast.Value, err error)

FunctionalBuiltin3 defines an interface for simple functional built-ins.

Implement this interface if your built-in function takes three inputs and produces one output.

If an error occurs, the functional built-in should return a descriptive message. The message should not be prefixed with the built-in name as the framework takes care of this.

type FunctionalBuiltinVoid2

type FunctionalBuiltinVoid2 func(op1, op2 ast.Value) (err error)

FunctionalBuiltinVoid2 defines an interface for simple functional built-ins.

Implement this interface if your built-in function takes two inputs and produces no outputs.

If an error occurs, the functional built-in should return a descriptive message. The message should not be prefixed with the built-in name as the framework takes care of this.

type Iterator

type Iterator func(*Topdown) error

Iterator is the interface for processing evaluation results.

type Op

type Op string

Op defines the types of tracing events.

const (
	// EnterOp is emitted when a new query is about to be evaluated.
	EnterOp Op = "Enter"

	// ExitOp is emitted when a query has evaluated to true.
	ExitOp Op = "Exit"

	// EvalOp is emitted when an expression is about to be evaluated.
	EvalOp Op = "Eval"

	// RedoOp is emitted when an expression, rule, or query is being re-evaluated.
	RedoOp Op = "Redo"

	// FailOp is emitted when an expression evaluates to false.
	FailOp Op = "Fail"
)

type QueryParams

type QueryParams struct {
	Context     context.Context
	Compiler    *ast.Compiler
	Store       *storage.Storage
	Transaction storage.Transaction
	Input       ast.Value
	Tracer      Tracer
	Path        ast.Ref
}

QueryParams defines input parameters for the query interface.

func NewQueryParams

func NewQueryParams(ctx context.Context, compiler *ast.Compiler, store *storage.Storage, txn storage.Transaction, input ast.Value, path ast.Ref) *QueryParams

NewQueryParams returns a new QueryParams.

func (*QueryParams) NewTopdown

func (q *QueryParams) NewTopdown(query ast.Body) *Topdown

NewTopdown returns a new Topdown object.

This function will not propagate optional values such as the tracer, input document, etc. Those must be set by the caller.

type QueryResult

type QueryResult struct {
	Result   interface{}            // Result contains the document referred to by the params Path.
	Bindings map[string]interface{} // Bindings contains values for variables in the params Input.
}

QueryResult represents a single query result.

func (*QueryResult) String

func (qr *QueryResult) String() string

type QueryResultSet

type QueryResultSet []*QueryResult

QueryResultSet represents a collection of query results.

func Query

func Query(params *QueryParams) (QueryResultSet, error)

Query returns the value of the document referred to by the params' path. If the params' input contains non-ground terms, there may be multiple query results.

Example

Code:play 

package main

import (
	"context"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
	"github.com/open-policy-agent/opa/topdown"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	compiler := ast.NewCompiler()

	// Define a dummy module with rules that produce documents that we will query below.
	module, err := ast.ParseModule("my_module.rego", `package opa.example

p[x] { q[x]; not r[x] }
q[y] { a = [1, 2, 3]; y = a[_] }
r[z] { b = [2, 4]; z = b[_] }`,
	)

	mods := map[string]*ast.Module{
		"my_module": module,
	}

	if compiler.Compile(mods); compiler.Failed() {
		fmt.Println(compiler.Errors)
	}

	if err != nil {
		// Handle error.
	}

	// Instantiate the policy engine's storage layer.
	store := storage.New(storage.InMemoryConfig())

	// Create a new transaction. Transactions allow the policy engine to
	// evaluate the query over a consistent snapshot fo the storage layer.
	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Prepare query parameters. In this case, there are no additional documents
	// required by the policy so the input is nil.
	var input ast.Value
	params := topdown.NewQueryParams(ctx, compiler, store, txn, input, ast.MustParseRef("data.opa.example.p"))

	// Execute the query against "p".
	v1, err1 := topdown.Query(params)

	// Inspect the result.
	fmt.Println("v1:", v1[0].Result)
	fmt.Println("err1:", err1)

}

Output:

v1: [1 3]
err1: <nil>

func (*QueryResultSet) Add

func (qrs *QueryResultSet) Add(qr *QueryResult)

Add inserts a result into the query result set.

func (QueryResultSet) Undefined

func (qrs QueryResultSet) Undefined() bool

Undefined returns true if the query did not find any results.

type Resolver

type Resolver interface {
	Resolve(ref ast.Ref) (value interface{}, err error)
}

Resolver defines the interface for resolving references to base documents to native Go values. The native Go value types map to JSON types.

type Topdown

type Topdown struct {
	Query    ast.Body
	Compiler *ast.Compiler
	Input    ast.Value
	Index    int
	Previous *Topdown
	Store    *storage.Storage
	Tracer   Tracer
	Context  context.Context
	// contains filtered or unexported fields
}

Topdown stores the state of the evaluation process and contains context needed to evaluate queries.

func New

func New(ctx context.Context, query ast.Body, compiler *ast.Compiler, store *storage.Storage, txn storage.Transaction) *Topdown

New returns a new Topdown object without any bindings.

func (*Topdown) Bind

func (t *Topdown) Bind(key ast.Value, value ast.Value, prev *Undo) *Undo

Bind updates t to include a binding from the key to the value. The return value is used to return t to the state before the binding was added.

func (*Topdown) Binding

func (t *Topdown) Binding(k ast.Value) ast.Value

Binding returns the value bound to the given key.

func (*Topdown) Child

func (t *Topdown) Child(query ast.Body) *Topdown

Child returns a new Topdown object to evaluate query without bindings from t.

func (*Topdown) Closure

func (t *Topdown) Closure(query ast.Body) *Topdown

Closure returns a new Topdown object to evaluate query with bindings from t.

func (*Topdown) Current

func (t *Topdown) Current() *ast.Expr

Current returns the current expression to evaluate.

func (*Topdown) Resolve

func (t *Topdown) Resolve(ref ast.Ref) (interface{}, error)

Resolve returns the native Go value referred to by the ref.

func (*Topdown) Step

func (t *Topdown) Step() *Topdown

Step returns a new Topdown object to evaluate the next expression.

func (*Topdown) Unbind

func (t *Topdown) Unbind(undo *Undo)

Unbind updates t by removing the binding represented by the undo.

func (*Topdown) Vars

func (t *Topdown) Vars() map[ast.Var]ast.Value

Vars returns bindings for the vars in the current query

func (*Topdown) WithInput

func (t *Topdown) WithInput(input ast.Value) *Topdown

WithInput returns a new Topdown object that has the input document set.

func (*Topdown) WithTracer

func (t *Topdown) WithTracer(tracer Tracer) *Topdown

WithTracer returns a new Topdown object that has a tracer set.

type Tracer

type Tracer interface {
	Enabled() bool
	Trace(t *Topdown, evt *Event)
}

Tracer defines the interface for tracing in the top-down evaluation engine.

type Undo

type Undo struct {
	Key   ast.Value
	Value ast.Value
	Prev  *Undo
}

Undo represents a binding that can be undone.

type Vars

type Vars map[ast.Var]ast.Value

Vars represents a set of var bindings.

func (Vars) Diff

func (vs Vars) Diff(other Vars) Vars

Diff returns the var bindings in vs that are not in other.

func (Vars) Equal

func (vs Vars) Equal(other Vars) bool

Equal returns true if vs is equal to other.

Source Files

aggregates.go arithmetic.go binary.go builtins.go casts.go doc.go eq.go errors.go ineq.go input.go regex.go sets.go strings.go topdown.go trace.go

Directories

PathSynopsis
topdown/builtinsPackage builtins contains utilities for implementing built-in functions.
topdown/explainPackage explain contains utilities for post-processing traces emitted by the evaluation engine.
Version
v0.4.6
Published
Mar 12, 2017
Platform
js/wasm
Imports
13 packages
Last checked
10 seconds ago

Tools for package owners.