package topdown

import "github.com/open-policy-agent/opa/topdown"

Package topdown provides query evaluation support.

The topdown implementation is a (slightly modified) version of the standard "top-down evaluation" algorithm used in query languages such as Datalog. The main difference in the implementation is in the handling of Rego references.

At a high level, the topdown implementation consists of (1) term evaluation and (2) expression evaluation. Term evaluation involves handling expression terms in isolation whereas expression evaluation involves evaluating the terms in relation to each other.

During the term evaluation phase, the topdown implementation will evaluate references found in the expression to produce bindings for:

  1. Variables appearing in references
  2. References to Virtual Documents
  3. Comprehensions

Once terms have been evaluated in isolation, the overall expression can be evaluated. If the expression is simply a term (e.g., string, reference, variable, etc.) then it is compared against the boolean "false" value. If the value IS NOT "false", evaluation continues. On the other hand, if the expression is defined by a built-in operator (e.g., =, !=, min, max, etc.) then the appropriate built-in function is invoked to determine if evaluation continues and bind variables accordingly.

Index

Examples

Constants 

const (

	// InternalErr represents an unknown evaluation error.
	InternalErr = iota

	// ConflictErr indicates multiple (conflicting) values were produced
	// while generating a virtual document. E.g., given two rules that share
	// the same name: p = false :- true, p = true :- true, a query "p" would
	// evaluate p to "true" and "false".
	ConflictErr = iota

	// TypeErr indicates evaluation stopped because an expression was applied to
	// a value of an inappropriate type.
	TypeErr = iota
)

Functions

func Continue 

func Continue(t *Topdown, key, value ast.Value, iter Iterator) error

Continue binds key to value in t and calls the iterator. This is a helper function for simple cases where a single value (e.g., a variable) needs to be bound to a value in order for the evaluation the proceed.

func ContinueN 

func ContinueN(t *Topdown, iter Iterator, x ...ast.Value) error

ContinueN binds N keys to N values. The key/value pairs are passed in as alternating pairs, e.g., key-1, value-1, key-2, value-2, ..., key-N, value-N.

func Eval 

func Eval(t *Topdown, iter Iterator) error

Eval evaluates the query in t and calls iter once for each set of bindings that satisfy all of the expressions in the query.

Example

Code:play  

package main

import (
	"bytes"
	"context"
	"encoding/json"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
	"github.com/open-policy-agent/opa/topdown"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	compiler := ast.NewCompiler()

	// Define a dummy query and some data that the query will execute against.
	query, err := compiler.QueryCompiler().Compile(ast.MustParseBody("data.a[_] = x, x >= 2"))
	if err != nil {
		// Handle error.
	}

	var data map[string]interface{}

	// OPA uses Go's standard JSON library but assumes that numbers have been
	// decoded as json.Number instead of float64. You MUST decode with UseNumber
	// enabled.
	decoder := json.NewDecoder(bytes.NewBufferString(`{"a": [1,2,3,4]}`))
	decoder.UseNumber()

	if err := decoder.Decode(&data); err != nil {
		// Handle error.
	}

	// Instantiate the policy engine's storage layer.
	store := storage.New(storage.InMemoryWithJSONConfig(data))

	// Create a new transaction. Transactions allow the policy engine to
	// evaluate the query over a consistent snapshot fo the storage layer.
	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Prepare the evaluation parameters. Evaluation executes against the policy
	// engine's storage. In this case, we seed the storage with a single array
	// of number. Other parameters such as the request, tracing configuration,
	// etc. can be set on the Topdown object.
	t := topdown.New(ctx, query, compiler, store, txn)

	result := []interface{}{}

	// Execute the query and provide a callbakc function to accumulate the results.
	err = topdown.Eval(t, func(t *topdown.Topdown) error {

		// Each variable in the query will have an associated "binding" in the context.
		x := t.Binding(ast.Var("x"))

		// The bindings are ast.Value types so we will convert to a native Go value here.
		v, err := topdown.ValueToInterface(x, t)
		if err != nil {
			return err
		}

		result = append(result, v)
		return nil
	})

	// Inspect the query result.
	fmt.Println("result:", result)
	fmt.Println("err:", err)

}

Output:

result: [2 3 4]
err: <nil>

func MakeRequest 

func MakeRequest(pairs [][2]*ast.Term) (ast.Value, error)

MakeRequest returns a request value for the given key/value pairs. Assumes keys are valid import paths.

func PlugExpr 

func PlugExpr(expr *ast.Expr, binding Binding) *ast.Expr

PlugExpr returns a copy of expr with bound terms substituted for the binding.

func PlugHead 

func PlugHead(head *ast.Head, binding Binding) *ast.Head

PlugHead returns a copy of head with bound terms substituted for the binding.

func PlugTerm 

func PlugTerm(term *ast.Term, binding Binding) *ast.Term

PlugTerm returns a copy of term with bound terms substituted for the binding.

func PlugValue 

func PlugValue(v ast.Value, binding func(ast.Value) ast.Value) ast.Value

PlugValue returns a copy of v with bound terms substituted for the binding.

func PrettyTrace 

func PrettyTrace(w io.Writer, trace []*Event)

PrettyTrace pretty prints the trace to the writer.

func RegisterBuiltinFunc 

func RegisterBuiltinFunc(name ast.Var, fun BuiltinFunc)

RegisterBuiltinFunc adds a new built-in function to the evaluation engine.

func ResetQueryIDs 

func ResetQueryIDs()

ResetQueryIDs resets the query ID generator. This is only for test purposes.

func ResolveRefs 

func ResolveRefs(v ast.Value, t *Topdown) (ast.Value, error)

ResolveRefs returns the AST value obtained by resolving references to base documents.

func ValueToFloat64 

func ValueToFloat64(v ast.Value, resolver Resolver) (float64, error)

ValueToFloat64 returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage.

func ValueToInt 

func ValueToInt(v ast.Value, resolver Resolver) (int64, error)

ValueToInt returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage.

func ValueToInterface 

func ValueToInterface(v ast.Value, resolver Resolver) (interface{}, error)

ValueToInterface returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage. Composite AST values such as objects and arrays are converted recursively.

func ValueToJSONNumber 

func ValueToJSONNumber(v ast.Value, resolver Resolver) (json.Number, error)

ValueToJSONNumber returns the underlying Go value associated with the AST value. If the value is a reference, the reference is fetched from storage.

func ValueToSlice 

func ValueToSlice(v ast.Value, resolver Resolver) ([]interface{}, error)

ValueToSlice returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage.

func ValueToString 

func ValueToString(v ast.Value, resolver Resolver) (string, error)

ValueToString returns the underlying Go value associated with an AST value. If the value is a reference, the reference is fetched from storage.

func ValueToStrings 

func ValueToStrings(v ast.Value, resolver Resolver) ([]string, error)

ValueToStrings returns a slice of strings associated with an AST value.

Types

type Binding 

type Binding func(ast.Value) ast.Value

Binding defines the interface used to apply term bindings to terms, expressions, etc.

type BufferTracer 

type BufferTracer []*Event

BufferTracer implements the Tracer interface by simply buffering all events received.

func NewBufferTracer 

func NewBufferTracer() *BufferTracer

NewBufferTracer returns a new BufferTracer.

func (*BufferTracer) Enabled 

func (b *BufferTracer) Enabled() bool

Enabled always returns true.

func (*BufferTracer) Trace 

func (b *BufferTracer) Trace(t *Topdown, evt *Event)

Trace adds the event to the buffer.

type BuiltinFunc 

type BuiltinFunc func(t *Topdown, expr *ast.Expr, iter Iterator) (err error)

BuiltinFunc defines the interface that the evaluation engine uses to invoke built-in functions. Users can implement their own built-in functions and register them with the evaluation engine.

Callers are given the current evaluation Context t with the expression expr to be evaluated. Callers can assume that the expression has been plugged with bindings from the current context. If the built-in function determines that the expression has evaluated successfully it should bind any output variables and invoke the iterator with the context produced by binding the output variables.

type Error 

type Error struct {
	Code    int
	Message string
}

Error is the error type returned by the Eval and Query functions when an evaluation error occurs.

func (*Error) Error 

func (e *Error) Error() string

type Event 

type Event struct {
	Op       Op            // Identifies type of event.
	Node     interface{}   // Contains AST node relevant to the event.
	QueryID  uint64        // Identifies the query this event belongs to.
	ParentID uint64        // Identifies the parent query this event belongs to.
	Locals   *ast.ValueMap // Contains local variable bindings from the query context.
}

Event contains state associated with a tracing event.

func (*Event) Equal 

func (evt *Event) Equal(other *Event) bool

Equal returns true if this event is equal to the other event.

func (*Event) HasBody 

func (evt *Event) HasBody() bool

HasBody returns true if the Event contains an ast.Body.

func (*Event) HasExpr 

func (evt *Event) HasExpr() bool

HasExpr returns true if the Event contains an ast.Expr.

func (*Event) HasRule 

func (evt *Event) HasRule() bool

HasRule returns true if the Event contains an ast.Rule.

func (*Event) String 

func (evt *Event) String() string

type Iterator 

type Iterator func(*Topdown) error

Iterator is the interface for processing evaluation results.

type Op 

type Op string

Op defines the types of tracing events. 

const (
	// EnterOp is emitted when a new query is about to be evaluated.
	EnterOp Op = "Enter"

	// ExitOp is emitted when a query has evaluated to true.
	ExitOp Op = "Exit"

	// EvalOp is emitted when an expression is about to be evaluated.
	EvalOp Op = "Eval"

	// RedoOp is emitted when an expression, rule, or query is being re-evaluated.
	RedoOp Op = "Redo"

	// FailOp is emitted when an expression evaluates to false.
	FailOp Op = "Fail"
)

type QueryParams 

type QueryParams struct {
	Context     context.Context
	Compiler    *ast.Compiler
	Store       *storage.Storage
	Transaction storage.Transaction
	Request     ast.Value
	Tracer      Tracer
	Path        ast.Ref
}

QueryParams defines input parameters for the query interface.

func NewQueryParams 

func NewQueryParams(ctx context.Context, compiler *ast.Compiler, store *storage.Storage, txn storage.Transaction, request ast.Value, path ast.Ref) *QueryParams

NewQueryParams returns a new QueryParams.

func (*QueryParams) NewTopdown 

func (q *QueryParams) NewTopdown(body ast.Body) *Topdown

NewTopdown returns a new Topdown object that can be used to do evaluation.

type QueryResult 

type QueryResult struct {
	Result   interface{}            // Result contains the document referred to by the params Path.
	Bindings map[string]interface{} // Bindings contains values for variables in the params Request.
}

QueryResult represents a single query result.

func (*QueryResult) String 

func (qr *QueryResult) String() string

type QueryResultSet 

type QueryResultSet []*QueryResult

QueryResultSet represents a collection of query results.

func Query 

func Query(params *QueryParams) (QueryResultSet, error)

Query returns the value of document referred to by the params Path field. If the params' Request field contains values that are non-ground (i.e., they contain variables), then the result may contain multiple entries.

Example

Code:play  

package main

import (
	"context"
	"fmt"

	"github.com/open-policy-agent/opa/ast"
	"github.com/open-policy-agent/opa/storage"
	"github.com/open-policy-agent/opa/topdown"
)

func main() {
	// Initialize context for the example. Normally the caller would obtain the
	// context from an input parameter or instantiate their own.
	ctx := context.Background()

	compiler := ast.NewCompiler()

	// Define a dummy module with rules that produce documents that we will query below.
	module, err := ast.ParseModule("my_module.rego", `

	    package opa.example

	    p[x] :- q[x], not r[x]
	    q[y] :- a = [1,2,3], y = a[_]
	    r[z] :- b = [2,4], z = b[_]

	`)

	mods := map[string]*ast.Module{
		"my_module": module,
	}

	if compiler.Compile(mods); compiler.Failed() {
		fmt.Println(compiler.Errors)
	}

	if err != nil {
		// Handle error.
	}

	// Instantiate the policy engine's storage layer.
	store := storage.New(storage.InMemoryConfig())

	// Create a new transaction. Transactions allow the policy engine to
	// evaluate the query over a consistent snapshot fo the storage layer.
	txn, err := store.NewTransaction(ctx)
	if err != nil {
		// Handle error.
	}

	defer store.Close(ctx, txn)

	// Prepare query parameters. In this case, there are no additional documents
	// required by the policy so the request is nil.
	var request ast.Value
	params := topdown.NewQueryParams(ctx, compiler, store, txn, request, ast.MustParseRef("data.opa.example.p"))

	// Execute the query against "p".
	v1, err1 := topdown.Query(params)

	// Inspect the result.
	fmt.Println("v1:", v1[0].Result)
	fmt.Println("err1:", err1)

}

Output:

v1: [1 3]
err1: <nil>

func (*QueryResultSet) Add 

func (qrs *QueryResultSet) Add(qr *QueryResult)

Add inserts a result into the query result set.

func (QueryResultSet) Undefined 

func (qrs QueryResultSet) Undefined() bool

Undefined returns true if the query did not find any results.

type Resolver 

type Resolver interface {
	Resolve(ref ast.Ref) (value interface{}, err error)
}

Resolver defines the interface for resolving references to base documents to native Go values. The native Go value types map to JSON types.

type Topdown 

type Topdown struct {
	Query    ast.Body
	Compiler *ast.Compiler
	Request  ast.Value
	Locals   *ast.ValueMap
	Index    int
	Previous *Topdown
	Store    *storage.Storage
	Tracer   Tracer
	Context  context.Context
	// contains filtered or unexported fields
}

Topdown stores the state of the evaluation process and contains context needed to evaluate queries.

func New 

func New(ctx context.Context, query ast.Body, compiler *ast.Compiler, store *storage.Storage, txn storage.Transaction) *Topdown

New returns a new Topdown object without any bindings.

func (*Topdown) Bind 

func (t *Topdown) Bind(key ast.Value, value ast.Value, prev *Undo) *Undo

Bind updates t to include a binding from the key to the value. The return value is used to return t to the state before the binding was added.

func (*Topdown) Binding 

func (t *Topdown) Binding(k ast.Value) ast.Value

Binding returns the value bound to the given key.

func (*Topdown) Child 

func (t *Topdown) Child(query ast.Body, locals *ast.ValueMap) *Topdown

Child returns a new Topdown object to evaluate a query that was referred to by the query of t.

func (*Topdown) Current 

func (t *Topdown) Current() *ast.Expr

Current returns the current expression to evaluate.

func (*Topdown) Resolve 

func (t *Topdown) Resolve(ref ast.Ref) (interface{}, error)

Resolve returns the native Go value referred to by the ref.

func (*Topdown) Step 

func (t *Topdown) Step() *Topdown

Step returns a new Topdown object to evaluate the next expression.

func (*Topdown) Unbind 

func (t *Topdown) Unbind(undo *Undo)

Unbind updates t by removing the binding represented by the undo.

type Tracer 

type Tracer interface {
	Enabled() bool
	Trace(t *Topdown, evt *Event)
}

Tracer defines the interface for tracing in the top-down evaluation engine.

type Undo 

type Undo struct {
	Key   ast.Value
	Value ast.Value
	Prev  *Undo
}

Undo represents a binding that can be undone.

Source Files

aggregates.go arithmetic.go builtins.go casts.go doc.go eq.go ineq.go regex.go request.go sets.go strings.go topdown.go trace.go

Directories

PathSynopsis
topdown/explainPackage explain contains utilities for post-processing traces emitted by the evaluation engine.
Version
v0.3.1
Published
Dec 23, 2016
Platform
js/wasm
Imports
13 packages
Last checked
51 seconds ago –

Tools for package owners.