package watchdog

import "gvisor.dev/gvisor/pkg/sentry/watchdog"

Package watchdog is responsible for monitoring the sentry for tasks that may potentially be stuck or looping inderterminally causing hard to debug hangs in the untrusted app.

It works by periodically querying all tasks to check whether they are in user mode (RunUser), kernel mode (RunSys), or blocked in the kernel (OffCPU). Tasks that have been running in kernel mode for a long time in the same syscall without blocking are considered stuck and are reported.

When a stuck task is detected, the watchdog can take one of the following actions:

1. LogWarning: Logs a warning message followed by a stack dump of all goroutines. If a tasks continues to be stuck, the message will repeat every minute, unless a new stuck task is detected
2. Panic: same as above, followed by panic()

Index ¶

• Variables
• type Action
• func (a *Action) Get() any
• func (a *Action) Set(v string) error
• func (a Action) String() string
• type Opts
• type Watchdog
• func New(k *kernel.Kernel, opts Opts) *Watchdog
• func (w *Watchdog) Start()
• func (w *Watchdog) Stop()

Variables ¶

var DefaultOpts = Opts{

	TaskTimeout:       3 * time.Minute,
	TaskTimeoutAction: LogWarning,

	StartupTimeout:       30 * time.Second,
	StartupTimeoutAction: LogWarning,
}

DefaultOpts is a default set of options for the watchdog.

Types ¶

type Action ¶

type Action int

Action defines what action to take when a stuck task is detected.

const (
	// LogWarning logs warning message followed by stack trace.
	LogWarning Action = iota

	// Panic will do the same logging as LogWarning and panic().
	Panic
)

func (*Action) Get ¶

func (a *Action) Get() any

Get implements flag.Value.

func (*Action) Set ¶

func (a *Action) Set(v string) error

Set implements flag.Value.

func (Action) String ¶

func (a Action) String() string

String returns Action's string representation.

type Opts ¶

type Opts struct {
	// TaskTimeout is the amount of time to allow a task to execute the
	// same syscall without blocking before it's declared stuck.
	TaskTimeout time.Duration

	// TaskTimeoutAction indicates what action to take when a stuck tasks
	// is detected.
	TaskTimeoutAction Action

	// StartupTimeout is the amount of time to allow between watchdog
	// creation and calling watchdog.Start.
	StartupTimeout time.Duration

	// StartupTimeoutAction indicates what action to take when
	// watchdog.Start is not called within the timeout.
	StartupTimeoutAction Action
}

Opts configures the watchdog.

type Watchdog ¶

type Watchdog struct {
	// Configuration options are embedded.
	Opts
	// contains filtered or unexported fields
}

Watchdog is the main watchdog class. It controls a goroutine that periodically analyses all tasks and reports if any of them appear to be stuck.

func New ¶

func New(k *kernel.Kernel, opts Opts) *Watchdog

New creates a new watchdog.

func (*Watchdog) Start ¶

func (w *Watchdog) Start()

Start starts the watchdog.

func (*Watchdog) Stop ¶

func (w *Watchdog) Stop()

Stop requests the watchdog to stop and wait for it.

Source Files ¶

watchdog.go