package godirwalk

import "github.com/karrick/godirwalk"

Package godirwalk provides functions to read and traverse directory trees.

In short, why do I use this library?

* It's faster than `filepath.Walk`.

* It's more correct on Windows than `filepath.Walk`.

* It's more easy to use than `filepath.Walk`.

* It's more flexible than `filepath.Walk`.

USAGE

This library will normalize the provided top level directory name based on the os-specific path separator by calling `filepath.Clean` on its first argument. However it always provides the pathname created by using the correct os-specific path separator when invoking the provided callback function.

dirname := "some/directory/root"
err := godirwalk.Walk(dirname, &godirwalk.Options{
    Callback: func(osPathname string, de *godirwalk.Dirent) error {
        fmt.Printf("%s %s\n", de.ModeType(), osPathname)
        return nil
    },
})

This library not only provides functions for traversing a file system directory tree, but also for obtaining a list of immediate descendants of a particular directory, typically much more quickly than using `os.ReadDir` or `os.ReadDirnames`.

dirname := "some/directory"
names, err := godirwalk.ReadDirnames()

Index ¶

• Constants
• Variables
• func ReadDirnames(osDirname string, _ []byte) ([]string, error)
• func Walk(pathname string, options *Options) error
• type Dirent
• func NewDirent(osPathname string) (*Dirent, error)
• func (de Dirent) IsDevice() bool
• func (de Dirent) IsDir() bool
• func (de Dirent) IsRegular() bool
• func (de Dirent) IsSymlink() bool
• func (de Dirent) ModeType() os.FileMode
• func (de Dirent) Name() string
• type Dirents
• func ReadDirents(osDirname string, _ []byte) (Dirents, error)
• func (l Dirents) Len() int
• func (l Dirents) Less(i, j int) bool
• func (l Dirents) Swap(i, j int)
• type ErrorAction
• type Options
• type Scanner
• func NewScanner(osDirname string) (*Scanner, error)
• func (s *Scanner) Dirent() (*Dirent, error)
• func (s *Scanner) Err() error
• func (s *Scanner) Name() string
• func (s *Scanner) Scan() bool
• type WalkFunc

Constants ¶

const DefaultScratchBufferSize = 0

DefaultScratchBuffer is a deprecated config parameter, whose usage was obsoleted by the introduction of the Scanner struct, and migrating ReadDirents, ReadDirnames, and Walk to use Scanner for enumerating directory contents.

Variables ¶

var MinimumScratchBufferSize = os.Getpagesize()

MinimumScratchBufferSize specifies the minimum size of the scratch buffer that Walk, ReadDirents, ReadDirnames, and Scandir will use when reading file entries from the operating system. It is initialized to the result from calling `os.Getpagesize()` during program startup.

Functions ¶

func ReadDirnames ¶

func ReadDirnames(osDirname string, _ []byte) ([]string, error)

ReadDirnames returns a slice of strings, representing the immediate descendants of the specified directory. If the specified directory is a symbolic link, it will be resolved.

The second parameter was an optional scratch buffer, but is no longer used because ReadDirents invokes Scanner to enumerate the contents of the directory.

Note that this function, depending on operating system, may or may not invoke the ReadDirents function, in order to prepare the list of immediate descendants. Therefore, if your program needs both the names and the file system mode types of descendants, it will always be faster to invoke ReadDirents directly, rather than calling this function, then looping over the results and calling os.Stat or os.LStat for each entry.

children, err := godirwalk.ReadDirnames(osDirname, nil)
if err != nil {
    return nil, errors.Wrap(err, "cannot get list of directory children")
}
sort.Strings(children)
for _, child := range children {
    fmt.Printf("%s\n", child)
}

func Walk ¶

func Walk(pathname string, options *Options) error

Walk walks the file tree rooted at the specified directory, calling the specified callback function for each file system node in the tree, including root, symbolic links, and other node types.

This function is often much faster than filepath.Walk because it does not invoke os.Stat for every node it encounters, but rather obtains the file system node type when it reads the parent directory.

If a runtime error occurs, either from the operating system or from the upstream Callback or PostChildrenCallback functions, processing typically halts. However, when an ErrorCallback function is provided in the provided Options structure, that function is invoked with the error along with the OS pathname of the file system node that caused the error. The ErrorCallback function's return value determines the action that Walk will then take.

func main() {
    dirname := "."
    if len(os.Args) > 1 {
        dirname = os.Args[1]
    }
    err := godirwalk.Walk(dirname, &godirwalk.Options{
        Callback: func(osPathname string, de *godirwalk.Dirent) error {
            fmt.Printf("%s %s\n", de.ModeType(), osPathname)
            return nil
        },
        ErrorCallback: func(osPathname string, err error) godirwalk.ErrorAction {
        	// Your program may want to log the error somehow.
        	fmt.Fprintf(os.Stderr, "ERROR: %s\n", err)

        	// For the purposes of this example, a simple SkipNode will suffice,
        	// although in reality perhaps additional logic might be called for.
        	return godirwalk.SkipNode
        },
    })
    if err != nil {
        fmt.Fprintf(os.Stderr, "%s\n", err)
        os.Exit(1)
    }
}

Types ¶

type Dirent ¶

type Dirent struct {
	// contains filtered or unexported fields
}

Dirent stores the name and file system mode type of discovered file system entries.

func NewDirent ¶

func NewDirent(osPathname string) (*Dirent, error)

NewDirent returns a newly initialized Dirent structure, or an error. This function does not follow symbolic links.

This function is rarely used, as Dirent structures are provided by other functions in this library that read and walk directories, but is provided, however, for the occasion when a program needs to create a Dirent.

func (Dirent) IsDevice ¶

func (de Dirent) IsDevice() bool

IsDevice returns true if and only if the Dirent represents a device file.

func (Dirent) IsDir ¶

func (de Dirent) IsDir() bool

IsDir returns true if and only if the Dirent represents a file system directory. Note that on some operating systems, more than one file mode bit may be set for a node. For instance, on Windows, a symbolic link that points to a directory will have both the directory and the symbolic link bits set.

func (Dirent) IsRegular ¶

func (de Dirent) IsRegular() bool

IsRegular returns true if and only if the Dirent represents a regular file. That is, it ensures that no mode type bits are set.

func (Dirent) IsSymlink ¶

func (de Dirent) IsSymlink() bool

IsSymlink returns true if and only if the Dirent represents a file system symbolic link. Note that on some operating systems, more than one file mode bit may be set for a node. For instance, on Windows, a symbolic link that points to a directory will have both the directory and the symbolic link bits set.

func (Dirent) ModeType ¶

func (de Dirent) ModeType() os.FileMode

ModeType returns the mode bits that specify the file system node type. We could make our own enum-like data type for encoding the file type, but Go's runtime already gives us architecture independent file modes, as discussed in `os/types.go`:

Go's runtime FileMode type has same definition on all systems, so that
information about files can be moved from one system to another portably.

func (Dirent) Name ¶

func (de Dirent) Name() string

Name returns the basename of the file system entry.

type Dirents ¶

type Dirents []*Dirent

Dirents represents a slice of Dirent pointers, which are sortable by name. This type satisfies the `sort.Interface` interface.

func ReadDirents ¶

func ReadDirents(osDirname string, _ []byte) (Dirents, error)

ReadDirents returns a sortable slice of pointers to Dirent structures, each representing the file system name and mode type for one of the immediate descendant of the specified directory. If the specified directory is a symbolic link, it will be resolved.

The second parameter was an optional scratch buffer, but is no longer used because ReadDirents invokes Scanner to enumerate the contents of the directory.

children, err := godirwalk.ReadDirents(osDirname, nil)
if err != nil {
    return nil, errors.Wrap(err, "cannot get list of directory children")
}
sort.Sort(children)
for _, child := range children {
    fmt.Printf("%s %s\n", child.ModeType, child.Name)
}

func (Dirents) Len ¶

func (l Dirents) Len() int

Len returns the count of Dirent structures in the slice.

func (Dirents) Less ¶

func (l Dirents) Less(i, j int) bool

Less returns true if and only if the Name of the element specified by the first index is lexicographically less than that of the second index.

func (Dirents) Swap ¶

func (l Dirents) Swap(i, j int)

Swap exchanges the two Dirent entries specified by the two provided indexes.

type ErrorAction ¶

type ErrorAction int

ErrorAction defines a set of actions the Walk function could take based on the occurrence of an error while walking the file system. See the documentation for the ErrorCallback field of the Options structure for more information.

const (
	// Halt is the ErrorAction return value when the upstream code wants to halt
	// the walk process when a runtime error takes place. It matches the default
	// action the Walk function would take were no ErrorCallback provided.
	Halt ErrorAction = iota

	// SkipNode is the ErrorAction return value when the upstream code wants to
	// ignore the runtime error for the current file system node, skip
	// processing of the node that caused the error, and continue walking the
	// file system hierarchy with the remaining nodes.
	SkipNode
)

type Options ¶

type Options struct {
	// ErrorCallback specifies a function to be invoked in the case of an error
	// that could potentially be ignored while walking a file system
	// hierarchy. When set to nil or left as its zero-value, any error condition
	// causes Walk to immediately return the error describing what took
	// place. When non-nil, this user supplied function is invoked with the OS
	// pathname of the file system object that caused the error along with the
	// error that took place. The return value of the supplied ErrorCallback
	// function determines whether the error will cause Walk to halt immediately
	// as it would were no ErrorCallback value provided, or skip this file
	// system node yet continue on with the remaining nodes in the file system
	// hierarchy.
	//
	// ErrorCallback is invoked both for errors that are returned by the
	// runtime, and for errors returned by other user supplied callback
	// functions.
	ErrorCallback func(string, error) ErrorAction

	// FollowSymbolicLinks specifies whether Walk will follow symbolic links
	// that refer to directories. When set to false or left as its zero-value,
	// Walk will still invoke the callback function with symbolic link nodes,
	// but if the symbolic link refers to a directory, it will not recurse on
	// that directory. When set to true, Walk will recurse on symbolic links
	// that refer to a directory.
	FollowSymbolicLinks bool

	// Unsorted controls whether or not Walk will sort the immediate descendants
	// of a directory by their relative names prior to visiting each of those
	// entries.
	//
	// When set to false or left at its zero-value, Walk will get the list of
	// immediate descendants of a particular directory, sort that list by
	// lexical order of their names, and then visit each node in the list in
	// sorted order. This will cause Walk to always traverse the same directory
	// tree in the same order, however may be inefficient for directories with
	// many immediate descendants.
	//
	// When set to true, Walk skips sorting the list of immediate descendants
	// for a directory, and simply visits each node in the order the operating
	// system enumerated them. This will be more fast, but with the side effect
	// that the traversal order may be different from one invocation to the
	// next.
	Unsorted bool

	// Callback is a required function that Walk will invoke for every file
	// system node it encounters.
	Callback WalkFunc

	// PostChildrenCallback is an option function that Walk will invoke for
	// every file system directory it encounters after its children have been
	// processed.
	PostChildrenCallback WalkFunc

	// ScratchBuffer is a deprecated config parameter, whose usage was obsoleted
	// by the introduction of the Scanner struct, and migrating ReadDirents,
	// ReadDirnames, and Walk to use Scanner for enumerating directory contents.
	ScratchBuffer []byte

	// AllowNonDirectory causes Walk to bypass the check that ensures it is
	// being called on a directory node, or when FollowSymbolicLinks is true, a
	// symbolic link that points to a directory. Leave this value false to have
	// Walk return an error when called on a non-directory. Set this true to
	// have Walk run even when called on a non-directory node.
	AllowNonDirectory bool
}

Options provide parameters for how the Walk function operates.

type Scanner ¶

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner is an iterator to enumerate the contents of a directory.

func NewScanner ¶

func NewScanner(osDirname string) (*Scanner, error)

NewScanner returns a new directory Scanner that lazily enumerates the contents of a single directory.

scanner, err := godirwalk.NewScanner(dirname)
if err != nil {
    fatal("cannot scan directory: %s", err)
}

for scanner.Scan() {
    dirent, err := scanner.Dirent()
    if err != nil {
        warning("cannot get dirent: %s", err)
        continue
    }
    name := dirent.Name()
    if name == "break" {
        break
    }
    if name == "continue" {
        continue
    }
    fmt.Printf("%v %v\n", dirent.ModeType(), dirent.Name())
}
if err := scanner.Err(); err != nil {
    fatal("cannot scan directory: %s", err)
}

func (*Scanner) Dirent ¶

func (s *Scanner) Dirent() (*Dirent, error)

Dirent returns the current directory entry while scanning a directory.

func (*Scanner) Err ¶

func (s *Scanner) Err() error

Err returns the error associated with scanning a directory.

func (*Scanner) Name ¶

func (s *Scanner) Name() string

Name returns the name of the current directory entry while scanning a directory.

func (*Scanner) Scan ¶

func (s *Scanner) Scan() bool

Scan potentially reads and then decodes the next directory entry from the file system.

When it returns false, this releases resources used by the Scanner then returns any error associated with closing the file system directory resource.

type WalkFunc ¶

type WalkFunc func(osPathname string, directoryEntry *Dirent) error

WalkFunc is the type of the function called for each file system node visited by Walk. The pathname argument will contain the argument to Walk as a prefix; that is, if Walk is called with "dir", which is a directory containing the file "a", the provided WalkFunc will be invoked with the argument "dir/a", using the correct os.PathSeparator for the Go Operating System architecture, GOOS. The directory entry argument is a pointer to a Dirent for the node, providing access to both the basename and the mode type of the file system node.

If an error is returned by the Callback or PostChildrenCallback functions, and no ErrorCallback function is provided, processing stops. If an ErrorCallback function is provided, then it is invoked with the OS pathname of the node that caused the error along along with the error. The return value of the ErrorCallback function determines whether to halt processing, or skip this node and continue processing remaining file system nodes.

The exception is when the function returns the special value filepath.SkipDir. If the function returns filepath.SkipDir when invoked on a directory, Walk skips the directory's contents entirely. If the function returns filepath.SkipDir when invoked on a non-directory file system node, Walk skips the remaining files in the containing directory. Note that any supplied ErrorCallback function is not invoked with filepath.SkipDir when the Callback or PostChildrenCallback functions return that special value.

Source Files ¶

debug_release.go dirent.go doc.go modeType.go modeTypeWithoutType.go nameWithoutNamlen.go readdir.go scandir_unix.go scanner.go symdir.go walk.go

Directories ¶