package scanner

import "github.com/gorilla/css/scanner"

Package gorilla/css/scanner generates tokens for a CSS3 input.

It follows the CSS3 specification located at:

http://www.w3.org/TR/css3-syntax/

To use it, create a new scanner for a given CSS string and call Next() until the token returned has type TokenEOF or TokenError:

s := scanner.New(myCSS)
for {
	token := s.Next()
	if token.Type == scanner.TokenEOF || token.Type == scanner.TokenError {
		break
	}
	// Do something with the token...
}

Following the CSS3 specification, an error can only occur when the scanner finds an unclosed quote or unclosed comment. In these cases the text becomes "untokenizable". Everything else is tokenizable and it is up to a parser to make sense of the token stream (or ignore nonsensical token sequences).

Note: the scanner doesn't perform lexical analysis or, in other words, it doesn't care about the token context. It is intended to be used by a lexer or parser.

Index ¶

• Constants
• type Scanner
• func New(input string) *Scanner
• func (s *Scanner) Next() *Token
• type Token
• func (t *Token) String() string

Constants ¶

const (
	// Scanner flags.
	TokenError tokenType = iota
	TokenEOF
	// From now on, only tokens from the CSS specification.
	TokenIdent
	TokenAtKeyword
	TokenString
	TokenHash
	TokenNumber
	TokenPercentage
	TokenDimension
	TokenURI
	TokenUnicodeRange
	TokenCDO
	TokenCDC
	TokenS
	TokenComment
	TokenFunction
	TokenIncludes
	TokenDashMatch
	TokenPrefixMatch
	TokenSuffixMatch
	TokenSubstringMatch
	TokenChar
	TokenBOM
)

The complete list of tokens in CSS3.

Types ¶

type Scanner ¶

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner scans an input and emits tokens following the CSS3 specification.

func New ¶

func New(input string) *Scanner

New returns a new CSS scanner for the given input.

func (*Scanner) Next ¶

func (s *Scanner) Next() *Token

Next returns the next token from the input.

At the end of the input the token type is TokenEOF.

If the input can't be tokenized the token type is TokenError. This occurs in case of unclosed quotation marks or comments.

type Token ¶

type Token struct {
	Type   tokenType
	Value  string
	Line   int
	Column int
}

Token represents a token and the corresponding string.

func (*Token) String ¶

func (t *Token) String() string

String returns a string representation of the token.

Source Files ¶

doc.go scanner.go