package tag

import "github.com/jdkato/prose/tag"

Package tag implements functions for tagging parts of speech.

Index ¶

• type AveragedPerceptron
• func NewAveragedPerceptron(weights map[string]map[string]float64, tags map[string]string, classes []string) *AveragedPerceptron
• type PerceptronTagger
• func NewPerceptronTagger() *PerceptronTagger
• func NewTrainedPerceptronTagger(model *AveragedPerceptron) *PerceptronTagger
• func (pt *PerceptronTagger) Classes() []string
• func (pt *PerceptronTagger) Tag(words []string) []Token
• func (pt *PerceptronTagger) TagMap() map[string]string
• func (pt *PerceptronTagger) Train(sentences TupleSlice, iterations int)
• func (pt *PerceptronTagger) Weights() map[string]map[string]float64
• type Token
• type TupleSlice
• func ReadTagged(text, sep string) TupleSlice
• func (t TupleSlice) Len() int
• func (t TupleSlice) Swap(i, j int)

Examples ¶

• ReadTagged

Types ¶

type AveragedPerceptron ¶

type AveragedPerceptron struct {
	// contains filtered or unexported fields
}

AveragedPerceptron is a Averaged Perceptron classifier.

func NewAveragedPerceptron ¶

func NewAveragedPerceptron(weights map[string]map[string]float64,
	tags map[string]string, classes []string) *AveragedPerceptron

NewAveragedPerceptron creates a new AveragedPerceptron model.

type PerceptronTagger ¶

type PerceptronTagger struct {
	// contains filtered or unexported fields
}

PerceptronTagger is a port of Textblob's "fast and accurate" POS tagger. See https://github.com/sloria/textblob-aptagger for details.

func NewPerceptronTagger ¶

func NewPerceptronTagger() *PerceptronTagger

NewPerceptronTagger creates a new PerceptronTagger and loads the built-in AveragedPerceptron model.

func NewTrainedPerceptronTagger ¶

func NewTrainedPerceptronTagger(model *AveragedPerceptron) *PerceptronTagger

NewTrainedPerceptronTagger creates a new PerceptronTagger using the given model.

func (*PerceptronTagger) Classes ¶

func (pt *PerceptronTagger) Classes() []string

Classes returns the model's classes in the form

["EX", "NNPS", "WP$", ...]

func (*PerceptronTagger) Tag ¶

func (pt *PerceptronTagger) Tag(words []string) []Token

Tag takes a slice of words and returns a slice of tagged tokens.

func (*PerceptronTagger) TagMap ¶

func (pt *PerceptronTagger) TagMap() map[string]string

TagMap returns the model's classes in the form

{
  "four": "CD",
  "facilities": "NNS",
  ...
}

func (*PerceptronTagger) Train ¶

func (pt *PerceptronTagger) Train(sentences TupleSlice, iterations int)

Train an Averaged Perceptron model based on sentences.

func (*PerceptronTagger) Weights ¶

func (pt *PerceptronTagger) Weights() map[string]map[string]float64

Weights returns the model's weights in the form

{
  "i-1 suffix ity": {
    "MD": -0.816,
    "VB": -0.695,
    ...
   }
   ...
}

type Token ¶

type Token struct {
	Text string
	Tag  string
}

Token represents a tagged section of text.

type TupleSlice ¶

type TupleSlice [][][]string

TupleSlice is a slice of tuples in the form (words, tags).

func ReadTagged ¶

func ReadTagged(text, sep string) TupleSlice

ReadTagged converts pre-tagged input into a TupleSlice suitable for training.

{
	tagged := "Pierre|NNP Vinken|NNP ,|, 61|CD years|NNS"
	fmt.Println(ReadTagged(tagged, "|"))
	// Output: [[[Pierre Vinken , 61 years] [NNP NNP , CD NNS]]]
}

[[[Pierre Vinken , 61 years] [NNP NNP , CD NNS]]]

func (TupleSlice) Len ¶

func (t TupleSlice) Len() int

Len returns the length of a Tuple.

func (TupleSlice) Swap ¶

func (t TupleSlice) Swap(i, j int)

Swap switches the ith and jth elements in a Tuple.

Source Files ¶

aptag.go tag.go