@yozora/tokenizer-break produces Break nodes. See documentation for details.

Install

npm install --save @yozora/tokenizer-break

Usage

@yozora/tokenizer-break is built into @yozora/parser, @yozora/parser-gfm-ex, and @yozora/parser-gfm.

With DefaultParser

A tokenizer must be registered with an IParser before use.

import { DefaultParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import BreakTokenizer from '@yozora/tokenizer-break'

const parser = new DefaultParser()
  .useFallbackTokenizer(new ParagraphTokenizer())
  .useFallbackTokenizer(new TextTokenizer())
  .useTokenizer(new BreakTokenizer())

parser.parse(`
foo
baz

foo\
baz

foo
baz
`)

With @yozora/parser

import YozoraParser from '@yozora/parser'

const parser = new YozoraParser()
parser.parse(`
foo
baz

foo\
baz

foo
baz
`)

With @yozora/parser-gfm

import GfmParser from '@yozora/parser-gfm'

const parser = new GfmParser()
parser.parse(`
foo
baz

foo\
baz

foo
baz
`)

With @yozora/parser-gfm-ex

import GfmExParser from '@yozora/parser-gfm-ex'

const parser = new GfmExParser()
parser.parse(`
foo
baz

foo\
baz

foo
baz
`)

Options

| Name | Type | Required | Default | | :--------- | :------- | :------- | :------------------------------ | | name | string | No | "@yozora/tokenizer-break" | | priority | number | No | TokenizerPriority.SOFT_INLINE |

• name: Unique tokenizer name. Generated tokens retain it so the parser can dispatch later phases to the same tokenizer.

• priority: Processing priority. Tokenizers with higher values run first. During block matching, a higher-priority tokenizer can interrupt a lower-priority tokenizer.

Related

• @yozora/ast
• @yozora/parser
• @yozora/parser-gfm
• @yozora/parser-gfm-ex
• @yozora/react-break
• @yozora/react-markdown
• Live Examples
• Break | Yozora AST
• Documentation