docast

Documentation Abstract Syntax Tree.

docast is a specification for representing comments as abstract syntax trees.

It implements the unist spec.

Contents

• Introduction
  • Where this specification fits
• Integration
• Nodes (abstract)
  • Node
  • Literal
  • Parent
• Nodes
  • CodeSegment
    • CodeSegmentName
  • Comment
  • CommentCloser
  • CommentLinePrefix
  • CommentMarker
    • CommentMarkers
    • SerializedCommentMarker
    • SerializedMarker
  • CommentOpener
  • CommentPadding
  • Identifier
  • InlineTag
  • Namepath
  • NamepathConnector
    • SerializedNamepathConnector
  • Root
  • Summary
  • Tag
  • TagName
    • SerializedTagName
  • TypeMetadata
• Content model
  • CommentContent
  • InlineTagContent
  • PhrasingContent
  • RootContent
  • SummaryContent
  • TagContent
  • TypeExpression
• Glossary
• List of utilities
• Project
  • Version
  • Contribute
  • Sponsor

Introduction

This document defines a format for representing comments as abstract syntax trees. Development of docast started in October 2022. This specification is written in a TypeScript-like grammar.

Where this specification fits

docast extends unist, a format for syntax trees, to benefit from its ecosystem of utilities. It also integrates with mdast, a specification for representing markdown.

docast relates to JavaScript and TypeScript in that both languages support docblock comments. docast is language-agnostic, however, and can be used with any source language that supports comments.

docast relates to JSDoc, TSDoc, and typedoc in that these tools parse docblock comments. These tools also define or recognize sets of tags with established semantics. If developers already have a set of tags they're using, they must spend additional time configuring those tags for their chosen tool. docast, however, does not enforce any tag semantics — the user does. Tag specifications can be left to an ESLint rule, or a setting akin to jsdoc/check-tag-names or jsdoc.structuredTags.

Integration

TypeScript users can integrate docast type definitions into their project by installing the appropriate packages:

yarn add @flex-development/docast

Nodes (abstract)

Node

interface Node extends unist.Node {}

Node (unist.Node) is a syntactic unit in docast syntax trees.

Literal

interface Literal extends Node {
  value: bigint | boolean | number | string | null | undefined
}

Literal is an abstract interface in docast containing a scalar value.

Parent

interface Parent extends unist.Parent {
  children: Child[]
}

Parent (unist.Parent) is an abstract interface in docast containing other nodes (said to be children).

Its content is limited to docast content and mdast content.

Nodes

CodeSegment

interface CodeSegment extends Parent {
  children: [CodeSegment | Comment, ...(CodeSegment | Comment)[]]
  data?: CodeSegmentData | undefined
  name?: CodeSegmentName extends never ? string : CodeSegmentName | undefined
  type: 'codeSegment'
}

CodeSegment (Parent) is an abstract representation of a source language AST (or CST) node that a Comment documents.

CodeSegmentName

type CodeSegmentName = CodeSegmentNameMap[keyof CodeSegmentNameMap]

Union of registered source language AST or CST node types.

When developing source language parsers compatible with docast, the CodeSegmentNameMap should be augmented (and exported! :wink:) to register custom node types:

declare module '@flex-development/docast' {
  interface CodeSegmentNameMap {
    arrayType: ArrayType['type']
    assertionPredicate: AssertionPredicate['type']
    bigint: BigIntLiteral['type']
    boolean: BooleanLiteral['type']
    conditionalType: ConditionalType['type']
    constructorType: ConstructorType['type']
    functionType: FunctionType['type']
    genericType: GenericType['type']
    identifier: Identifier['type']
    inferType: InferType['type']
    intersectionType: IntersectionType['type']
    nonNullableType: NonNullableType['type']
    null: NullLiteral['type']
    nullableType: NullableType['type']
    number: NumberLiteral['type']
    objectLiteralType: ObjectLiteralType['type']
    optionalType: OptionalType['type']
    parenthesizedType: ParenthesizedType['type']
    propertyAccessType: PropertyAccessType['type']
    string: StringLiteral['type']
    super: Super['type']
    templateLiteral: TemplateLiteral['type']
    this: This['type']
    tupleType: TupleType['type']
    typeOperation: TypeOperation['type']
    typePredicate: TypePredicate['type']
    typeSymbol: TypeSymbol['type']
    undefined: UndefinedLiteral['type']
    unionType: UnionType['type']
    variadicType: VariadicType['type']
  }
}

Comment

interface Comment extends Parent {
  children:
    | [open: CommentOpener, ...content: CommentContent[]]
    | [open: CommentOpener, ...content: CommentContent[], close: CommentCloser]
  data?: CommentData | undefined
  type: 'comment'
}

Comment (Parent) represents a comment in source content.

Comment can be used in root nodes.
Its content model is comment content.

CommentCloser

interface CommentCloser extends Parent {
  children: CommentMarkers
  data?: CommentCloserData | undefined
  type: 'commentCloser'
}

CommentCloser (Parent) represents the closing syntax of a comment.

It consists of one or more comment marker nodes.

CommentCloser can be used as the final child of comment nodes.

CommentLinePrefix

interface CommentLinePrefix extends Parent {
  children:
    | CommentMarkers
    | [...markers: CommentMarkers, right: CommentPadding]
    | [left: CommentPadding, ...markers: CommentMarkers]
    | [left: CommentPadding, ...markers: CommentMarkers, right: CommentPadding]
  data?: CommentLinePrefixData | undefined
  type: 'commentLinePrefix'
}

CommentLinePrefix (Parent) represents syntax preceding the content of a line within a comment.

It consists of one or more comment marker nodes, optionally surrounded by comment padding.

CommentLinePrefix can be used in comment nodes.

CommentMarker

interface CommentMarker extends Literal {
  data?: CommentMarkerData | undefined
  value: SerializedCommentMarker extends never ? SerializedMarker : SerializedCommentMarker
}

CommentMarker (Literal) represents a marker in comment syntax.

Its value is the serialized marker.

CommentMarker can be used in comment opener, comment closer, and comment line prefix nodes.

CommentMarkers

type CommentMarkers = [marker: CommentMarker, ...markers: CommentMarker[]]

A list of CommentMarker child nodes.

At least one child is required.

SerializedCommentMarker

type SerializedCommentMarker = CommentMarkerMap[keyof CommentMarkerMap]

Union of registered serialized comment markers.

When developing compilers and utilities compatible with docast, the CommentMarkerMap should be augmented (and exported! :wink:) to register custom comment markers:

declare module '@flex-development/docast' {
  interface CommentMarkerMap {
    sass: '*' | '/' | ' '
  }
}

SerializedMarker

type SerializedMarker = string

Any serialized marker.

CommentOpener

interface CommentOpener extends Parent {
  children: CommentMarkers
  data?: CommentOpenerData | undefined
  type: 'commentOpener'
}

CommentOpener (Parent) represents the opening syntax of a comment.

It consists of one or more comment marker nodes.

CommentOpener can be used as the first child of comment nodes.

CommentPadding

interface CommentPadding extends Node {
  data?: CommentPaddingData | undefined
  type: 'commentPadding'
}

CommentPadding (Node) represents paddingwithin a comment line prefix.

Identifier

interface Identifier extends Literal {
  data?: IdentifierData | undefined
  type: 'identifier'
  value: string
}

Identifier (Literal) represents an identifier.
It cannot contain any children — it is a leaf.

Identifier can be used in tag name and namepath nodes.

InlineTag

interface InlineTag extends Parent {
  children: [name: TagName, ...InlineTagContent[]]
  data?: InlineTagData | undefined
  name: string
  type: 'inlineTag'
}

InlineTag (Parent) represents inline metadata.

Inline tags are denoted by wrapping a TagName and any tag content in curly braces ({ and }).

InlineTag can be used in comment, summary and tag nodes.

Namepath

interface Namepath extends Parent {
  children: [identifier: Identifier, ...(Identifier | NamepathConnector)[]]
  data?: NamepathData | undefined
  type: 'namepath'
}

Namepath (Parent) represents a source language namepath.

A namepath consists of one or more identifier nodes separated by namepath connectors.

Namepath can be used in tag nodes.

NamepathConnector

interface NamepathConnector extends Literal {
  data?: NamepathConnectorData | undefined
  type: 'namepathConnector'
  value: SerializedNamepathConnector
}

NamepathConnector (Literal) represents a connector between identifier nodes in a namepath.

It cannot contain any children — it is a leaf.

SerializedNamepathConnector

type SerializedNamepathConnector = '#' | '.' | '~'

The serialized form of a namepath connector.
Its value is one of #, ., or ~.

Root

interface Root extends Parent {
  children: RootContent[]
  data?: RootData | undefined
  type: 'root'
}

Root (Parent) represents a documentation fragment or an entire documented file.

A documented file, also known as a documentation root, is any source file containing comments.
In docast, all comments are considered documentation, with info comments being comments identified as documentation by the surrounding source language.
Parser extensions and other tools can be used to differentiate between info comments and their counterpart, petty comments.

Root can be used as the root of a tree, never as a child.
It can contain code segment and comment nodes.

Summary

interface Summary extends Parent {
  children: SummaryContent[]
  data?: SummaryData | undefined
  type: 'summary'
}

Summary (Parent) represents text at the beginning of a comment.
It starts and ends before any other comment syntax, and may contain markdown content.

Summary can be used in comment nodes.
Its content model is summary.

Tag

interface Tag extends Parent {
  children:
    | [name: TagName, ...UntypedTagContent[]]
    | [name: TagName, Namepath | TypeMetadata, ...UntypedTagContent[]]
    | [name: TagName, type: TypeMetadata, namepath: Namepath, ...UntypedTagContent[]]
  data?: TagData | undefined
  type: 'tag'
}

Tag (Parent) represents top-level metadata.

Tags should be the only element on their line, except in cases where special meaning is assigned to succeeding text. All text following the tag name, up until the start of the next tag name or a comment closer, is considered to be tag content.

Tag can be used in comment nodes. Its content model is tag content.

TagName

interface TagName extends Parent {
  children: [identifier: Identifier]
  data?: TagNameData | undefined
  type: 'tagName'
}

TagName (Parent) represents a tag name.

A tag name consists of an at-sign (@) followed by an identifier.

TagName can be used in tag and inline tag nodes.

SerializedTagName

type SerializedTagName<Identifier extends string = string> = `@${Identifier}`

The serialized form of a tag name.

TypeMetadata

interface TypeMetadata extends Parent {
  children: [expression: TypeExpression]
  data?: TypeMetadataData | undefined
  raw: string
  type: 'typeMetadata'
}

TypeMetadata (Parent) represents an inline type expression (e.g. {number}).

A raw field must be present.
Its value is the raw type expression (e.g. number).

TypeMetadata can be used in tag nodes.
Its content model is type expression.

Content model

type Content =
  | CommentContent
  | InlineTagContent
  | PhrasingContent
  | RootContent
  | SummaryContent
  | TagContent
  | TypeExpression

Nodes are grouped by content type, if applicable.
Each node in docast falls into one or more categories of Content.

CommentContent

type CommentContent = Summary | SummaryContent | Tag

Comment content represents content that can occur in a comment.

FreeformCommentContent

type FreeformCommentContent = Exclude<CommentContent, Summary>

FreeformComment content represents content that can occur in a comment without a summary.

InlineTagContent

type InlineTagContent = Exclude<PhrasingContent, CommentLinePrefix | InlineTag> | Identifier

InlineTag content represents content that can occur inside an inline tag.

It consists of phrasing content and identifier nodes, but cannot contain comment line prefixes or nested inline tags.

PhrasingContent

type PhrasingContent = CommentLinePrefix | InlineTag | mdast.PhrasingContent

Phrasing content represents inline text and markup.

RootContent

type RootContent = CodeSegment | Comment

Root content represents content that can occur in at the root of a tree.

It consists of code segment and comment nodes.

SummaryContent

type SummaryContent = PhrasingContent | mdast.RootContent

Summary content represents summary text and its markup.

TagContent

type TagContent = PhrasingContent | TypeMetadata

Tag content represents tag text and its markup.

UntypedTagContent

type UntypedTagContent = Exclude<TagContent, TypeMetadata>

UntypedTag content represents tag content that does not contain type metadata.

TypeExpression

type TypeExpression = TypeExpressionMap[keyof TypeExpressionMap]

TypeExpression content is a type expression.

When developing compilers and utilities compatible with docast, the TypeExpressionMap should be augmented (and exported! :wink:) to register custom nodes:

declare module '@flex-development/docast' {
  interface TypeExpressionMap {
    arrayType: ArrayType
    assertionPredicate: AssertionPredicate
    bigint: BigIntLiteral
    boolean: BooleanLiteral
    conditionalType: ConditionalType
    constructorType: ConstructorType
    functionType: FunctionType
    genericType: GenericType
    identifier: Identifier
    inferType: InferType
    intersectionType: IntersectionType
    nonNullableType: NonNullableType
    null: NullLiteral
    nullableType: NullableType
    number: NumberLiteral
    objectLiteralType: ObjectLiteralType
    optionalType: OptionalType
    parenthesizedType: ParenthesizedType
    propertyAccessType: PropertyAccessType
    string: StringLiteral
    super: Super
    templateLiteral: TemplateLiteral
    this: This
    tupleType: TupleType
    typeOperation: TypeOperation
    typePredicate: TypePredicate
    typeSymbol: TypeSymbol
    undefined: UndefinedLiteral
    unionType: UnionType
    variadicType: VariadicType
  }
}

Glossary

See the unist glossary for more terms.

Comment

A region of source content used to provide additional information.

Tag content

Text following a tag name (e.g. @example, @param) up until the start of the next tag or comment closer, or text following an inline tag name up until the closing punctuator (}).

List of utilities

See the unist list of utilities for more utilities.

• docast-util-from-comments — parse comments

Project

Version

docast adheres to semver.

Contribute

See CONTRIBUTING.md.

Ideas for new utilities and tools can be posted in docast/ideas.

This project has a code of conduct.
By interacting with this repository, organization, or community you agree to abide by its terms.

Sponsor

Small primitives power larger systems. Support long-term stability by sponsoring Flex Development.