TS-BIOFILEDETECTOR

Overview

A set of function to predict file formats

Formats included:

• Fasta (proteic/nucleic)
• GFF3 (annotation/alignment/interproscan)
• BED
• VCF
• TSV, with specific typing of :
  • OrthoFinder
  • InterProScan
  • Nicknames
  • PAF
  • Differential Gene Expression Table
• BAM
• BigWig

Installation

npm i @lipme/ts-biofiledetector

Requires Node.js 18 or later.

Usage

import { getFileFormat } from "@lipme/ts-biofiledetector";

const result = await getFileFormat("path/to/file");

Compressed files (gzip, bzip2, xz) are detected from their content, whatever their extension.

The first 1000 lines are read once to select the candidate formats, so a file is usually read in full only once.

Each detector can also be called on its own (isFasta, isGff3, isVcf, isBed, isTsv, isBam, isBigwig); it returns the result, or null when the file is not in that format.

Output

All results share this shape:

{
  format: "fasta|gff3|bed|vcf|tsv|bam|bigwig|unknown",
  valid: true|false,          // false when there are errors
  errors?: [ issue ],         // first 100 errors only
  errorCount?: number,        // total number of errors
  warnings?: [ issue ],       // first 100 warnings only
  warningCount?: number,      // total number of warnings
  metadata: { ... }           // format specific, see below
}

where an issue is:

{
  code: "INVALID_LINE",       // stable identifier, see the list below
  message: "Not a BED line",  // human readable
  line?: 12,                  // 1-based line number, when about one line
  content?: "chr1 abc 20"     // the line, cut to 200 characters
}

| Format | Error codes | Warning codes | | --- | --- | --- | | Fasta | DUPLICATE_ID, UNKNOWN_CHARACTERS | | | GFF3 | INVALID_LINE, INVALID_ATTRIBUTE, EMPTY_ATTRIBUTE | FASTA_SECTION (lines after ##FASTA are not parsed) | | BED | INVALID_LINE, START_AFTER_END, MIXED_COLUMN_COUNTS | TRAILING_WHITESPACE, MIXED_SEPARATORS | | VCF | INVALID_LINE, HEADER_AFTER_DATA | | | TSV | WRONG_FIELD_COUNT | | | BAM | TRUNCATED_HEADER, CORRUPTED_HEADER | |

Format specific results:

• Fasta

{
  format: "fasta",
  valid: true|false,
  metadata: {
    type: "dna|protein",
    category: "sequence",
    seqids: ['list','of','seqids'],
    alignment: true|false, // true if >= 2 sequences, all of the same length, with gaps ('-' or '.')
    alignmentLength?: "alignmentLength", // only when alignment is true
  }
}

• GFF3

{
    format: "gff3",
    valid: true|false,
    metadata: {
    type: "annotation|alignment|interproscan",
    category: "annotation|alignment|protein_match",
    seqids: ['list','of','seqids'],
    features: ['list', 'of', 'features'],
    attributes: {'list' : 'cardinality', 'of' : 'cardinality', 'attributes' : 'cardinality'}
  }
}

• BED

{
  format: "bed",
  valid: true|false,
  metadata: {
    type: "bed3|...|bed12", // "bed" when the number of columns varies
    category: "annotation",
    seqids: ['list','of','seqids'],
    columns: numberOfColumns, // list of the numbers of columns when it varies
    withBrowser: true|false, //browser line found
    withTrack: true|false, //track line found
  }
}

• VCF

{
  format: "vcf",
  valid: true|false,
  metadata: {
    type: "VCFversion",
    category: "polymorphism",
    seqids: ['list','of','seqids'],
    samples: ['list','of','samples'], // empty for a sites-only VCF (no FORMAT/sample columns)
    fields: {'INFO', 'and', 'FORMAT', 'metadata'}
    }
}

• TSV

{
  format: "tsv",
  valid: true|false,
  metadata: {
    type: "unknown|orthofinder|interproscan|nicknames|paf|dge",
    category: "table|alignment", // alignment for paf
    columns: numberOfColumns,
    lines: numberOfLines,
    explicitHeader: true|false, //true if first line stars with '#'
    header: ['list','of','field','names'],
    }
}

• BAM

{
  format: "bam",
  valid: true|false,
  metadata: {
    type: "alignment",
    category: "alignment",
    seqids: ['list', 'of', 'seqids'], // reference names from the binary header
    programs: ['list', 'of', 'programs']  //@PG lines
  }
}

• BigWig

{
  format: "bigwig",
  valid: true,
  metadata: {
    type: "quantitative",
    category: "quantitative",
    seqids: ['list', 'of', 'seqids'], // chromosomes
    version: 'bigwigVersion'
  }
}

Development

npm ci
npm run typecheck   # tsc
npm run lint        # ESLint
npm test            # Jest, with snapshots in tests/__snapshots__ (update them with `npx jest -u`)
npm run build       # tsup, to dist/
npm run test:dist   # loads dist/ through ESM import and CommonJS require

The same steps run in GitLab CI (.gitlab-ci.yml). See CHANGELOG.md for the changes between versions.

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

Contact

For questions or feedback, please open an issue or contact us at [email protected]