npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@earthranger/react-native-audio-form-fill

v0.0.1

Published

Offline, on-device dictation that fills JSON Schema forms using Whisper and llama.cpp

Readme

react-native-audio-form-fill

On-device speech-to-form filling for React Native. Records speech, transcribes it with Whisper, and fills JSON Schema form fields with a small local language model whose output is constrained to the schema. Runs offline after a one-time model download.

Install

yarn add @earthranger/react-native-audio-form-fill react-native-fs react-native-audio-api@^0.13.0 \
  whisper.rn@^0.7.4 llama.rn@^0.12.9
cd ios && pod install

Peer dependencies: react-native-fs downloads and stores models, react-native-audio-api records speech, whisper.rn (whisper.cpp) transcribes it and llama.rn (llama.cpp) fills the form. The React Native New Architecture is required by both native model libraries.

Recording needs microphone permission: NSMicrophoneUsageDescription in the iOS Info.plist, and android.permission.RECORD_AUDIO in the Android manifest. The hook asks for permission the first time it records.

whisper.rn 0.7.4 patch

whisper.rn 0.7.4 needs a patch until it is fixed upstream. Copy docs/patches/whisper.rn+0.7.4.patch into your app's patches/ folder and apply it with patch-package. It fixes two problems:

  • Its package.json exports map has no "." or "./package.json" entry, so React Native codegen silently skips the library (iOS fails with RNWhisperSpec/RNWhisperSpec.h not found) and Jest/Node cannot resolve it.
  • Its podspec copies headers from every slice of the prebuilt xcframework, which collide under use_frameworks! :linkage => :static ("Multiple commands produce …/Headers/*.h").

Extraction

Pure TypeScript with no native dependencies:

| Export | Purpose | |---|---| | getDictationFields(jsonSchema) | Lists the top-level fields a model can fill (text, number, integer, boolean, date/time, single and multi choice) and the ones it skips (objects such as locations, collections, attachments, deprecated/hidden fields, choice lists over MAX_DICTATION_CHOICE_OPTIONS). | | buildExtractionSchema(fields) | JSON Schema for the model's output, for grammar-constrained decoding (e.g. llama.cpp response_format). Choice fields are limited to their option titles. | | buildExtractionMessages(fields, transcript, currentValues, now) | System and user chat messages for the extraction prompt. | | parseExtractionOutput(text) | Parses the model's JSON reply, tolerating code fences or leading text. | | coerceExtractedValues(fields, extracted) | Maps choice titles back to stored values and enforces min/max, pattern, max length and date formats. Returns values to merge into the form and rejected values with a reason. |

Choice options are read from v1 enum + enumNames (array or index-keyed object) and from v2 oneOf / anyOf: [{ oneOf }] { const, title } lists, as produced by @earthranger/react-native-jsonforms-formatter.

import {
  buildExtractionMessages,
  buildExtractionSchema,
  coerceExtractedValues,
  getDictationFields,
  parseExtractionOutput,
} from '@earthranger/react-native-audio-form-fill';

const { fields } = getDictationFields(jsonSchema);
const messages = buildExtractionMessages(fields, transcript, formData, new Date());
const schema = buildExtractionSchema(fields);

// Run any chat model with `messages`, constraining its output to `schema`, then:
const { values, rejected } = coerceExtractedValues(fields, parseExtractionOutput(modelOutput));

Models

| Export | Purpose | |---|---| | LOCAL_MODELS | Built-in registry: Whisper base and small (speech), Qwen3 1.7B (the default) and Qwen2.5 1.5B Instruct (form filling), downloaded from Hugging Face. | | getModelsForType(type, models?) | Models of type 'speech' or 'language'. Pass your own list to use different models. | | getModelForType(type, id, models?) | The model with a saved id, falling back to the first model of that type. | | downloadModel(model, onProgress) | Downloads to <Documents>/local-ai-models. Writes to a .partial file and only moves it into place after a 200 response, so an interrupted download never looks complete. | | isModelDownloaded(model), getModelPath(model), deleteModel(model) | Locate and remove downloaded files. |

import { downloadModel, getModelForType, isModelDownloaded } from '@earthranger/react-native-audio-form-fill';

const speechModel = getModelForType('speech', savedSpeechModelId);
if (!(await isModelDownloaded(speechModel))) {
  await downloadModel(speechModel, (fraction) => setProgress(fraction));
}

Engine

| Export | Purpose | |---|---| | transcribeAudio(model, audioPath, shouldTranslateToEnglish) | Transcribes a WAV file with Whisper (language auto-detected; optionally translated to English). Returns the trimmed transcript, detected language and timings. | | extractFormValues(model, fields, transcript, currentValues) | Fills fields from the transcript with a llama.cpp model whose output is constrained to buildExtractionSchema(fields), then checks it with coerceExtractedValues. Returns values, rejected, the model's rawOutput, and load, generation and token timings. | | releaseDictationModels() | Frees the loaded models. |

One speech model and one language model stay loaded between calls, because loading a 1 GB model takes several seconds; selecting a different model releases the previous one. The language model runs with every layer offloaded to the GPU (Metal on iOS) where llama.cpp supports it, and on the CPU otherwise.

import { extractFormValues, getDictationFields, transcribeAudio } from '@earthranger/react-native-audio-form-fill';

const { transcript } = await transcribeAudio(speechModel, wavPath, true);
const { values, rejected } = await extractFormValues(languageModel, getDictationFields(jsonSchema).fields,
  transcript, formData);

Dictation hook

useFormDictation(options) drives a dictation button: one tap starts recording, the next stops it, transcribes, fills the form and calls onFilled with the checked values. It records a 16 kHz mono WAV file and deletes it afterwards.

| Option | | |---|---| | jsonSchema | The form's JSON Schema, as passed to the form renderer. | | currentValues | The form's current data, sent to the model as context. Read when dictation finishes, so values typed meanwhile are included. | | onFilled(values) | Called when at least one field was filled. Merge values into the form data. | | speechModel, languageModel | Downloaded models to use (see Models). | | shouldTranslateToEnglish | Translate speech to English before filling, for forms whose options are in English. | | logger | Optional { debug, info, warn, error }. Receives status changes, the field summary, transcript, filled and rejected values, the raw model output and a metrics line. | | getUsedMemory | Optional () => Promise<bytes> (e.g. DeviceInfo.getUsedMemory), added to the metrics line. |

Returns status ('idle' | 'recording' | 'transcribing' | 'extracting'), fieldCount (hide the button when it's 0) and toggleRecording.

const { status, fieldCount, toggleRecording } = useFormDictation({
  jsonSchema,
  currentValues: formData,
  onFilled: (values) => setFormData((data) => ({ ...data, ...values })),
  speechModel,
  languageModel,
  shouldTranslateToEnglish: true,
  logger: console,
});

Model settings hook

useLocalModels({ storage, models?, logger? }) manages which speech and form fill models are selected, whether speech is translated to English (default on), and model downloads. Selections are saved through an AudioFormFillStorage you provide, under AUDIO_FORM_FILL_STORAGE_KEYS.

interface AudioFormFillStorage {
  getString: (key: string) => string | undefined;
  getBoolean: (key: string) => boolean | undefined;
  set: (key: string, value: string | boolean) => void;
  // Optional: report changes so every screen using the hook stays in sync. Returns an unsubscribe function.
  subscribe?: (onChange: (key: string) => void) => () => void;
}

Create the storage once (e.g. at module level), not inside a component. With MMKV:

import { MMKV } from 'react-native-mmkv';
import { AudioFormFillStorage, useLocalModels } from '@earthranger/react-native-audio-form-fill';

const mmkv = new MMKV();

export const audioFormFillStorage: AudioFormFillStorage = {
  getString: (key) => mmkv.getString(key),
  getBoolean: (key) => mmkv.getBoolean(key),
  set: (key, value) => mmkv.set(key, value),
  subscribe: (onChange) => {
    const listener = mmkv.addOnValueChangedListener(onChange);
    return () => listener.remove();
  },
};

const {
  speechModel, languageModel, areModelsReady, shouldTranslateToEnglish,
  speechModels, languageModels, downloadedModelIds, downloadProgress, downloadError,
  onSelectModel, onDownloadModel, setShouldTranslateToEnglish, refreshDownloadedModels,
} = useLocalModels({ storage: audioFormFillStorage });

Downloaded models are checked when the hook mounts. Call refreshDownloadedModels when a screen regains focus (e.g. from React Navigation's useFocusEffect) to pick up downloads made on another screen.

Development

npm install      # also points git hooks at .githooks (lint on commit)
npm run lint
npm run typecheck
npm test
npm run build    # compiles src/ to dist/ (CommonJS + type declarations)

Metro resolves the TypeScript source through the react-native field in package.json; Node and Jest use dist/.

License

Apache-2.0