VE Digital Human MCP

MCP server covering the full digital-human chain via the ve-backend proxy: clone (real-person speaking video → avatar with extracted reference voice + transcript), voice synthesis (text → wav in the cloned voice), and avatar lipsync (driving video + audio → talking-head MP4).

Environment

• API_URL: backend API root, default https://api.visionengine-tech.com/api/v1
• API_KEY: user API key from VisionEngine backend (required for submit/query and remote upload)
• WORKDIR: local workspace root, default ./
• DEFAULT_OUTPUT_DIR: default relative output directory, default public/videos
• FILE_MODE: local file handling mode, local or remote, default remote
• REMOTION_WORK_DIR: shared mount root used in local mode, default /vec
• BASE_URL: backend public base url used for /save and /shared links, default https://api.visionengine-tech.com
• REMOTE_VIDEO_UPLOAD_PATH: remote upload path for local videos in remote mode, default public/videos
• REMOTE_AUDIO_UPLOAD_PATH: remote upload path for local audios in remote mode, default public/audios

Tools

Six tools in three stages. Tools never poll internally: after a submit, call the matching query tool, wait the suggested interval, and query again until a terminal status.

| Stage | Submit | Query | | ----------------------------- | ---------------- | --------------- | | Clone (avatar-prep) | clone_submit | clone_query | | Voice synthesis (voice-clone) | voice_submit | voice_query | | Avatar lipsync | lipsync_submit | lipsync_query |

Typical chain (cloned voice, talking-head video)

1. clone_submit with a real-person speaking video (local file / URL / storage path) → avatarId. Free of charge; background voice extraction + transcript generation.
2. Poll clone_query until status: "ready". The result carries avatar.audioUrl (freshly signed ~1-hour reference audio URL), avatar.referenceAudioText (word-for-word transcript), and avatar.videoPath / avatar.audioPath (uid-prefixed storage paths that never expire).
3. voice_submit with text, promptAudioUrl: avatar.audioUrl, promptText: avatar.referenceAudioText. Charged up front per-10k-chars; refunded automatically on failure.
4. Poll voice_query until SUCCEEDED. It downloads the wav by default and returns resultStoragePath (never expires) plus resultUrl (~1 hour, re-query to refresh).
5. lipsync_submit with videoPath: avatar.videoPath and audioPath: <voice resultStoragePath>. 500 credits pre-charged, settled per output second (difference refunded/charged).
6. Poll lipsync_query until SUCCEEDED. It downloads the MP4 by default; expect billingStatus: "settled".

Media input kinds

Every media parameter (videoPath, audioPath, promptAudioUrl) accepts, in order of detection:

• http(s) URL — passed through as-is (including /shared/...?download=true links produced by this server).
• existing local file path — handled by FILE_MODE:
  • local: the path is rewritten to a /shared download URL after removing the REMOTION_WORK_DIR prefix.
  • remote (default): the file is uploaded through the backend /save endpoint and submitted as a /shared URL.
• uid-prefixed Supabase storage path — any string that is neither a URL nor an existing local file is passed through verbatim as a storage-path reference (e.g. avatar.videoPath or a voice result resultStoragePath). Ownership is enforced by the backend; pass chain-produced paths exactly as returned.

promptAudioUrl only accepts http(s) URLs or local files (voice-clone requires a plain URL); to reuse a cloned voice, pass the avatar.audioUrl from clone_query.

Upstream constraints

• Clone driving video: one person speaking clearly, >=8s recommended (>=720p), <=200MB recommended (mirror hard cap 2GB), mp4/mov/webm. Free stage.
• Voice synthesis: text <=3000 characters; reference audio wav/mp3/flac/m4a (24kHz mono wav ideal); output 24kHz mono wav; wait up to ~25 minutes worst case (cold start included).
• Lipsync driving video: same constraints as clone. Audio: wav/mp3/flac/m4a, keep well under 30 minutes. Output: H.264+AAC MP4 at the driving video resolution. Duration: tens of seconds to ~8 minutes; first request after idle may include a cold start; stale tasks auto-fail and refund after 2 hours.

Result URLs

Result URLs are signed on every query and valid ~1 hour — re-run the query tool to refresh. Local downloads (default on for voice_query / lipsync_query) land in DEFAULT_OUTPUT_DIR (public/videos) relative to WORKDIR unless an outputPath is given.