@saco/file-worker

浏览器里按 SHA-256 鉴定文件。整文件那一串和文件服务落盘时算出来的一致，创建分片任务时当作 fileId。

安装

pnpm add @saco/file-worker

类型在 @saco/file-worker。脚本还是 @saco/file-worker/worker.js，一份 ES Module。用 createFileHashWorker 建出来的 Worker，onmessage 的 event.data 是 FileHashReply，postMessage 只接受 FileHashRequest。

Vite

import { createFileHashWorker } from '@saco/file-worker'

const worker = createFileHashWorker(
	new URL('@saco/file-worker/worker.js', import.meta.url),
	{ type: 'module' },
)

?worker 会让 Vite 单独打出 Worker 包。那个构造函数没有消息类型，建好后交给 createFileHashWorker：

import ViteWorker from '@saco/file-worker/worker.js?worker'
import { createFileHashWorker } from '@saco/file-worker'

const worker = createFileHashWorker(new ViteWorker())

Webpack

Webpack 5 认 new Worker(new URL(...))，由它自己把脚本打成 Worker 包。不要加 { type: 'module' }，产物不是浏览器原生模块。

import { createFileHashWorker } from '@saco/file-worker'

const worker = createFileHashWorker(
	new URL('@saco/file-worker/worker.js', import.meta.url),
)

算法

• 整文件是 SHA-256，64 个小写十六进制字符。这就是创建任务时的 fileId。
• 默认按 10MB（10485760 字节）切开。每一片单独再算 SHA-256。最后一片可以不足 10MB。
• 整文件哈希是各片按顺序拼起来再算，不是把每片的哈希串起来。chunkSize 只影响每一片的 fileId。
• 空文件的哈希是 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855，chunks 是空数组。

每一片的内容在 chunks[].blob 里，上传时直接放进表单。服务端会按收到的字节再算一遍哈希。blob 是文件上的一段视图，不会把整份文件再复制一份。

消息

发给 Worker：

| 消息 | 字段 | 说明 | | -------- | ----------- | -------------------------------- | | hash | file | 要鉴定的 File | | hash | chunkSize | 可以不传。传了必须是正整数，字节 | | cancel | | 停掉当前这一次 |

Worker 回：

| type | 字段 | 说明 | | ----------- | ------------------------------------------------------- | ------------------------------ | | progress | loaded、total | 已经读过的字节和文件总字节 | | done | fileId、fileName、fileSize、chunkSize、chunks | 鉴定完成 | | error | message | 请求认不出来，或分片大小不合法 | | cancelled | | 收到了 cancel |

chunks 里每一项是 chunkIndex（从 0 开始）、fileId、chunkSize、blob。blob 就是这一片，不用再按 chunkSize 去切原文件。

同一个 Worker 同时只鉴定一个文件。再发一条 hash 会停掉上一条，上一条不会再回 done，也不会回 cancelled。cancel 才会回 cancelled。没有正在跑的任务时发 cancel，同样回 cancelled。

在 Vue 里的分片上传示例

鉴定和上传放在一个 hook 里。不超过默认一片（10MB）的文件直接 POST /upload，不建 Worker，也不走分片。更大的文件才让 Worker 算哈希，done 之后用 axios 创建任务。callback 收到一个对象：type 是阶段，status 表示这一阶段是否成功。阶段有 upload、hash、create、existed、chunk、merge、cancel。按 type 收窄之后，只剩这一阶段自己的字段，例如 hash 上没有 chunkIndex，chunk 上才有。整文件上传结束回调 upload，fileId 用接口返回的那一串。创建任务会带回 progress：existed 为 true 表示这份文件已经在库里，回调 existed，这就是秒传。chunks 里 status 为 1 的片已经传过，跳过。剩下的片同时上传，每一片一个 UploadShard，每一片结束回调 chunk。全部片都到齐再 POST /upload/split/merge，合并结束回调 merge。分片这条的 fileId 用整文件那一串。file 是 <input type="file"> 里的那个 File。

请求写相对路径。开发时 Vite、Webpack 把 /upload 代理到文件服务，生产由 nginx 转发。令牌放在 token 头里。Worker 不上传，也不读取令牌。下面按 Vite 建 Worker；Webpack 去掉 { type: 'module' }。

upload(file, callback) 立刻返回这一份的状态。id 是字符串，取消时传给 cancel，并回调 cancel。进度放在一个响应式数组里。不超过一片时数组里只有这一项，跟着整文件的 onUploadProgress 涨。分片时每一片一项，uploadedSize 把各片已传字节加起来。uploadPercentage 用它除以文件大小。整文件上传时，loading 表示这一次请求还没结束。分片时，loading 是创建任务、传片、合并这三段里任意一段还在进行。

再传一份就再调一次 upload，各自一份状态，不会停掉前一份。超过一片的各自一个 Worker。cancel(id) 中止这一份的请求，有 Worker 时关掉它。组件卸掉时全部停掉。

useFileWorker.ts：

import { computed, onUnmounted, reactive, ref, type Ref } from 'vue'
import axios from 'axios'
import { createFileHashWorker, type FileHashWorker } from '@saco/file-worker'

// 文件服务 ChunkUploadStatus.success
const CHUNK_UPLOADED = 1
// 和 Worker 默认分片同一档。不超过这一档只有一片，直接整文件上传
const FILE_CHUNK_SIZE = 10 * 1024 * 1024

type FileServiceRes<T> = {
	code: number
	message: string
	data: T
}
type StoredFile = {
	fileId: string
	fileName: string
	fileSize: number
}
type SplitChunk = {
	chunkIndex: number
	status: number
}
type SplitProgress = {
	existed: boolean
	chunks: SplitChunk[]
}
type SplitTask = {
	taskId: string
	progress: SplitProgress
}
type UploadShard = {
	fileId: string
	fileName: string
	fileSize: number
	uploadedSize: number
	loading: boolean
}
type UploadCallbackPayload =
	| {
			type: 'upload'
			status: false
	  }
	| {
			type: 'upload'
			status: true
			fileId: string
	  }
	| {
			type: 'hash'
			status: false
	  }
	| {
			type: 'hash'
			status: true
			fileId: string
	  }
	| {
			type: 'create'
			status: false
			fileId: string
	  }
	| {
			type: 'create'
			status: true
			fileId: string
			taskId: string
	  }
	| {
			type: 'existed'
			status: true
			fileId: string
			taskId: string
	  }
	| {
			type: 'chunk'
			status: boolean
			fileId: string
			taskId: string
			chunkIndex: number
	  }
	| {
			type: 'merge'
			status: boolean
			fileId: string
			taskId: string
	  }
	| {
			type: 'cancel'
			status: false
	  }
type UploadTask = {
	shards: UploadShard[]
	worker: FileHashWorker | null
	controller: AbortController
	createLoading: Ref<boolean>
	mergeLoading: Ref<boolean>
	callback: (payload: UploadCallbackPayload) => void
}

/**
 * 鉴定并上传。不超过一片时直接整文件上传。
 * @param token 文件服务令牌
 * @returns { upload, cancel }
 */
export const useFileWorker = (token: string) => {
	const workerUrl = new URL('@saco/file-worker/worker.js', import.meta.url)
	const taskMap = new Map<string, UploadTask>()

	// 组件卸掉时停掉还在传的文件
	onUnmounted(() => {
		taskMap.forEach((task) => {
			task.controller.abort()
			task.worker?.terminate()
		})
		taskMap.clear()
	})

	const instance = {
		/**
		 * 鉴定并上传一份文件。不超过一片时不走分片。
		 * @param file 文件
		 * @param callback 每个阶段结束时调用，参数是 {@link UploadCallbackPayload}
		 */
		upload(file: File, callback: (payload: UploadCallbackPayload) => void) {
			const id = Math.random().toString(36).substring(2, 15)
			const shards = reactive<UploadShard[]>([])
			const controller = new AbortController()
			const { signal } = controller
			const createLoading = ref(true)
			const mergeLoading = ref(false)
			const shardLoading = computed(() => shards.some((shard) => shard.loading))
			const uploadedSize = computed(() => {
				return shards.reduce((sum, shard) => sum + shard.uploadedSize, 0)
			})
			const result = {
				id,
				loading: computed(() => {
					return createLoading.value || shardLoading.value || mergeLoading.value
				}),
				uploadedSize,
				uploadPercentage: computed(() => {
					if (!file.size) return 0
					return Math.min(
						100,
						Math.round((uploadedSize.value / file.size) * 100),
					)
				}),
			}
			// 不超过一片时只有这一片，直接整文件上传。再走分片要多一次建任务和合并。
			if (file.size <= FILE_CHUNK_SIZE) {
				taskMap.set(id, {
					shards,
					worker: null,
					controller,
					createLoading,
					mergeLoading,
					callback,
				})
				const shard: UploadShard = {
					fileId: '',
					fileName: file.name,
					fileSize: file.size,
					uploadedSize: 0,
					loading: true,
				}
				shards.push(shard)
				const body = new FormData()
				body.append('file', file, file.name)
				axios
					.post<FileServiceRes<StoredFile>>('/upload', body, {
						headers: { token },
						signal,
						onUploadProgress(progress) {
							shard.uploadedSize = Math.min(progress.loaded, file.size)
						},
					})
					.then((res) => {
						if (signal.aborted) return
						const stored = res.data.data
						shard.fileId = stored.fileId
						shard.uploadedSize = file.size
						callback({ type: 'upload', status: true, fileId: stored.fileId })
					})
					.catch((error: unknown) => {
						if (axios.isCancel(error) || signal.aborted) return
						callback({ type: 'upload', status: false })
					})
					.finally(() => {
						shard.loading = false
						createLoading.value = false
					})
				return result
			}
			const worker = createFileHashWorker(workerUrl, { type: 'module' })
			taskMap.set(id, {
				shards,
				worker,
				controller,
				createLoading,
				mergeLoading,
				callback,
			})
			worker.onmessage = (event) => {
				const message = event.data
				if (message.type === 'error') {
					createLoading.value = false
					callback({ type: 'hash', status: false })
					return
				}
				if (message.type !== 'done') return
				const { fileId, fileName, fileSize, chunks } = message
				callback({ type: 'hash', status: true, fileId })
				const mergeFile = (taskId: string) => {
					if (signal.aborted || mergeLoading.value) return
					mergeLoading.value = true
					axios
						.post(
							'/upload/split/merge',
							{ taskId },
							{ headers: { token }, signal },
						)
						.then(() => {
							callback({ type: 'merge', status: true, fileId, taskId })
						})
						.catch((error: unknown) => {
							if (axios.isCancel(error) || signal.aborted) return
							callback({ type: 'merge', status: false, fileId, taskId })
						})
						.finally(() => {
							mergeLoading.value = false
						})
				}
				axios
					.post<FileServiceRes<SplitTask>>(
						'/upload/split/create-task',
						{ fileId, fileName, fileSize },
						{ headers: { token }, signal },
					)
					.then((res) => {
						if (signal.aborted) return
						const splitTask = res.data.data
						// 秒传
						if (splitTask.progress.existed) {
							callback({
								type: 'existed',
								status: true,
								fileId,
								taskId: splitTask.taskId,
							})
							return
						}
						callback({
							type: 'create',
							status: true,
							fileId,
							taskId: splitTask.taskId,
						})
						// 片都已在服务端，文件还没合并
						const pending = splitTask.progress.chunks.some(
							(item) => item.status !== CHUNK_UPLOADED,
						)
						if (!pending) {
							mergeFile(splitTask.taskId)
							return
						}
						// 逐片上传
						chunks.forEach((chunk) => {
							const shard: UploadShard = {
								fileId: chunk.fileId,
								fileName,
								fileSize: chunk.chunkSize,
								uploadedSize: 0,
								loading: true,
							}
							shards.push(shard)
							// 已上传过的分片
							const status = splitTask.progress.chunks[chunk.chunkIndex]?.status
							if (status === CHUNK_UPLOADED) {
								shard.uploadedSize = chunk.chunkSize
								shard.loading = false
								return
							}
							// 未上传过的分片，创建上传
							const body = new FormData()
							body.append('file', chunk.blob, fileName)
							body.append('taskId', splitTask.taskId)
							body.append('chunkIndex', String(chunk.chunkIndex))
							axios
								.post('/upload/split/chunk', body, {
									headers: { token },
									signal,
									onUploadProgress(progress) {
										shard.uploadedSize = Math.min(
											progress.loaded,
											chunk.chunkSize,
										)
									},
								})
								.finally(() => {
									shard.loading = false
								})
								.then(() => {
									shard.uploadedSize = chunk.chunkSize
									callback({
										type: 'chunk',
										status: true,
										fileId,
										taskId: splitTask.taskId,
										chunkIndex: chunk.chunkIndex,
									})
									// 还有片在传就先不合并
									if (shardLoading.value) return
									mergeFile(splitTask.taskId)
								})
								.catch((error: unknown) => {
									if (axios.isCancel(error) || signal.aborted) return
									callback({
										type: 'chunk',
										status: false,
										fileId,
										taskId: splitTask.taskId,
										chunkIndex: chunk.chunkIndex,
									})
								})
						})
					})
					.catch((error: unknown) => {
						if (axios.isCancel(error) || signal.aborted) return
						callback({ type: 'create', status: false, fileId })
					})
					.finally(() => {
						createLoading.value = false
					})
			}
			worker.postMessage({ type: 'hash', file })
			return result
		},

		/**
		 * 取消一份上传。
		 * @param uploadId upload 返回的编号
		 */
		cancel(uploadId: string) {
			const task = taskMap.get(uploadId)
			if (!task) return
			task.controller.abort()
			task.worker?.terminate()
			task.callback({ type: 'cancel', status: false })
			task.createLoading.value = false
			task.mergeLoading.value = false
			task.shards.forEach((shard) => {
				shard.loading = false
			})
			taskMap.delete(uploadId)
		},
	}
	return instance
}

页面 upload.vue。useFileWorker 要在 <script setup> 里调用，组件卸掉时才会停掉上传。

<script setup lang="ts">
import { ref } from 'vue'
import { useFileWorker } from './useFileWorker'

const token = '文件服务令牌'
const fileInstance = useFileWorker(token)
const tasks = ref<ReturnType<typeof fileInstance.upload>[]>([])

const onFile = (event: Event) => {
	const input = event.target
	if (!(input instanceof HTMLInputElement) || !input.files) return
	for (const file of input.files) {
		const task = fileInstance.upload(file, (payload) => {
			// payload.type 是阶段，payload.status 是这一阶段是否成功
		})
		tasks.value.push(task)
	}
}
</script>

<template>
	<input type="file" multiple @change="onFile" />
	<div v-for="item in tasks" :key="item.id">
		<span>{{ item.uploadPercentage }}%</span>
		<span>{{ item.uploadedSize }}</span>
		<button type="button" @click="fileInstance.cancel(item.id)">取消</button>
	</div>
</template>