WebM Blob 序列化/反序列化无法正常工作

发布于 2025-01-12 18:45:56 字数 2526 浏览 0 评论 0原文

我正在使用 MediaRecorder 记录 WebM 音频数据块。这些块以斑点的形式提供给我。我正在尝试将它们转换为 Base64 字符串并通过网络发送。我也接受将它们转换为任何东西并通过网络发送的概念。

这是我的 UI 代码

  private attachRecorderDataAvailableListener(): void {
    if (!this.mediaRecorder) {
      return;
    }

    const blobtoBase64 = (blob: Blob): Promise<string> =>
      new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend = () => resolve(reader.result as string);
        reader.readAsDataURL(blob);
      });

    const base64ToBlob = (dataURI: string) => {
      // convert base64 to raw binary data held in a string
      // doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
      const byteString = atob(dataURI.split(",")[1]);

      // separate out the mime component
      const mimeString = dataURI.split(",")[0].split(":")[1].split(";")[0];

      // write the bytes of the string to an ArrayBuffer
      const ab = new ArrayBuffer(byteString.length);

      // create a view into the buffer
      const ia = new Uint8Array(ab);

      // set the bytes of the buffer to the correct values
      for (let i = 0; i < byteString.length; i++) {
        ia[i] = byteString.charCodeAt(i);
      }

      // write the ArrayBuffer to a blob, and you're done
      const blob = new Blob([ab], { type: "audio/webm;codecs=opus" });
      return blob;
    };
    this.mediaRecorder.addEventListener(
      "dataavailable",
      async (event: BlobEvent) => {
        console.log("Media Recorder Blob: ", event.data);
        const base64Str = await blobtoBase64(event.data);
        const dt = new Date().getTime();
        saveAs(event.data, `original-${dt}.webm`);
        saveAs(base64ToBlob(base64Str), `modified-${dt}.webm`);
        if (event.data.size > 0) {
          const input: any = {
            data: await event.data.text(),
          };
          const decoder = new TextDecoder("utf-8");
          await store.dispatch(
            "WebsocketModule/emitMeetingSocketEvent",
            {
              eventType:
                MEETING_WEBSOCKET_EVENT_TYPE.TRANSCRIPTION_AUDIO_RECEIVED,
              eventPayload: input,
            },
            {
              root: true,
            }
          );
        }
      }
    );
  }

问题是 blob 未正确序列化。当我在 VLC 中打开原始文件时,我会得到一个音频短片。当我打开修改后的文件(已在此处和背面翻译)时,由于“解复用器错误”,我收到一个无法播放的音频文件。这与我后端发生的情况一致。

有谁知道如何正确序列化这个 blob,以便我可以通过 websocket 发送它。

I am using MediaRecorder to record chunks of WebM audio data. These chunks are given to me as blobs. I am trying to transform them into base64 strings and send them over the wire. I would also accept the concept of transforming them into anything and send them over the wire.

Here's my code for the UI

  private attachRecorderDataAvailableListener(): void {
    if (!this.mediaRecorder) {
      return;
    }

    const blobtoBase64 = (blob: Blob): Promise<string> =>
      new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend = () => resolve(reader.result as string);
        reader.readAsDataURL(blob);
      });

    const base64ToBlob = (dataURI: string) => {
      // convert base64 to raw binary data held in a string
      // doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
      const byteString = atob(dataURI.split(",")[1]);

      // separate out the mime component
      const mimeString = dataURI.split(",")[0].split(":")[1].split(";")[0];

      // write the bytes of the string to an ArrayBuffer
      const ab = new ArrayBuffer(byteString.length);

      // create a view into the buffer
      const ia = new Uint8Array(ab);

      // set the bytes of the buffer to the correct values
      for (let i = 0; i < byteString.length; i++) {
        ia[i] = byteString.charCodeAt(i);
      }

      // write the ArrayBuffer to a blob, and you're done
      const blob = new Blob([ab], { type: "audio/webm;codecs=opus" });
      return blob;
    };
    this.mediaRecorder.addEventListener(
      "dataavailable",
      async (event: BlobEvent) => {
        console.log("Media Recorder Blob: ", event.data);
        const base64Str = await blobtoBase64(event.data);
        const dt = new Date().getTime();
        saveAs(event.data, `original-${dt}.webm`);
        saveAs(base64ToBlob(base64Str), `modified-${dt}.webm`);
        if (event.data.size > 0) {
          const input: any = {
            data: await event.data.text(),
          };
          const decoder = new TextDecoder("utf-8");
          await store.dispatch(
            "WebsocketModule/emitMeetingSocketEvent",
            {
              eventType:
                MEETING_WEBSOCKET_EVENT_TYPE.TRANSCRIPTION_AUDIO_RECEIVED,
              eventPayload: input,
            },
            {
              root: true,
            }
          );
        }
      }
    );
  }

The problem is that the blob is not serializing correctly. When I open the original file in VLC, I get a short clip of my audio. When I open the modified (which has been translated there and back), I get an unplayable audio file due to a "demux error". This is consistent with what happens on my backend.

Does anyone know how to correctly serialize this blob so that I can send it over a websocket.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

旧时模样 2025-01-19 18:45:56

我可能只是一个白痴,没有正确阅读我复制/粘贴的内容。给我一点时间

编辑

是的,我是个白痴。我需要对数据 URI 调用 fetch 才能正确序列化它。

I might just be an idiot who didn't read what I copied/pasted correctly. Gimme a sec

EDIT

Yeah I'm an idiot. I need to call fetch on the data URI to serialize it correctly.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文