当前位置：文江博客话题详情

annotations ReactJS text-to-speech

当音频叙述时，如何实时按照网站上的音频强调文本

发布于 2025-01-23 19:15:32 字数 348 浏览 0 评论 0 原文

我正在尝试根据音频来弄清楚用哪种技术来启动文本。很像 https://spechify.com/ 正在做什么。

这是假设我能够运行TTS算法，并且可以将文本转换为语音。我尝试过多种资源，但我无法确定在音频说的时突出文本的确切技术或方法。

任何帮助都将不胜感激。我已经在互联网上浪费了2天来解决这个问题，但没有运气:(

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

紅太極 2025-01-30 19:15:32

一种简单的方法是使用用香草JS强调单词。发射的事件为我们提供了炭指数，因此无需使用Regexes或Super AI的东西疯狂:)

在其他任何事情之前，请确保API可用，

const synth = window.speechSynthesis
if (!synth) {
  console.error('no tts for you!')
  return
}

TTS话语发出了“边界”事件，我们可以使用它来突出显示文本。

let text = document.getElementById('text')
let originalText = text.innerText
let utterance = new SpeechSynthesisUtterance(originalText)
utterance.addEventListener('boundary', event => {
  const { charIndex, charLength } = event
  text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
})
synth.speak(utterance)

完整示例：

const btn = document.getElementById("btn")

const highlight = (text, from, to) => {
  let replacement = highlightBackground(text.slice(from, to))
  return text.substring(0, from) + replacement + text.substring(to)
}
const highlightBackground = sample => `<span style="background-color:yellow;">${sample}</span>`

btn && btn.addEventListener('click', () => {
  const synth = window.speechSynthesis
  if (!synth) {
    console.error('no tts')
    return
  }
  let text = document.getElementById('text')
  let originalText = text.innerText
  let utterance = new SpeechSynthesisUtterance(originalText)
  utterance.addEventListener('boundary', event => {
    const { charIndex, charLength } = event
    text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
   })
  synth.speak(utterance)
})

codesandbox link

是非常基本的，您可以（并且应该）改进它。

编辑

oops，我忘记了这被标记为reactjs。这是与React相同的示例（CODESANDBOX链接在注释中）：

import React from "react";

const ORIGINAL_TEXT =
  "Call me Ishmael. Some years ago—never mind how long precisely—having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world.";

const splitText = (text, from, to) => [
  text.slice(0, from),
  text.slice(from, to),
  text.slice(to)
];

const HighlightedText = ({ text, from, to }) => {
  const [start, highlight, finish] = splitText(text, from, to);
  return (
    <p>
      {start}
      <span style={{ backgroundColor: "yellow" }}>{highlight}</span>
      {finish}
    </p>
  );
};

export default function App() {
  const [highlightSection, setHighlightSection] = React.useState({
    from: 0,
    to: 0
  });
  const handleClick = () => {
    const synth = window.speechSynthesis;
    if (!synth) {
      console.error("no tts");
      return;
    }

    let utterance = new SpeechSynthesisUtterance(ORIGINAL_TEXT);
    utterance.addEventListener("boundary", (event) => {
      const { charIndex, charLength } = event;
      setHighlightSection({ from: charIndex, to: charIndex + charLength });
    });
    synth.speak(utterance);
  };

  return (
    <div className="App">
      <HighlightedText text={ORIGINAL_TEXT} {...highlightSection} />
      <button onClick={handleClick}>klik me</button>
    </div>
  );
}

A simple approach would be to use the event listener provided by the SpeechSynthesisUtterance boundary event to highlight words with vanilla JS. The emitted event gives us char indices, so no need to go crazy with regexes or super AI stuff :)

Before anything else, make sure the API is available

const synth = window.speechSynthesis
if (!synth) {
  console.error('no tts for you!')
  return
}

The tts utterance emits an 'boundary' event, we can use it to highlight text.

let text = document.getElementById('text')
let originalText = text.innerText
let utterance = new SpeechSynthesisUtterance(originalText)
utterance.addEventListener('boundary', event => {
  const { charIndex, charLength } = event
  text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
})
synth.speak(utterance)

Full example:

const btn = document.getElementById("btn")

const highlight = (text, from, to) => {
  let replacement = highlightBackground(text.slice(from, to))
  return text.substring(0, from) + replacement + text.substring(to)
}
const highlightBackground = sample => `<span style="background-color:yellow;">${sample}</span>`

btn && btn.addEventListener('click', () => {
  const synth = window.speechSynthesis
  if (!synth) {
    console.error('no tts')
    return
  }
  let text = document.getElementById('text')
  let originalText = text.innerText
  let utterance = new SpeechSynthesisUtterance(originalText)
  utterance.addEventListener('boundary', event => {
    const { charIndex, charLength } = event
    text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
   })
  synth.speak(utterance)
})

CodeSandbox link

This is pretty basic, and you can (and should) improve it.

Edit

Ooops, I forgot that this was tagged as ReactJs. Here's the same example with React (codesandbox link is in the comments):

import React from "react";

const ORIGINAL_TEXT =
  "Call me Ishmael. Some years ago—never mind how long precisely—having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world.";

const splitText = (text, from, to) => [
  text.slice(0, from),
  text.slice(from, to),
  text.slice(to)
];

const HighlightedText = ({ text, from, to }) => {
  const [start, highlight, finish] = splitText(text, from, to);
  return (
    <p>
      {start}
      <span style={{ backgroundColor: "yellow" }}>{highlight}</span>
      {finish}
    </p>
  );
};

export default function App() {
  const [highlightSection, setHighlightSection] = React.useState({
    from: 0,
    to: 0
  });
  const handleClick = () => {
    const synth = window.speechSynthesis;
    if (!synth) {
      console.error("no tts");
      return;
    }

    let utterance = new SpeechSynthesisUtterance(ORIGINAL_TEXT);
    utterance.addEventListener("boundary", (event) => {
      const { charIndex, charLength } = event;
      setHighlightSection({ from: charIndex, to: charIndex + charLength });
    });
    synth.speak(utterance);
  };

  return (
    <div className="App">
      <HighlightedText text={ORIGINAL_TEXT} {...highlightSection} />
      <button onClick={handleClick}>klik me</button>
    </div>
  );
}

回复收藏 0 原文

蔚蓝源自深海 2025-01-30 19:15:32

tts-react 提供了接受a usetts 代码> MarkTextAssPoken 参数将突出显示所说的单词。

这是一个例子：

import { useTts } from 'tts-react'

const TTS = ({ children }) => {
  const { ttsChildren, play } = useTts({ children, markTextAsSpoken: true })

  return (
    <div>
      <button onClick={play}>
        Click to hear the text spoken
      </button>
      {ttsChildren}
    </div>

  )
}

const App = () => {
  return <TTS>Some text to be spoken.</TTS>
}

您也可以从CDN加载它：

<!DOCTYPE html>
<html lang="en-US">
  <head>
    <title>tts-react UMD example</title>
    <script src="https://unpkg.com/react@18/umd/react.development.js"></script>
    <script src="https://unpkg.com/react-dom@18/umd/react-dom.development.js"></script>
    <script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
    <script src="https://unpkg.com/[email protected]/dist/umd/tts-react.min.js"></script>
  </head>
  <body>
    <div id="root"></div>
    <script type="text/babel">
      const root = ReactDOM.createRoot(document.getElementById('root'))
      const { TextToSpeech, useTts } = TTSReact
      const CustomTTS = ({ children }) => {
        const { play, ttsChildren } = useTts({ children, markTextAsSpoken: true })

        return (
          <>
            <button onClick={() => play()}>Play</button>
            <div>{ttsChildren}</div>
          </>
        )
      }

      root.render(
        <>
          <CustomTTS>
            <p>Highlight words as they are spoken.</p>
          </CustomTTS>
          <TextToSpeech markTextAsSpoken>
            <p>Highlight words as they are spoken.</p>
          </TextToSpeech>
        </>
      )
    </script>
  </body>
</html>

tts-react provides a hook useTts that accepts a markTextAsSpoken argument that will highlight the words being spoken.

Here's an example:

import { useTts } from 'tts-react'

const TTS = ({ children }) => {
  const { ttsChildren, play } = useTts({ children, markTextAsSpoken: true })

  return (
    <div>
      <button onClick={play}>
        Click to hear the text spoken
      </button>
      {ttsChildren}
    </div>

  )
}

const App = () => {
  return <TTS>Some text to be spoken.</TTS>
}

You can load it from a CDN too:

<!DOCTYPE html>
<html lang="en-US">
  <head>
    <title>tts-react UMD example</title>
    <script src="https://unpkg.com/react@18/umd/react.development.js"></script>
    <script src="https://unpkg.com/react-dom@18/umd/react-dom.development.js"></script>
    <script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
    <script src="https://unpkg.com/[email protected]/dist/umd/tts-react.min.js"></script>
  </head>
  <body>
    <div id="root"></div>
    <script type="text/babel">
      const root = ReactDOM.createRoot(document.getElementById('root'))
      const { TextToSpeech, useTts } = TTSReact
      const CustomTTS = ({ children }) => {
        const { play, ttsChildren } = useTts({ children, markTextAsSpoken: true })

        return (
          <>
            <button onClick={() => play()}>Play</button>
            <div>{ttsChildren}</div>
          </>
        )
      }

      root.render(
        <>
          <CustomTTS>
            <p>Highlight words as they are spoken.</p>
          </CustomTTS>
          <TextToSpeech markTextAsSpoken>
            <p>Highlight words as they are spoken.</p>
          </TextToSpeech>
        </>
      )
    </script>
  </body>
</html>