使用exoplayer for USAGE_ASSISTANT和content_type_speech的最佳方法

发布于 2025-01-27 14:24:44 字数 3015 浏览 3 评论 0原文

我正在开发Kotlin中Android的应用程序,该应用程序将用户通过例程。他们可以在执行任务时选择音频帮助。设计此问题的最佳方法是什么?想想偶尔会偶尔射击语音说明之类的东西。

以下是一些要求:

  • 用户可以在后台播放音乐或播客,因此每次我的应用程序都会说话时,内容都必须躲避。
  • 用户无法以任何方式控制或与音频或音频队列进行交互。这实际上只是他们执行的任务的语音助手。
  • 用户可以选择离开该应用程序,并使用其他应用程序,同时仍在接收音频提示。因此,系统永远不应杀死该过程。
  • 音频队列需要有些聪明,因此,如果用户选择跳过步骤,我们可以从队列中删除该提示。

到目前为止,这是我在 exoplayer文档的帮助下开发的体系结构。 href =“ https://github.com/android/uamp” rel =“ nofollow noreferrer”> uamp示例 和 raywenderlich绑定服务教程

  • koin中的媒体模块注入音频服务,下载管理器和我用于保持音频队列的类。然后,我在我的ViewModel中调用Audiomanager以添加项目。
    val mediaModule = module {
        single { AudioPlayerConnection(androidApplication()) }
        single { DownloadManager(get(), get(), get()) }
        single { AudioManager(get(), get()) }
    }
  • audioplayerConnnection是连接器类,创建audioplayerservice,提供绑定,打开调用和访问服务功能。

  • AudioPlayerService是播放从Web下载的MP3音频文件的Audio服务。它包含exoplayerPlayerNotificationManagerAudioPlayerBinder。在创建基本上,基本上会执行以下操作:

    override fun onCreate() {
        super.onCreate()
    
        initializeAudioPlayer() 
        audioManager = applicationContext.getSystemService(Context.AUDIO_SERVICE)
         as AudioManager
    exoPlayer?.volume = 1.0f
    val audioAttributes = android.media.AudioAttributes.Builder()
            .setUsage(android.media.AudioAttributes.USAGE_ASSISTANT)
            .setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
            .build()
    focusRequest = 
        AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK)
            .setAcceptsDelayedFocusGain(false)
            .setAudioAttributes(audioAttributes)
            .setForceDucking(true)
            .build()
    }
  • audiomanager保留@volatile列表,当ViewModel决定给出用户提示时,该列表从ViewModel添加到了ViewModel。它负责下载音频寿命的URL,并将项目添加到列表中。然后,它pings audioplayerConnection可以播放媒体。当audioservice的播放器在player.state_ended中我做以下操作:
    exoPlayer?.clearMediaItems()
    exoPlayer?.seekToDefaultPosition()
    if (audioManager != null && focusRequest != null) {
        audioManager!!.requestAudioFocus(focusRequest!!)
    }
    exoPlayer?.addMediaItem(item.mediaItem)

这似乎很笨拙地处理exoplayer中的队列,但是我有没有找到更好的方法。

当前解决方案的问题是:

  • 显示了下载媒体和播放媒体的无用通知。请注意,用户真的不应该对音频服务一无所知。他们可以在设置中选择以删除提示,但是除此之外,不应将音频播放器功能暴露在它们上。
  • 我很难保留媒体队列中项目的参考,并将其删除,以便如果用户跳过步骤,则不会播放它们。如何以编程方式将对象的引用保留在系工队列中,我可以删除。
  • 该服务似乎没有正确销毁,因此泄漏的人在抱怨。
  • 如果服务崩溃,我不知道在哪里拦截它并再次创建它,以便用户可以继续接收语音帮助。

谢谢您的帮助和建议!

I'm developing an application for Android in Kotlin that takes the user through a routine. They can choose to have audio assistance while they are performing the tasks. What is the best way to design this? Think something like Google Maps shooting off voice directions every once in a while.

Here are a few requirements:

  • Users can play music or podcast in the background, so each time my app speaks, the content must duck.
  • Users cannot control or interact with audio or audio queue in any way. It's really just voice assistant for the tasks they're performing.
  • Users can choose to leave the app, and use other apps, while still receiving audio cues. So the system should never kill the process.
  • The audio queue needs to be somewhat smart, so that if the user chooses to skip the step, we can remove that cue from the queue.

So far, this is the architecture that I have developed with the help of ExoPlayer documentation, UAMP sample, and RayWenderlich binding services tutorial

  • Media module in Koin injects the Audio service, download manager, and a class I use for keeping audio queue. I then call the AudioManager in my ViewModel to add items to it.
    val mediaModule = module {
        single { AudioPlayerConnection(androidApplication()) }
        single { DownloadManager(get(), get(), get()) }
        single { AudioManager(get(), get()) }
    }
  • AudioPlayerConnection is Connector class that creates AudioPlayerService, provides binding, unbinding calls, and access to service's functionality.

  • AudioPlayerService is Audio service that plays mp3 audio files downloaded from the web. It contains ExoPlayer, PlayerNotificationManager and AudioPlayerBinder. On create it basically does the following:

    override fun onCreate() {
        super.onCreate()
    
        initializeAudioPlayer() 
        audioManager = applicationContext.getSystemService(Context.AUDIO_SERVICE)
         as AudioManager
    exoPlayer?.volume = 1.0f
    val audioAttributes = android.media.AudioAttributes.Builder()
            .setUsage(android.media.AudioAttributes.USAGE_ASSISTANT)
            .setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
            .build()
    focusRequest = 
        AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK)
            .setAcceptsDelayedFocusGain(false)
            .setAudioAttributes(audioAttributes)
            .setForceDucking(true)
            .build()
    }
  • AudioManager keeps @Volatile list that is added to from the ViewModel, when the ViewModel decides to give user cues. It is responsible for downloading the url at which the audio lives, and adds the items to the list. It then pings AudioPlayerConnection that the media is ready to be played. When AudioService's players is in Player.STATE_ENDED I do the following:
    exoPlayer?.clearMediaItems()
    exoPlayer?.seekToDefaultPosition()
    if (audioManager != null && focusRequest != null) {
        audioManager!!.requestAudioFocus(focusRequest!!)
    }
    exoPlayer?.addMediaItem(item.mediaItem)

This seems rather clumsy handling of the queue in ExoPlayer, but I have found no better way.

The problems with the current solution are:

  • There is a useless notification shown for downloading the media, and playing the media. Note, the user really shouldn't know anything about audio service. They can choose in the settings to remove the cues, but other than that, no audio player functionality should be exposed to them.
  • I have a hard time keeping the reference for the items in the media queue, and removing them so they are not played if the user skips the step. How do I programmatically keep references of the objects in the exoplayer queue, that I can just remove.
  • The service does not seem to be correctly destroyed, so LeakCanary is complaining.
  • If the service crashes, I have no idea where to intercept that and create it again, so that the user can continue receiving the voice assistance.

Thank you for help and suggestions!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文