实时艺术项目---输入：声音---输出：图像（更好的标题？）
我不太确定是否应该在这里问这个问题。 我想做一个艺术项目。 我想使用语音作为输入，使用图像作为输出。 图像随着声音而变化。
我怎样才能意识到这一点？因为我需要实时或低于 50 毫秒的延迟。 起初我认为使用微控制器会更好。 但我想计算巨大的图像，也许我的微控制器无法做到这一点。 例如我想计算 10.000 个移动物体。 我可以用 windows/linux/mircocontroller 实现这一点吗？
如果我能使用Python那就太好了。 或者您认为加工是更好的选择？
您需要更多详细信息吗？
I am not quite sure if I should ask this question here.
I want to make an art project.
I want to use voice as an input, and an image as output.
The image changes accordingly to the sound.
How can I realise this? Because I need realtime or a delay under 50 ms.
At first I thougt it would be better to use a micro controller.
But I want to calculate huge images, maybe I microcontroller can't do this.
For example I want to calculate 10.000 moving objects.
Could I realise this with windows/linux/mircocontroller?
It would be very good if I could use Python.
Or do you thing processing is a better choice?
Do you need more details?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
您是否考虑过使用图形数据流环境，例如 Pure Data (Pd) 或 最大？ Max是商业产品，但Pd是免费的。
即使您最终没有在最终项目中使用 Pd，它也是一个出色的快速原型设计工具。虽然 Pd 的图形处理能力有限，但有一些扩展，例如 Gridflow 和 Gem，这可能会对您有所帮助。当然，使用 Pd，您可以使用 [fiddle~] 对象分析传入的声音，该对象将为您提供各个分音的整体音调和频率/幅度，以及 [env~]，它将为您提供 RMS 幅度。然后，您可以在 Gem 或 Gridflow 中轻松地将声音（音高、振幅、音色）的变化映射到图像的各种属性，例如颜色、形状、元素数量等。
即使在现代桌面 GPU 上，10k 个移动物体听起来也很多！即时计算所有这些位置将消耗大量资源。我认为即使使用像 openFrameworks 这样的专用 C++ 图形库，这也可能是一场斗争。您可能需要考虑优化策略，例如图像的预渲染方面，以及使用实时音频控制来确定在任何给定时间显示哪些预渲染组件。这可能会给人一种控制 10k 个对象的错觉，而实际上其中大部分都是预先渲染的。
祝你好运！
Have you thought about using a graphical dataflow environment like Pure Data (Pd) or Max? Max is a commercial product, but Pd is free.
Even if you don't end up using Pd for your final project, it makes an excellent rapid prototyping tool. Whilst the graphics processing capabilities of Pd are limited, there are extensions such as Gridflow and Gem, which may help you. Certainly with Pd you can analyse incoming sound using the [fiddle~] object, which will give you the overall pitch and frequency/amplitude of individual partials and [env~], which will give you RMS amplitude. You could then very easily map changes in sound (pitch, amplitude, timbre) to various properties of an image such as colour, shapes, number of elements and so on in Gem or Gridflow.
10k moving objects sounds like a heck of a lot even on a modern desktop GPU! Calculating all of those positions on-the-fly is going to consume a lot of resources. I think even with a dedicated C++ graphics library like openFrameworks, this might be a struggle. You might want to consider an optimisation strategy like pre-rendering aspects of the image, and using the real-time audio control to determine which pre-rendered components are displayed at any given time. This might give the illusion of control over 10k objects, when in reality much of it is pre-rendered.
Good luck!
上面的答案是一个很好的答案，PD 非常灵活，但如果你想要更多面向代码并且更适合与 MCU 混合的东西，处理可能会更好。
另一种好方法是将 Csound 与 Csound Python API 结合使用。 Csound 具有陡峭的学习曲线，但具有大量的音频分析功能，并且非常适合低延迟运行。您绝对可以实时分析输入信号，然后将控制值发送到图形环境，并使用 Python 编写脚本。
The above answer is a good one, PD is very flexible, but if you want something more code oriented and better suited to mixing with MCUs, processing might be better.
Another good way to do it would be to use Csound with the Csound Python API. Csound has a steep learning curve, but tons tons of audio analysis functionality and it's very good for running with low latency. You could definitely analyse an input signal in real time and then send control values out to a graphic environment, scripting both with Python.