基于手部动作感应和即时音频合成的虚拟演奏系统

        我们以EC5-1719CLDNA平台为基础开发了一套虚拟的音乐演奏系统。所谓虚拟，是指这套系统展示在你面前的只有一个虚拟的场景，而在你手中的也不再是沉重的吉他、笛子，而只有一副手套。这副手套允许你在空气中进行乐器的演奏。

        系统从音乐演奏的客观需求出发，综合考虑各种乐器的演奏特点，提出了吉他、钢琴、笛子三种类型乐器在同一双手套上实现的“一体化”演奏方案。该方案利用多种传感器和机器视觉对手部动作和位置进行感应，并辅以ZigBee模块实现了信号的无线传输。系统还发挥了平台双核处理器的并行处理优势和多媒体功能，在Visual Studio 2005下利用DirectX SDK开发了它的核心部分——3D演奏场景以及音频合成引擎。系统最终实现了自由演奏，乐谱引导下演奏以及演奏过程中乐谱实时创作的功能。此外系统还加入了基于TCP/IP协议的网络功能，完成了乐谱的远程创作及传输。

We have developed a set of hypothesized music performance system which takes the EC5-1719CLDNA platform as the foundation. It is called hypothesized, because this set of system display a hypothesized scene for the users. There will not be any heavy guitar or bamboo flute but just a pair of gloves for playing.

Embarking from the demand of music performance, considering musical instruments’ performance characteristic, the system proposed an integration performance plan which realizes guitar, piano and bamboo flute with just a pair of gloves. This plan utilizes kinds of sensors and computer vision to induce hand movement, as well as hand position. What is more, wireless transmission of signals is carried out by the employment of ZigBee module. The plan has also fully displayed Dual-core processors' parallel processing superiority and the multimedia functions of the platform. It develops the 3D performance scene and audio synthesis engine, using Visual Studio 2005 and DirectX SDK. The system finally realizes free performance function, performance under music guidance and real-time score creation function. In addition, with network interconnection based on TCP/IP protocol, the system possesses long-distance music score creation and transmission function.


linux操作系统文章专题:linux操作系统详解（linux不再难懂）