PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Granular Synthesis for Display of Time-Varying Probability Densities
John Williamson and Roderick Murray-Smith
IEEE multimedia Volume in press, 2004.


Abstract—We present a method for displaying time-varying probabilistic information to users using asynchronous granular synthesis.We extend the basic synthesis technique to include distribution over waveform source, spatial position, pitch, and time inside waveforms. To enhance the synthesis in interactive contexts, we “quicken” the display by integrating predictions of user behaviour into the sonification. This includes linear predictions in goal space, and more sophisticated prediction using Monte Carlo sampling to predict future user states in nonlinear dynamic systems. These techniques can be used to improve user performance in continuous control systems and in the interactive exploration of high dimensional spaces. The method provides feedback from users’ potential goals, and their progress toward achieving them; modulating the feedback with prediction can help shape users actions toward achieving these goals. We have applied these techniques to challenging control problems as well as to the sonification of online probabilistic gesture recognition. We are using these displays in mobile, gestural interfaces, where visual display is often impractical. The granular synthesis approach is theoretically elegant and easily applied in contexts where dynamic probabilistic displays are required.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Multimodal Integration
ID Code:303
Deposited By:Roderick Murray-Smith
Deposited On:02 December 2004