Yoav Y. Schechner: Research

Home
Contact

Audio Inpainting

The popularity of voice over internet protocol (VoIP) systems is continuously growing. Such systems depend on unreliable internet communication, in which chunks of data often get lost during transmission. Various solutions to this problem were proposed, most of which are better suited to small rates of lost data. This work addresses this problem by filling in missing data using examples taken from prior recorded audio of the same user. Our approach also harnesses statistical priors and data inpainting smoothing techniques. The effectiveness of the proposed solution is demonstrated experimentally, even in large data-gaps, which cannot be handled by standard packet loss concealment techniques.

Publication

    Yuval Bahat, Yoav Y. Schechner and Michael Elad, "Self-content-based audio inpainting,'' Signal Processing 111, pp. 61-72 (2015).

Presentation

  1. "Multi-modal audio inpainting" (25 Mb, PowerPoint)

    The presentation discusses, apart from audio inpainting, some results from subsequent work on incorporating visual features, in addition to the audio features.

Data

    The following are links to sample raw material of our experiments. You can use it in your work if you clearly acknowledge the source by citing our paper "Self-Content-Based Audio Inpainting" detailed above.
  1. Raw material and results using different PLC techniques and configurations (23.8 Mb, Zip).

Related Research

  1. Cross-modal Denoising
  2. Harmony in Motion
  3. Pixels that Sound
2.6MB AVI
Story experiment: pierced data, uncorrected. Instances of piercing marked in red.
2.6MB AVI
Result of our audio inpainting, including regularization
2.5MB AVI
Packet Loss Concealement using state of the art G.723.1 PLC