AI & ML
FoleyForge: Free-form, Controllable, Rectified-flow Editing of Sound Effects
arXiv cs.SD0 views
arXiv:2611.08695v1 Announce Type: new Abstract: We present FoleyForge, a generative model for instruction-based editing of sound effects. Given an input clip and a free-form instruction describing the desired change, it applies the edit while preserving the input's identity. A single model handles semantic, stylistic, and DSP-like changes. We introduce a data pipeline that guides an audio language model to author both DSP-based acoustic edits and similarity-preserving semantic edits, thus addressing the lack of publicly available audio editing data. Such data is then used to fine-tune a rectified-flow audio generation model, which we extend to expose attribute controls for loudness change, edit strength, and an input-derived contour envelope, each tunable via classifier-free guidance, yielding a more expressive editing model. FoleyForge achieves state-of-the-art instruction adherence at competitive audio quality and source faithfulness.
Source:https://arxiv.org/abs/2611.08695PreviousInstance-Optimal Estimation with Multiple LLM Judges on a BudgetNextAircraftWitness: How Often Aircraft Can Locate Recordings
Related posts
AI Shakes Volunteer Sites
0Hacker News New
Suitable for Taycan Turbo GT model carbon fiber tail
0Hacker News New
The Most Advanced AI Rabbi
0Hacker News New
Nvidia investigating reports that CEO Jensen Huang was targeted by Chinese intelligence
0Phys.org Tech
Under fire over AI, effective altruism wants to tell its own story
0Phys.org Tech

'I am deeply sorry': Jensen Huang affair with suspected Chinese spy sees Nvidia open investigation into its CEO — but what happens next?
0TechRadar