0

I would like to build a function that does something based on the current value of the waveform of an audio.

Inputs:

  • An audio file: like a speaker object or a path on the hard drive or a track name in VSE or whatever.
  • Frame number.

Outputs:

  • A number that represents how high or low the audio is. But I don't need to get the volume which is constant by default (100%). I need to get a number that represents the current point on the waveform.

Is there a python code that does the following? I'll use it for something like visualization and auto move the mouth (auto speak).

user2824371
  • 796
  • 1
  • 7
  • 27
  • 1
    https://blender.stackexchange.com/search?q=bake+sound+to+fcurve – batFINGER Feb 24 '21 at 08:39
  • @batFINGER But that's too direct. What If I need to play with fcurve after baking? Is there an easy method to change the key frames (add/remove/edit value)? – user2824371 Feb 24 '21 at 08:41
  • once a sound is "baked" can get any value with fcurve.evaluate(frame) . Didn't download but suggest it relates to "I need".. and no to little research. Personally find the aubio module very good for this and have put together a lot of new code to replace blenders bake ... Some related links https://blender.stackexchange.com/questions/127895/how-can-you-invert-an-f-curve https://blender.stackexchange.com/questions/39231/whats-the-range-of-values-on-a-sound-baked-f-curve/39241#39241 https://blender.stackexchange.com/questions/171087/how-to-add-a-minimum-value-to-f-curves – batFINGER Feb 24 '21 at 08:46
  • Re un-baking an fcurve have posted re this .. couldn't find, however found this link https://blender.stackexchange.com/questions/19357/smoothing-a-baked-f-curve (** didn't down-vote (not download)) – batFINGER Feb 24 '21 at 08:50
  • @batFINGER well, I did my own research but I didn't find something useful. I was thinking about scripted expressions (drivers) from my research but not that useful. I'll try to read your links and get back to you. Thanks for helping. – user2824371 Feb 24 '21 at 08:54
  • Re scripting drivers. Spent far too much time on this one https://github.com/batFINGER/batFINGER-blender-addons IIRC also added lip-sync. Have since moved to using aubio, scipy, numpy, matplotlib .. but still way back on back burner. Can dig up some code re aubio -> blender fcurve. – batFINGER Feb 24 '21 at 08:59

1 Answers1

1

I would recommend looking at other forums on python for how to read audio files. This is not possible with blender’s normal API usage. You would need to go through Python.

  1. Pygame. I believe that pygame.mixer can return a sound as an array which will give you the individual samples. This is what you need.
  2. Wave. A module for reading the data in wave audio files.

You may not be able to install these Python packages to blender’s copy of Python, But you might be able to run python externally to convert an audio file into your own, native-Python-readable format for blender to open.

TheLabCat
  • 6,219
  • 2
  • 14
  • 28
  • "You may not be able to install these Python packages to blender’s copy of Python" - according to this source: https://docs.blender.org/api/current/info_tips_and_tricks.html you can remove Blender Python subdirectory in order for Blender to fall back to system Python, in which you could install needed packages. – Markus von Broady Feb 24 '21 at 21:33
  • 1
    Oh! thx for that. – TheLabCat Feb 24 '21 at 21:36