Personally, I started by writing externals for Pure Data, then started to contribute to the core. Later I took the same path for SuperCollider.

You may start with simple audio plugins. Have a look at JUCE (https://juce.com/)!

Realtime audio programming has some rather strict requirements that you don't have in most other software. Check out this classic article: http://www.rossbencina.com/code/real-time-audio-programming-...

The Will Pirkle books are good.