I'm also a newcomer and a self-learner in this field. I suppose there are two subfields: speech and "others", where "others" can include things like music. I have little experience with the "others" subfield so I will just focus on speech. I think the key thing to note is that audio, like images, can be converted to a series of discrete signals, and hence it falls under the umbrella of signal processing generally.

For example, with 44.1kHz sampling, we have a vector of length 44100 for each second of recording (per channel). For the purpose of analysis, this vector is often turned into a spectrogram representation through discrete Fourier analysis. Typically these spectrograms are based on overlapping "windows", or views of the original sequence, and are calculated every 10ms or 25ms. For speech, we have further transformations to highlight the human voice frequencies, so a special type of spectrogram called the MFCC is usually used.

Many (probably most) speech processing algorithms take this standard transformation as input, so it is possible to go a long way to just focus on these algorithms without worrying too much about the transformation from the raw audio to the MFCC representation.

These days, the best algorithms are done using neural networks, and I find it very helpful to go through the material on Recurrent Neural Networks. I recommend Andrew Ng's coursera course on sequence model. Although its focus is on Language models, I think most of the concepts apply to speech or any audio sequences also.

If you are interested in speech, I think it will also be important to understand the GMM-HMM paradigm, for which I recommend a series of articles by Jonathn Hui on his blog, e.g. this and this.