In the book I am reading "Foundations of Deep Reinforcement Learning" by Laura Graesser & Wah Loon Keng, I saw the formula to optimize the $\theta$

$$ \theta \leftarrow \theta + \alpha \nabla_\theta J(\pi_\theta) \\ \nabla_\theta J(\pi_\theta) = E_(\tau ~ \pi_\theta)[\sum_{t=0}^T R_t(\tau)\nabla_\theta\log\pi_\theta(a_t|s_t)] $$

So, is the policy a function? Does it have to be it to calculate the gradient? So, how do I identify that policy function?

I still don't get the math. I searched and saw that people usually use the softmax function, but the the parameter in that softmax function is something called a scoring function.

How do I identify the scoring function?