I'm working on reusing the paper Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation, which deforms a 3D mesh to make it fit some 2D images, in another scenario than the original one.

The loss function is comprised of four terms : the chamfer loss, the cosine loss, a Laplacian regularization and an edge length regularization (description here).

These 4 terms have very different scales and are somewhat unbound, meaning that I can't normalize all of them using a maximum value and that if I leave them as-is, some of them will be lost (lol) in the optimization. To add to the problem, not all parts of the loss are equally important : the chamfer loss, minimizing the distance between the expected and actual meshes, should for example have a higher weight in the optimization when compared to the regularization factors.

In the github repository, some coefficients were found that may or may not be optimal for the problem at hand. I'm guessing they're not bad, considering that's what the authors decided to use. However, the method used to find these coefficients isn't explained anywhere (I even tried opening a github issue regarding this but I didn't get any answer).

Now, I've read many questions here regarding optimization of the questions of multi-loss optimization and loss scaling, but I still don't know how to approach the problem of finding the optimal coefficients.

I thought about using a hyperparameter optimizer like Hyperband to find the best coefficients, but what obviously happens is that the engine takes the lower bound of each coefficient, leading to a small loss, but not a better result.

I read about using a balance factor like $ \lambda\times \text{loss}_1 + (1 - \lambda)\times \text{loss}_2 $, but I don' think this cannot be applied to more than two losses.

Finally, I tried setting coefficients by myself to get better results, but this looks like an endless search.

So, my question would be : What are ways to find good coefficients for a loss function with multiple parts where not all parts have the same scale?