Does L1/L2 (NAdam weight decay) really make the model "unlearn"?

Sort of. These regularisation methods help to make it "forget", but are not required. This is something it will do anyway - unless repeatedly shown the same data, neural networks will preferentially match to more recent data to at least some degree. This is often desirable, and allows neural networks to be used in situations that require online learning.

A possible solution I'm thinking is like keeping the weights of the model before and/or after the training I want to make it unlearn and then somehow use it to do the unlearning but not sure.

For a neural network, this is the only workable solution.

You could, in principle, keep the individual weight step calculations, and reverse them. It would have to be done strictly in order, like an "undo" step in a word processor. However, you cannot calculate what the previous weight step was from the current network plus a set of data. So this will normally keep a data structure the same size as the neural network parameters for each undo step. Keeping a backup copy of the network to restore will use less data than two undo steps, and is less mathematically complicated therefore less prone to error in code.

Most neural network libraries will let you clone a network, so you use that feature to assign to a variable and/or save to disk. If you need to restore then copy the backup (cloning again, so that the two sets of weights are not shared) over the variable tracking the current learning network.