If the question is actually a statistical topic disguised as a coding question, then OP should edit the question to clarify this. After the statistical content has been clarified, the question is eligible for reopening.

I have an Autoencoder model that receives 4 different vectors { (1,0,0,0) , (0,1,0,0) , (0,0,1,0) , (0,0,0,1) }

The encoder transforms the vectors to vectors size 2, which get inside an "NLPN channel" which is a channel with nonlinear phase noise, and then goes to the decoder that reconstructs the original vector.

Architecture: 4->50->2->NLPN->2->50->4 (Softmax)

The loss function is mse between the input, and the input ( self.model.fit(self.x, self.x) )

The code is pretty basic: the first cell is just some simple functions defined, then it's the autoencoder class, then it's the training, and then it's the prediction which shows that for each vector 1,0,0,0 and so on, the decoder predicts (~0.25,~0.25,~0.25,~0.25) which shows that the NN has no idea what it's doing.

CODE: https://colab.research.google.com/drive/1nFymmEloUSa1yjS7e5CPGsKwqP1HjS57?usp=sharing