I have a basic autoencoder structure. I want to change it to a stacked autoencoder. From what I know the stacked AE differs in 2 ways:
- It is made up of layers of sparse vanilla AEs
- It does layer-wise training.
I want to know if sparsity is a necessity for stacked AEs or just increasing number of hidden layers in vanilla AE structure will make it a stacked AE?
class Autoencoder(Chain):
def __init__(self):
super().__init__()
with self.init_scope():
# encoder part
self.l1 = L.Linear(1308608,500)
self.l2 = L.Linear(500,100)
# decoder part
self.l3 = L.Linear(100,500)
self.l4 = L.Linear(500,1308608)
def forward(self,x):
h = self.encode(x)
x_recon = self.decode(h)
return x_recon
def __call__(self,x):
x_recon = self.forward(x)
loss = F.mean_squared_error(h, x)
return loss
def encode(self, x, train=True):
h = F.dropout(self.activation(self.l1(x)), train=train)
return self.activation(self.l2(x))
def decode(self, h, train=True):
h = self.activation(self.l3(h))
return self.l4(x)
