Table 1: The number of parameters and performance of networks
Network type
#Parameters (M)
Performance(
ms
)
fully-connected 1024
4.91
1.08
fully-connected 2048
16.12
3.04
mixture of experts
8.47
1.39
RNN
7.78
1.91
transformer
6.96
2.49