In this paper, the gain in LD-CELP speech coding algorithm is predicted using three neural models, that are equipped by genetic and particle swarm optimization (PSO) algorithms to optimize the structure and parameters of neural networks. Elman, multi-layer perceptron (MLP) and fuzzy ARTMAP are the candidate neural models. The optimized number of nodes in the first and second hidden layers of Elman and MLP and also the initial weights and biases of these nets are determined by genetic algorithm (GA) and PSO. In the fuzzy ARTMAP, the choice parameter, a, learning rate, b, and vigilance parameter, r, are selected by GA and PSO, as well. In this way, the performance of GA and PSO are compared when using different neural architectures in this application. Empirical results show that when gain is predicted by Elman and MLP neural networks with GA/PSOoptimized parameters, the segmental signal to noise ratio (SNRseg) and mean opinion score (MOS) are improved as compared to traditional implementation based on ITU-T G.728 recommendation. On the other hand, fuzzy ARTMAP-based gain predictor reduces the computational complexity noticeably, with no significant degradations in SNRseg and MOS.