Surface electromyography(sEMG)-based gesture classification methods have been widely developed in neural decoding. However, these decoding methods are usually constrained to a fixed set of gestures, which hinders flexibility in practical application. This paper contributes to an incremental learning framework to make classifiers learn different gesture sets (new tasks) gradually without catastrophic forgetting. First, this study analyzes the existing neural decoding methods with deep learning, and introduces an early and late fusion convolutional neural network (ELFCNN) structure based on frequency spectrum. Then, a sEMG-based gesture classification with incremental learning is demonstrated, which modifies the end to end learning method with hybrid data over/down-sampling (HDOD) method. By combining ELFCNN and the HDOD progress, the incremental learning method can make a comparable performance without data dimension reduction, and mitigate catastrophic forgetting while reducing data storage. Experimental results on both small and large size situations show consistent classification accuracy improvement from 0.47% to 0.71% compared with other popular incremental learning methods.