From the past decade understanding of depressive state of humans is interesting topic of researchers. They are consider a different brain datasets which contain different set of features and apply machine learning models to identify the depressive state. But the machine learning models like SVM and KNN provided with the prediction accuracies of 98% and 78-85%, respectively. There is still need improvement in detection accuracy of identifying depressive state. In this paper, we utilize the EEG dataset with 19 channels. It can be utilized to produce the exact report on the level of depression. Our plan is to adapt and fine-tune the weights of networks to the target task with the small-sized dataset. Finally, to improve the recognition performance, an ensemble method based on majority voting of outputs of five mentioned deep TL architectures has been developed. Results indicate that the best performance among basic models achieved by DenseNet121 with accuracy, sensitivity and specificity of 95.74%, 95.56% and 95.64%, respectively. An Ensemble of these basic models created to surpass the accuracy obtained by each individual basic model.