Over the past few years, the quick development in Artificial Intelligence (AI) has devised novel techniques to manipulate multimedia. The misuse of a face swap approach named deepfake has created various cybercrimes like the spreading of fake news, identity theft, and financial crime. One promising countermeasure in opposition to deepfakes is termed as deepfake detection. Still, the detection of deepfake is complex due to the larger dataset. To resolve such issues, this paper develops the Fractional Secretary Bird Skill Optimization Algorithm-enabled Pyramid Deep Belief Network (FSBSOA-PyramidFDBNet)-based multi-face deepfake detection using Federated Learning (FL). The nodes and servers are the major parts of FL. In the training model, the video frames are subjected to face detection. The facial action units are detected with the utilization of Action Unit Network (AUNet). The feature extraction extracts the required features and the deep fake detection is done using the proposed FSBSOA-PyramidFDBNet. The updated weight from the local nodes is aggregated at the server. In addition, the FSBSOA-PyramidFDBNet-based multi face deepfake detection attained the optimal accuracy, loss function, Mean Square Error (MSE), True Negative Rate (TNR), and True Positive Rate (TPR) of 93.91%, 0.064, 0.179, 94.16%, and 92.35%.