Speech is the primary medium of human communication and involves the interpretation of both auditory and visual information. Lipreading, or visual speech recognition, aims to infer spoken content by analyzing lip movements. With the growing importance of speech confidentiality in security-sensitive environments and the increasing vulnerability to acoustic eavesdropping, lipreading has attracted significant research attention in recent years. This paper presents a novel visual speech recognition framework that integrates convolutional neural networks (CNNs) with Transformer architectures to recognize speech from lip movement images of multiple speakers. In the proposed framework, CNNs extract discriminative spatial features, which are subsequently processed by a Transformer-based model to generate the corresponding textual output. The proposed method is evaluated on the Persian PAVID dataset. A key contribution of this work is viseme-level lip movement recognition, which improves generalization capability and enables continuous speech reconstruction. Experimental results demonstrate that the proposed approach reduces training time while enhancing overall recognition performance, achieving an accuracy of 89% in phoneme recognition on the test set and outperforming existing state-of-the-art methods.