Impact of LLM Rephrasing of Spear-Phishing Emails on Email Filter Rule Activation

Keywords: filter-rule analysis, content-based filtering, corporate email, stylistic normalization, social-engineering indicators, automated threat detection

Abstract

Purpose. To assess how large language model rephrasing affects email-filter rule activation for spear-phishing messages under controlled laboratory conditions.

Method. A controlled experiment examined three groups of 150 messages: original phishing emails, rephrased versions, and legitimate business correspondence. Data were obtained from ePhishLLM, Nazario, and Enron. Following deduplication, network identifiers were replaced with standardized placeholders. A local language model rephrased messages while preserving their intent and requested actions. Standardized emails were evaluated using SpamAssassin and Rspamd, with a focus on classification outcomes and rule activation.

Results. Rspamd incorrectly flagged all legitimate control messages because of laboratory header characteristics, preventing meaningful binary comparisons. SpamAssassin detected two original and one rephrased phishing email. Following rephrasing, activations associated with urgency, brand impersonation, and phishing-related addresses decreased from 6 to 0, 3 to 0, and 2 to 1, respectively. Account- and security-related rules were maintained or had their activation frequency increased.

Theoretical implications. Rule-level analysis reveals changes obscured by binary classification metrics.

Practical implications. The approach supports laboratory evaluation of the robustness of content-based filtering.

Originality / value. The study identifies measurable changes in rule activation across paired phishing messages.

Limitations / future research. Findings are laboratory-specific and require validation across additional models and configurations.

Article type. Empirical research.

Downloads

Download data is not yet available.

References

Anti-Phishing Working Group. (2024, December 4). Phishing activity trends report: 3rd quarter 2024. https://docs.apwg.org/reports/apwg_trends_report_q3_2024.pdf

Apache Software Foundation. (n.d.). Apache SpamAssassin. Retrieved June 3, 2026, from https://spamassassin.apache.org/

Aseeri, A. M., & Bohacek, S. (2025). Using ensembles of large language models to detect phishing emails. In L. Barolli (Ed.), Advanced information networking and applications (pp. 21–34). Springer. https://doi.org/10.1007/978-3-031-87778-0_3

Bountakas, P., & Xenakis, C. (2023). HELPHED: Hybrid ensemble learning phishing email detection. Journal of Network and Computer Applications, 210, Article 103545. https://doi.org/10.1016/j.jnca.2022.103545

Butavicius, M., Taib, R., & Han, S. J. (2022). Why people keep falling for phishing scams: The effects of time pressure and deception cues on the detection of phishing emails. Computers & Security, 123, Article 102937. https://doi.org/10.1016/j.cose.2022.102937

Butt, U. A., Amin, R., Aldabbas, H., Mohan, S., Alouffi, B., & Ahmadian, A. (2023). Cloud-based email phishing attack using machine and deep learning algorithm. Complex & Intelligent Systems, 9(3), 3043–3070. https://doi.org/10.1007/s40747-022-00760-3

Carnegie Mellon University. (n.d.). Enron email dataset. Retrieved June 3, 2026, from https://www.cs.cmu.edu/~enron/

European Union Agency for Cybersecurity. (2024). ENISA threat landscape 2024: July 2023 to June 2024. Publications Office of the European Union. https://doi.org/10.2824/0710888

Eze, C. S., & Shamir, L. (2024). Analysis and prevention of AI-based phishing email attacks. Electronics, 13(10), Article 1839. https://doi.org/10.3390/electronics13101839

Fairbanks, J., & Serra, E. (2024). Generating phishing attacks and novel detection algorithms in the era of large language models. In W. Ding, C.-T. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, & X. Hu (Eds.), Proceedings of the 2024 IEEE International Conference on Big Data (BigData 2024) (pp. 2314–2319). IEEE. https://doi.org/10.1109/BigData62323.2024.10825007

Gutierrez, R., Villegas-Ch, W., & Govea, J. (2026). Cross-model evaluation of phishing detectors against large language model-generated emails. Frontiers in Big Data, 9, Article 1883452. https://doi.org/10.3389/fdata.2026.1883452

Han, W., Zhu, J., Zhang, C., Zhang, Z., Mei, Y., & Wang, L. (2025). Phish-Master: Leveraging large language models for advanced phishing email generation and detection. Applied Sciences, 15(22), Article 12203. https://doi.org/10.3390/app152212203

Heiding, F., Schneier, B., Vishwanath, A., Bernstein, J., & Park, P. S. (2024a). Devising and detecting phishing emails using large language models. IEEE Access, 12, 42131–42146. https://doi.org/10.1109/ACCESS.2024.3375882

Heiding, F., Lermen, S., Kao, A., Schneier, B., & Vishwanath, A. (2024b). Evaluating large language models’ capability to launch fully automated spear-phishing campaigns: Validated on human subjects. arXiv. https://doi.org/10.48550/arXiv.2412.00586

Jamal, S., Wimmer, H., & Sarker, I. H. (2024). An improved transformer-based model for detecting phishing, spam and ham emails: A large language model approach. Security and Privacy, 7(5), Article e402. https://doi.org/10.1002/spy2.402

Koide, T., Nakano, H., & Chiba, D. (2024a). ChatPhishDetector: Detecting phishing sites using large language models. IEEE Access, 12, 154381–154400. https://doi.org/10.1109/ACCESS.2024.3483905

Koide, T., Fukushi, N., Nakano, H., & Chiba, D. (2024b). ChatSpamDetector: Leveraging large language models for effective phishing email detection. arXiv. https://doi.org/10.48550/arXiv.2402.18093

National Institute of Standards and Technology. (n.d.-a). Social engineering. Computer Security Resource Center Glossary. Retrieved June 3, 2026, from https://csrc.nist.gov/glossary/term/social_engineering

National Institute of Standards and Technology. (n.d.-b). Spear phishing. Computer Security Resource Center Glossary. Retrieved June 3, 2026, from https://csrc.nist.gov/glossary/term/spear_phishing

Nazario, J. (n.d.). Phishing corpus. Monkey.org. Retrieved June 3, 2026, from https://monkey.org/~jose/phishing/

Ollama. (2024). llama3.1:8b (Version 3.1) [Large language model]. https://ollama.com/library/llama3.1:8b

Opara, C., Modesti, P., & Golightly, L. (2025). Evaluating spam filters and stylometric detection of AI-generated phishing emails. Expert Systems with Applications, 276, Article 127044. https://doi.org/10.1016/j.eswa.2025.127044

Pajola, L., Caripoti, E., Banzer, S., Pizzi, S., Conti, M., & Apruzzese, G. (2025). E-PhishGen: Unlocking novel research in phishing email detection. arXiv. https://doi.org/10.48550/arXiv.2509.01791

Qi, Q., Luo, Y., Xu, Y., Guo, W., & Fang, Y. (2025). SpearBot: Leveraging large language models in a generative-critique framework for spear-phishing email generation. Information Fusion, 122, Article 103176. https://doi.org/10.1016/j.inffus.2025.103176

Rspamd. (n.d.). Actions and scores. Rspamd Documentation. Retrieved June 3, 2026, from https://docs.rspamd.com/configuration/metrics/

Schmitt, M., & Flechais, I. (2024). Digital deception: Generative artificial intelligence in social engineering and phishing. Artificial Intelligence Review, 57(12), Article 324. https://doi.org/10.1007/s10462-024-10973-2

Shahriar, S., Mukherjee, A., & Gnawali, O. (2022). Improving phishing detection via psychological trait scoring. arXiv. https://doi.org/10.48550/arXiv.2208.06792

Tarapiah, S., Abbas, L., Mardawi, O., Atalla, S., Himeur, Y., & Mansoor, W. (2025). Evaluating the effectiveness of large language models versus machine learning in identifying and detecting phishing email attempts. Algorithms, 18(10), Article 599. https://doi.org/10.3390/a18100599

Uddin, M. A., Mahiuddin, M., & Sarker, I. H. (2026). An explainable transformer-based model for phishing email detection: A large language model approach. Computer Networks, 277, Article 112061. https://doi.org/10.1016/j.comnet.2026.112061

Wood, T., Basto-Fernandes, V., Boiten, E., & Yevseyeva, I. (2022). Systematic literature review: Anti-phishing defences and their application to before-the-click phishing email detection. arXiv. https://doi.org/10.48550/arXiv.2204.13054


Abstract views: 7
PDF Downloads: 3
Published
2026-10-31
How to Cite
Zhyshko , R., Cheremnykh, Y., & Сусукайло, В. (2026). Impact of LLM Rephrasing of Spear-Phishing Emails on Email Filter Rule Activation. Social Development and Security, 16(5), 343-360. https://doi.org/10.33445/sds.2026.16.5.16
Section
Information and Cyber Security