Evaluating Benchmark Cheating and the Superiority of MAMBA over Transformers in Bayesian Neural Networks: An in-depth Analysis of AI Performance
1 Department of Electrical and Electronics Engineering, University of Ibadan, Ibadan, Nigeria.
2 College of Arts and Sciences, Department of Chemistry and Biochemistry, North Dakota State University, USA.
3 College of Arts and Sciences, Department of Chemistry, Southern Illinois University Edwardsville, USA.
4 Department of Research and Development, The Energy Connoisseur L.L.C, Houston, Texas, USA.
5 Department of Physics, Joseph Sarwuan Tarka University, Makurdi, Nigeria.
6 Department of Architecture, College of Architecture Construction and Planning, The University of Texas at San Antonio, Texas, USA.
Review
World Journal of Advanced Engineering Technology and Sciences, 2024, 12(01), 372–389.
Article DOI: 10.30574/wjaets.2024.12.1.0254
Publication history:
Received on 10 May 2024; revised on 16 June 2024; accepted on 18 June 2024
Abstract:
Artificial Intelligence (AI) models have seen unprecedented advancements with the rise of architectures like Transformers and Bayesian Neural Networks (BNNs). However, these innovations have also given rise to concerns over benchmark cheating, potentially skewing results that influence model selection in practical applications. This review paper provides an in-depth analysis of benchmark cheating and explores the relative performance of the Multi-resolution Aggregated Memory and Boundary-Aware Architecture (MAMBA) compared to Transformers within the context of Bayesian Neural Networks. The paper begins with an exploration of benchmark cheating, outlining its manifestations in different AI research settings and its impact on evaluating model performance. It investigates how overfitting, data leakage, and selective benchmark reporting can distort comparative analyses. The subsequent section delves into the architecture and advantages of MAMBA over Transformers, highlighting its memory aggregation and boundary-awareness strategies that potentially make it superior in certain contexts.
Keywords:
Bayesian Neural Networks; Multi-resolution Aggregated Memory and Boundary-Aware Architecture; Transformers; AI; Benchmark Cheating; Model Evaluation; Overfitting; Data Leakage
Full text article in PDF:
Copyright information:
Copyright © 2024 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0