The 17th Annual Modeling, Simulation and Visualization Student Capstone Conference (MSVSCC 2024) took place at the Virginia Modeling, Analysis, and Simulation Center (VMASC) on April 11, 2024. The conference was held in-person and it was my second time attending the MSVSCC. This year, I, Tarannum Zaki, along with Lawrence Obiuwevwi, Brian Llinas, and Jhon Botello represented WS-DL at the conference to present our work.
We’re excited to kick off the 17th annual @ODU Modeling, Simulation & Visualization Student Capstone Conference today! Welcome to students, professionals, and enthusiasts for a day packed with innovation, learning, and networking. #ODUInnovate#MSVSCCpic.twitter.com/zF3jxCZhQ3
— ODU Office of Enterprise Research and Innovation (@oeriatodu) April 11, 2024
The MSV Student Capstone Conference provides a platform that features research of both undergraduate and graduate students in modeling and simulation, across various disciplines from different departments of Old Dominion University (ODU). The conference also invites participation of programs from other colleges and universities. Students are required to submit research papers (either a two page extended abstract or a 12-page full paper) to present their work at the conference. There were seven conference paper tracks this year: Transportation, Business & Industry, General Sciences & Engineering, Education & Training, Virtual Environments & Visualization, Security & Military, Health Care, and AI & Autonomous Systems. Papers were reviewed by research and faculty track chairs from ODU and VMASC.
The conference kicked off with a discussion about the history, challenges, and future of modeling and simulation. The keynote speaker for this welcome session was Dr. Bowen Loftin.
Welcome to our keynote speaker, Dr. Bowen Loftin, who is no stranger to Modeling and Simulation. As a former Executive Director for @vmasc_odu, Dr. Loftin was responsible for @ODU’s graduate programs in M&S! #ODUInnovate#MSVSCCpic.twitter.com/dnw92cmU90
— ODU Office of Enterprise Research and Innovation (@oeriatodu) April 11, 2024
The AI & Autonomous Systems track was introduced this year. In this blog post, I provide a summary of the presentations of the AI & Autonomous Systems track that were covered in the morning session.
Papers
Among Us: A Design of AI-Enabled Smart Multidisciplinary Team Member
Authors:Lawrence Obiuwevwi, Krzysztof Rechowicz, Sampath Jayarathna
Performance Analysis of Fine-Tuning LLM on Domain Specific Task
Authors:Saumya Dabhi, Joseph Martinez, Faryaneh Poursardar, Jose Padilla, Krzysztof Rechowicz
Web Archives for Verifying Attribution in Twitter Screenshots
Authors:Tarannum Zaki, Michael L. Nelson, Michele C. Weigle
Assessing Error Types in LLM-Generated Narratives
Authors: Erik Jensen, Christopher Lynch, Ross Gore, James Leathrum
AI-Driven App for Accessibility in Education: Converting Scanned Documents to Readable Text for Students with Low Vision
Authors: Brian Llinas, Jhon Botello, Joseph Martínez, Meaghan McLeod Mozingo, Melissa Miller-Felton, Erika Frydenlund, Jose Padilla
Evaluation of Simulated Weather Effects on Autonomous Ground Vehicles
Authors:Jalen Caldwell, Jared Cross, Patrick Oglesbee, Brandon Thompson
Optimizing Sentiment Analysis: A Novel Approach Using Lipschitz Recurrent Neural Networks and Learning to Search
Authors:Mahmudul Hasan, Sachin Shetty
Presentations
Presentation 1
Lawrence Obiuwevwi from the NIRDS Lab of the WS-DL research group presented an extended abstract paper titled “Among Us: A Design of AI-Enabled Smart Multidisciplinary Team Member.” This is a collaborative work with the Storymodelers team of VMASC. Conflict resolution is an important factor that impacts team performance. Early signs of conflicts include a variety of physiological changes. In this paper, the authors investigated the feasibility of using artificial intelligence (AI) as a team member to detect and mediate conflicts in interdisciplinary team dynamics. Current research progress of this work involves developing a real-time speech recognition model using open source speech dataset like Hugging Face. It further involves the use of Whisper and ChatGPT models to transcribe real-time speech to text and analyze sentiment. Future work of this research targets to synchronize real-time emotion detection model and body language model along with the speech recognition model.
.@LObiuwevwi is now presenting their work "Among Us: A Design of AI-Enabled Smart Multidisciplinary Team Member" at the 17th Annual Student Capstone Conference, #MSVSCC2024 at @VMASC_ODU!@OpenMaze@NirdsLab@WebSciDL@oducs@storymodelerspic.twitter.com/finhzr1Bx1
— Tarannum Zaki (@tarannum_zaki) April 11, 2024
Presentation 2
Saumya Dabhi from the Storymodelers team of VMASC presented an extended abstract titled “Performance Analysis of Fine-Tuning LLM on Domain Specific Task.” The motivation of this research is to understand the causes and impacts of Venezuelan migration crisis in Columbia. To look into this matter, the authors presented an experimental study where they used two fine-tuning methods for Large Language Models (LLMs) on migration related news data. The data includes a dataset of 322 news articles extracted from El Tiempo related to Venezuelan migration in Colombia from 2015 to 2021 and a Question & Answer dataset created using ChatGPT from news articles. The first method involves self-supervised fine-tuning the Llama2 model on the news articles’ dataset first and then supervised fine-tuning on the Q&A dataset. The second method involves fine-tuning the Llama2 model directly on the Q&A dataset incorporating contextual information in the responses. The authors further assessed the effectiveness of these approaches and found that the second approach provides more consistent responses than the first approach.
Presenting our research "Performance Analysis of Fine Tuning LLM on Domain Specific Task" at the 17th Annual Student Capstone Conference, #MSVSCC2024 at @VMASC_ODU@Faryane#josephmartinez@storymodelerspic.twitter.com/gjR4nU8NgC
— Saumya (@SaumyaDabhi) April 11, 2024
Presentation 3
I presented an extended abstract paper titled “Web Archives for Verifying Attribution in Twitter Screenshots.” Screenshots on social media have become a common way of sharing information. Users rarely verify before sharing screenshots of a fabricated tweet and this eventually leads to misinformation and disinformation spread on social media. The goal of this research is to develop an automated tool to estimate the probability that the shared screenshot was really posted by the alleged author. To validate the attribution of screenshots, I discuss a method in this paper on how finding a tweet in the web archives can be useful to prove that the account actually made the alleged post in the screenshot. I conducted an experimental study on 108 single tweet screenshot images from our collected dataset and demonstrated the method. The results showed that a text similarity score threshold of 60% between tweet text of screenshot and archived tweets produced the highest F1 score (0.69).
I was excited to win the Best Presentation award in the AI & Autonomous Systems track.
Happening Now! Presenting an extended abstract "Web Archives for Verifying Attribution in Twitter Screenshots" related to our current research work at the 17th Annual Student Capstone Conference, #MSVSCC2024 at @VMASC_ODU!@LrningEngneer@weiglemc@phonedude_mln@WebSciDL@oducspic.twitter.com/7V74pj4ycq
— Tarannum Zaki (@tarannum_zaki) April 11, 2024
Presentation 4
Erik Jensen from VMASC presented their work titled “Assessing Error Types in LLM-Generated Narratives.” The authors here explore the accuracy and different types of errors in narratives that are generated by LLMs. The work is more focused on evaluating accurate and relevant responses from Structured Narrative Prompts (SNPs) such as forms, templates etc. generated from simulated life events. This would benefit certain domains/applications that deal with structured prompts, for example, patient diagnosis. The methodology of assessing the effectiveness of SNPs involves first manually classifying the LLM-generated narratives based on how they comply with the expectations. Next, the manually classified narratives are used to train machine learning models for automatic narrative classification. Some common identified errors in LLM-generated narratives are: incorrect event description, temporal inaccuracies, and age-inappropriateness. This research provides an overview of the potential as well as limitations of LLM-generated narratives.
Presentation 5
Brian Llinas from the ODU WS-DL research group presented their work titled “AI-Driven App for Accessibility in Education: Converting Scanned Documents to Readable Text for Students with Low Vision.” This is a collaborative work with the Storymodelers team of VMASC. Jhon Botello of ODU WS-DL research group is also a co-author of this paper. The goal of this project is to develop an AI-based app that would aid students with low vision to access educational materials that are primarily visual. The project is mainly intended to reduce the challenges faced by students in the ODU World Languages and Cultures Department. The development of the app includes a user-friendly interface that converts scanned documents into a readable format that is compatible with users’ existing accessibility tools. The conversion is done using Nougat (Neural Optical Understanding for Academic Documents), an Optical Character Recognition (OCR) model that converts scientific documents into a markup language. The authors evaluated the performance of the app based on successful extraction of text and obtained a BLEU score of 0.918.
.@bllin001 is now presenting their work "AI-Driven App for Accessibility in Education: Converting Scanned Documents to Readable Text for Students with Low Vision" at the 17th Annual Student Capstone Conference, #MSVSCC2024 at @VMASC_ODU!@Jhon_gbm12@WebSciDL@storymodelerspic.twitter.com/P3ekfdSgTT
— Tarannum Zaki (@tarannum_zaki) April 11, 2024
Presentation 6
Jalen Caldwell, Jared Cross, and Patrick Oglesbee from the West Point Department of Systems Engineering, United States Military Academy presented their work titled “Evaluation of Simulated Weather Effects on Autonomous Ground Vehicles.” Autonomous Ground Vehicles (AGV) can significantly reduce risk to soldiers and increase logistical capabilities in military missions. These AGVs require thorough testing under different conditions before being incorporated into missions alongside soldiers. The purpose of this study is to assist the Engineering Research and Development Center (ERDC) to improve virtual testing through simulation and modeling. This is done by testing the usabilityof the AGVs through ERDC’s developed Software-in-the-Loop (SIL) simulation. The authors conducted a usability study and a design of experiments to test the effects of simulated weatheron vehicle performance under three levels of terrain. The usability study was conducted using a survey questionnaire that consisted of 5 army officers. The design of experiments involved a simulation analysis where 90 of them were successful 118 simulation runs. The results of the virtual testing showed that AGVs performed better under simulated rain and fog weather conditions and on low vegetation terrain.
Presentation 7
Mahmudul Hasan from VMASC presented their work titled “Optimizing Sentiment Analysis: A Novel Approach Using Lipschitz Recurrent Neural Networks and Learning to Search.” The objective of this paper is to capture the temporal dynamics of natural language to perform sentiment classification accurately. To achieve this, the authors introduced an advanced sentiment analysis methodology which combines Lipschitz Recurrent Neural Network (LRNN) with Learning to Search (L2S). They demonstrated that through dynamic parameter optimization and adaptive learning, this method significantly improves sentiment classification accuracy. The authors further provided an empirical validation on the IMDB dataset specifying the potential of integrating LRNN and L2S for handling complex natural language processing tasks. The authors lastly proposed a future direction regarding the application of this advanced sentiment analysis methodology that would enhance human-robot interactions (HRI) where robots can adapt their responses based on nuanced emotional cues from human interactions.
Awards
The conference ended with highlighting the remarkable research works presented by the student authors and an award ceremony. There were awards for both Best Presentation and Best Paper for each track as well as an award for Best Overall Paper across all seven conference tracks. I won the award for the Best Presentation in AI & Autonomous Systems track for “Web Archives for Verifying Attribution in Twitter Screenshots.” The award ceremony was preceded by a showcase session where research faculty from both ODU and VMASC shared their innovative research to members of the modeling and simulation community from academic, industry, and government backgrounds.
🏆Received "Best Presentation Award" back to back for another year. This time for the AI & Autonomous Systems track! 🥳 pic.twitter.com/e1DcbUYngV
— Tarannum Zaki (@tarannum_zaki) April 11, 2024
Wrap-up
The MSV Student Capstone Conference provides a great opportunity to share ideas and gain knowledge about various research works going on at ODU relevant to modeling and simulation engineering. Last year, the ODU WS-DL research group participated for the first time in the MSVSCC and won all the awards (see MSVSCC 2023 trip report). This year too, the ODU-WSDL research group won the Best Presentation award for the AI & Autonomous Systems track. I am extremely grateful to the organizer of the MSV Student Capstone Conference Dr. Jessica Johnson, ODU-VMASC’s Director of STEM and Educational Partnerships, for providing such an amazing platform to present our research work. I am delighted to win the Best Presentation award for two consecutive years. I am excited about participating at the conference next year!