In this project, we aim to extend our previous work on time-series analysis by incorporating Large Language Model (LLM)-centric agents. We will explore different architectures in aligning time-series foundation models with LLM reasoning capabilities in making meaningful and explainable inferences. Based on recentmultimodal SSL works, COCOA [1] and ImageBind[2], we target various sources of data and integrate them with open-sourced LLMs such as LLAMA to interpret and reason about the underlying sensor data[3].
In this project, we expect the student to develop an evaluation setup to examine the capability of our proposed LLM-based model in interpreting and extracting insights from physical activity and physiological data. The project requires the student to be familiar with deep learning concepts and have experience in using either TensorFlow or Pytorch.
School
Computer Science and Engineering
Research Area
Deep learning | Large language models | Time-series sensor data
- Research environment
- Expected outcomes
- Supervisory team
- Reference material/links
The research will be conducted at the School of CSE together with the supervision team.
The outcomes by the end of the project will include a technical report, a code package, and a video demo (to explain the projects, technologies, and achievements). We are planning to extend or write outcomes as a technical paper targeting a publication in a top venue.
