LLMs之Agent:Personal_LLM_Agents_Survey的简介、使用方法之详细攻略
导读:该项目包含了针对个人型LLM代理(Personal LLM Agents)的相关论文列表。通过查询相关论文,可以了解这一新兴技术方向的最新研究进展,比如在对话能力、知识表示、隐私保护等方面如何进行优化,从而提升用户体验。通过论文也可以了解这一技术的应用案例、难点以及解决方法。例如如何将LLM代理应用在教育或医疗助手等领域,如何使其对话能力更加逼真自然,或者如何保护用户隐私不被滥用等都是值得关注的问题。
总的来说,此项目给出了一个系统整理的个人LLM代理相关论文列表,从多个角度论述了这个新技术方向的发展现状和未来走势,有助于研究人员和开发者更好地把握趋势并开展工作。
目录
Personal_LLM_Agents_Survey的使用方法
个人LLM代理(智能体)被定义为一种特殊类型的基于LLM的代理,它与个人数据、个人设备和个人服务深度集成。它们最好部署到资源受限的移动/边缘设备和/或由轻量级AI模型提供支持。个人LLM代理的主要目的是协助最终用户并增强其能力,帮助他们更专注、更出色地处理有趣和重要的事务。
这份论文清单涵盖了个人LLM代理的几个主要方面,包括能力、效率和安全性。
GitHub地址:https://github.com/MobileLLM/Personal_LLM_Agents_Survey
任务自动化是个人LLM代理的核心能力,它决定了代理能够多好地响应用户命令和/或自动执行用户任务。由于UI-based任务自动化代理在这个列表中很受欢迎并与个人设备密切相关,我们专注于这方面。
理解当前上下文的能力对于个人LLM代理提供个性化、上下文感知的服务至关重要。这包括感知用户活动、心理状态、环境动态等技术。
“Afective State Prediction from Smartphone Touch and Sensor Data in the Wild” (Wampfler et al., 2022, p. 1)?CHI'22
“Mobile Localization Techniques for Wireless Sensor Networks: Survey and Recommendations” (Oliveira et al., 2023, p. 361)?ACM Transactions on Sensor Networks
“Are You Killing Time? Predicting Smartphone Users’ Time-killing Moments via Fusion of Smartphone Sensor Data and Screenshots” (Chen et al., 2023, p. 1)?CHI'23
“Remote Breathing Rate Tracking in Stationary Position Using the Motion and Acoustic Sensors of Earables” (Ahmed et al., 2023, p. 1)?CHI'23
“SAMoSA: Sensing Activities with Motion and Subsampled Audio” (Mollyn et al., 2022, p. 1321)?IMWUT
“A Systematic Survey on Android API Usage for Data-Driven Analytics with Smartphones” (Lee et al., 2023, p. 1)?ACM Computing Surveys
“A Multi-Sensor Approach to Automatically Recognize Breaks and Work Activities of Knowledge Workers in Academia” (Di Lascio et al., 2020, p. 781)?IMWUT
“Robust Inertial Motion Tracking through Deep Sensor Fusion across Smart Earbuds and Smartphone” (Gong et al., 2021, p. 621)?IMWUT
“DancingAnt: Body-empowered Wireless Sensing Utilizing Pervasive Radiations from Powerline” (Cui et al., 2023, p. 873)?ACM MobiCom'23
“DeXAR: Deep Explainable Sensor-Based Activity Recognition in Smart-Home Environments” (Arrotta et al., 2022, p. 11)?IMWUT
“MUSE-Fi: Contactless MUti-person SEnsing Exploiting Near-field Wi-Fi Channel Variation” (Hu et al., 2023, p. 1135)?IMWUT
“SenCom: Integrated Sensing and Communication with Practical WiFi” (He et al., 2023, p. 903)?ACM MobiCom'23
“SleepMore: Inferring Sleep Duration at Scale via Multi-Device WiFi Sensing” (Zakaria et al., 2022, p. 1931)?IMWUT
“COCOA: Cross Modality Contrastive Learning for Sensor Data” (Deldari et al., 2022, p. 1081)?ACM MobiCom'23
“M3Sense: Affect-Agnostic Multitask Representation Learning Using Multimodal Wearable Sensors” (Samyoun et al., 2022, p. 731)?IMWUT
“Predicting Subjective Measures of Social Anxiety from Sparsely Collected Mobile Sensor Data” (Rashid et al., 2020, p. 1091)?IMWUT
“Attend and Discriminate: Beyond the State-of-the-Art for Human Activity Recognition Using Wearable Sensors” (Abedin et al., 2021, p. 11)?IMWUT
“Fall Detection based on Interpretation of Important Features with Wrist-Wearable Sensors” (Kim et al., 2022, p. 1)?IMWUT
“PowerPhone: Unleashing the Acoustic Sensing Capability of Smartphones” (Cao et al., 2023, p. 842)?ACM MobiCom'23
“I Spy You: Eavesdropping Continuous Speech on Smartphones via Motion Sensors” (Zhang et al., 2022, p. 1971)?IMWUT
“Watching Your Phone’s Back: Gesture Recognition by Sensing Acoustical Structure-borne Propagation” (Wang et al., 2021, p. 821)?IMWUT
“Gesture Recognition Method Using Acoustic Sensing on Usual Garment” (Amesaka et al., 2022, p. 411)?IMWUT
“A Multi-Sensor Approach to Automatically Recognize Breaks and Work Activities of Knowledge Workers in Academia” (Di Lascio et al., 2020, p. 781)?IMWUT
Mobile and Wearable Sensing Frameworks for mHealth Studies and Applications: A Systematic Review” (Kumar et al., 2021, p. 81)?ACM Transaction on Computing for Healthcare
“Afective State Prediction from Smartphone Touch and Sensor Data in the Wild” (Wampfler et al., 2022, p. 1)?CHI'22
“Are You Killing Time? Predicting Smartphone Users’ Time-killing Moments via Fusion of Smartphone Sensor Data and Screenshots” (Chen et al., 2023, p. 1)?CHI'23
“FeverPhone: Accessible Core-Body Temperature Sensing for Fever Monitoring Using Commodity Smartphones” (Breda et al., 2022, p. 31)?IMWUT
“Guard Your Heart Silently: Continuous Electrocardiogram Waveform Monitoring with Wrist-Worn Motion Sensor” (Cao et al., 2022, p. 1031)?IMWUT
“Listen2Cough: Leveraging End-to-End Deep Learning Cough Detection Model to Enhance Lung Health Assessment Using Passively Sensed Audio” (Xu et al., 2021, p. 431)?IMWUT
“HealthWalks: Sensing Fine-grained Individual Health Condition via Mobility Data” (Lin et al., 2020, p. 1381)?IMWUT
“Identifying Mobile Sensing Indicators of Stress-Resilience” (Adler et al., 2021, p. 511)?IMWUT
“MoodExplorer: Towards Compound Emotion Detection via Smartphone Sensing” (Zhang et al., 2018, p. 1761)?IMWUT
“mTeeth: Identifying Brushing Teeth Surfaces Using Wrist-Worn Inertial Sensors” (Akther et al., 2021, p. 531)?IMWUT
“Detecting Job Promotion in Information Workers Using Mobile Sensing” (Nepal et al., 2020, p. 1131)?IMWUT
“First-Gen Lens: Assessing Mental Health of First-Generation Students across Their First Year at College Using Mobile Sensing” (Wang et al., 2022, p. 951)?IMWUT
“Predicting Personality Traits from Physical Activity Intensity” (Gao et al., 2019, p. 1)?IEEE Computer
“Predicting Symptom Trajectories of Schizophrenia using Mobile Sensing” (Wang et al., 2017, p. 1101)?IMWUT
“Predictors of Life Satisfaction based on Daily Activities from Mobile Sensor Data” (Yürüten et al., 2014, p. 1)?CHI'14
“SmartGPA: How Smartphones Can Assess and Predict Academic Performance of College Students” (Wang et al., 2015, p. 1)?UbiComp'15
“Social Sensing: Assessing Social Functioning of Patients Living with Schizophrenia using Mobile Phone Sensing” (Wang et al., 2020, p. 1)?CHI'20
“SmokingOpp: Detecting the Smoking ‘Opportunity’ Context Using Mobile Sensors” (Chatterjee et al., 2020, p. 41)?IMWUT
记忆是个人LLM代理保持关于用户信息的能力,使代理能够提供更定制的服务并根据用户偏好自我演变。
LLM代理的效率与LLM推理、LLM训练/定制以及内存管理的效率密切相关。
LLM推理/训练的效率已经在现有调查中得到全面总结(例如此链接)。因此,在这个列表中,我们省略了这部分内容。
在这里,我们主要列举与高效内存管理相关的论文,这是LLM代理的重要组成部分。
(with vector library, vector DB, and others)
Vector Library
Vector Database
Other Forms of Memory
AI/ML的安全与隐私是一个庞大的领域,涉及大量相关论文。在这里,我们只关注与LLM和LLM代理相关的论文。