Finetuning LLMs for Mental Health Counsel
Tags: Healthcare AI · NLP
Recognizing the inherent bias of most LLMs towards European languages and ethnicities and the low resources of structured Bengali data, I focused on refining open-source models like LLaMA using different parameter-efficient fine-tuning (PEFT) techniques (e.g., Adapter injections and LoRA). I successfully fine-tuned LLaMA for Bengali mental health consultation using QLoRA, resulting in a more optimized model that can be served on low GPU memory—ensuring equitable access to healthcare technologies across diverse linguistic communities.