Design and Implementation of a Fine-Tuned Llama-Based AI Chatbot with Voice and Text Interaction Using Streamlit and Ollama
Polamarasetty Harshavardhan1, Mandala Usha Sri2, Madiya Someswararao3,
Koyya Ravi Kumar4
Department of Computer Science and Engineering, Raghu Engineering College, Andhra Pradesh, India.
EmailId:itsmeviratvardhan@gmail.com , ushamandala18@gmail.com , ravikumarkoyya10@gmail.com , somesh14082@gmail.com
Abstract—Natural language processing (NLP) drives artificial intelligence (AI)driven chatbots that have gained great popularity in many industries in recent years [15], therefore improving humancomputer interactions. Optimized for quick conversational reactions, this paper describes an AIdriven chatbot driven by a finely tuned Llama 3.2 model. Using Streamlit for an interactive user interface, Ollama for model deployment, and SpeechRecognition and pyttsx3 for smooth voice input and texttospeech (TTS) output [5][10[11]], the chatbot combines voice and textbased communication [2][4].
Using Sloth, a dedicated framework for finetuning big language models (LLMs), the chatbot model is trained in a Google Colab environment. Exported in GGUF format and deployed using the Ollama runtime, the trained model helps run inference efficiently. Streamlit produces a strong user interface including chat history management, voicebased interaction toggle, live response streaming, and downloadable conversation logs [7][8].
The techniques used for chatbot finetuning, model deployment, and improvement of realtime interaction are brought forward in this study. The accuracy of responses, user experience evaluations, and performance improvements mentioned. The research shows how connecting cuttingedge NLP models with current deployment platforms improves chatbot usability and performance [1][9][12][15].
Keywords—Artificial Intelligence (AI), Natural Language Processing (NLP), Conversational AI, AI Chatbots, Large Language Models (LLMs), Llama 3.2, Fine-Tuning, Sloth Framework, Google Colab, Ollama Deployment, GGUF Format, Streamlit UI, Speech-to-Text (STT), Text-to-Speech (TTS), SpeechRecognition, pyttsx3, Model Optimization, User Interaction, Real-time Response, Voice-Based Chatbot
