Fine-tuning Gemini Pro for Text Emotion classification[AI for Autism]
Autism Spectrum Disorder (ASD) is a cognitive disease characterized by communication and exchange difficulties. One obstacle which people with this condition encounter is interpretation and articulation of emotions which may affect their social lives adversely. Recent technological advancements in Artificial Intelligence (AI) have made it possible to address some of the challenges faced by people living with ASD, for instance employing AI in emotional text classification. By doing so it is anticipated that individuals will be able to improve their understanding of feelings as well as other conversational approaches hence enhancing their interaction within the society and improving their quality of life generally.
What is Text emotion classification?
Text emotion classification focuses on applying AI algorithms to analyze written text and identify the underlying emotions from this text. These algorithms can recognize the subtleties in language such as context, word choice and structure of sentences enabling emotion identification. For individuals with ASD who may struggle with interpreting nonverbal cues like facial expressions and body language, text emotion classification can be a useful alternative for understanding others’ emotions.
Gemini Pro model
Gemini Pro is a high powered large language model (LLM) developed by Google AI that is capable of revolutionizing data analysis and visualization. It has gone beyond typical tools by incorporating natural language processing (NLP) and advanced reasoning along with analytical capabilities.
Fine-tuning
Google has recently released the fine-tuning ability of Gemini Pro. But why do we need fine-tuning? There are multiple benefits of finetuning large models. By finetuning we can achieve excellent results by using very less amount of data since we are leveraging the original data of the mode. Generally the models are trained on large datasets and by fine-tuning we can build upon it and use the model on a downstream task.
We can finetune the Gemini pro model either through the API in Python or using the AI studio. In this blog we will delve into both of these methods.
If using the API first we need to setup the credentials and for finetuning you will need to setup the Oauth credentials in you project.
import os
if 'COLAB_RELEASE_TAG' in os.environ:
from google.colab import userdata
import pathlib
pathlib.Path('client_secret.json').write_text(userdata.get('CLIENT_SECRET'))
!gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'
else:
!gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'We will be using the Goemotions dataset here.
import pandas as pd
df = pd.read_csv('/content/data/full_dataset/goemotions_1.csv')
df = df[df['example_very_unclear'] == False]
df.drop(['id', 'author', 'subreddit' ,'link_id' ,'parent_id' ,'created_utc' ,'rater_id', 'example_very_unclear'],inplace=True,axis=1)
df['emotion'] = (df.iloc[:, 1:] == 1).idxmax(1)
df = df[['text','emotion']]
df.head()
After this we setup our base model for finetuning.
import google.generativeai as genai
genai.configure(transport='grpc')
base_model = [
m for m in genai.list_models()
if "createTunedModel" in m.supported_generation_methods][0]
base_modelNext up we create our training set in the correct format of the prompts.
training_data= []
for i in range(df.shape[0]):
training_data.append({
'text_input': df.iloc[i,0],
'output': df.iloc[i,1],
})
If using the AI studio, we can directly import the CSV file to create our training prompt
After creating our training prompt, we setup our model and feed the dataset into the model. This is also supported in both the API and AI studio.
name = 'generate-num-1'
operation = genai.create_tuned_model(
source_model=base_model.name,
training_data=training_data,
id = name,
epoch_count = 20,
batch_size=4,
learning_rate=0.001,
)
In the AI studio, we can set the hyperparameters using the Advanced settings.
After tuning our model we can see that the new finetuned model is created in our library.
Now we can test our model. We can also use the API to load our tuned model and test it thoroughly.
model = genai.GenerativeModel(model_name=f'tunedModels/{name}')
result = model.generate_content('I am elated')
result.text
And bravo! It is done. We can clearly see that by using just 500 examples we are able to finetune the Gemini Pro model for Text emotion classification and that too with such high accuracy.
