Member-only storyNabanita RoyFollowTowards Data Science--ShareOne of the key challenges of Machine Learning(ML) is unbalanced data and the biases they introduce in ML models. With the advent of powerful Generative AI (GenAI) models, we can augment imbalanced training data with synthetic data easily, especially for Natural Language Processing(NLP) tasks. As a result, we can train models using classic ML algorithms for better performances in scenarios where Deep Learning models or directly using LLMs are not an option for reasons like the cost of computation, memory, availability of infrastructure or model explainability. Besides, despite the great efficacy shown by LLMs, we still don’t fully trust them. However, we can use LLMs to aid our work as data professionals and overcome roadblocks in building NLP systems.In this article, I have demonstrated how we can improve model performance for minority classes in imbalanced datasets using GenAI and Python to generate synthetic data and how we can iteratively engineer prompts to generate the desired outcome.Long story short, ML models need enough examples to learn patterns and predict accurately. If the data contains fewer examples, then the model does not generalise and consequently, perform well. In such scenarios, the model can overfit for classes that have more examples and underfit for classes with fewer examples. To tackle unbalanced data we traditionally use statistical sampling methods like over or under-sampling, typically using SMOTE.There are several examples of classification tasks in NLP where data is imbalanced and we have to resort to under-sampling to overcome this challenge. Information loss is the fundamental problem with under-sampling. While SMOTE implements efficient over-sampling strategies for numerical datasets, it is not…----Towards Data ScienceData Scientist @ EY (UK & Ireland) | Education Lead @ Women in AI Ireland | ❤ NLPNabanita RoyinTowards Data Science--Giuseppe ScalamognainTowards Data Science--14Heiko HotzinTowards Data Science--16Nabanita RoyinTowards Data Science--Dominik PolzerinTowards Data Science--8Jyotsna Choudhary--Manoranjan Rajguru--1Rei InamotoinUX Collective--16Tushit Dave--1Adam LoulyinLevel Up Coding--4HelpStatusWritersBlogCareersPrivacyTermsAboutText to speechTeams