Skip to content

Classification

LLMs are powerful tools for classification tasks. They possess a deep understanding of context, language structure, and semantics, enabling them to make informed predictions. By finetuning with lot of examples, these models can accurately categorize and classify text based on different criteria. For tasks like sentiment analysis, toxicity detection, spam detection, and intent recognition, LLMs can be finetuned to automate the process of classification, saving time and effort. The potential applications for finetuned LLMs in classification are extensive.

Binary Classification

Prompt Completion
You are so dumb. Can't you understand a simple concept? 1
I disagree with your opinion, but let's have a respectful discussion about it. 0
You're such an idiot. How can you not grasp such a basic concept? 1
I understand your viewpoint, but I have a different perspective. Let's discuss it respectfully. 0
Are you mentally challenged? It's not that difficult to comprehend. 1
I respect your opinion, although I hold a different stance. Let's engage in a civil conversation. 0

With the above examples, the model will learn to classify the text into two classes: 1 and 0. You can also use labels like positive - negative, toxic - non-toxic, spam - ham, etc.

Multi-Class Classification

Prompt Completion
the movie is good. positive
I don't like the movie at all. negative
Okay for one time watch. neutral
Loved the acting of Leo in this movie. positive
I would not recommend this movie to anyone. negative
I had high expectations from this movie, but it was not good as I expected. Kind of in the middle area neutral

Best Practices

Here are some best practices to consider when working with classification problems:

  • In classification problems, each input in the prompt should be classified into one of the predefined classes.
  • Choose classes that map to a single token. At inference time, specify max_tokens=1 since you only need the first token for classification. Ensure that label names correspond to a single token by checking them with the OpenAI Tokenizer.
  • Aim for at least ~100 examples per class.
  • To get class log probabilities, you can specify logprobs=5 (for 5 classes) when using your model.
  • For classification into a large number of categories, it is recommended to convert those categories into numbers, which work well up to ~500 categories.