Skip to content

Version Management

What is a version?

In EasyLLM, a version is a data-centric version control system for doing data versioning and model versioning.

To create a version, you first create a dataset. This allows you to create fine-tuned models using the dataset. In a particular version, you will have one dataset and one or many fine-tuned LLM models created using that particular dataset. You can choose the best model in a particular version based on different metrics that you can configure.

For new versions, you can add a new dataset, and this will be added on top of the previous versions datasets. Your dataset will increase incrementally like stacking up pizza boxes. This allows you to fine-tune new models with a higher dataset size.

For example, let's say you are working on a project to analyze customer reviews and predict their sentiment. You could create a version with a dataset of 1,000 reviews and a fine-tuned model trained on that dataset. Later, you could create a new version with an additional 500 reviews, resulting in a dataset of 1,500 reviews and a new fine-tuned model trained on the larger dataset.

This abstraction allows you to easily maintain datasets and the associated fine-tuned models easily and track changes over time. You can create multiple versions under a project, each with its dataset and fine-tuned models, and compare the performance of each version.

Version management is important because it allows you to track changes to your models over time, compare performance between versions, and easily roll back to previous versions if necessary. This can help you make more informed decisions about which model to use in production and ensure that your models are always performing at their best.

Dashboard Page

The dashboard page displays all versions within a particular project, sorted by version number in ascending order. You can view version details, including version number, created date, created by, and type. If a version is marked as production, it will have a "Production" label.

Create New Version

To create a new version, click on the "Create Finetune" floating button. This will take you into the Finetune Wizard and allow you to add a new dataset and create fine-tuned models using that dataset.

For example, let's say you have a dataset of customer reviews for a product and sentiments for that reviews. You can create a new version by adding a new dataset of customer reviews with sentiments for creating a finetuned LLM model to find sentiment of a given review.

Actions on Version

You can do the following actions on a version

Show progress

To view the progress of a version, click on the 3-dot menu next to the version you want to view and select "Show Progress". This will show you the progress of the version, including the dataset creation status and the sweep(hyperparameter tuning) status with status of each finetune creation.

Show Best Fineune

To view the best fine-tune of a version, click on the 3-dot menu next to the version you want to view and select "Show Best Fine-tune". This will show you the best finetune of the version based on the metric you have configured.

You can use this finetune name in your OpenAPI Completion API call for model parameter. Refere to OpenAPI Completion API for more details.

Python code to call OpenAPI Completion API with the best fine-tune model value

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")
openai.Completion.create(
  model="ada:ft-organization:project-v-1-s-1-f-1-2023-01-15-07-25-02",
  # replace above value with your best finetune model name
  prompt="Say this is a test",
  max_tokens=7,
  temperature=0
)

Mark a Version as Production

To mark a version as production, click on the 3-dot menu next to the version you want to mark and select "Mark as Production". This will allow you to easily identify the version that is currently in use.

For example, let's say you have two versions of your model, version 1 and version 2. You have tested both versions and found that version 2 performs better. You can mark version 2 as production to ensure that it is the version that is currently in use.

Revert to Previous Version

If you need to undo changes, you can revert to a previous version. To do this, click on the 3-dot menu next to the version you want to revert to and select "Revert to this Version".

For example, let's say you have created version 2 and Version 3, but you find that the changes have negatively impacted performance and there is no potential improvements. You can revert to version 1 to undo the changes and restore the previous version.

Delete Finetunes

If you are not happy with the results from created fine-tunes and you want to remove the underperforming fine-tunes, you can delete them. To do this, click on the 3-dot menu next to the latest version and select "Delete Fine-tunes". This will not delete the datasets, and you can retry the finetuning again in the Finetune Wizard.

For example, let's say you have created several fine-tunes for a particular version, but you find that none of them are suiting your needs. You can delete the finetunes entirely.

Delete Full Version

If you are not happy with the results and want to modify the dataset, you can delete the latest version. To do this, click on the 3-dot menu next to the latest version and select "Delete". This will delete the version fully, along with the dataset and all fine-tunes.

For example, let's say you have created a new version, but you find that the dataset is not suitable for your needs. You can delete the version to remove the dataset and start again with a new dataset.