Versions
What is a version?
EasyLLM projects follow a data-centric version control system for doing data versioning and model versioning.
To create a version, you first create a dataset. This allows you to create finetuned LLMs using the dataset. In a particular version, you will have one dataset and one or many finetuned LLMs created using that particular dataset. You can choose the best finetuned LLM in a particular version based on evaluation results.
For further versions, you can add a new dataset, and this will be added on top of the previous versions datasets. Your dataset will increase incrementally like stacking up pizza boxes. This allows you to finetune new LLMs with a higher dataset size.
For example, let's say you are working on a project to analyze customer reviews and predict their sentiment. You could create a version 1 with a dataset of 1,000 reviews and finetune few LLMs with that dataset. Later, you could create version 2 with an additional 500 reviews, resulting in a combined dataset of 1,500 reviews and finetune LLMs with the dataset.
This abstraction allows you to easily maintain datasets and the associated finetuned LLMs easily and track changes and improvements over time. You can create multiple versions under a project, each with its dataset and finetuned LLMs, and compare the performance of each version.
You can compare performance between versions, and easily roll back to previous versions if necessary. This can help you make more informed decisions about which finetuned LLM to use in production and ensure that your LLMs are always performing at their best.
Dashboard Page
The dashboard page displays all versions within a particular project, sorted by version number in ascending order. You can view version details, including version number, created date, created by, and type. If a version is marked as production, it will have a "Production" label.
Create New Version
To create a new version, click on the "Create Version" floating button. This will take you into the Finetune Wizard and allow you to add a new dataset. As soon as you add a new dataset in the "Create Dataset" step, the version will be created. You can continue on the wizard to finetune LLMs with the dataset.
For example, let's say you have a dataset of customer reviews for a product and sentiments for that reviews. You can create a new version by adding a new dataset of customer reviews with sentiments.
Actions on Version
You can do the following actions on a version
Show progress
To view the progress of a version, click on the 3-dot menu next to the version you want to view and select "Show Progress". This will show you the progress of the version, including the dataset creation status and the sweep(hyperparameter tuning) status with status of each finetune run.
Show Best Fineune
To view the best finetuned LLM of a version, click on the 3-dot menu next to the version you want to view and select "Show Best finetune". This will show you the best finetune of the version based on the metric you have configured.
You can use this finetune name in your LLM inference API code. For example:
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="ft-gpt-4o-mini", # replace with your finetuned LLM name
messages=[
{"role": "user", "content": "I did not like the product. It was not working as shown in the video."}
]
)
- Azure OpenAI: Finetuned LLMs aren't automatically available for inference. You must deploy them, which incurs hourly charges. You can deploy across different regions, subscriptions and resources (Refer this for more details).
- AWS Bedrock: Similar to Azure, you need to deploy finetuned LLMs, incurring hourly charges.
Resume on finetune wizard
In a specific version, once you have created a dataset, you can resume on the finetune wizard by clicking on the 3-dot menu next to the version and select "Resume on Finetune Wizard". This will take you back to the Finetune Wizard and you can continue from where you left off.
This can be also used to perform multiple Sweeps (hyperparameter tuning) on the same dataset.
Mark as Production
To mark a version as production version, click on the 3-dot menu next to the version you want to mark and select "Mark as Production". This will allow you to easily identify the version that is currently in use.
For example, let's say you have two versions, version 1 and version 2. Based on the evaluation results and your testing, you found that version 2 performs better. You can mark version 2 as production to ensure that it is the version that is currently in use.
Revert to Previous Version
If you need to undo changes, you can revert to a previous version. To do this, click on the 3-dot menu next to the version you want to revert to and select "Revert to this Version".
For example, let's say you have created version 2 and Version 3, but you find that the changes have negatively impacted performance and there is no potential improvements. You can revert to version 1 to undo the changes and restore the previous version.
Delete Finetunes
If you are not happy with the results from created finetunes and you want to remove the underperforming finetunes and have a clean slate, you can delete them. To do this, click on the 3-dot menu next to the latest version and select "Delete finetunes". This will not delete the datasets, and you can retry the finetuning again in the Finetune Wizard.
For example, let's say you have created several finetunes for a particular version, but you find that none of them are suiting your needs. You can delete the finetunes entirely.
Delete Version
If you are not happy with the results and want to modify the dataset, you can delete the latest version. To do this, click on the 3-dot menu next to the latest version and select "Delete". This will delete the version fully, along with the dataset and all finetunes.
For example, let's say you have created a new version, but you find that the dataset is not suitable for your needs. You can delete the version to remove the dataset and start again with a new dataset.
Continuous Finetuning
For all the new versions you create after creating the first version (version > 1), Users can choose between continuous finetuning or new finetune.
- Choosing
Continuous finetuningwill do finetuning on the previously finetuned LLM with new data. - Choosing
New finetunewill let you choose the from the normal list of LLMs to finetune. Here the data from current version and previous versions will be combined to finetune the LLM.