Augmenting Data with AI

According to the OpenAI official documentation, a JSONL dataset should contain at least 10 JSON objects, and conducting fine-tuning with only 50-100 high-quality data points can yield good results.

However, manually creating a dataset is time-consuming and costly, which is why using data augmentation techniques to expand the dataset and then refining the augmented data is more efficient.

Data augmentation is a technique for generating new data based on existing data, allowing for an increased dataset size, preventing overfitting, and enhancing the generalization performance of the AI model.

In the past, implementing data augmentation required writing complex code to utilize a program. These days, data augmentation can be easily accomplished by instructing text-generating AI to create new data based on existing data.

Codefriends offers a feature that allows you to perform complex data augmentation with just one click.

You can augment 10 lines of JSON data in the Codefriends fine-tuning practice environment through the 3 steps below.

When augmenting data, generative AI is used to create new training data based on the JSON data that has been generated so far.