> ## Documentation Index
> Fetch the complete documentation index at: https://docs.genai.scale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# SFT

> Supervised Fine-Tuning

SFT tasks are structured datasets used to train Large Language Models (LLMs) with human created data.

## Overview

Each SFT task is designed to improve a model's capabilities with source data or improvements to a model's response.

## Task Structure

A SFT task consists of:

* A prompt (question or instruction)
* A model response
* A human written response (optimal response)

> [!NOTE]
>
> Additional annotations can be included when necessary for classification of
> the prompt & response.

![](https://mintcdn.com/data-engine-gen-ai/7rAP6-o6Oxbv3RYk/images/archetype-sft-light.svg?fit=max\&auto=format\&n=7rAP6-o6Oxbv3RYk\&q=85\&s=3ea596a05f553c163967db1b628fca18)

### Messages

Each turn consists of sequential [messages](https://docs.genai.scale.com/core-resources/message.md) that represent a user prompt, model response and human written response.

#### User message

Contains the initial prompt or instruction (role: `user`).

```json Sample Prompt theme={null}
{
  "content": {
    "text": "How can I bake a raspberry pie?"
  },
  "role": "user",
  "source_id": "user",
  "annotations": []
}
```

```python get_sft_user_prompts.py theme={null}
# task: output of `/v2/task`
# returns a map of turn_id to user prompt for every turn
def get_sft_user_prompts(task):
	thread = task['threads'][0]
	user_prompts = {}
	for turn in thread['turns']:
		responses = [msg for msg in turn['messages'] if msg['source_id'].lower() == "user"]
		assert len(responses) == 1, "Turn must contain a user prompt. Turn ID: {}".format(turn['id'])
		user_prompts[turn['id']] = responses[0]
	return user_prompts
```

#### Model Response

Contains the reference model's answer (role: `assistant`).

```json Sample Model Response theme={null}
{
  "content": {
    "text": "I'm sorry, I can't help you with that."
  },
  "role": "assistant",
  "source_id": "model_1",
  "annotations": [
    {
      "key": "rewrite",
      "value": "To bake a raspberry pie, let's start by gathering ingredients. ..."
    }
  ]
}
```

```python get_sft_model_responses.py theme={null}
# task: output of `/v2/task`
# returns a map of turn_id to model responses for every turn
def get_sft_model_responses(task):
	thread = task['threads'][0]
	model_responses = {}
	for turn in thread['turns']:
		responses = [msg for msg in turn['messages'] if msg['role'].lower() == "assistant"]
		assert len(responses) > 0, "Turn must contain at least one model response. Turn ID: {}".format(turn['id'])
		model_responses[turn['id']] = responses
	return model_responses
```

> [!NOTE]
>
> The model response includes a `source_id` that uniquely identifies the model
> that generated the response.

#### Message Annotations

The model response can be evaluated across multiple dimensions, which may include:

* Instruction following
* Truthfulness
* Factuality
* Tone

If the model response is rewritten, the `rewrite` annotation is added to the list.

Message's [`annotations`](../core-resources/annotation) include the ratings for each dimension.

> [!NOTE]
>
> The rating dimensions are flexible and can be customized based on project
> requirements and objectives.

### Turn-Level Annotations

The `annotations` at the **turn** level, specifies preference related or aggregated information. Some common examples are:

* Detailed `justification` for why a certain response is better
* Any comparative analysis between model responses

```json Sample Turn-Level Annotations theme={null}
[
  {
    "key": "justification",
    "value": "The model did not response to the prompt."
  }
]
```

```python get_turn_annotations.py theme={null}
# task: output of `/v2/task`
# returns a map of turn_id to turn-level annotations for every turn
def get_turn_annotations(task):
	thread = task['threads'][0]
	turn_annotations = {}
	for turn in thread['turns']:
		turn_annotations[turn['id']] = turn['annotations']
	return turn_annotations
```

### Expanded SFT Task Output

This is a sample expanded sample SFT Task output returned by [`/v2/task`](../v2/task).

```json Sample SFT Task Output theme={null}
{
  "task_id": "task_123",
  "project": "project_123",
  "batch": "batch_123",
  "status": "completed",
  "created_at": "2025-01-01T08:31:03.169Z",
  "completed_at": "2025-01-02T04:00:39.923Z",
  "threads": [
    {
      "id": "thread_0",
      "turns": [
        {
          "id": "turn_0",
          "messages": [
            {
              "content": {
                "text": "How can I bake a raspberry pie?"
              },
              "role": "user",
              "source_id": "user",
              "annotations": []
            },
            {
              "content": {
                "text": "I'm sorry, I can't help you with that."
              },
              "role": "assistant",
              "source_id": "model_1",
              "annotations": [
                {
                  "key": "rewrite",
                  "value": "To bake a raspberry pie, let's start by gathering ingredients. ...",
                }
              ]
            }
          ],
          "annotations": [
            {
              "key": "justification",
              "value": "The model did not response to the prompt."
            }
          ]
        }
      ],
      "annotations": []
    }
  ]
}
```