Proses Training Fine-tuning LLM

Fine-tuning mengambil model pre-trained dan melatihnya lebih lanjut dengan data spesifik domain Anda. Hasilnya: model yang memahami konteks bisnis Anda.

Fine-tuning via OpenAI API

1. Upload Dataset

curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="fine-tune" \
  -F file="@training_data.jsonl"

2. Start Training Job

curl https://api.openai.com/v1/fine_tuning/jobs \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "training_file": "file-abc123",
    "model": "gpt-4o-mini-2024-07-18",
    "hyperparameters": {
      "n_epochs": 3,
      "batch_size": 1,
      "learning_rate_multiplier": 1.8
    }
  }'

3. Monitor Progress

# Check status
curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123 \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# Stream events
curl https://api.openai.com/v1/fine_tuning/jobs/ftjob-abc123/events \
  -H "Authorization: Bearer $OPENAI_API_KEY"

4. Use Fine-tuned Model

const completion = await openai.chat.completions.create({
  model: "ft:gpt-4o-mini-2024-07-18:org:custom-suffix:id", // Fine-tuned model ID
  messages: [
    { role: "system", content: "Kamu adalah asisten Web3." },
    { role: "user", content: "Apa itu gas fee?" },
  ],
});

Hyperparameters

Epochs

• 1-3 epochs: Fine-tuning ringan, tetap general
• 3-5 epochs: Sweet spot untuk dataset kecil (100-500 contoh)
• 5+ epochs: Risk overfitting, hanya untuk dataset besar

Learning Rate

• Default biasanya cukup baik
• Lebih kecil (0.5x-1x): Jika dataset sangat berbeda dari base model
• Lebih besar (1.5x-2x): Jika dataset mirip dengan base model

Batch Size

• 1: Dataset kecil, lebih stabil
• 4-8: Dataset menengah, lebih cepat
• 16+: Dataset besar, butuh GPU besar

LoRA / QLoRA (Efficient Fine-tuning)

Untuk model open-source, LoRA lebih efisien:

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                    # Rank — semakin tinggi semakin expressif
    lora_alpha=32,           # Scaling factor
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

model = get_peft_model(base_model, lora_config)
# Hanya train ~0.1% dari parameter!

Cost Estimation

OpenAI GPT-4o-mini fine-tuning:
- Training: $3.00 per 1M tokens
- Inference input: $0.30 per 1M tokens  
- Inference output: $1.20 per 1M tokens

Contoh: 100 contoh × 500 tokens = 50K tokens
Training cost: ~$0.15 (sangat murah!)

Latihan

Fine-tune GPT-4o-mini dengan 50 contoh Q&A tentang blockchain. Evaluasi apakah output lebih konsisten dibanding base model.