Fine-Tuning Large Language Models (LLMs) with .NET Core, Python, and Azure

Table of Contents

1. Introduction
2. Why fine-tune large language models?
3. Solution Overview
4. Environment Settings
5. Training and fine-tuning using Python
6. Integrate fine-tuned models in .NET Core
7. Deploy to Azure
8. Best Practices
9. Conclusion

---

1. Introduction

Large-scale language models (LLMs) have received widespread attention for their ability to understand and generate human-like text. However, many organizations have unique, domain-specific data sets and vocabularies that may not be fully captured by generic models. Fine-tuning enables developers to adapt these large models to specific environments or industries, improving accuracy and relevancy.

This article explores how to fine-tune an LLM using Python, then integrate and deploy the resulting model into a .NET Core C# application, all done on Microsoft Azure for scalability and Convenience.

---

2. Why fine-tune large language models?

3. Domain Specificity: LLM can be fine-tuned to use industry-specific terminology, product names, or jargon.

4. Performance improvements: Fine-tuning often reduces errors and improves relevancy in use cases such as customer service, research, and analytics.

5. Reduce costs: Instead of building a model from scratch, you can customize an existing powerful LLM.

6. Improving efficiency: You take advantage of pre-trained weights and only adjust the final layer or parameters, thus speeding up the process.

---

3. Solution Overview

Components and Technologies

1. Python for fine-tuning

   • Commonly used libraries (e.g. Hugging Face Transformers, PyTorch)
   • Simplified the process of loading and tuning pre-trained models

2. .NET Core C# for integration

   • Expose a backend service or API for fine-tuning the model
   • Strongly typed language, familiar to many enterprise developers

3. Azure Services

   • Azure Machine Learning for training and model management
   • Azure Storage for data and model artifacts
   • Azure App Service or Azure Function for hosting .NET Core applications
   • Azure Key Vault (optional) for protecting credentials

---

4. Environment settings

Prerequisites

• Azure Subscription: Required to create resources such as Machine Learning Workspace and App Service.
• Python 3.8 : Installed locally for model fine-tuning.
• .NET 6/7/8 SDK: For creating and running .NET Core C# applications.
• Visual Studio 2022 or Visual Studio Code: Recommended IDE.
• Azure CLI: Used to configure and manage Azure services through the terminal.
• Docker (optional): Can be used to containerize your application if needed.

---

5. Training and fine-tuning using Python

This example uses Hugging Face Transformers - one of the most widely adopted LLM fine-tuning libraries.

5.1 Set up virtual environment

<code>python -m venv venv
source venv/bin/activate  # 在 Windows 上：venv\Scripts\activate</code>

5.2 Install dependencies

<code>pip install torch transformers azureml-sdk</code>

5.3 Create an Azure Machine Learning workspace

1. Resource Group and Workspace:

<code>   az group create --name LLMFinetuneRG --location eastus
   az ml workspace create --name LLMFinetuneWS --resource-group LLMFinetuneRG</code>

1. Configure the local environment to connect to the workspace (using a config.json file or environment variables).

5.4 Fine-tuning script (train.py)

<code>import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer
from azureml.core import Workspace, Run

# 连接到 Azure ML
ws = Workspace.from_config()
run = Run.get_context()

model_name = "gpt2"  # 示例模型
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# 加载自定义数据集（本地或来自 Azure 存储）
# 示例：Azure ML 中的文本文件或数据集
train_texts = ["此处输入您的特定领域文本..."]  # 简化版
train_encodings = tokenizer(train_texts, truncation=True, padding=True)

class CustomDataset(torch.utils.data.Dataset):
    def __init__(self, encodings):
        self.encodings = encodings
    def __len__(self):
        return len(self.encodings["input_ids"])
    def __getitem__(self, idx):
        return {k: torch.tensor(v[idx]) for k, v in self.encodings.items()}

train_dataset = CustomDataset(train_encodings)

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=2,
    save_steps=100,
    logging_steps=100
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
)

trainer.train()

# 保存微调后的模型
trainer.save_model("./fine_tuned_model")
tokenizer.save_pretrained("./fine_tuned_model")</code>

5.5 Register model in Azure

<code>from azureml.core.model import Model

model = Model.register(
    workspace=ws,
    model_path="./fine_tuned_model",
    model_name="myFineTunedLLM"
)</code>

At this point, your fine-tuned model is stored in Azure Machine Learning for easy access and version control.

---

6. Integrate fine-tuned models in .NET Core

6.1 Create .NET Core Web API project

<code>dotnet new webapi -n FineTunedLLMApi
cd FineTunedLLMApi</code>

6.2 Add dependencies

• HttpClient for calling Azure endpoints or local inference API
• Newtonsoft.Json (if you prefer to use JSON.NET for serialization)
• Azure.Storage.Blobs or Azure.Identity for secure access to Azure resources

<code>dotnet add package Microsoft.Extensions.Http
dotnet add package Microsoft.Azure.Storage.Blob
dotnet add package Newtonsoft.Json</code>

6.3 ModelConsumerService.cs

Assume you have deployed your fine-tuned model as a web service (for example, using Azure Container Instance or a custom endpoint in Azure ML). The following code snippet calls the service to get completion results.

<code>using Newtonsoft.Json;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

public class ModelConsumerService
{
    private readonly HttpClient _httpClient;

    public ModelConsumerService(IHttpClientFactory httpClientFactory)
    {
        _httpClient = httpClientFactory.CreateClient("FineTunedModel");
    }

    public async Task<string> GetCompletionAsync(string prompt)
    {
        var requestBody = new { prompt = prompt };
        var content = new StringContent(
            JsonConvert.SerializeObject(requestBody),
            Encoding.UTF8, 
            "application/json");

        var response = await _httpClient.PostAsync("/predict", content);
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStringAsync();
    }
}</code>

6.4 LLMController.cs

<code>using Microsoft.AspNetCore.Mvc;
using System.Threading.Tasks;

[ApiController]
[Route("[controller]")]
public class LLMController : ControllerBase
{
    private readonly ModelConsumerService _modelService;

    public LLMController(ModelConsumerService modelService)
    {
        _modelService = modelService;
    }

    [HttpPost("complete")]
    public async Task<IActionResult> CompletePrompt([FromBody] PromptRequest request)
    {
        var result = await _modelService.GetCompletionAsync(request.Prompt);
        return Ok(new { Completion = result });
    }
}

public class PromptRequest
{
    public string Prompt { get; set; }
}</code>

6.5 Configuring .NET Core Applications

In Program.cs or Startup.cs:

<code>var builder = WebApplication.CreateBuilder(args);

// 注册 HttpClient
builder.Services.AddHttpClient("FineTunedModel", client =>
{
    client.BaseAddress = new Uri("https://your-model-endpoint/");
});

// 注册 ModelConsumerService
builder.Services.AddTransient<ModelConsumerService>();

builder.Services.AddControllers();
var app = builder.Build();

app.MapControllers();
app.Run();</code>

---

7. Deploy to Azure

8. Azure App Service:

   • For many .NET Core applications, this is the easiest path.
   • Create a new Web App from the Azure portal or via the CLI.

<code>python -m venv venv
source venv/bin/activate  # 在 Windows 上：venv\Scripts\activate</code>

1. Azure Function (optional):

   • Ideal for running serverless, event-driven logic if your usage is intermittent or scheduled.

2. Azure Kubernetes Service (AKS) (Advanced):

   • Ideal for large-scale deployment.
   • Containerize your application using Docker and push it to Azure Container Registry (ACR).

---

8. Best Practices

9. Data Privacy: Ensure responsible handling of sensitive or proprietary data, especially during model training.

10. Monitoring and Logging: Integrate with Azure Application Insights to monitor performance, track usage, and detect anomalies.

11. Security: Use Azure Key Vault to store keys (API keys, connection strings).

12. Model Versioning: Track different fine-tuned versions of your model in Azure ML; rollback to older versions if needed.

13. Hint Engineering: Refine your hints to get the best results from your fine-tuned model.

---

9. Conclusion

Fine-tune LLM using Python and Azure Machine Learning and then integrate them into .NET Core applications, allowing you to build powerful domain-specific AI solutions. This combination is an excellent choice for organizations looking to take advantage of Python’s AI ecosystem and the enterprise capabilities of .NET, all powered by the extensibility of Azure.

With careful planning for security, data governance, and DevOps, you can launch a production-ready solution that meets real-world needs, delivering accurate domain-specific language functionality in a powerful and easy-to-maintain framework.

The above is the detailed content of Fine-Tuning Large Language Models (LLMs) with .NET Core, Python, and Azure. For more information, please follow other related articles on the PHP Chinese website!