Skip to main content
W&B Inference allows you to use a custom LoRA with some base models. This tutorial demonstrates how to create a fine-tuned LoRA using supervised post-training via the TRL library. This example uses a dataset of query/response pairs to fine-tune a model to respond like a cowboy. You can also create a LoRA using W&B Training which provides serverless reinforcement learning.

Post-training dataset

The following dataset contains 50 query and response pairs that are formatted as a list of messages, such as:
User: “What is your favorite color?”
Assistant: “Well, pardner, my favorite color’s the blazin’ orange of a desert sunset…”
The example file contains one JSON object per line. Save the following data in your working directory as cowboy_examples.jsonl.
cowboy_examples.jsonl

Post-training

This script trains a LoRA adapter on examples from the JSONL file and uploads it to W&B as an artifact for use with the W&B Inference API or Playground. At a high level, this script does the following:
  1. Logs in to W&B. The Hugging Face Transformers integration of W&B Models automatically records training progress and metrics.
  2. Loads the base model (OpenPipe/Qwen3-14B-Instruct) from Hugging Face.
  3. Configures the LoRA using hyperparameters, such as rank and alpha defined as constants near the top of the file.
  4. Loads the examples from the file into a dataset and then runs SFTTrainer. By default, the script uses all examples.
  5. Saves the LoRA and uploads it to W&B as an Artifact so it can be used with W&B Inference.
When the script completes, open the last printed URL in your browser to see the persisted Artifact. It looks like this: Artifact URL: https://wandb.ai/<yourentity>/create-lora-tutorial/artifacts/lora/OpenPipe_Qwen3-14B-Instruct_cowboy/v0 Save the following program as create_lora.py and update the ENTITY value with your W&B entity. To simplify execution the script uses inline script metadata to declare dependencies.
create_lora.py
Run the script using uv:
Execution time depends on hardware. You can add the --max-examples=10 argument to speed up training but it affects how well the LLM responds in character.

Using the LoRA

You can quickly try your new LoRA in the W&B Weave Playground. When you open the artifact URL, click the “Try in playground” button.
LoRA in Artifacts UI
Then input your prompt at the bottom of the chat interface.
LoRA in Playground UI
To use your newly created LoRA from code, see the Use Serverless LoRA Inference guide for step-by-step instructions.

Next steps

Once you have created the LoRA, you can experiment with its training, such as:
  • Training the LoRA with a smaller number of examples to see if it still provides the desired effect.
  • Changing the responses in the dataset to showcase another character, such as a pirate or a ninja.