Hands on Large Language Models

Dec 14, 2025··

13 min read

Hands-On Large Language Models (LLMs)

A practical, in-depth guide for researchers, engineers, and practitioners who want to understand, build, fine-tune, evaluate, deploy, and safely operate large language models (LLMs). This article covers history and theory, architectures, training data, hands-on code examples, fine-tuning strategies (LoRA / QLoRA), retrieval-augmented generation (RAG), deployment and optimization, safety & evaluation, current state, and future directions.

Table of Contents

Introduction and scope
A brief history of LLMs
Key concepts and terminology
Theoretical foundations
Data: pretraining and fine-tuning corpora
Transformer architectures and attention mechanics
Practical setup: hardware, software, and tooling
Hands-on examples
- Inference: Hugging Face Transformers pipeline
- Inference: OpenAI API example
- Fine-tuning: classic supervised fine-tune
- Parameter-efficient fine-tuning: LoRA / QLoRA with PEFT
- RAG: building a retrieval-augmented pipeline with FAISS
- Prompt engineering patterns and chain-of-thought
Evaluation: metrics and adversarial testing
Deployment & optimization: quantization, batching, streaming
Safety, ethics, and governance
Current landscape and trends
Future directions and open research problems
Practical checklist and best practices
References and further reading

Introduction and scope

Large Language Models (LLMs) are neural networks trained on massive text corpora that can generate text, answer questions, summarize, translate, reason, and perform many language tasks. This guide emphasizes practical, hands-on knowledge: how LLMs work, how to run them, fine-tune them, integrate them with retrieval, evaluate them, and deploy them responsibly.

Target audience:

ML engineers and researchers implementing LLM pipelines
Product teams building LLM-powered features
Data scientists experimenting with fine-tuning and evaluation
Students learning about modern NLP

A brief history of LLMs

Pre-Transformer era: n-gram models, RNNs, LSTMs. These struggled with long-range dependencies.
2017: "Attention is All You Need" (Vaswani et al.) introduced the Transformer architecture — scalable self-attention.
2018-2019: GPT-1, BERT. GPT introduced autoregressive LM pretraining; BERT used masked language modeling.
2019-2021: GPT-2, GPT-3 demonstrated scaling behavior; GPT-3 (175B parameters) showed few-shot abilities, prompting research in emergent capabilities.
2022 onward: LLaMA, Mistral, GPT-4, open-source derivatives, and techniques for PEFT (LoRA, adapters) and quantization (8-bit, 4-bit, QLoRA).
2023-2024: RAG pipelines and instruction-tuning (e.g., RLHF) became standard for usable assistants.

Key concepts and terminology

Autoregressive vs. Encoder-Decoder vs. Masked models
- Autoregressive: predict next token (GPT family).
- Encoder-decoder: sequence-to-sequence (T5, BART).
- Masked: predict masked tokens (BERT).
Tokenization: Byte-Pair Encoding (BPE), SentencePiece, or WordPiece; subword tokenization strategies.
Self-Attention: computation of token-token affinities using queries, keys, and values.
Fine-tuning: adapting a pretrained LM to a downstream task.
Instruction tuning: fine-tuning on instruction-response pairs to make the model follow instructions.
RLHF (Reinforcement Learning from Human Feedback): uses human preferences to reward and steer output quality.
LoRA (Low-Rank Adapters): inject low-rank updates into weights for efficient fine-tuning.
QLoRA: LoRA combined with quantization for extreme memory efficiency.
RAG (Retrieval-Augmented Generation): combines vector search retrieval of documents with LLM generation.
Quantization: lowering numeric precision (8-bit, 4-bit) for memory and inference efficiency.
Emergent abilities: behaviors that appear at scale and not in smaller models.

Theoretical foundations

Self-Attention: key building block. Given input embeddings X, compute Q = XWq, K = XWk, V = XWv; attention = softmax(QK^T / sqrt(dk)) V.
Positional encoding: transforms or embeddings that inject sequence order.
Residual connections and LayerNorm for stable deep training.
Scaling laws: model loss scales predictably with compute, parameters, and dataset size (Kaplan et al.). Bigger models generally improve performance across many tasks.
Emergence: complex behaviors not present in smaller models may appear once models reach certain sizes.
Generalization vs. memorization: risks of memorizing training data (privacy/injection).
Calibration and uncertainty: softmax probability is not a well-calibrated measure of answer correctness; various calibration techniques exist.

Data: pretraining and fine-tuning corpora

Pretraining data: large crawls (CommonCrawl), books, Wikipedia, code repositories (GitHub), conversation datasets. Data quality matters — noisy web data often needs deduplication and filtering.
Fine-tuning data: labeled datasets (SQuAD, GLUE), instruction-following datasets (SuperNI, Anthropic HH), domain-specific corpora.
Data governance: licensing, PII removal, copyright, and fairness considerations.

Transformer architectures and attention mechanics

Multi-head attention: parallel attention heads capture different relations.
Feed-forward networks between attention layers.
Decoder-only (GPT-style): causal (masked) attention for autoregressive generation.
Encoder-decoder: cross-attention allows attending over source sequences.
Efficient variants: sparsity, linear attention, sliding-window attention (Longformer), etc., for long-context handling.

Practical setup: hardware, software, and tooling

Hardware:
- GPUs (NVIDIA A100, H100) for training/fine-tuning.
- Consumer GPUs (3090/4090) can run smaller LLMs or quantized large models for inference & finetuning with QLoRA.
- CPU inference possible with quantized models and ONNX runtime but typically slower.
Software:
- Python, PyTorch (or JAX/Flax), Hugging Face Transformers, Datasets, Accelerate, PEFT, bitsandbytes (bnb), SentenceTransformers, FAISS.
- Tools: LangChain, LlamaIndex, Weaviate, Pinecone for RAG and orchestration.
Installation (typical):
- pip install transformers accelerate bitsandbytes peft datasets sentence-transformers faiss-cpu

Example:

Bash

pip install transformers accelerate bitsandbytes peft datasets sentence-transformers faiss-cpu

Hands-on examples

1) Simple inference with Hugging Face Transformers

A minimal local inference using an available model (CPU/GPU dependent).

Python example:

Python

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model_name = "gpt2"  # replace with larger model like "gpt2-xl" or an LLaMA-variant via HF hub
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

generator = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)  # device=0 for first GPU
out = generator("Explain attention in simple terms:", max_length=150, do_sample=True, temperature=0.7)
print(out[0]['generated_text'])

Notes:

For large models, use device_map="auto" (with accelerate or Transformers 4.30+) and pass load_in_8bit=True with bitsandbytes.

2) Inference with OpenAI API

Quick usage example (replace with your key).

Python

import openai
openai.api_key = "sk-..."

resp = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize the main ideas of the Transformer paper in 3 bullets."}
    ],
    temperature=0.2,
    max_tokens=150
)
print(resp['choices'][0]['message']['content'])

3) Classic supervised fine-tuning (transformers Trainer)

Fine-tune an encoder-decoder for summarization or a causal LM for next-token prediction. For brevity, a sketch:

Python

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, Trainer, TrainingArguments

model_name = "t5-small"
dataset = load_dataset("cnn_dailymail", "3.0.0", split="train[:1%]")
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

def preprocess(batch):
    inputs = ["summarize: " + doc for doc in batch["article"]]
    model_inputs = tokenizer(inputs, max_length=512, truncation=True)
    labels = tokenizer(batch["highlights"], max_length=128, truncation=True)
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

dataset = dataset.map(preprocess, batched=True)
training_args = TrainingArguments(
    output_dir="./t5-finetune",
    num_train_epochs=1,
    per_device_train_batch_size=4,
    fp16=True
)
trainer = Trainer(model=model, args=training_args, train_dataset=dataset)
trainer.train()

4) Parameter-efficient fine-tuning: LoRA & QLoRA (PEFT)

LoRA allows training low-rank updates so you can fine-tune large models on a single GPU. QLoRA adds 4-bit quantization to reduce memory.

High-level steps:

Quantize the base model (bitsandbytes).
Apply PEFT/LoRA adapters.
Train only adapter parameters.

Example with Hugging Face PEFT (sketch):

Python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
import bitsandbytes as bnb

model_name = "meta-llama/Llama-2-7b"  # example (check HF license)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)

model = AutoModelForCausalLM.from_pretrained(model_name, load_in_4bit=True, device_map="auto",
                                             quantization_config=bnb.QuantizationConfig(load_in_4bit=True))
model = prepare_model_for_kbit_training(model)

lora_config = LoraConfig(
    r=8,  # rank
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)

# Now train only the LoRA params using accelerate/trainer.

For full QLoRA workflow, Hugging Face has an example repo "trlx" and community scripts. Using 48 GB GPU (or even 24 GB with QLoRA on a 3090/4090) is often enough for practical experiments.

5) Retrieval-Augmented Generation (RAG) with FAISS + Sentence-Transformers

RAG pipeline: embed docs, store in vector store (FAISS), retrieve nearest neighbors at query time, and condition the LLM on these passages.

Steps:

Embed corpus with sentence-transformers.
Index with FAISS.
At query time, embed query, search index, build context prompt, pass to LLM.

Example (simplified):

Python

from sentence_transformers import SentenceTransformer
import faiss
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

# 1) Embed corpus
corpus = ["Doc 1 text ...", "Doc 2 text ..."]  # load real docs
embed_model = SentenceTransformer("all-MiniLM-L6-v2")
corpus_embeddings = embed_model.encode(corpus, convert_to_numpy=True)

# 2) Build FAISS index
d = corpus_embeddings.shape[1]
index = faiss.IndexFlatIP(d)
faiss.normalize_L2(corpus_embeddings)
index.add(corpus_embeddings)

# 3) Query and generate
query = "What is attention?"
q_emb = embed_model.encode([query], convert_to_numpy=True)
faiss.normalize_L2(q_emb)
k = 3
D, I = index.search(q_emb, k)
retrieved = [corpus[i] for i in I[0]]

# 4) Build prompt
context = "\n\n---\n\n".join(retrieved)
prompt = f"Use the following documents to answer the question.\n\nDocuments:\n{context}\n\nQuestion: {query}\nAnswer:"

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
print(generator(prompt, max_length=200)[0]['generated_text'])

For production, use hybrid retrieval, chunking, reranking, and caching.

Prompt engineering: patterns and best practices

System vs. user messages: set assistant behavior via system prompt.
Instruction format: give clear, concise tasks. Use examples (few-shot).
Temperature/top-p: control creativity. Lower temp for deterministic outputs; higher for creativity.
Chain-of-thought (CoT): ask model to reason step-by-step; can be enabled by prompt or use few-shot CoT.
Self-consistency: sample multiple CoTs and aggregate.
Output constraints: ask for specific format (JSON schema) to make parsing easier.
Tool-use scaffolding: when LLM has tool access, provide tool descriptions and invocation format.

Example instruction prompt:

YAML

System: You are a helpful assistant that answers in short bullet points.
User: Summarize the following article in 3 bullets: <article_text>

Chain-of-thought example:

YAML

User: Solve: if 3x + 5 = 20, what's x? Show reasoning.
Model: First, subtract 5 from both sides -> 3x = 15. Then divide both sides by 3 -> x = 5.

Prompt engineering patterns:

Zero-shot: concise instruction.
Few-shot: include 2–5 examples of desired behavior.
Chain-of-thought prompting for reasoning.
Role play / persona injection.

Evaluation: metrics and adversarial testing

Task metrics:
- Text generation: BLEU, ROUGE, METEOR (for translation/summarization).
- Perplexity for LM modeling.
- Exact Match / F1 for QA.
- Human evaluation: fluency, relevance, correctness, factuality.
- Embedding-based metrics: BERTScore, MoverScore.
Safety/regression tests:
- Prompt injection, malicious instructions.
- Hallucination tests: ask for verifiable facts.
- Bias audits: evaluate different demographic contexts.
A/B testing in production for user-facing features.
Red-teaming: adversarial probing by specialized human testers.

Example evaluation code (BLEU):

Python

from datasets import load_metric
bleu = load_metric("bleu")
predictions = [["the", "cat", "sat"]]
references = [[["the", "cat", "is", "sitting"]]]
bleu.compute(predictions=predictions, references=references)

Note: Use multiple metrics; no single number captures quality.

Deployment & optimization

Key considerations:

Latency vs. throughput tradeoffs.
Batch requests where possible.
Use quantized models (8-bit, 4-bit) for inference to reduce memory and cost.
Use model sharding for very large models across multiple GPUs (device_map="auto").
Use GPU A100/H100 for low-latency or high-throughput needs.
Use server frameworks: Triton, NVIDIA TensorRT, ONNX Runtime, FastAPI + Uvicorn for REST.
Autoscaling, caching, and rate limiting in production.
Observability: logs, metrics (latency, throughput, correctness), traceability of prompts/responses.

Quantization strategies:

FP16: halves memory, good performance on modern GPUs.
8-bit (int8) or bfloat16: common via bitsandbytes.
4-bit (QLoRA): combined with LoRA to train and serve large models on modest hardware.
Post-training quantization or quantization-aware training.

Streaming:

Use streaming APIs (OpenAI) or server push for incremental output.
Token-by-token streaming reduces perceived latency.

Safety, ethics, and governance

Privacy: avoid exposing PII in training data; use differential privacy where required.
Copyright: ensure content licensing compliance for training corpora and deployment.
Hallucinations: RAG and grounding reduce hallucinations by providing source docs.
Toxicity & bias: filter training data, apply safety layers, and maintain content policies.
Explainability: provide provenance for answers (source links) whenever possible.
Human-in-the-loop: require human review for high-stakes outputs.
Auditability: keep records of model versions, dataset snapshots, and prompt histories.
Regulatory compliance: CCPA, GDPR, and other jurisdictional laws.

Practical mitigation:

Use classifiers for harmful content moderation.
Add safety prompts that refuse certain harmful requests.
Red-team your model and maintain documented incident response.

Current landscape and trends (as of 2024–2026)

Proliferation of open-source LLMs (LLaMA family, Mistral, Falcon, phi-2, etc.).
Advances in parameter-efficient finetuning (LoRA/Adapters) and quantization (QLoRA).
Specialized LLMs for code (Codex/Codegen), reasoning (dedicated architectures), and multimodal models (text+image+audio).
Tool-augmented LLMs: models that call external tools (search, code execution).
More emphasis on retrieval + grounding to mitigate hallucinations.
Growing ecosystem of vector DBs (Pinecone, Weaviate, Milvus) and frameworks (LangChain, LlamaIndex).
Commercialization of LLMs with pay-as-you-go APIs and hosted model inference.

Future directions and open research problems

True grounded reasoning: tighter integration of symbolic reasoning, knowledge graphs, and LLMs.
Long-context models: efficient attention for documents spanning millions of tokens.
Better evaluation suites for truthfulness, reasoning, and long-term memory.
Personalization while preserving privacy: federated fine-tuning and on-device personalization.
Energy-efficient training and inference.
Robustness to adversarial inputs and distribution shift.

Case studies and practical examples

Customer support assistant:
- Fine-tune on support transcripts and use RAG with company knowledge base for updated responses.
- Safety: restrict to policy-based responses for refunds or legal advice; escalate to human when necessary.
Code assistant:
- Use code-pretrained model (CodeT5, CodeGen) fine-tuned on internal repo.
- RAG for large codebases: retrieve relevant file snippets.
- Use static analysis tools as tools invoked by the LLM.
Medical summarization (high-stakes):
- Use domain-specific fine-tuning on deidentified EHR notes.
- Always require clinician-in-the-loop and provide sources.
- Use differential privacy and HIPAA-compliant infrastructure.

Practical checklist and best practices

Start small: prototype with smaller models and then scale.
Use PEFT for practical fine-tuning on limited hardware.
Always have a retrieval and grounding strategy for factual tasks.
Instrument logging: store prompts, responses, model version, and metadata.
Evaluation: combine automatic metrics and human review.
Maintain versioned datasets and model cards (data provenance, license).
Design clear ownership/incident response for harmful outputs.
Monitor cost — quantify inference cost per request and optimize.

Example: Full RAG pipeline with LangChain (conceptual)

Ingest documents -> chunk -> embed -> index (FAISS).
At query time -> embed query -> search -> re-rank -> build context.
Call LLM with prompt template using retrieved citations.
Parse output; present generated answer plus citations and confidence.

Code snippet (LangChain-esque pseudo-code):

Python

from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.llms import HuggingFaceLLM

emb = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
doc_store = FAISS.from_documents(docs, embedding=emb)
retriever = doc_store.as_retriever(search_kwargs={"k": 5})
llm = HuggingFaceLLM(model_name="meta-llama/Llama-2-7b", temperature=0.0)
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
resp = qa.run("Tell me the steps for X and cite sources.")

References and further reading

Vaswani et al., "Attention is All You Need" (2017).
Brown et al., "Language Models are Few-Shot Learners" (GPT-3).
Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models" (2021).
Dettmers et al., "QLoRA" (2023).
RAG and related papers: Lewis et al., "Retrieval-Augmented Generation" (2020).
Hugging Face docs: Transformers, Datasets, Accelerate, PEFT examples.
SentenceTransformers, FAISS, LangChain.

Final notes

LLMs offer powerful capabilities but require careful engineering, ethical consideration, and continuous evaluation. Hands-on experimentation — from simple inference to RAG and PEFT workflows — is the fastest way to learn. Start with public models and datasets, build reproducible experiments, document datasets and model versions, and prioritize safety and interpretability when moving to production.

If you'd like, I can:

Provide a complete, runnable QLoRA fine-tuning script tailored to your GPU (specify GPU memory).
Create a step-by-step walkthrough for building a production RAG system with LangChain + Pinecone.
Generate prompt templates for specific use cases (summarization, code generation, customer support).