projects
GAVEL — Graph-Augmented Paragraph Retrieval for Legal Judgments
Extends LexCLiPR with dual Graph Attention Networks (local 5-hop adjacency + global BERTopic topic nodes) trained under contrastive loss, outperforming a fully fine-tuned bi-encoder with a frozen mDPR encoder.
pythonpytorchPyGGATcontrastive-learningmDPR
SmoLLM — 109M Parameter Language Model from Scratch
A Llama-style decoder-only transformer built from scratch — custom BPE tokenizer, RoPE, RMSNorm, SwiGLU — pretrained on FineWeb-Edu, then instruction-tuned on databricks-dolly-15k.
pythonpytorchtransformersRoPESwiGLUBPE