I am an MSc student in Integrated Machine Learning Systems at University College London, supervised by Prof. Miguel Rodrigues. My research interest is in AI systems that can take progressively greater responsibility for their own improvement. I am interested in how models can choose useful experience, assess its reliability, and use it to guide further learning with less continuous human supervision.
My recent projects span executable reasoning, post training, multimodal retrieval, and GUI grounding. Together, they have shaped my interest in agents that can reason, act, and adapt across structured, visual, and interactive environments while keeping learning and decision making auditable and reliable. Before moving into machine learning, I completed a BEng at Nanjing University, where I worked with Prof. Jingbo Wu on programmable terahertz devices.
Reliable AI Agents · Verifiable Reasoning · Multimodal Grounding · Learning from Verified Experience
Built an executable QA system over 166,277 UK procurement releases. I created a provenance-preserving knowledge graph and a 12,828-question program-first benchmark, then trained local 8B planners only on programs that passed deterministic execution and oracle checks. The best system reaches 85.65% on the sealed test, compared with 69.76% for the cloud teacher, and the same execution framework transfers to WikiTableQuestions.
Multimodal Document Reranking with Vision-Language Models
Jan 2026 — Mar 2026
Research Project · UCL
Built a multimodal reranker for RAG using ColQwen2 for candidate retrieval and Qwen2.5-VL-7B for second-stage ranking over text, tables, figures, and layout. I curated 7.2K SFT examples and 3K GRPO samples. MMDocIR Macro Recall@1 improves from 64.81% to 69.51% (+4.70 pp), above a Qwen2.5-VL-32B CoT reranker.
GUI Element Localisation with Vision-Language Models
Sep 2025 — Nov 2025
Course Research Project · UCL
Built a DOM-independent GUI grounding model that maps screenshots and natural-language instructions to click coordinates across desktop, web, and mobile interfaces. I trained Qwen2.5-VL-3B on ~24K screenshots with resolution-aware coordinate alignment. It reaches 84.9% on ScreenSpot and 86.4% on ScreenSpot-v2, improving the base model by 29.4 pp.
Physics-Constrained Vision Models for Pasture Biomass under Geographic Domain Shift
Apr 2026 — Jun 2026
Applied Machine Learning Systems II · UCL · 85/100
Built a physics-constrained regression model whose outputs always satisfy biomass decomposition identities. I compared CNN and ViT baselines under limited data and tested geographic transfer. ViT-Small reaches 0.556 ± 0.023 weighted R² IID but −0.005 on held-out Tasmania; a metadata-only Ridge baseline reaches +0.251.
KG-GNN-DPO: Knowledge Graph-Grounded Preference Optimisation for Multi-Hop QA
Apr 2026 — Jun 2026
Deep Learning for NLP · UCL · 85/100
Built 387 preference pairs without human annotation by combining R-GCN structural scores with DeBERTa NLI scores, then used them for DPO on Qwen2-0.5B. No knowledge graph is used at inference. On MetaQA 3-hop, the model reaches 5.27% EM and 44.90% KG path consistency, a 25.5% relative EM gain over SFT.
06 · DEPLOYAuditable answerEvidence or typed abstention
Self-improvement loop: once a local student exists, its own candidate plans return to the same verification gate. Accepted survivors become new supervision, so continued teacher access is not required to generate training signals.
Problem and design
Many procurement questions require exact set construction, multi-hop joins and exhaustive aggregation. Top-k retrieval can surface convincing records without recovering the full population a count, sum or ranking needs; a free-form LLM can also emit a program that executes cleanly while answering the wrong question. I therefore separate language from factual computation: the model proposes a typed computation, while deterministic components define the data semantics, ground the plan, execute it exhaustively and return the answer with provenance.
The same execution contract also becomes a training-data engine. Teacher and student models may propose candidates, but neither is trusted as the source of truth: only candidates that survive structural checks, grounding, execution and oracle comparison are allowed to become supervision.
85.65%Local 8B · sealed test
+15.89 ppOver cloud teacher
78.31%PACS challenge
51.80%WikiTableQuestions
99.88%Oracle agreement
01 · Executable data layer
Raw records are not query-ready. The 166,277 OCDS releases contain fragmented organisation identities, repeated monetary values across procurement stages, missing fields and role-sensitive relations. A precision-first normalisation pipeline resolves identifiers, preserves stored names and provenance, records value source and additivity, and materialises a typed buyer–supplier–contract–category graph that becomes the executor's canonical world state.
This data contract is part of the reasoning system rather than preprocessing hidden from it: deduplication rules determine what a count means, additivity guards determine which values may be summed, and unresolved evidence can lead to a typed ambiguous, unsupported or no-result outcome instead of a guess.
02 · Program-first benchmark
Instead of asking an LLM to invent both questions and answers, I instantiate legal programs in a closed algebra of 17 typed operators, execute them first, and only then verbalise the computation into a question. The resulting 12,828-question benchmark retains each gold program, answer contract, grounded entities and relations, executor output, evidence, provenance, answerability status and composition metadata.
A separately implemented evaluator reconstructs answers without sharing execution code and agrees with the benchmark oracle on 99.88% of 14,770 audited cases. This keeps the gold signal outside the model and makes it possible to distinguish an executable plan from a semantically correct one.
03 · Typed planning and deterministic execution
A reader first turns the question into a structured briefing; a planner composes a typed program under grammar-constrained decoding; grounding resolves fields and entities; the executor runs the computation exhaustively with deduplication, exact arithmetic and provenance; and a runtime verifier checks uniqueness, additivity and coverage before an answer is released. Repair is narrow and triggered only by typed, diagnosable failures.
This scaffold changes the task even before learning: on development, retrieval reaches 31.5%, while an untrained local 8B model routed through the typed interface and executor reaches 70.4%. The data-access ablation explains why: retrieval can answer many single-record questions but collapses below 4% on exhaustive aggregation because top-k evidence does not enumerate the complete denotation.
04 · Verification as training-data filtering
Teacher distillation is useful but noisy: a plausible trace can hallucinate an entity, omit a dependency or execute the wrong semantics. My acceptance rule therefore treats model output as a proposal rather than truth. A candidate enters the training pool only if it has a valid typed structure, grounds to the question and data, executes successfully and reproduces the audited oracle.
The candidate source can then be switched from teacher to student without changing the verifier. Rejection-sampling/self-harvest reruns the same loop on student-generated plans and trains on the verified survivors, so the system can continue producing task-specific supervision without continued cloud-teacher access. This is the core self-improvement mechanism: generate → verify → learn → generate again.
05 · Sealed evaluation
The strongest fully local Qwen3-8B system reaches 85.65% on the sealed 2,285-question procurement test, compared with 69.76% for the cloud teacher that supplied training candidates. On the generator-independent PACS challenge, the frozen local planner reaches 78.31% versus 50.33% for the teacher.
The interpretation is deliberately narrow: the 8B model is not a more capable foundation model in general. Rather, deterministic filtering removes teacher failures and converts a stronger but noisy proposer into cleaner task-specific supervision, allowing the student to become more accurate on the executable task it is trained to solve.
06 · Transfer to WikiTableQuestions
To test portability, I keep the typed execution architecture and replace the procurement adapter with table views and a column-aware linker for WikiTableQuestions. The untrained local base scores 22.51%; the procurement checkpoint transfers zero-shot to 27.33%; answer-only training reaches 44.43%; and gold-program training reaches 51.80% on the official test.
A differential audit against reference SQL programs shows that the remaining limit is largely representation and program-language coverage, not executor correctness: after compiler closure, 39.6% of reference programs still lie outside the language. The transfer therefore demonstrates a reusable architecture while making its coverage ceiling explicit.
Summary
This project combines a typed execution layer, a program-first benchmark, and deterministic verification. The same verifier checks answers at inference time and filters training traces, so the local planner can learn from generated programs without treating model output as ground truth.
BibTeX
@mastersthesis{zhou2026auditable,
title = {Auditable Compositional Question Answering over UK Public Procurement},
author = {Zhou, Hai},
school = {University College London},
year = {2026},
note = {Manuscript in preparation}
}
Multimodal Document Reranking with Vision-Language Models
First-stage retrievers often retrieve the right region of a document but fail to place the most useful page at Rank 1, especially when relevance depends on a table, chart, figure, or page layout rather than plain text. I built a retriever-agnostic multimodal reranker that takes the candidate pages returned by a retriever, explicitly reasons about each page–query pair, and produces a refined ranking for downstream RAG. The system combines quality-controlled reasoning supervision with a two-stage SFT–GRPO training pipeline around Qwen2.5-VL-7B, improving MMDocIR Macro Recall@1 from 64.81% to 69.51%.
69.51%Macro Recall@1
+4.70 ppOver retriever
7,200SFT traces
3,000RL samples
01 · Reranking after retrieval
Problem. A retriever is optimized to find plausible candidates efficiently, not to perform expensive page-level reasoning. In multimodal documents, high lexical or visual similarity can therefore push a wrong page above the page that actually contains the answer.
Design. I kept retrieval and reasoning separate. The first stage proposes a small candidate set; the second-stage VLM reranker spends more computation on those candidates, explains their relevance, and reorders them before they are passed to RAG. This makes the reranker plug-and-play: the downstream component does not depend on one specific retriever.
02 · Candidate retrieval on MMDocIR
The training source is MMDocIR, a large-scale document retrieval benchmark built for complex visual documents. Its training split contains 6,878 documents and 73,843 question–answer pairs aggregated from seven document-VQA datasets. The evaluation set contains 313 long documents averaging roughly 65 pages, 1,658 expert-written queries, and ten domains including academic, legal, and financial material.
Candidate pages are retrieved with ColQwen2, a late-interaction visual document retriever. Instead of compressing a page into one vector, ColQwen2 keeps multiple query/page representations and aggregates fine-grained similarities, producing a relevance score for every candidate page. I use those scores both to form the reranking set and to mine hard negatives: pages scored highly by the retriever but marked irrelevant by the ground truth.
03 · Reasoning supervision
Why not reason over all pages at once? Direct multi-image prompting degraded supervision quality: the VLM tended to focus on the most obvious positive page and under-analyse the other candidates. That creates shortcut rationales that may predict the correct winner without learning why the hard negatives are wrong.
Single-page reasoning. Each candidate page is therefore judged independently against the query. This forces the teacher to produce explicit evidence for every positive and negative candidate instead of relying on cross-image shortcuts.
Reasoning refinement. Independent analyses are much cleaner but also repetitive—roughly one paragraph per page. The page-level rationales are concatenated and compressed into a concise chain of thought that preserves the decisive positive evidence and the reason the remaining candidates are not relevant. This pipeline yields 7,200 high-quality SFT examples rather than simply treating model-generated text as supervision.
04 · Supervised fine-tuning
SFT uses a conversational training format with a system instruction, a user prompt containing the query and candidate pages, and an assistant response containing explicit reasoning followed by the predicted ranking. Qwen2.5-VL-7B is adapted with LoRA so that the model learns the reranking task and output pattern without updating the full base model.
At this stage the goal is behavioural acquisition: teach the VLM to inspect each page, articulate the discriminative evidence, and produce a complete ranking in a stable format. SFT gives the model this reasoning policy, but it does not directly optimize the Top-1 ranking metric used at evaluation.
05 · Resolution-balanced RL sampling
The RL stage starts from the SFT checkpoint, but naive random sampling is unstable because document pages have highly heterogeneous resolutions. High-resolution pages create many more visual tokens and consume disproportionately more compute and memory, while low-resolution documents can become under-represented.
I sort samples by average image resolution, divide them into ten resolution buckets, and sample proportionally across all buckets. The resulting 3,000-example RL set keeps different resolution regimes in the optimization loop and reduces batch-to-batch variance caused by visual-token imbalance.
06 · Reward design
Result Reward. The reranker should place relevant pages as early as possible, with Rank 1 carrying much more value than later positions. A relevant page at rank r contributes 1 / r³, normalized by the ideal ordering of all relevant pages. The first position contributes 1.0, while the second contributes only 0.125, making the reward strongly aligned with Top-1 performance.
Format Reward. Ranking quality alone is not enough if the output cannot be parsed. The response must first satisfy the hard <think>...</think><answer>[...]</answer> structure. If that structure is valid, soft length and ID-range terms reward producing exactly one legal ranking entry per candidate. The final RL signal combines ranking correctness with output validity.
07 · GRPO and distributed training
GRPO samples multiple candidate completions for the same prompt, scores each completion, and normalizes rewards within the group to obtain a relative advantage. This is useful for reranking because query difficulty varies substantially: an absolute reward that looks low for a hard query may still be the best completion in its group. Group-relative comparison supplies an adaptive baseline without training a separate critic network.
The training pipeline uses LoRA and mixed precision across 4× NVIDIA A100 GPUs. SFT uses a smaller adaptation capacity to learn task structure and reasoning format; RL uses a larger LoRA capacity so policy optimization can make stronger changes to ranking preferences. Avoiding a separate critic also matters operationally for a 7B vision-language model, where image tokens already make memory a major constraint.
08 · Evaluation
At inference time, the retriever first returns a small ordered set of page images. The reranker receives the query and those candidates, generates a compact relevance rationale, then outputs a new page ordering. Evaluation uses Recall@K, with Macro Recall@K computed independently for each query and then averaged so that queries with different numbers of relevant pages contribute equally.
On 1,658 MMDocIR evaluation queries, ColQwen2 retrieval alone reaches 64.81% Macro Recall@1. The final SFT+GRPO reranker reaches 69.51%, a +4.70 percentage-point improvement, while Macro Recall@5 rises from 87.66% to 88.42%. The final 7B model also exceeds a Qwen2.5-VL-32B CoT reranking baseline, showing that the training and supervision pipeline matters more than model scale alone.
What the ablation shows
SFT and RL are complementary rather than interchangeable. SFT teaches the model how to perform explicit page-level relevance reasoning; RL then aligns those behaviours with the ranking objective. The combined system produces the strongest result.
64.79%CoT baseline
66.73%SFT only
65.86%RL only
69.51%SFT + GRPO
Summary
The project shows that a compact VLM can improve first-stage visual retrieval when the training data and ranking objective are designed for page-level relevance. The reranker sits between retrieval and generation, so the first-stage retriever does not need to be replaced.
Physics-Constrained Vision Models for Pasture Biomass Prediction under Geographic Domain Shift
Hai Zhou
Course Research · Apr — Jun 2026
Applied Machine Learning Systems II · University College London · 85/100
Pasture biomass is a small-data multimodal regression problem in which five targets are physically coupled and visual appearance changes sharply across geography. I designed a model family that compares CNN and ViT inductive biases while enforcing biomass decomposition identities directly in the output head. The study combines physics-constrained prediction, zero-inflated clover modelling, metadata fusion, and a deliberately hard geographic OOD evaluation with Tasmania held out.
0.556 ± 0.023ViT · IID weighted R²
−0.005ViT · Tasmania OOD
+0.251Metadata-only Ridge · OOD
357Field images
01 · Physics-constrained output parameterisation
A free five-output regressor can produce physically impossible predictions such as Total biomass below one of its components. The constrained head predicts only Green, Dead, and Clover, then derives GDM = Green + Clover and Total = GDM + Dead algebraically. Physical consistency is therefore exact at inference, with no post-hoc correction or tunable penalty.
02 · Architecture and sparse-target ablations
Eight controlled variants compare ResNet-50, EfficientNet-B0, and ViT-Small under a shared training protocol, then isolate multi-task learning, metadata fusion, and a zero-inflated clover gate. ViT-Small is the clear IID winner at 0.556 ± 0.023 weighted R², while the gate improves clover modelling in some seeds but remains unstable under the small sample size.
03 · Geographic OOD evaluation
Holding out all Tasmanian images produces a much harsher test than a random split. Every neural image model falls to negative or near-zero R²; the strongest IID ViT drops from 0.556 to −0.005, while a metadata-only Ridge baseline reaches +0.251. The negative result is the main finding: in-situ NDVI and sward height transfer across geography more reliably than visual features learned from the limited mainland imagery.
Summary
The physics constraint guarantees valid output relationships, but it does not solve geographic shift. The held-out Tasmania results show that strong IID performance can still depend on location-specific visual cues.
KG-GNN-DPO: Knowledge Graph-Grounded Preference Optimisation for Multi-Hop Question Answering
Hai Zhou
Course Research · Apr — Jun 2026
Deep Learning for NLP · University College London · 85/100
This project asks whether knowledge-graph structure can shape a language model during training without requiring graph retrieval at deployment. A two-layer R-GCN scores structural reachability, a DeBERTa NLI cross-encoder scores semantic consistency, and their fused signal constructs 387 human-annotation-free preference pairs for Qwen2-0.5B. On MetaQA 3-hop, KG-GNN-DPO achieves the best result among five controlled systems on exact match (5.27%), token F1 (5.91%), and KG path consistency (44.90%), while also exposing a 44.90% structural-consistency vs. 3.93% NLI-faithfulness gap.
Approach
Candidate answers are scored twice — structurally, by an R-GCN over the graph neighbourhood, and textually, by an NLI entailment model against retrieved evidence. Disagreements are discarded rather than averaged, which keeps the resulting pairs clean enough for reference-free DPO on a 0.5B model.
Finding
An answer can be textually entailed while remaining structurally implausible, and vice versa. A single grounding score hides this failure mode, so reporting both is necessary when claiming that a multi-hop answer is grounded.
GUI Element Localisation with Vision-Language Models
Traditional GUI automation relies on DOM trees, XPath/CSS selectors, or recorded coordinates, which become brittle under dynamic layouts and across desktop, web, and mobile platforms. I built a DOM-independent visual grounding system that takes a screenshot and a natural-language instruction such as “click the search button” and directly predicts a clickable coordinate. The project combines cross-platform data design, multi-turn grounding supervision, resolution-aware coordinate alignment, and distributed parameter-efficient training around Qwen2.5-VL-3B. The final model reaches 84.9% on ScreenSpot and 86.4% on ScreenSpot-v2.
84.9%ScreenSpot
86.4%ScreenSpot-v2
+29.4 ppOver base 3B
30×Max turns / image
01 · Visual grounding without DOM selectors
Problem. XPath and CSS selectors work when a stable DOM is available, but they are expensive to maintain when layouts change and do not naturally transfer to native desktop or mobile interfaces. Recorded pixel coordinates are even more fragile because they break as soon as window size or resolution changes.
Design. I reformulated element localisation as multimodal grounding: the model sees only the rendered interface plus a natural-language target description and outputs the point that should be clicked. This makes the same interface contract usable across desktop applications, web pages, and mobile apps without requiring DOM access at inference.
02 · Cross-platform data
The training mixture contains roughly 24K UI screenshots drawn from four sources and is built around diversity rather than a single dominant platform. It covers Windows and macOS desktops, browser interfaces, Android and iOS apps, text and icon controls, predominantly English plus some Chinese interfaces, and resolutions from roughly 720p to 4K.
I used a balanced multi-source mixture so that additional data from one platform would not overwhelm the others. Each dataset follows the same image/metadata directory contract and is registered through a configuration-to-directory mapping, allowing new sources to be added without rewriting the core loader. Parallel data loading keeps preprocessing from becoming the training bottleneck.
03 · Multi-turn grounding supervision
A conventional pipeline creates one training example per screenshot–element pair, repeatedly encoding the same image even though a single interface can contain dozens of clickable controls. I instead sample up to 30 annotated elements from a screenshot and organise them into a multi-turn dialogue. The first turn provides the image and full task prompt; later turns provide only a new target description while reusing the visual context.
This raises the potential supervision density from one to as many as 30 element-level interactions per screenshot—up to about 720K grounding interactions from a 24K-image pool—while also resembling real GUI use, where several actions are usually taken on the same page. During causal-LM training, system/user tokens are masked with the loss ignore index and only assistant coordinate responses contribute to the objective.
04 · Prompt variation
Training randomly samples from 15 semantically equivalent system prompts that define the grounding task and coordinate output format. This prevents the model from overfitting to one fixed instruction template and encourages it to follow the target semantics instead of memorising phrasing.
Evaluation uses a deterministic prompt so model comparisons are reproducible and consistent with the benchmark protocol. Integer point coordinates are used as the default generated representation because they are compact, directly executable, and empirically easier for an autoregressive language model to predict reliably than long floating-point values.
05 · Resolution-aware coordinate alignment
Qwen2.5-VL does not necessarily encode a screenshot at its original pixel dimensions. Its visual preprocessing dynamically resizes the image under a visual-token budget and uses dimensions compatible with a 28-pixel factor. That creates two coordinate spaces: the original screen and the resized image actually seen by the model.
During training, ground-truth points are mapped from original-image space into the resized model space before becoming labels. At inference, generated points are transformed in the opposite direction using the actual resize ratios and then clipped to the original screen boundary. This symmetric original → model → original mapping prevents systematic click drift across heterogeneous resolutions.
06 · VLM adaptation
Qwen2.5-VL-3B-Instruct is adapted with parameter-efficient fine-tuning rather than full-model updates. The vision encoder remains frozen while task adaptation is concentrated in selected language-transformer modules, preserving the pretrained multimodal representation while keeping the trainable parameter budget small.
Training is split into two complementary stages. Stage 1 mixes all platforms to establish general GUI understanding and grounding behaviour. Stage 2 uses a smaller learning rate, larger effective batch, and more passes over a targeted subset to improve difficult high-resolution cases without discarding the broad capability learned in Stage 1.
07 · Distributed training
The VLM is trained with DeepSpeed ZeRO-2 and BF16 mixed precision, partitioning optimizer state across workers while reducing memory and compute overhead for multimodal batches. The data pipeline, model processor, chat template, coordinate conversion, and distributed trainer are kept as separate components so dataset or model changes do not require rewriting the entire stack.
Deployment mirrors evaluation: load the base model plus the learned adapter, switch to evaluation mode, run autoregressive generation under no_grad, parse the generated coordinate, invert the resize transform, validate the screen boundary, and hand the final pixel point to the GUI executor.
08 · Click-point evaluation
The model generates a point autoregressively rather than predicting a detection box. A prediction is counted as correct when the transformed click point lies inside the annotated target rectangle: x₁ ≤ x̂ ≤ x₂ and y₁ ≤ ŷ ≤ y₂. This point-in-box metric matches GUI interaction more directly than IoU because a click succeeds anywhere inside the actionable element.
ScreenSpot reports six platform/type slices: Mobile, Desktop, and Web crossed with Text and Icon elements. The model reaches 84.9% overall on ScreenSpot and 86.4% on ScreenSpot-v2. Against the untuned Qwen2.5-VL-3B base model at 55.5%, this is a +29.4 percentage-point gain on ScreenSpot. It also exceeds the strong UI-TARS-2B baseline by +2.6 pp on ScreenSpot and +1.7 pp on ScreenSpot-v2.
Where the model is strongest
Text-labelled controls are consistently easier than icon-only targets because the model can combine visual layout with OCR and language matching. Mobile interfaces also perform strongly, plausibly because touch targets and layout conventions are often more regular than dense desktop toolbars or browser chrome.
96.3%Mobile text
93.8%Desktop text
89.5%Web text
81.6%Mobile icon
74.2%Desktop icon
74.2%Web icon
Summary
The model grounds natural-language GUI instructions directly to click points without DOM or accessibility-tree input. Most of the gain comes from cross-platform data, dense multi-turn supervision, and keeping coordinate labels consistent with the VLM’s image resizing.