I'm a grad student researcher focusing on LLM reasoning and self-improvement, as well as Machine Learning in general.
Aside from completing my M.Sc. in AI/ML at PSL University (under ENS, Mines and Dauphine), I'm currently working on semi-supervised learning with INSA Lyon (DISP Lab); and on reasoning methods in small LLMs independently.
Recurrent Attention Uncertainty-Triggered Reasoning explores RAUQ-driven rollbacks that trigger compact chain-of-thought only when the model spikes in uncertainty. I am pairing the controller with QLoRA fine-tuning to lift math, code, and QA accuracy on small LLMs without bloating the token budget.
Semi-Supervised Pipeline for Etiological Diagnosis of Uveitis, building a data-efficient flow from patch-level embeddings to graph attention maps for ultra-wide-field fundus scans. The project combines MTSN contrastive training, One Vote Veto pseudo-labelling, and GACNN localisation with collaborators at INSA Lyon.
Starting with a pure python/pytorch re-implementation of GPT-2 as described in the original OpenAI paper, this project was extended to allow fine-tuning on messages to try to replicate given personalities.
This series reimplements feedforward networks in C++ and NumPy for handwriting recognition, music tagging, and more. Several optimizations (Dynamic tanH instead of LayerNorm for regularization, dropout, weight tying) improved the speed and performance of the C++ implementation to be far faster than their numpy counterparts.
A fine-tuned PatchCore pipeline that flags industrial soldering anomalies using anomalib's memory-bank k-NN scoring. I tuned thresholds and evaluation scripts so operators can catch rare defects without expanding labeling budgets.
py-scripts is a grab bag of Python utilities I rely on for automation and quick experiments. Expect everything from data wrangling helpers to small personal productivity hacks.
This is still being written...
Despite their seemingly endlessly increasing sizes, human-engineered neural-network based systems remain easy to modelize, track and "explain".
Though we can't track their every weight at every time in a perfectly deterministic way, we can shape them to the best (or at least a local best) of a loss landscape we get to define.
It's now almost undisputed by anyone with a technical understanding of LLMs that they are simply very proficient pattern-matching machines; even their finetuning can be seen as simply bending the patterns they produce.
On the other hand, we don't understand our own brains nearly as well as the networks we're producing.
We more or less know that specific areas of the brain specialize in specific tasks and responsibilities, we know that their building block is the neuron, we know there are different speeds of reasoning and types of actions but that's an entirely superior tier of ignorance to not knowing exactly which attention head attends to what.
We don't understand them because we didn't build them from scratch like we did transformers, and thus, we can't truly tell if they perform something fundamentally above extremely proficient and wide pattern matching.
Regardless of how little we know about our own brains, we can't help but observe so many similarities between the systems we build in search of human-like (or better-than-human) intelligence, and the systems that have already achieved human intelligence:
Reinforcement learning, gradient descent, and evolution are three names for the same fundamental process.
Neural nets learn behavior by watching thousands of examples, until the right connections are made between their components to functionally reproduce that pattern, through reward systems. In the same way, the best way to learn as a human is to practice, to do, literally reinforcing the neural circuits involved in that action until they've mastered the behavior.
Our bodies, which are so well crafted that many theists use them as an argument for the existence of a supernatural creator, are but the fruit of the same process once again.
At first, it's hard to conceive of the human brain as a unidirectional neural network, with a fixed number of outputs. If its inputs are fixed, surely they are of an immense number, and of a non-discrete form - the images in our eyes aren't exactly arrays of pixels, our sense of taste is hard to express as the span of a finite basis, same goes for our proprioception, hearing, sense of touch, etc, doesn't it?
And yet, the brain receives external information in discrete spikes along finite nerves, predetermined and of a given throughput. What about the brain's "outputs"? Intuition would tempt us to say that it is more than simply motor commands to the body (again, transmitted by specific nervous channels), to treat thoughts as products in their own right. But unless a thought is transduced into something more than neural activity, is there any route other than motor control by which it becomes real? (Sure, you could mention brain-computer interfaces, but that's the same as sampling inner activations — we do that with LLMs too.)
But not all apparent differences can be overcome. Transformers and human brains radically differ in self-modification. This is probably what allows us to learn complex workflows, methodologies and behaviors in no more than a few exposures, when neural nets need thousands, millions or billions of examples.
Where transformers are fundamentally discrete, operating one token at a time, one iteration of attention before the other, the brain is time-continuous in its operation, and orchestrates itself in a web of nuance, parallel neural circuits and hierarchical speeds that is hard to rival.
The century or so that has occurred since the discovery of the perceptron is nowhere near enough to truly rival 4 billion years of iteration and refinement. And yet — multi-scale models are beginning to see the day. The hierarchical reasoning model, coupling two LLMs of different scales to model thinking fast and slow, has been impressive on its performance (albeit on very restricted and specific tasks). As we work on size and computing efficiency, where we currently lag so far behind the human brain it can seem inconceivable to ever catch up, maybe all of the fundamentally "different" and "superior" abilities we take pride in will manifest themselves in intelligent, artificial systems, one by one.
This is still being written...