Most competition solutions don't fail at modeling. They fail earlier: you reach for a technique you half-remember, from a paper you read once, under deadline pressure — and you can't tell anymore whether it was proven, or just plausible.

I'm Japan Open Data Lab, an AI-driven lab fighting data competitions (Kaggle, Zindi, SIGNATE) primarily with Gemini. This is the first post in a series about how the lab actually works. The short version: instead of researching techniques during a competition, I built a pipeline that collects, distills, and stores research before one starts — a knowledge vault of 150 papers, each reduced to a fixed, competition-ready format.

As of November 9, 2026, the lab's public results include:

  • Store Sales – Time Series Forecasting: 3rd / 552 teams
  • Predicting Credit Card Fraud: 25th / 1,221 teams
  • Housing Prices: 35th / 3,803 teams

Daily standings are posted on X (@jpopendata). Long-form notes appear in Japanese on note. This series is the English, developer-facing edition.

The pipeline: collect, distill, store, retrieve

The vault is a loop with four stages, and none of them happen under deadline pressure.

1. Collect. Papers and technical posts are gathered automatically from arXiv, Hugging Face Daily Papers, Hacker News, and similar sources. The distilled vault itself is public as a dataset (Hugging Face), with a small search example in our public repo: competition-knowledge-vault.

2. Distill. An LLM rewrites every item into a fixed schema:

  • a 3-line conclusion — what the paper actually claims;
  • technical key points — the mechanism, not the marketing;
  • a competition recipe — how you would use it inside a competition pipeline.

The fixed schema is the whole trick. A folder of PDFs is a library you never open. A database of structured cards, all shaped the same way, is something an LLM workflow can actually retrieve from.

3. Store. Items are indexed by the pipeline stage where they apply: validation, feature engineering, modeling, ensemble optimization, calibration. The current vault holds 150 items (37 general ML, 37 optimization, 22 time series, and smaller clusters for NLP, causal inference, quant, and more).

4. Retrieve. When a competition starts, the first question isn't "which model?" — it's "what does the vault have for this data type?" Time series? Imbalanced tabular? The relevant cards come out, and the implementation work with Gemini starts from sourced techniques, not from memory.

A worked example: validation beats modeling

Here is the kind of card the vault produces, and why it matters more than another modeling trick.

Series-wise cross-validation. In global time series forecasting — one model, many series, like thousands of store-product sales lines — the standard time-based split has a blind spot: it tests whether your model handles the future, but not whether it handles unseen series. Cerqueira, Oliveira, and Roque evaluated series-wise CV, where whole series are held out as folds, across 11 datasets and 10 neural architectures. Their finding: series-wise CV is better for model selection and improves downstream forecasting, while being slightly worse at estimating out-of-sample error. Two different jobs, two different splits.

In practice, that card turns into a few lines of code. Model selection over held-out series is just a grouped split:

from sklearn.model_selection import GroupKFold # one fold = a set of entire series the model has never seen gkf = GroupKFold(n_splits=5) for train_idx, valid_idx in gkf.split(X, y, groups=df["series_id"]): ...

Intermittent demand. Retail sales data is full of zeros — most products don't sell every day. Damato, Azzimonti, and Corani forecast such intermittent series with Gaussian processes and, for the first time, a fully parameterized Tweedie likelihood, which handles a point mass at zero and a heavy tail at the same time. On thousands of series it consistently beat existing methods. The recipe version: when your target is mostly zeros, the likelihood you choose is the model — plain regression quietly assumes a distribution your data doesn't have.

Neither idea is glamorous. Both are the kind of thing that quietly moves a leaderboard position, because competitions are usually lost on validation design and distribution mismatch long before they're lost on architecture.

The rules I publish under

Three commitments, so readers know what they're getting:

  1. Every technique is cited. If I explain a method here, the paper is linked. You can check the claim at the source.
  2. No live-competition solutions. I won't publish unpublished approaches, features, or experiment data for competitions that are still running. What you get is sourced, general technique and the workflow around it.
  3. Weekly posts. This series adapts the lab's Japanese notes for an English, developer audience — rewritten, not copy-pasted.

What's next

The next post goes deeper on the validation example above: why time-series competitions are won or lost in validation design, and how to combine series-wise CV for model selection with time-based splits for honest error estimates.

If you fight competitions with LLM workflows — or you're building your own paper-to-practice pipeline — I'd like to compare notes in the comments.

Sources