This article explains how I built a Git repository of Chile's entire legal code: more than 300,000 laws, decrees and resolutions in a corpus that reaches back to 1810, with one commit per legislative publication, so the text of any law can be read as it stood on any date. It covers scraping the Biblioteca del Congreso Nacional (BCN) and LeyChile, building the history with git fast-import, the limits of Git and GitHub that the repository hit, and the web UI, API and MCP server built on top of it. The repository is pisanvs/ley-chile, and you can browse it at leyes.pisanvs.cl.

Why Version Control for Legislation?

I think most programmers would agree that working—in a team—without a version control system like Git, is as horrifying as ripping your nails out with bare hands. Git has become a ubiquitous tool in the software industry, not because it’s amazing, but because it’s undoubtedly necessary.

As coders and hackers, we tend to believe in the delusion that we can fix others’ problems by working our fingers against the keyboard. This is partly (mostly) the reason why one of my obsessions is making other fields of work more efficient.

One of these fields of work is law. You see, the lawyers I know religiously use either Google Docs’ or MS Word’s implementations of version control systems. This doesn’t look like a bad idea or use-case at first glance, but upon further examination we can see the shortcomings of this approach.

MS Word

Git

Chronological History

✅

✅

Author attribution

🚧 (“blame” doesn’t exist, you must rely on version history)

✅

Branching

❌ (need to create a separate file)

✅

Multi-file tracking

❌

✅

Intentional & Directed commits

❌ “versions” are just point-in-time objects

✅ Can stage specific hunks, write commit messages, explain changes.

Selective historical undo

❌ Can only revert to a specific version

✅

Cherry-picking

❌

✅

Now, we programmers use these features regularly, but other fields of work don’t. They don’t have the tools and they have to find increasingly complex workarounds.

A good place to start the Git crusade, in my opinion, is the law.

Existing Git Repositories of Laws: US Code and Legalize

There are some projects around the world that have tried doing this to varying levels of success.

nickvido/us-code

US Code repository on GitHub

nickvido’s us-code repository on GitHub has one substantial shortcoming: it only has 16 commits.

How many bills have been passed to modify laws in the US Code? Thousands. At least a few hundred per two-year congressional term. But this doesn’t check out. Why? Because this repo is built from the point-in-time snapshots from the Library of Congress, instead of having its history built per bill.

In my humble opinion, although this repo is accurate, it does not provide a value-add over what the Library of Congress snapshots already provide.

Legalize.dev

Legalize.dev Chilean law repository

Legalize is probably the most complete solution that has scaled this up to multiple countries. But sadly, their Chilean repo is a bit behind: it only has 3k laws and reforms in it, compared to the more than 300k norms in the complete Chilean corpus.

I really appreciate the work Legalize has done. If I had been aware of their existence before I started this project, I probably would’ve made a fork instead of starting from scratch.

Apart from being mostly unaware of other solutions, the reason I built this project this way was because I had some very particular requirements and constraints. No good engineering challenge has few constraints, after all!

Some of these requirements are considered (rightfully so) a bit outrageous by some friends of mine. I would agree for the most part!

  1. It must fully live within Git
  2. The code should be as simple and compact as possible
  3. Maintainers need not worry about updates to the law

The main purpose of these requirements is to have an easily clone-able, replicable and iterable source for the data and the scraper/generator alike. This enables some benefits of distribution and accessibility, which the traditional sources don’t have.

A fourth bonus requirement was to have the pipeline run alongside the repo. The default option for this was of course GitHub Actions. Having a self-contained GitHub repository for the whole “ecosystem” makes it easy for anyone to clone, modify, and distribute as needed.

Now, having this pipeline run completely within GitHub actions generates its own set of weird constraints. Contrary to popular belief, it has actually brought on some benefits, especially related to the fact that public facing IPs on each job tend to be distinct, and thus lets you circumvent some rate limiters by spreading your work across IPs.

Lucky for the Chilean folk, our National Library of Congress keeps in-depth, updated, accurate and densely linked data on the legal history of the country. They’ve got laws spanning from the country’s first provisional government in 1810 to the present day. The founding act of the First Government Junta is dated 18 September 1810, the day Chile now celebrates as its national day. The Acta de la Instalación de la Primera Junta de Gobierno is one example from the beginning of the corpus.

Another early entry shows how broad the words “legal code” are in the Library. They file “Medidas contra la sífilis” as a decree, but it’s really a letter from the government to a marquis, dated 13 January 1813. Providence, they say, put the cure to syphilis in the hot springs on the marquis’ estate, and the government would like the sick to be able to reach them. So it asks him what road work is needed between Rancagua and the baths, and how it could be paid for. It is an early example of a government text with practical force, even though it might look like it’s very informal.

The Chilean Library of Congress, BCN from now on, exposes multiple APIs, to the point where it’s not possible to have a single source that trumps the rest.

The first two APIs are used and each serves a specific purpose in this project. The only API that is not being used is the API-key-restricted one.

datos.bcn.cl, the SPARQL endpoint, is the fastest way to get the list of everything that exists on the server. With one query we can get every norm's leychileCode, type and publication date. Unfortunately, the server times out on deep OFFSETs, so the script pages with a keyset cursor asking for 500 sequential codes at a time.

curl https://datos.bcn.cl/sparql \ --data-urlencode 'format=application/json' \ --data-urlencode 'query= PREFIX bcn: <http://datos.bcn.cl/ontologies/bcn-norms#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT ?code ?tipo ?date WHERE { ?s bcn:leychileCode ?code . FILTER(xsd:integer(?code) > 0) OPTIONAL { ?s bcn:type ?t . BIND(STR(?t) AS ?tipo) } OPTIONAL { ?s bcn:publishDate ?date } } ORDER BY xsd:integer(?code) LIMIT 500'

This query is from our first stage of the process, which is building the catalog.

nuevo.leychile.cl, the norm JSON, gives me all data corresponding to one norm, and the text of any version of it. The same endpoint serves both the metadata and the versioned text. The difference is one parameter:

# metadata and current text of a norm curl 'https://nuevo.leychile.cl/servicios/Navegar/get_norma_json?idNorma=<ID>' # the same norm as it read on a given date curl 'https://nuevo.leychile.cl/servicios/Navegar/get_norma_json?idNorma=<ID>&idVersion=2009-04-01&agrupa_partes=1'

The response is an HTML-like tree with law fragments plus a list of the dated versions the norm has had. Inside the HTML are callouts with links to the norms that modified it, and that's where I get the edges of the graph (more on that in the next section).

The third API, bcn.cl/leychile/api/v1, I never used. It needs an API key, and the quota is 500 requests per day per key. At 358,000 norms, one pass would take about two years of quota, before counting any versions. This is a separate issue, but when scraping doesn’t affect in a negative way the infrastructure of a service, and the second best way to get the data is actively harmful for performance or results, it’s probably a symptom of a system that isn’t really open, independent of how much the orgs publicize the “open data” aspect of it.

Scraping Chilean Laws from LeyChile and the BCN from GitHub Actions

Ingesting all of this data is an interesting task. It is weirdly shaped, not always complete, or accurate, or standardized.

The scraper is, in essence, four idempotent scripts that run in order.

  1. build_catalog.py pages through SPARQL and writes catalog.json, the list of every norm ID, its type and its publication date.
  2. fetch_normas.py downloads each norm's metadata into graph.json. For every norm it pulls the links out of the HTML tree, and each link to another norm becomes an edge: "this norm was modified by that one". The result is the amendment graph of the whole legal system.
  3. fetch_versions.py downloads every dated version of every norm and computes the diffs between them.
  4. build_history.py turns the cache into Git commits.

A note on counts, since they appear throughout: in the main branch’s builder, the catalog contained about 358,000 norms. Roughly 19,000 lacked a usable publication date or had only sentinel dates, leaving about 339,000 that this builder could place in history. Those are pipeline figures, not a count of what the deployed site can show; its Railway deployment uses a different path. The search index in a later section was loaded when it stood at 333,026 norms.

SPARQL ─► catalog.json ─► get_norma_json ─► graph.json (amendment edges) │ ▼ get_norma_json?idVersion=… ─► cache/versions + cache/diffs │ ▼ git fast-import stream ─► branch historial

Now, trying to scrape data means that it’s probably going to fight back the whole way. Some of these examples are the following.

  • The endpoint is behind a flaky firewall: My first runs on GitHub Actions came back with Expecting value: line 1 column 1. I assumed those were dead IDs. They weren't. From my home connection, every URL in the failure log returned valid JSON. From a GitHub runner (Azure IPs), the WAF answered with HTTP 200 and a 264-byte block page. The fix was to send a full Chrome-style header set.
  • A missing norm sometimes also returns 200 with an empty body, and a few return 500 with a “not found” message. These looked identical to WAF blocks, so dead IDs were being retried forever and throttling the whole crawl. Now both are marked permanent after one failure and don't count as a load signal.
  • My rate limiter deadlocked. A run with 16,629 tasks went silent for two hours right after the first resize from 3 to 4 parallel slots. The fix was to change the live semaphore instead of replacing it.
  • 358,000 tasks at once is a bad idea for GitHub actions (lol) Implementing parallel task scheduling was key to unlock better performance.
  • We restored catalog.json with cp, which resets the file's modification time on every run. The "is the catalog stale?" check read the mtime, so the catalog always looked brand new and was never refreshed. The fix was to stamp a fetched_at timestamp inside the file and read the age from that, and to make CI warn if the newest data is more than seven days behind.

Because every stage can be resumed, the work gets spread across GitHub Action runs. The pipeline runs every 3 hours in three phases. A first job refreshes the catalog and graph when they're more than a day old. Then the version fetch runs on 8 parallel runners to avoid IP rate-limiting, since each runner gets a different IP. The last job merges the eight shard branches back into pipeline-cache, rebuilds historial, and pushes it. A concurrency group makes sure only one run can write historial at a time.

An average library, with 10,000 books, should store around 20 GB of just text. There’s a reason why we have a Library of Congress.

The ley-chile repo stores almost 20 GB of pure normative text.

At this scale, some things start to break.

For example:

Git repository limits diagram

To some extent, the ley-chile repo has broken two of these limits: the size of its tree objects and the width of its directories.

git cat-file --batch-all-objects --batch-check='%(objecttype) %(objectsize)' | awk '{s[$1]+=$2; t+=$2} END {for(k in s) printf "%s %.2f GiB\n",k,s[k]/2^30; printf "total %.2f GiB\n",t/2^30}' commit 0.12 GiB tree 375.27 GiB blob 17.24 GiB total 392.63 GiB

This is the uncompressed size of the repo. Note that the on-disk value is much smaller:

du -sh .git 7.8G .git

What this means is that if you were to count every version of every file ever committed, you would have around 17 GiB of data in total (blobs). You can see that the trees take up the most space. Why is this?

Git never edits objects in place. When you change one file, Git writes a new blob for it. Then it writes a new tree for every directory on the path from that file up to the root, because each parent's listing now holds a new hash.

For a normal repo that costs little. Here, directories like cache/versions/ or leyes/ list thousands of entries each. Every commit that touches even one law rewrites those huge listings in full. Thousands of commits * multi-MB listings = 375 GiB uncompressed.

Which leads us to the second limit broken: directory width. The three biggest folders in the repo are all inside the pipeline-cache branch (an orphan branch that holds the downloaded data; the rebuilt history of the laws lives in a second orphan branch, historial, described below). They all have more than 357k entries.

In the history branch, with the “rendered” repository, you will see the biggest folder by entries is etc/res with all of the resolutions from different state organisms.

A few more things that broke or had me bang my head against a wall along the way:

  • Two resolutions with the same number in one folder. Res N°20, from the “Municipalidad de Tocopilla” and Res N°20, from the DGA, both wrote to etc/res/20/. The second one silently overwrote the first with no notice. I fixed this with a forward-pass over the whole 357,000-node graph that hands out paths in ID order, so the first norm keeps etc/res/20/ and the second becomes etc/res/20-1163150/.
  • Typing hell. A helper returned ley for an empty type, so every untyped norm landed in leyes/. A related bug dropped any law with no number into leyes/ with identical names, and they all overwrote each other's metadata.json. Unknown types now go to etc/unknown/<id>/, and number-less laws are filed by ID.
  • GitHub's 100 MB blob limit. At some point, pushes of pipeline-cache started being rejected because the diffs of three large codes were over the limit: norm 6368 at 100.11 MB, 8201 at 117 MB and 207436 at 137 MB. They're HTML-heavy, so gzip shrank them about tenfold (76 MB down to 7.6 MB), and the cache now stores .json.gz.
  • HTTP 500 on git push. Git's default http.postBuffer is 1 MB. Above that it switches to chunked transfer encoding, which GitHub's endpoint often rejects for big packs. I raised the buffer to 2 GB and cut the chunk size from 500 commits to 100, since 500 commits of multi-MB texto.md files made 1–2 GB packs. Then I found that git push can exit 0 with "Everything up-to-date" after a mid-stream disconnect, so the script now checks with git ls-remote that the branch actually moved and retries up to 5 times.
  • The runner ran out of disk. The pipeline-cache worktree has around 1.1 million files. A run died at 92% of the checkout with "No space left on device" on a runner with about 14 GB free. I ended up using an action template to delete the runner's preinstalled Android SDK and .NET toolchains (about 30 GB), and on top of that, used a sparse checkout that skips the per-norm metadata folder.

Git History: One Commit per Publication

The useful unit of history is a publication, not a law or an edit within a single file. A new Chilean norm can amend several older ones at once, so I made each publication a single commit containing the new norm and every change it caused. Open that commit and you can see the legal change as one event.

I wrote about the mechanics of generating those commits with git fast-import in my previous article. Here, that commit model matters because it powers the diffs and historical lookups below.

Git Date Limits: Commits Before 1970 and After 2099

The full story, with the code, is in my previous article as well. Three things shaped this project:

  • Git can't hold a date before 1970 in a repo that GitHub will accept. (The failures, as I showed in my previous article: --date silently falls back to the current year, the environment variables give a misleading "invalid date format", and fsck rejects what fast-import lets through.) GitHub runs git fsck --strict on every push, and negative timestamps fail it. In the main pipeline, my fix collapses every pre-1970 commit into the first few seconds of 1970, one second apart so they still sort in order, and keeps the real date in two places Git doesn't police: the metadata.json stored next to each law (fechaPublicacion), and the commit subject as publicada (YYYY-MM-DD) for every amendment. The main pipeline’s index builder reads the date from the subject, not from Git's own timestamp.
  • Same-day collisions. Commit timestamps have to be unique, so publications that share a day are shifted by a second. At one point this also moved some real post-1970 dates by a day. Reading the date from the subject fixed that too.
  • Past 2099, Git's --date parsing fails, while raw timestamps keep working. This hasn't hit the project, because I filter anything above 2100 as the 2222-02-02 sentinel.

I also tried the alternatives for the dates, as I described in my previous article. Mercurial accepts 1901 through 2038, which is a bit wider than Git for the past but closes the future sooner. Fossil stores dates as Julian day numbers in SQLite, so it took every date in the corpus, including early nineteenth-century dates.

Web UI, API and MCP Server for Searching Chilean Law

For our users to actually make good use of the data, we needed to expose it in a more familiar interface. Given Git’s and GitHub’s constraints, the fact that the existing GitHub UI wasn’t made for exploring data of this shape, and the hard limits around datetimes, metadata, linking, etc, I decided to go forward building a Web UI for the project. This was probably the best decision I made.

Ley Chile web interface

The web UI makes it easy for lawyers and regular users to take advantage of Git’s utility, while not necessarily having to interact with it. Common interactions are implemented, with Git-specific functionality available for power users who are willing to clone the repo.

A counterpart of the Web UI is an API and an MCP server. These can get quick access to the corpus with the main functionality being search. Search is a big challenge, especially when storing not only the latest versions but also the full history.

The directory structure of ley-chile only serves quick lookup when the user previously knows the text’s type and id. For any semantic search, another layer is needed. Just grep-ing over the whole directory is an immensely slow operation. The challenge then becomes building a search engine over the corpus.

The site started with two search tiers. Meilisearch stored the top ~8% of laws that people actually read, 28,730 of 333,026, and Postgres held the rest. This was supposed to be an 80/20 solution to the search problem, but it actually became very expensive!

Size

Corpus covered

Meilisearch volume

4.76 GB

under 10%

Postgres full-text index on articles (articulo_tsv_idx)

304 MB

100%

That is about sixteen times the disk for under a tenth of the content. There was a second cost in memory. Meilisearch memory-maps its index, so its resident memory climbs to cover the working set and never comes back down, and Railway bills observed memory, so the mapped index was a continuous charge. A premium price to pay even with no usage being registered.

What settled it was an outage. On 2026-09-07 Meilisearch didn't come back from a Railway region migration. Search returned nothing for every query, including a bare law number that Postgres could have answered, because the code awaited the Meilisearch tier unconditionally and swallowed the error. A dependency serving under a tenth of the corpus took down 100% of search, and did it with a 200 response and an empty list.

So I moved all of it into Postgres, in two parts:

  • Typeahead works at the level of a whole norm, not article text. A table of about 333,000 rows holds the title, the common names people use ("ley de partidos") and the number, with a small text-search function over it. Ranking is by type (laws and codes outrank decrees and resolutions, because "1" alone matches around 450 decrees), by how often the law has been amended, and by how often people look it up. Typeahead takes about 3 ms.
  • Deep search over article text has two query shapes whose benefits measurably balance each other. For a common word like "trabajo", walking the norms in ID order finds 20 matches in about 2.5 ms. That same shape on a rare word never finds 20 and degenerates into a full table scan. The other shape ranks everything through the full-text index first, which is fast for a rare word and slow for a common one. The deep search function estimates how many articles will match from a table of word frequencies, and picks the shape. The crossover sits near the square root of the corpus size, so the threshold is computed when data loads (it resolved to 1,868).

Measured on production:

Query

Before

After

"geotermia" (149 matches)

21,458 ms

114.5 ms

"contrato" (60,644 matches)

118 ms

8.2 ms

The old query took 21 seconds because Postgres chose the wrong index and removed 524,720 rows by filter while never touching the full-text index at all. Making one query shape the default would have made common words 38 times slower, this justifies the logical overhead.

This was the point when I removed the Meilisearch client and its volume. It's not perfect. When a common word has thousands of matches, the fast shape picks which 20 laws to show by lowest ID, not by relevance. This probably justifies the decision to build a different engine at some point, but with an LLM in front of the search, as with the MCP server, it works well enough most of the time.

The fact that we had to build all of this separate infrastructure on top goes to show how Git really isn’t the best format to store this type of data. But alas, it is very useful for a specific purpose.

  • "What changed, word for word, between these two points in time?" - Creating diffs is easy in Git. The web UI has a redline view that compares two versions of a law, with deletions in red and additions in green. The diff_versions tool does the same for agents.
  • "What touches this law?" - The data is structured such that a single commit will show this. Amended laws arrive along with their reform texts. In the MCP get_modifications lists the norms that amended a law and the ones it amends. It’s very useful for understanding the evolution of legislation around a topic.
  • A JSON API for developers. https://leyes.pisanvs.cl/api/v1 has search, norm metadata and article endpoints, an OpenAPI 3.1 spec at /api/v1/openapi.json (the docs page is at /api), and an asOf=YYYY-MM-DD parameter that searches the text in force on that date. Keys are issued by hand. Here's a search for "partidos politicos":
curl -H "Authorization: Bearer $LEYCHILE_KEY" \ 'https://leyes.pisanvs.cl/api/v1/search?q=partidos+politicos&limit=2'

The first result is idNorma 29994, "Ley Orgánica Constitucional de los Partidos Políticos" (Ley 18603). Note that every endpoint takes an idNorma, not a type and number. (tipo, numero) doesn't identify a Chilean norm: there are 227 “DFL 1”, 75 “DFL 4” and 525 “DTO 1”. At one point the endpoint for law 20780 /ley/20780 resolved to an unrelated decree whose internal ID happened to be 20780.

  • Agents. There is a read-only MCP server at https://leyes.pisanvs.cl/api/mcp, with no authentication and eight tools: search_laws, get_law, get_article, search_articles, diff_versions, get_modifications, list_versions and get_raw_link. To add it to Claude Code:
claude mcp add --transport http ley-chile https://leyes.pisanvs.cl/api/mcp

There's also a /llms.txt with instructions written for agents. This is how most users discover this project, and is probably the most useful use case that impacts efficiency.

Should You Put Legislation in Git?

You should, if you care about the text of the law being traceable. A Git history gives you something a PDF can't: every version has a hash, every change has a cause, and "the law as of date X" is a command instead of a research project. Lawyers don't need better word processors for this. They need blame, diff and log, which they have never had.

You shouldn't expect Git to be the product. My lawyer friends will keep using Word. What made the project usable for them was the web UI on top, so Git ended up as the engine and not the interface. And it's not even a good database: search needed Postgres, and my first search engine (Meilisearch) cost about 16 times the disk for under a tenth of the corpus and took search down with it when it went offline. In the final design Git is the single source of truth, and everything else (Postgres, the search tables) is derived and can be thrown away and rebuilt. Right now, the MCP seems to be the biggest product.

You should check the limits before you start. GitHub caps blobs at 100 MB, rejects pushes with big packs, and runs fsck on everything you send it. Read the repository limits first. I learned most of them from rejected pushes.

The code is on GitHub and the site is at leyes.pisanvs.cl. Issues and pull requests are welcome.