AI Engineering

The context window is not a feature. It’s a constraint you have to design around. Here’s what I’ve learned so far.

Rule 1: Front-load the important stuff. Models pay more attention to the beginning and end of context. Put your critical instructions at the top, not buried in the middle of a system prompt.

Rule 2: Don’t fill it just because you can. A 200K context window doesn’t mean you should dump 200K tokens in. Signal-to-noise ratio matters more than volume. Every token of noise dilutes the signal.

Rule 3: Structure beats prose. Markdown headers, bullet lists, and clear sections outperform walls of text every time. The model parses structure faster than it parses nuance.

Rule 4: Memory is not context. Shoving your entire conversation history into context is not memory. It’s hoarding. Curate. Summarize. Keep what matters, discard what doesn’t.

Rule 5: The system prompt is sacred ground. Treat it like expensive real estate. Every instruction should earn its place. If you haven’t revisited your system prompt in a month, it’s probably full of dead weight.

More to come. This is a living document.

Your health data is scattered across a dozen apps. Oura knows your sleep. Strava knows your runs. COROS knows your cadence and ground contact time. Apple Health tries to be the hub but mostly just collects dust.

I wanted all of it in one place. Not a dashboard — a queryable data store I control.

The stack is deliberately simple: Python scripts pull from each API, land raw JSON in a bronze layer, then transform to Parquet files in a silver layer. DuckDB provides SQL access without a server. The whole thing runs on a machine with 512MB of memory.

Why Parquet? Columnar storage is perfect for time-series health data. Compresses well, queries fast, and you can read it with anything — Python, R, DuckDB, even Excel.

The real insight came when I started joining datasets. Overlaying sleep quality on training load. Correlating HRV trends with mileage ramps. Seeing how a bad night of sleep shows up two days later in your running power.

Your body generates incredible data. The least you can do is keep it somewhere you can actually use it.

Everyone’s building chatbots. The interesting work is in agents.

A chatbot answers questions. An agent does things. The difference matters because the hard problems in AI right now aren’t about generating text — they’re about acting reliably in the world. Reading files. Making API calls. Handling errors. Knowing when to ask for help versus when to push through.

The architecture that works: give the agent tools, a clear mandate, and a workspace. Let it figure out the execution. Don’t micromanage the steps — define the outcome.

What I’ve learned running agents in production:

1. Memory is everything. An agent without persistent memory is a goldfish with superpowers. It’ll do amazing things and then forget all of them.

2. Tool use > reasoning. The smartest model in the world is useless if it can’t read a file or make an HTTP request. Give it hands before you give it a bigger brain.

3. Trust is gradual. Start with read-only access. Expand to internal actions. Only give external capabilities (sending emails, posting publicly) after you’ve seen it behave.

4. Failure modes matter more than success modes. Any agent can succeed on the happy path. The question is what happens when the API returns a 500, or the file doesn’t exist, or the user asks for something ambiguous.

The chatbot era was the warm-up. The agent era is where it gets real.

I don’t use LLMs to write for me. I use them to argue with me.

The most valuable thing a model can do is push back. “Have you considered the opposite?” “What’s the weakest part of this argument?” “What would someone who disagrees say?”

Most people prompt for agreement. They write their thesis, feed it in, and get back a polished version of what they already believe. That’s not thinking — that’s intellectual comfort food.

The setup that works for me: give the model an explicit contrarian role. Tell it to find the holes. Tell it you don’t want validation. Then actually listen to what comes back.

Sometimes the pushback is shallow. Sometimes it’s profound. The point isn’t that the model is always right — it’s that it forces you to defend your position. And if you can’t defend it, maybe it doesn’t deserve defending.

The best thought partner is one that isn’t trying to make you feel good. That’s as true for humans as it is for machines.