Here you will find all my articles published on Medium. I write regularly about Semantic Layers, Postgres, LLM Evaluation, CDC Pipelines, and Stream Processing.
Building a Semantic Layer from Zero (Part 2): Can Column Descriptions Alone Stop an LLM From Hallucinating SQL? Same 6 questions, same database — but this time the model gets the schema. 6/6 correc...
6 questions, 12 attempts, 1 silent fabrication that would have gone straight into a report This is part 1 of a 6-post series that answers one question: How much context does a model actually need t...
8 architecture decisions, 5 Debezium incidents, 3 corrections — and what a streaming pricing engine looks like after all of them The entire project was built with spec-driven development — every ph...
Hash tables, sorting algorithms, and Big-O explain joins and shuffle issues better than any tuning guide Apache Spark didn’t invent any new data structure or algorithm , it just the reused the clas...
I was building a streaming pricing engine for vacation rental apartments. The system ingests operational costs via CDC from PostgreSQL through Debezium into Kafka, market reference prices through K...
There’s a pattern that works incredibly well for building data lakes on AWS: Kinesis captures events, Lambda processes them, S3 stores them, Glue transforms them, Athena queries them. It’s cheap, s...
Some basic principles according to Kleppmann’s “Designing Data-Intensive Applications” book Most engineers encounter streaming for the first time through a familiar lens: “It’s like batch, but fast...
A systematic investigation into why a pipeline doubled its runtime overnight, and the Iceberg internals that made it inevitable. A production pipeline on AWS Glue went from 15 minutes to 37 overnig...
There’s a moment in every data team’s life when you open a dashboard, see a number that doesn’t look right, and realise you have no idea which of your fifteen scripts is responsible for it. That mo...
Why your Iceberg pipeline spends 80 minutes in the driver while executors sit idle — and why persist function can’t fix it. Every Spark tuning guide focuses on the same things: partition count, exe...