Aiko Mori
@aikoInference · serving · GPUs
Throughput per dollar. Batches, quantisation, and the unglamorous work of p99.
on Lessons learned: 3,000 AI review comments, ranked by usefulness · 5d ago
Great write-up, though I'd push back on the 8s timeout. Some of our prompts legitimately need 20s.
on Lessons learned: 3,000 AI review comments, ranked by usefulness · 5d ago
Solid. Two things I'd add: jitter on the client retry, and a hard cap on queue depth.
on Tutorial: hybrid search with pgvector and reciprocal rank fusion · 5d ago
Seconding this. Cost visibility was the thing that unlocked the budget conversation.
on Tutorial: hybrid search with pgvector and reciprocal rank fusion · 6d ago
Would love a follow-up on how this behaves during a region failover.
on Tutorial: hybrid search with pgvector and reciprocal rank fusion · 6d ago
This matches our experience almost exactly, down to the retry storm on a Friday.
on Tutorial: hybrid search with pgvector and reciprocal rank fusion · 6d ago
Would you still build it in-house today, or reach for something off the shelf?
on Lessons learned: what 18 months of eval data actually predicts · 10d ago
Question on the checkpointing: how do you deal with side effects that already happened?
on Lessons learned: what 18 months of eval data actually predicts · 11d ago
Bookmarked. The verification checklist is going straight into our onboarding doc.
on Lessons learned: 6% of our first LLM outputs were quietly malformed · 25d ago
Question on the checkpointing: how do you deal with side effects that already happened? (64)
on Architecture notes: planner–executor–verifier without the framework tax · 27d ago
What does your alerting look like for this? We can't distinguish 'slow' from 'stuck'. (21)
on Architecture notes: planner–executor–verifier without the framework tax · 27d ago
We did the boring version of this — a queue and a worker — and it held for two years. (44)