Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
stas 
posted an update 1 day ago
Post
95
The Art of Debugging Open Free book is now available in pdf/epub and finally sports a book cover

https://github.com/stas00/the-art-of-debugging#ebook-versions-of-the-book

While a lot of the focus is on Unix/Python/Pytorch, the methodology chapter is applicable to any Software Debugging.

It currently sports 161 packed pages in 5 solid chapters and more coming...

The methodology chapter is the actual book. Unix/Python/Pytorch is just where it gets stress-tested.

The rule that has saved me most: never trust the layer you are reading. A generation bug read off the decoded string is a lie, you diff the raw input_ids. A clean agent transcript hides a truncated tool-call JSON one layer down.

Does the methodology chapter cover the case where the instrumentation itself is wrong, the probe ages or the log silently drops a field, so the bug lives exactly where you stopped looking?

·

I'm new to using AI so I don't yet have enough experience to answer your question.

Until now all the debugging I did was done manually so nothing was aged or silently dropped, haven't had this problem.

I imagine that with time there will be new sections covering AI-use specifics, but I need to gain more experience first.

That distinction is the whole point. Manual debugging, you are the probe, so nothing ages or drops silently. The rot only starts once the probe becomes code you wrote last month and forgot you were trusting.

So it may not need to be an AI-specific chapter at all. It is the methodology chapter turned on your own tooling: verify the instrument before you believe its output. Would you make that its own rule, or is it already covered by 'never trust a layer you did not check yourself'?

·

I'm not sure, In my book I share practices that work for me, I have never had a "dull" tool in my debugging practice thus it's not there.

Fair, and that filter is why the book will hold up. You only wrote down what you actually hit.

My dull tool was never dull when I picked it up. It was a sharp one that went dull under me. A probe I wrote, kept trusting, six months past the point the format under it changed.

So maybe it is not a missing practice. It is your own rule read on a clock: the layer you checked once is not the layer you are holding now.

Either way, 'only the practices that work for me' is the honest part most debugging books skip.

·

I hear you. I suppose I have never been in a situation where I continued using the same debug code for months in a row, usually something is broken, I diagnose writing appropriate tools, fix it and move on. The next time the situation is different, so I always adapted any previous tooling.

Thank you for the kind appreciation and supporting me limiting the scope to what I know, Dipankar.

That last paragraph is the cleanest answer my question could get. Throwaway tooling can't age. You never hold an instrument long enough for the format under it to move.

The aging probe is only a problem for standing instrumentation, the dashboards and assertions that outlive the bug they were written for. Your workflow sidesteps it by construction: new bug, new tool, discard.

So it was never a missing chapter. It was a failure mode you designed out. Thank you, Stas, genuinely enjoyed this one.