"What does good output look like? If you can't write it, you are not ready to build it." This is one of the most overlooked foundation pieces. Thank you for touching on that.
Thanks Lee, glad it resonated! Often times as tech folks we get into the complex, more stimulating and challenging parts before getting the bare basics out of the way.
Defining what quality means for your specific user and your specific workflow may help push products beyond the demo phase, driving actual impact!
I can highly recommend Ankita's Substack as well! She thinks outside the box and connects the dots across disciplines. Ankita speaks from experience, and everything she writes about has been tested in practice beforehand. And additionally, she is a really nice person, we once had a video call together.
This is something I’ve been thinking about as a solo builder.
In a team, evaluation design can be shared across product managers, engineers, testers, and users. When you’re alone, all of those roles collapse into one person.
That’s actually one of the reasons I stopped short of shipping my first AI-assisted app. The coding part was manageable. Defining quality was harder.
If the AI gives me an answer, how do I know it’s correct?
How many examples do I need to test?
What edge cases am I missing?
What happens when a real user does something I never thought of?
And beyond quality, there’s another question that kept bothering me:
How do I know it’s safe?
The app may appear to work perfectly. The demo may go smoothly. But working and being safe are not the same thing.
The more I learned, the more I realized that “it seems to work” is not the same thing as knowing it works.
For a solo developer, evaluation isn’t just about measuring outputs. It’s also about questioning your own assumptions.
"What does good output look like? If you can't write it, you are not ready to build it." This is one of the most overlooked foundation pieces. Thank you for touching on that.
Thanks Lee, glad it resonated! Often times as tech folks we get into the complex, more stimulating and challenging parts before getting the bare basics out of the way.
Defining what quality means for your specific user and your specific workflow may help push products beyond the demo phase, driving actual impact!
this class is 2 weeks homework for me! (while im house/pet sitting) THANK YOU.
I can highly recommend Ankita's Substack as well! She thinks outside the box and connects the dots across disciplines. Ankita speaks from experience, and everything she writes about has been tested in practice beforehand. And additionally, she is a really nice person, we once had a video call together.
You are the best Bianca! Thanks for cheering me on, love the support friends bring to this forum.
Can’t wait to collaborate with you again!
We are very happy to highlight her work with this story! Have a wonderful friday Bianca.
This is something I’ve been thinking about as a solo builder.
In a team, evaluation design can be shared across product managers, engineers, testers, and users. When you’re alone, all of those roles collapse into one person.
That’s actually one of the reasons I stopped short of shipping my first AI-assisted app. The coding part was manageable. Defining quality was harder.
If the AI gives me an answer, how do I know it’s correct?
How many examples do I need to test?
What edge cases am I missing?
What happens when a real user does something I never thought of?
And beyond quality, there’s another question that kept bothering me:
How do I know it’s safe?
The app may appear to work perfectly. The demo may go smoothly. But working and being safe are not the same thing.
The more I learned, the more I realized that “it seems to work” is not the same thing as knowing it works.
For a solo developer, evaluation isn’t just about measuring outputs. It’s also about questioning your own assumptions.
I’m still learning how to do that well.