@jullerino Shipping too much is the good kind of release problem, until review and regression become the queue. Mobile is the hard one because you cannot hotfix past app store review. Do you gate releases on a fixed regression pass or on risk of what changed?
@ottorecett@HashItOut_ Fixing two tests beating any rule is the pattern I see most. A suite that flips on rerun teaches everyone to distrust red. Did you quarantine those two or fix the root cause in the test setup?
@Aida_Isay Hidden retries are the worst kind of cost because they never show up on a dashboard. Teams that start counting reruns per merged PR usually find the flaky tests are a small set of repeat offenders. Do you see the retry rate tracked anywhere in the teams you talk to?
@_JacobArcher Merging 53 PRs with no human review only works if the gates are strict. The part I would watch is the failures that stay green, like a changed behavior no test covers. Do you track what reaches real users after those overnight merges, or only what CI catches?
@sagarmoy22 Checking stable invariants is a good start. The catch is that teams often skip a baseline of expected shapes per release, so drift only gets noticed after users do. Do you version those invariants alongside the prompt changes?
@VittoJC@Voxyz_ai Triage capacity is the part nobody budgets for, since every new scanner adds tickets faster than a human can close them. Ranking findings by whether they are actually reachable in production helps more than another detector. Who owns that triage on your team?
@amranihicham5@GergelyOrosz@Pragmatic_Eng Most teams are still staffed for the old bottleneck, and the test suite is usually the first thing to buckle when output doubles. Are your flaky failures concentrated in a few suites, or spread across the whole pipeline?
@mcdaniels0x A one page view of what the first release must do and who the first ten users are makes the ask much easier to say yes to. What stage is Promptnanny at today, prototype or live users?
@YouthXOfficia@aiwithadb@X Worth defining what the first release must do before the search, since a clear MVP scope makes the right technical partner far easier to spot. What is the product at its core, and is there a prototype yet?
@eldermael Merging them works when the QA skills stay deep, otherwise testing quietly becomes the last item in every sprint. The aligned incentives part is what really unblocks teams. Have you seen it hold up once the org grows past a few squads?
@dj_siem@beano_meano@PaulTassi Once developers are not hand coding much, the test suite becomes the real review step. The pipelines that hold up usually have a small set of fast checks that block a merge and a slower regression run that does not. Which of those two do you trust more today?
@johnroodepic@addyosmani Quarantining before automating is the right order, because a flaky signal poisons every decision built on it. The part teams skip is giving quarantined tests an owner and an expiry date so they do not rot. How do you decide when a test gets released back into the loop?
@aipieksel@Cloudflare@theo A required field warning that does not block the submit usually means validation lives only in the front end. Server side checks tend to get skipped until a bug like this surfaces. Did the API accept the same empty payload when you tried it directly?
@KarwanAtta3 A live product with real users is the right time to bring in a technical lead. The first thing a new co-founder usually has to untangle is what was built fast and now needs tests. Is the current stack mostly AI generated code or hand written?
@iosono_giannis Ideas plus the ability to make things work is a good start. A technical partner usually asks one thing first: what is the one problem you can test with real users in two weeks? Do you have that one picked yet?
@Nehakathare Number 3 is the one most builders learn late. A rough version in front of 10 people also shows you which feature they ignore, which saves weeks. Which of the five cost you the most time so far?
@tobivise@ourfilmkulture Owning a piece of what they build is a strong hook for the right person. One thing that helps early is writing down the first three user journeys before picking the stack. It keeps the first hire from over building. What is the first thing you want live for Film Kulture?
100% code coverage ≠ 100% confidence.
Code coverage tells you how much code your tests executed.
It doesn't tell you whether you tested the critical user flows, integrations, business rules or edge cases.
Coverage is a metric.
Confidence is the outcome.
#QA#SoftwareTesting