Best Practices & FAQ
Guidelines for effective AI usage and answers to common questions.
Best Practices
- Always review AI-generated content. AI output is a starting point, not a finished product. Review every generated test case for accuracy, completeness, and relevance to your specific application before saving.
- Write detailed descriptions for generation. The more specific your input, the better the output. Include feature details, user roles, edge cases, and expected behaviors in your description.
- Generate in iterations. Start with a broad description for general coverage, then run focused generations for specific areas (security testing, negative testing, accessibility). This produces more comprehensive test suites than a single generation.
- Use insight reports with sufficient data. Insight reports require historical execution data to produce meaningful insights. Wait until you have completed at least 3-5 test cycles before relying on AI-powered analytics.
- Combine AI with human expertise. Use AI generation for routine and repetitive scenarios (form validation, CRUD operations), and write manual test cases for complex business logic, workflows, and domain-specific scenarios that require deep product knowledge.
- Monitor credit usage. Track how much of your allocation the credit-consuming features (Test Case Generation and the AI Agent) use, and adjust usage patterns. On a paid plan the balance resets on the 1st of each month and does not carry over; on a free organization the 100-credit grant is one-time and is never topped up. Insight reports do not consume credits, so you can run them freely.
- Use the AI Agent for exploration. The AI Agent is most valuable for brainstorming test scenarios, identifying coverage gaps, and getting testing strategy advice. Use it as a thinking partner rather than a test case factory.
- Keep AI features disabled when not needed. AI is enabled by default. If your team is in a documentation or planning phase with no testing activity, the organization owner can turn AI off under Organization → AI Credits to prevent accidental credit usage.
- Match the quality level to the job. The Basic / Balanced / Premium setting changes the cost of a call by roughly 12x for the AI Agent and 8x for test case generation. Use Basic for bulk drafting and reserve Premium for complex specs and critical flows.
FAQ
▶Do AI features work offline?
No. AI features require an active internet connection because the models run in the cloud. They do not require a paid plan to start — a free organization receives a one-time grant of 100 credits when it is created, and AI is enabled by default. A paid organization receives 100 credits per seat every month. Once a free organization has spent its grant, the only way to get more is to upgrade.
▶Are my test cases and project data sent to external AI services?
When you invoke an AI feature that calls a model — Test Case Generation or the AI Agent — the relevant data (your description, or your prompt plus project context) is sent to the model over an encrypted connection. Under our providers' API terms that data is not used to train their models. Providers may retain API request content for a short period for abuse monitoring, under their own terms. Insight reports send nothing anywhere — they are computed from your own data in TestKase.
▶Can I control which users can use AI features?
AI features are controlled at the organization level via the Enable AI Features toggle on Organization → AI Credits. AI is enabled by default, and only the organization owner can change the toggle. When enabled, all users can use AI features. Role-based restrictions on specific AI actions can be configured through the permissions matrix.
▶What happens when I run out of credits?
When your credit balance reaches zero, the credit-consuming AI features — Create with AI and the AI Agent — stop. What you are told depends on your plan:
- On a paid plan: "You've used this month's AI credits. They refresh on the 1st, or add seats to raise your monthly allowance."
- On a free organization: "You've used your free AI credits. To use additional AI credits, upgrade to a Premium plan."
Insight reports and the score badges keep working because they do not consume credits, and all other TestKase features continue to work normally. A paid balance is replenished automatically on the 1st of the next month; to raise the monthly allocation sooner, add seats — the per-seat rate is the same on Premium and Enterprise, so moving between those does not raise it. A free organization's grant is one-time and is not replenished.
▶Can I generate test cases in different languages?
The AI generates test cases in the language of your input description. If you write the description in English, output is in English. You can write descriptions in other languages and the AI will attempt to generate test cases in that language, though English produces the best results.
▶How accurate are AI-generated test cases?
AI-generated test cases are generally high quality for common scenarios (form validation, CRUD, authentication flows). However, they should always be reviewed for domain-specific accuracy. The AI may not know your specific business rules, infrastructure constraints, or regulatory requirements.
▶Do AI credits carry over month to month?
No. On a paid plan, credits are a monthly grant of 100 credits per seat, at the same rate on Premium and Enterprise. The balance is overwritten on the 1st of every month with a fresh allocation, so any unused credits from the previous month do not carry over. A free organization receives 100 credits once, when it is created, and gets no monthly grant — there is nothing to carry over. There is no separate credit-purchase flow: to change a paid allocation, change your seat count; a free organization gets credits again by upgrading.
▶Can I get a refund on unused credits?
Please refer to the TestKase billing and refund policy or contact support for information about credit refunds.
▶How are insight report results different from regular reports?
Regular reports (Tier 1 and 2) show what happened — counts, percentages, trends. Insight reports (Tier 3) apply advanced analytics and heuristic scoring on top of that data to rank risk, flag flaky tests, and highlight quality gaps. These are deterministic calculations computed from your execution history — not machine-learning predictions — and they consume no AI credits.
▶How often should I run insight reports?
As often as you like — Insight reports consume no credits and recalculate from live data every time you open them. In practice, reviewing them at the end of each sprint or test cycle works well. Reports like Flaky Tests and Suite Optimization are most meaningful when revisited periodically (monthly or quarterly) once enough history has accumulated.