How to review a codebase with ChatGPT without losing the real risk

Choose a review question with a clear consequence

Review my whole codebase is a broad request with no clear end. It often produces a mixture of style advice, guesses and real problems. Start with a path you care about: how a quantity reaches an order, how a user reads a record, or why a file upload can fail. Name the expected behaviour and the cost of getting it wrong.

A question such as Can malformed quantities reach the order total gives the review a route, an input and a consequence. It also gives you a way to check a finding. You can inspect the input rules, follow the call and test the result. That is more useful than a general quality score.

For an unfamiliar repository, first ask for a small map of entry points and relevant modules. Use that map to choose one path. Do not let the map itself become proof that the implementation was reviewed. A file name suggests a purpose; the code and its callers show what actually happens.

Provide the surrounding code that can change the answer

Share the function, its caller, the input rules and any test that defines the behaviour. Include the language, runtime and important framework versions. A dependency file can help establish versions, while a short file tree shows where the pieces live. Avoid uploading generated files or unrelated assets that add noise without helping the review.

Remove secrets and real customer data. Use sample identifiers and values that preserve the shape of the problem. Check configuration files and logs before sharing them. A password hidden in a stack trace is still a password. If the code cannot leave your approved environment, work in an authorized tool there or prepare a smaller example.

Ask ChatGPT to list the files it used and the context it did not have. If middleware is missing, an authorization finding may still be a hypothesis. If a caller validates the input before it reaches a helper, a suspected defect in that helper may not be reachable. Those details are part of the review.

Teaching example: a parser that accepts extra text

This teaching example is deliberately small. It is not a report of a test run. The requirement is: quantity must be a string containing only digits, must represent a whole number from one to one hundred, and must not accept leading or trailing spaces. The existing parser below reads a numeric prefix. Under JavaScript's defined parseInt behaviour, '2boxes' is parsed as 2, so the range check does not reject it.

That observation gives a concrete finding: src/quantity.js, readQuantity, accepts trailing text contrary to the stated contract. The possible downstream effect is an order being created from malformed input. Whether that happens in the full application depends on the caller and earlier checks. The finding should include that condition rather than claim unseen behaviour.

The example also accepts a decimal string such as '2.9' as 2. It therefore hides more than one invalid-input shape. A vague recommendation to improve validation misses what you need to prove. State the accepted format first, then check examples that should fall outside it.

// Teaching example: existing code, not an executed test
export function readQuantity(text) {
  const quantity = Number.parseInt(text, 10);
  if (!Number.isInteger(quantity) || quantity < 1 || quantity > 100) {
    throw new Error('Invalid quantity');
  }
  return quantity;
}

// Expected from the contract: reject '2boxes' and '2.9'.
// From parseInt semantics, the current function returns 2 for both.

Write findings that someone can verify

A useful finding needs a location, the condition that triggers it, the expected result, the observed or predicted result, and the consequence. Add a confidence label and the missing context. A suggestion such as add validation is too broad. A finding such as '2boxes' reaches readQuantity and returns 2 tells someone exactly what to inspect.

Do not confuse importance with certainty. A low-confidence authorization concern may be important enough to investigate first. A high-confidence spelling issue may have little impact. Rank findings by the harm they could cause and how easily the path can occur, while keeping certainty separate.

Ask what would disprove each claim. For the parser, an earlier check that rejects every non-digit string could make the malformed input unreachable on that route. You may still improve the helper's contract, but the user-visible defect is different. Reviewing counter-evidence saves you from changing code to fix an imagined failure.

Follow the input from entry point to effect

Trace the concrete input through the application. Where does it enter? Is it transformed? Which check runs? Which function uses it? What happens after an error? A code snippet can look unsafe in isolation but be protected by a caller. It can also look safe while a different path skips the check. Ask for the route, not just a comment on the helper.

For access control, follow how identity is established, how the requested record is selected and where permission is checked. Being signed in and being allowed to read this specific record are different conditions. Use harmless test records when checking this path. Include a denied case as well as a permitted one.

For concurrent updates, write the order of operations. Two requests reading the same stock count and then each writing a new count can behave differently from one protected update. Ask which operation guarantees the rule you need. A single normal request test cannot check a problem that depends on two requests overlapping.

Propose the smallest fix that matches the contract

For the teaching parser, a small repair checks the string format before conversion. It also checks the input type because the contract requires a string. The following code follows the stated rule: digits only, no spaces, and a value between one and one hundred. Leading zeros are allowed here because the example contract did not forbid them. If your real rule forbids them, make that rule explicit before changing the expression.

The repaired function is a proposal, not a claimed passing implementation. It uses a pattern that checks the whole string, then converts to a number and applies the range. This avoids reading only the numeric beginning of an invalid value. Keep the error behaviour consistent with the caller so the fix does not turn a handled error into an unexpected crash.

Do not combine the repair with a new framework, dependency upgrade and folder reorganization unless the defect truly needs those changes. A small change is easier to inspect and test. Broader improvements can be separate work after you have confirmed the bug.

// Teaching example: proposed repair
export function readQuantity(text) {
  if (typeof text !== 'string' || !/^[0-9]+$/.test(text)) {
    throw new Error('Invalid quantity');
  }
  const quantity = Number(text);
  if (!Number.isInteger(quantity) || quantity < 1 || quantity > 100) {
    throw new Error('Invalid quantity');
  }
  return quantity;
}

Use tests that distinguish the old code from the repair

A good regression test catches the failure you identified. For this parser, a test that only checks '2' passes in both versions and proves little about the repair. Add invalid examples such as '2boxes' and '2.9', plus the boundaries '1', '100', '0' and '101'. Include a non-string input because the contract names a type.

The expected cases below are derived from the teaching contract. They are not recorded execution results. Run equivalent tests in your actual test setup. Keep the failure output from before the fix and the passing output after it when practical. That record shows which behaviour changed.

For a database or network path, a helper test may not be enough. A test of the endpoint can show that invalid input produces the right response and does not create a record. Pick the level that exercises the real consequence without turning a small fix into an unrelated test project.

Separate suggested, executed and passing checks

ChatGPT may write a test without running it. It may also run a check in a tool-enabled environment. Those are different states. Ask for the command, environment and relevant output when execution is claimed. A sentence saying the code should pass is a prediction. A test process that finishes successfully is execution evidence for that particular check.

If you run the test yourself, provide the output when asking for the next step. Do not assume ChatGPT can see your terminal. If the check fails, include the failure message and the changed code. Avoid repeatedly asking whether the fix is correct without adding the evidence that can answer the question.

A passing parser test does not prove the entire checkout is correct. It proves the cases that test exercised under that environment. Check the caller's normal path and relevant error handling after the change. Broaden only when the defect or a new failure gives you a reason.

Repair false positives and version mistakes

A false positive is a reported problem that turns out not to be a real defect in the relevant path. If middleware already performs the claimed missing check, show that code and ask ChatGPT to update the finding. Keep a short note explaining why the concern was rejected. This prevents the same unsupported issue from returning in the next review.

Framework advice can also be wrong for your version. If a proposed API or option is unfamiliar, check the current official documentation for the version you use. Ask ChatGPT to cite the applicable page, rather than replacing one guessed method with another. A syntactically plausible patch may still call an unavailable API.

Treat style suggestions separately from defects. Renaming a variable may improve readability, but it is not evidence of data loss. You can accept useful cleanup without presenting it as a high-priority bug. The review should make decisions clearer rather than inflate the finding count.

Finish with a short record of what is known

Keep a finding list with location, trigger, effect, action and status. Useful statuses include confirmed by source inspection, reproduced, fixed and checked, rejected after more context, and unresolved. Do not label the whole application secure because a bounded review found no issues. State the path and files that were checked.

For the teaching parser, the final record would name the malformed-string issue, the proposed format check and the tests that need to run. If you actually perform those checks later, record the real results. The example in this article stops at a reasoned proposal; it does not invent local execution.

For larger reviews, use repository-aware developer tools where available so file locations, diffs and tests stay connected to the code. ChatGPT can still help explain a small path in an ordinary conversation. The important result is an issue someone can verify and a repair whose relevant behaviour has been checked.

Use the review finding to build a failure you can test. Reproduce and repair a specific bug →