Trust it like a stranger’s pull request.
Patchlit writes code with a model and pushes it to your repository. So the design assumes the model can be wrong, and can be fooled, and puts the checks outside the model.
Green means your tests passed. Not that it is right.
A fix counts as green only when your own CI passes on the exact commit Patchlit pushed, with every check on that commit finished, and when an audit of what actually landed in git finds nothing protected was touched. A model never grades its own work.
It still does not mean the change is correct. Patchlit never merges anything. A person reads the diff, and “merged as is” is reported as a second number beside the green rate, because no rule can inflate it.
What Patchlit can see and do on GitHub.
A GitHub App, installed only on the repositories you choose. It uses short-lived installation tokens, never your personal login.
- Contents: read and write
- To read the files a failure points at and push a fix to the PR's own branch.
- Pull requests: read and write
- To read the PR, post comments, and open a reproduction PR for an issue.
- Issues: read and write
- To read a labelled issue and say when it could not be reproduced.
- Actions, Checks and Commit statuses: read
- To read the failing job logs and to judge whether every check on a commit passed.
- Metadata: read
- GitHub requires it for any app.
To write a fix, the failing logs and the few files they name are sent to the AI model provider. Your whole repository is not. Signing in asks GitHub only for your identity.
Before the push, and again from what landed.
Every bypass we found came from a gap between what we thought a fix did and what git recorded. So the second check ignores what the model said and reads the result.
Vetting, before anything is pushed
The model’s output is treated as untrusted. It is checked against the refusals below, your size limits and your protected paths, and a fix that fails is never pushed.
Audit, after it lands
Once CI is green, Patchlit reads the commits it actually pushed, found by their trailer. If they touched something protected, the win is withdrawn and a person is told.
What it refuses to do
- Editing an existing test, or test tooling such as vitest, jest or playwright config
- CI and automation config (.github, .circleci, .husky), lockfiles, environment and secret files
- Changing or downgrading dependencies, or editing anything in package.json except scripts
- Deleting or emptying a file, or removing an exported function; guessing at a file it was never shown
- Pointing code at a web address the file did not already use, or building one at run time
- Anything larger than your limits, or inside a path you protect
- Pushing at all when your CI uses secrets, until you allow it
Your CI can read your secrets, so Patchlit asks first.
When Patchlit pushes a fix, your CI runs it, and CI can read your repository secrets. We tested this: a run triggered by Patchlit’s push sees them, and on a Dependabot branch it sees secrets GitHub deliberately withholds from Dependabot itself.
So the default is deny. If any workflow in a repository uses a secret, Patchlit pushes nothing and says why. You turn it on per repository, once you accept it. CI that runs on other systems, such as CircleCI or Jenkins, is not visible to Patchlit, so treat the switch accordingly.
What it does not catch.
A checker is only as good as what it knows to look for. Read every fix.
- A change to logic inside a file it was allowed to edit, such as quietly loosening a validation, passes every rule if your tests pass.
- New configuration for the very tool being upgraded can change how it behaves.
- Text in an issue or a CI log can try to steer the model. It is fenced as untrusted and the output is vetted, which lowers the odds and is not a guarantee.
- The host check cannot be complete: a determined encoding trick may get past it, which is why a person reads the diff.