SWE-bench Lite · 300 instances · Docker eval
142/300 resolved(47.3%)
Real GitHub issues. Official SWE-bench evaluation. Any model you choose.
142
Resolved
212
Patches
91%
Cache hit
$4.14
Total cost
Agent Comparison
| Agent | Score | Model | BYOK | OSS |
|---|---|---|---|---|
| Claude Code | 72.0% | Claude Sonnet 4 | No | No |
| Codex CLI | 69.1% | GPT-4.1 | No | Yes |
| Aider | 53.0% | Various | Yes | Yes |
| Kepleryou | 47.3% | DeepSeek V4 Flash | Yes | Yes |
| SWE-agent | 23.0% | Various | Yes | Yes |
Kepler v2.3 · DeepSeek V4 Flash · Updated 2026-07-15
Kepler across models
Same agent, same harness. Only the model changes.
| Metric | DeepSeek V4 Flashnow | Tencent HY3now | GLM 5.2soon |
|---|---|---|---|
| Vendor | DeepSeek | Tencent (free) | Zhipu |
| SWE-bench | 47.3% | 44.3% | — |
| Resolved | 142 / 300 | 133 / 300 | — |
| Patched | 212 (71%) | 230 (77%) | — |
| Cost / 300 | $4.14 | $0 (free) | — |
| Per instance | $0.014 | $0 | — |
| Cache hit | 91% | 93% | — |
| Date | 2026-07-15 | 2026-07-15 | pending |
Every model runs the same harness · Docker eval · SWE-bench Lite 300
How Kepler solves issues
1
Search
Find relevant files
2
Understand
Read the root cause
3
Fix
Surgical source edit
4
Verify
Run project tests
Resolved by project
django/django 68sympy/sympy 31scikit 13pytest 8matplotlib/matplotlib 7sphinx 5astropy/astropy 3mwaskom/seaborn 2pallets/flask 2pylint 2pydata/xarray 1
All 142 resolved instances
✓astropy__astropy-12907✓astropy__astropy-14995✓astropy__astropy-6938✓django__django-10914✓django__django-10924✓django__django-11001✓django__django-11039✓django__django-11049✓django__django-11099✓django__django-11133✓django__django-11179✓django__django-11283✓django__django-11422✓django__django-11583✓django__django-11620✓django__django-11630✓django__django-11999✓django__django-12125✓django__django-12184✓django__django-12284✓django__django-12286✓django__django-12453✓django__django-12497✓django__django-12700✓django__django-12856✓django__django-12908✓django__django-12915✓django__django-12983✓django__django-13028✓django__django-13033✓django__django-13158✓django__django-13230✓django__django-13447✓django__django-13551✓django__django-13590✓django__django-13658✓django__django-13660✓django__django-13757✓django__django-13768✓django__django-13925✓django__django-13933✓django__django-14238✓django__django-14382✓django__django-14411✓django__django-14580✓django__django-14608✓django__django-14672✓django__django-14730✓django__django-14752✓django__django-14787✓django__django-14855✓django__django-14915✓django__django-14999✓django__django-15347✓django__django-15498✓django__django-15789✓django__django-15814✓django__django-15819✓django__django-15851✓django__django-15902✓django__django-15996✓django__django-16041✓django__django-16046✓django__django-16139✓django__django-16255✓django__django-16379✓django__django-16527✓django__django-16595✓django__django-16873✓django__django-17051✓django__django-17087✓matplotlib__matplotlib-23964✓matplotlib__matplotlib-24149✓matplotlib__matplotlib-24265✓matplotlib__matplotlib-24970✓matplotlib__matplotlib-25442✓matplotlib__matplotlib-26011✓matplotlib__matplotlib-26020✓mwaskom__seaborn-2848✓mwaskom__seaborn-3010✓pallets__flask-4992✓pallets__flask-5063✓pydata__xarray-5131✓pylint-dev__pylint-5859✓pylint-dev__pylint-6506✓pytest-dev__pytest-11143✓pytest-dev__pytest-5227✓pytest-dev__pytest-5495✓pytest-dev__pytest-5692✓pytest-dev__pytest-7168✓pytest-dev__pytest-7220✓pytest-dev__pytest-7373✓pytest-dev__pytest-8906✓scikit-learn__scikit-learn-10297✓scikit-learn__scikit-learn-11281✓scikit-learn__scikit-learn-13142✓scikit-learn__scikit-learn-13241✓scikit-learn__scikit-learn-13439✓scikit-learn__scikit-learn-13496✓scikit-learn__scikit-learn-13584✓scikit-learn__scikit-learn-13779✓scikit-learn__scikit-learn-14092✓scikit-learn__scikit-learn-14894✓scikit-learn__scikit-learn-14983✓scikit-learn__scikit-learn-15512✓scikit-learn__scikit-learn-15535✓sphinx-doc__sphinx-10325✓sphinx-doc__sphinx-8273✓sphinx-doc__sphinx-8713✓sphinx-doc__sphinx-8721✓sphinx-doc__sphinx-8801✓sympy__sympy-12419✓sympy__sympy-12481✓sympy__sympy-13471✓sympy__sympy-13480✓sympy__sympy-13647✓sympy__sympy-13773✓sympy__sympy-13971✓sympy__sympy-14396✓sympy__sympy-14774✓sympy__sympy-15011✓sympy__sympy-15345✓sympy__sympy-15346✓sympy__sympy-16792✓sympy__sympy-16988✓sympy__sympy-17022✓sympy__sympy-17139✓sympy__sympy-18057✓sympy__sympy-18189✓sympy__sympy-18532✓sympy__sympy-19487✓sympy__sympy-20154✓sympy__sympy-20212✓sympy__sympy-20590✓sympy__sympy-21055✓sympy__sympy-21379✓sympy__sympy-21614✓sympy__sympy-21847✓sympy__sympy-23117✓sympy__sympy-23262✓sympy__sympy-24152✓sympy__sympy-24213
Methodology
Harness: Official swebench v4.1.0 with Docker containers. Each instance runs in isolation matching the repo's Python version.
Scoring: Resolved = ALL failing tests pass AND ALL existing tests still pass. No partial credit.
Score: 142/300 (47.3%) against all instances. 212 patches generated, 88 agent errors. 91% cache hit rate.
Try Kepler
Bring your own key. Pick any model.
npm i -g @axplusb/keplerWeb app