▸ MOAZ ESSAM / SOLARI INTERN CHALLENGE

Where does more CPU stop buying time?

That's the question I built solci to answer. It replays one GitHub Actions job on Solari microVMs at 1, 2, 4 and 8 vCPU and hands me the curve, so I pick a size from numbers instead of a runner label.

GitHub Install

moaz@solari ~ %

> 1 vCPU created

> cpu online

> npm ci 34.0s ok

> npm run typecheck 23.7s ok

> done

> 2 vCPU created

> cpu online

> npm ci 22.8s ok

> npm run typecheck 16.5s ok

> done

> 4 vCPU created

> cpu online

> npm ci 20.6s ok

> npm run typecheck 13.9s ok

> done

RESULTS / moazessam376-dev/Gym-App / typecheck

size	total	solari/run	status
1 vCPU	67.1 s	$0.0011	ok
2 vCPU	48.5 s	$0.0012	ok
4 vCPU	45.5 s	$0.0023	ok
GitHub ubuntu-latest	39.0 s	$0.0100	baseline

RECOMMENDATIONUse 2 vCPU: 48 s for $0.0012 per run, within 10% of the 4 vCPU time (46 s) at 53% of its cost.

click and type, try: help

moaz@solari ~ %

▸ measured on Solari microVMs one matrix cell native Linux steps

▤ 01 / THE EVIDENCE

The curve

For the Gym-App typecheck job the answer is two cores. The third and fourth buy three seconds between them.

GYM-APP / TYPECHECKlower is better

67.1 s

1 vCPU

$0.0011

48.5 s

2 vCPU

$0.0012

recommended

45.5 s

4 vCPU

$0.0023

39.0 s

GitHub

$0.0100

01 2 vCPU captures almost all of the speedup, at a fraction of the GitHub cost.

◎

crosstalk, another repo of mine: its test job ran 191 s at 1 vCPU and 197 s at 2 vCPU. Not CPU-bound. solci told me not to pay for more cores.

4 and 8 vCPU failed in the repo's own test suite, so those totals are not part of the recommendation.

◎ 02 / AGENT MODE

Agent mode

A workflow change backed by measurements, ready for a human to review.

I built this agent to measure a job and then propose only the change the numbers justify: solci agent owner/repo --job typecheck --cpu 1,2,4 --pr.

run

measure

solci runs the job at each requested vCPU size and records timing, cost, and findings.

67s / 48s / 45s at 1, 2, 4 vCPU

ask

propose

solci hands the evidence to Codex, with Gemini as fallback. The model proposes the smallest workflow change the numbers justify.

chose 2 vCPU, added timeout-minutes: 15 and a concurrency group

PR

It opens a pull request with the report and chart. It only edits the workflow file. It never merges. You do.

Gym-App PR #111, opened by solci, merged only by a human

Opensolci opened this pull request

ci.yml: bound typecheck and cancel superseded runs

moazessam376-dev/Gym-App (private) · typecheck · proposed change

@@ workflow top level
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
 jobs:
   typecheck:
+    timeout-minutes: 15

15 minutes is safely above the 57 second p90. Superseded runs are cancelled given 35% historical failures.

Gym-App is my own private repo, so that link only opens for me. solci opened the same kind of pull request on crosstalk #43, which is public and open to read.

▸ 03 / THE RUNNER

How it works

This is what happens when you run one job. Every step shows up in the report.

fetch workflow via gh

Read the workflow shape, job, runner label, history, and static findings.

pick job and first matrix cell

Pick one job and the first literal value for each matrix axis.

boot a microVM per size

Boot takes 1 s. vCPUs hot-plug in 1-15 s after boot, so the runner waits for cpu online.

clone + shims

Clone into /work/repo. Shim setup-node, setup-python, uv, pnpm, and bun.

noh

run each step natively

Long steps run under nohup with log polling, because exec is capped at 28 s.

out

curve, recommendation, findings

Write the evidence, recommendation, and findings, then delete every sandbox.

▤ 04 / STATIC CHECKS

Findings

Eleven checks run before any sandbox boots. Severity is a hint about what to fix first.

NO_CACHE_SETUPmedium

A supported setup action has no dependency caching.

UNPINNED_ACTIONlow

An action uses a mutable or missing ref.

NO_TIMEOUTlow

The job has no job-level timeout.

NO_CONCURRENCYlow

A pull-request workflow has no cancellation group.

FULL_CLONElow

Checkout requests the complete repository history.

SLOW_INSTALL_HINTinfo

pip install or npm install without a lockfile, which is slower.

HIGH_FAILURE_RATEhigh

The history baseline has a failure rate above 20%.

BIG_RUNNERmedium

The job requests a large cores or vcpu runner.

SERVICES_UNSUPPORTEDinfo

The job uses Docker or service containers, which Solari does not have.

MATRIX_NOTEinfo

Only the first matrix cell was measured.

BROWSER_JOBinfo

Browser tests install a local browser. Use --cloud-browser for Solari cloud Chrome.

▮ 05 / FIELD NOTES

What I learned about Solari

These are the things I ran into while building the runner. They are also why the report is explicit about what it did not measure.

$cpu/memMb

honored up to 16 vCPU / 16 GB

$nproc --wait

vCPUs hot-plug 1-15 s after boot

$exec

about 28 s wall-clock cap

$echo $HOME

HOME unset in a fresh sandbox

$df -h

about 2.2 GB disk available

$docker version

no Docker

$time sandbox

1 s sandbox boot

$git clone

sub-second git clone

▸ 06 / THE REST OF THE LAB

The rest of the lab

solari-lab and solari-playtest are open source and came before solci. The cloud Chrome work became a solci flag.

solab.mp4

solari-labopen source

doctor, bench, isolation, proxy, and cost CLI.

I built it to test what a browser session actually does.

demo.mp4

Cloud Chrome for browser jobs

shipped

solci run --cloud-browser runs a job's Playwright, Puppeteer or browser-use steps in Solari cloud Chrome over CDP. No change to the repo. solci injects the endpoint into the job. Two Playwright tests pass end to end, including one against the app's own localhost server through the sandbox preview URL. Chromium only for now.

solari-playtestopen source

Sandbox build plus cloud Chrome playtesting agent with MCP tools.

It found a real UI bug in a three.js game with no hints.

▸ 07 / EVERYTHING I BUILT

Everything I built

Five repos and two pull requests. I ran every one of them end to end.

solari-ci

CLI

solci. Inspect a workflow, replay one job on Solari at 1, 2, 4 and 8 vCPU, get the curve and findings, and let a model open the PR.

68 tests, 4 commands: doctor, inspect, run, agent

Gym-App PR #111

PR / private

The pull request solci agent opened on Gym-App, my own private repo, from the measured curve. Timeout and a concurrency group, nothing else. The link only opens for me, so the full diff is shown above.

67s / 48s / 45s at 1, 2, 4 vCPU

crosstalk PR #43

PR / public

The same thing on a public repo, so anyone can open it. solci measured the test job, found it was not CPU-bound, and proposed a job timeout and nothing else.

test job not CPU-bound, timeout only

solci-browser-demo

demo

A small Playwright repo I use to prove the cloud Chrome path. Two tests, one against the app's own localhost server through the sandbox preview URL.

2 passed, 1 browser session, 5.8s

solari-lab

CLI

Diagnostics for Solari sandboxes: doctor, bench, isolation, proxy and cost. This is where I learned the exec cap and the vCPU hot-plug delay.

solab doctor / bench / isolation / proxy / cost

solari-playtest

agent

A playtesting agent with MCP tools. It builds a game in a sandbox, drives it in cloud Chrome, and writes a report. It found a real UI bug with no hints.

128 sequences, 8 browser sessions, 1 confirmed bug

solari-showcase

site

This page. Static HTML with GSAP, an interactive terminal, and the GIFs from each tool. Hosted on GitHub Pages.

no build step, works with JS off

▸ 08 / GET STARTED

Install

Clone the repo, add your Solari and GitHub credentials, then ask one job for its curve. Five commands.

git clone https://github.com/moazessam376-dev/solari-ci && cd solari-ci && uv pip install -e .

cp .env.example .env

Add SOLARI_API_KEY to .env.

export GITHUB_TOKEN=$(gh auth token)

solci doctor

solci run owner/repo --job test --cpu 1,2,4,8