feat: setup local dev environment for opslevel-runner with kind#302
feat: setup local dev environment for opslevel-runner with kind#302archf wants to merge 11 commits into
Conversation
- configure local runner startup to use the same helper image as coding-agent jobs - improve script readability so run arguments are easier to inspect and maintain - clarify test job enqueueing purpose by naming it for runner jobs
- add a short sleep command so the test job remains available long enough for enqueue behavior to be observed
initial approach didn't work
- lower the default enqueue count to make local test runs faster and less resource-intensive - pass configurable job pod CPU and memory requests so local runner scripts can match constrained environments
| faktory: faktory | ||
| runner: go run main.go run --mode=faktory --queues=runner --job-pod-max-wait=300 No newline at end of file | ||
| runner: ../bin/opslevel-runner-runner | ||
| image-builder: watchexec --watch . --exts go,mod,sum --watch ../Dockerfile -- HELPER_IMAGE=localhost/opslevel-runner:local ../bin/build-helper-image.sh opslevel-runner |
There was a problem hiding this comment.
this seems like overkill - hot reloading the image in kind whenever anything changes?
There was a problem hiding this comment.
good point, I'll look to narrow this down
|
|
||
| ## Container / kind tooling | ||
|
|
||
| Prefer podman, fall back to docker. Under podman, kind needs the experimental |
There was a problem hiding this comment.
i wouldn't say this is necessarily true - generally we prefer docker
There was a problem hiding this comment.
That is for the agent. It's hard to go the other way around. Falling back to the assumed more widely adopted tech.
|
|
||
| HELPER_IMAGE="${OPSLEVEL_JOB_POD_HELPER_IMAGE:-localhost/opslevel-runner:local}" | ||
|
|
||
| echo "Starting runner (mode=faktory queues=$FAKTORY_QUEUES queue=${OPSLEVEL_QUEUE:-<none>}) ..." |
There was a problem hiding this comment.
we're adding a bunch of machinery, but at the end of the day what we really need is a way to bypass the opslevel api and test how jobs are enqueued to k8s... we could probably just rig up a way to push work to worker queues directly, or maybe a new test mode that we can just pipe json into stdin or something
2 participants
Motivation
Prior dev loop required manual cluster bring-up, manual helper image rebuilds, and manual restart on code changes to test
opselvel-runneragainst a kubernetes cluster.This PR enhances the
Taskfilewith commands to setup conveniently an easier development environment.Changes
Taskfile
start-faktory): brings upkindvia `setup-kind dep, then launches goremanbuild-helper-image,stop-kindinstall-depssplit per OS (darwin/linux); installsredis,watchexec,kindinstall-faktoryupgraded to v1.9.3, fetches asset URL via GitHub API + jq, idempotent via status: checkOPSLEVEL_GO_PKGbumped to v2026image-builder— rebuilds helper image on Go/Dockerfile changes, reloads into kindAdded
bin/for utility scriptskind-env.sh— shared sourcing helper: sources.env.localto exports KUBECONFIG or other local overrides, pickspodmanif available or fallback todockersetup-kind.sh— idempotent cluster create/start with lockfile mutex for concurrent goreman processes (multiple runningopslevel-runnerinstances;- stop-kind.sh — deletes job pods, stops control-plane containerbuild-helper-image.sh— checksums Go sources + DockerfileAdded
tests/(moved from src/scripts/)goreman)Repository structure modifications
src/Dockerfilemove to repository root Dockerfiledist/not to pollutesrc/Notes for reviewers
opslevel-runner; override via positional arg to scripts.bin/setup-kind.shuses a transient lockfile to serialize concurrent cluster creation whengoremanstarts multiple processes in parallellocalhost/opslevel-runner:localkindkubernetes cluster for testing.Testing performed
goremanprocesses start, image-builder loads helper into kindCaveats
Setup on Linux not fully supported at this time.