Add comuline-krl-data topic + jakarta-public-transport-data umbrella

Second audit under the Jakarta public transport umbrella: comuline.com replicates KAI Commuter's official partner API (api-partner.krl.co.id); its data has been frozen since ~2025-03-30. Adds the umbrella hub, the comuline topic (sources/analysis/recipes/next-steps), and links the existing transjakarta-live-data topic under the umbrella.
2026-10-12 12:17:01 +00:00 · 2026-10-12 12:17:01 +00:00 · 77b0a55c41
commit 77b0a55c41
parent 3a518be39f
7 changed files with 214 additions and 1 deletions
--- a/comuline-krl-data/analysis.md
+++ b/comuline-krl-data/analysis.md
@ -0,0 +1,36 @@
+# Analysis — how comuline works
+
+## Pipeline
+
+1. **Sync (daily cron, as designed):** `comuline/api/src/sync/station.ts` fetches `{KRL_ENDPOINT_BASE_URL}/krl-station`; `schedule.ts` then iterates all stations in batches of 5 (5 s sleep between) — CORS `OPTIONS` preflight, then `GET /schedule?stationid={id}&timefrom=00:00&timeto=23:00`. Requests carry browser-like headers plus a hardcoded bearer token, because the upstream is an authenticated partner API.
+2. **Transform:** stations → `st_krl_<id>` records (upstream `WIL` entries filtered; airport station BST manually added); schedules → `sc_krl_<station>_<train>` rows with origin/destination station IDs resolved by name, `departs_at`/`arrives_at` timestamps, and line color; upserted into Postgres.
+3. **Serve:** Hono API on Cloudflare Workers (Neon Postgres + Upstash Redis per-station read cache) exposes clean JSON with an OpenAPI spec; web/native apps consume it.
+
+`KRL_ENDPOINT_BASE_URL` = `https://api-partner.krl.co.id/krl-webs/v1` (from the repo's `.dev.example.vars`).
+
+## Data model (public API)
+
+- **Station:** `{ uid, id (e.g. MRI), name, type (KRL|LOCAL), metadata.origin.{daop,fg_enable}, active }`
+- **Schedule row:** `{ id, station_id, station_origin_id, station_destination_id, train_id (e.g. 1448B), line ("COMMUTER LINE BOGOR"), route ("JAKARTAKOTA-DEPOK"), departs_at, arrives_at, metadata.origin.color }`
+- **Route view** (`/v1/route/{train_id}`): ordered stop sequence (station name + departure time) for one train.
+
+## Observed state (2026-10-12)
+
+- API live; 113 stations served. Row counts sampled: MRI 1,045 · DP 730 · CLT 683 · JAKK 367 · BOO 341 · TNG 84 · BEK 0 (station present, no schedule rows).
+- **Freshness:** the newest `updated_at` values are bulk 2024-11-13 and partial re-sync 2025-03-30; nothing newer. Repo activity stops the same month (`api` last push 2025-03-30). Conclusion: the daily cron is not currently running; the data is a frozen snapshot (~18 months old as of this audit).
+- **No realtime tracking:** no position/GPS data in the bundle or API. The product is schedule lookup with locally saved stations.
+
+## Boundaries respected (deliberate limits)
+
+- Upstream `api-partner.krl.co.id` was probed once **without credentials** (returned 404) to characterize the auth wall.
+- No calls were made using the token published in the public repo. The token is not reproduced in this vault.
+- No attempts to bypass auth, rate limits, or access controls.
+
+## Confidence
+
+- **Established:** pipeline structure and architecture (public source + live API); coverage counts; staleness timestamps; absence of live tracking; upstream URL and auth wall.
+- **Inferred:** why the sync stopped (project dormancy vs token expiry vs upstream change — not observable from outside); whether the upstream data model changed since.
+
+## Artifacts
+
+- Evidence archive: `workspace/comuline-captures/` — page HTML + headers, web bundle, live API samples (`stations.json`, `sched_MRI.json`, `openapi.json`), sync sources (bearer token REDACTED in the archive copy), `dev.example.vars`.
--- a/comuline-krl-data/index.md
+++ b/comuline-krl-data/index.md
@ -0,0 +1,42 @@
+---
+title: "Comuline — KRL Commuter Line Data (schedule API replication)"
+description: "Reverse-engineering comuline.com: an open-source KRL schedule app whose public API replicates KAI Commuter's official partner API (api-partner.krl.co.id). Covers the pipeline, current staleness (data frozen ~Mar 2025), and usable endpoints."
+status: concluded
+category: technical
+tags: [comuline, krl, commuter-line, kai-commuter, kci, schedule, api, jabodetabek, yogyakarta, jakarta-public-transport-data]
+draft: true
+created: 2026-10-12
+updated: 2026-10-12
+---
+
+# Comuline — KRL Commuter Line Data
+
+## Summary
+
+[comuline.com](https://comuline.com) is an open-source KRL commuter-line schedule app (web + iOS + Android) by Abiel Zulio M (`github.com/comuline`). Its "own public API replication" is a **daily ETL copy of KAI Commuter's official partner API** (`api-partner.krl.co.id/krl-webs/v1` — an official, token-gated API on the krl.co.id domain), re-served publicly at `api.comuline.com`.
+
+**Status twist (verified 2026-10-12):** the sync last ran ~2025-03-30. The public API still serves that frozen snapshot, and the project has been dormant since roughly the same month.
+
+## Key Findings
+
+- **Upstream is official, token-gated:** the sync fetches `/krl-station` and `/schedule?stationid=…&timefrom=00:00&timeto=23:00` from `api-partner.krl.co.id` with a bearer token hardcoded in the public repo (value not reproduced here). Unauthenticated probe (no token) returned HTTP 404; no credentialed upstream calls were made during this audit.
+- **Architecture:** daily cron → Cloudflare Workers (Hono + Bun) → Neon Postgres (Drizzle ORM) + Upstash Redis read cache → public REST API with OpenAPI spec → Vite/React web app (plus native apps) consuming it via a typed `openapi-fetch` client.
+- **Coverage (from live API):** 113 stations — 95 daop-1 (Jabodetabek), 17 daop-6 (Yogyakarta area), 1 daop-2 (Purwakarta); 111 typed KRL + 2 LOCAL (Cikampek, Purwakarta). Sample row counts: MRI = 1,045 departures; DP = 730; CLT = 683; JAKK = 367; TNG = 84; BEK = 0.
+- **No live tracking:** schedules and departures only — no GPS/position data anywhere (bundle + API verified). "Real-time" in their copy means "current schedule lookup", not live trains.
+- **Data frozen:** schedule rows show last updates 2024-11-13 (bulk) + 2025-03-30 (partial passes); nothing newer. Repo activity stops 2025-03-30 (api) / 2025-04-03 (web).
+
+## Files
+
+- [[comuline-krl-data/sources|Sources]] — repos, endpoints, upstream, ecosystem
+- [[comuline-krl-data/analysis|Analysis]] — pipeline, data model, observed state, boundaries
+- [[comuline-krl-data/recipes|Recipes]] — using the public API (curl)
+- [[comuline-krl-data/next-steps|Next steps]] — open questions, risks, ideas
+
+## Related Research
+
+- [[jakarta-public-transport-data/index|Jakarta Public Transport Data]] — umbrella topic
+- [[transjakarta-live-data/index|TransJakarta Live Data]] — sibling audit (realtime bus feed)
+
+## Discussion
+
+Triggered 2026-10-12 from a Telegram request ("comuline.com which is including krl/jabotabek train data here — can we reverse engineer this as well?"). Documented as the second branch under the Jakarta public transport data umbrella. Method: artifact inspection (site bundle, response headers), public source reading (GitHub), live API sampling, and freshness checks via `updated_at` fields.
--- a/comuline-krl-data/next-steps.md
+++ b/comuline-krl-data/next-steps.md
@ -0,0 +1,20 @@
+# Next steps / open questions
+
+## Open questions
+
+- Why did comuline's sync stop (~2025-03-30)? Project dormancy, token expiry, or upstream change — not observable from outside. (Author is reachable via the links in [[comuline-krl-data/sources|Sources]].)
+- Current shape of the upstream partner API's data model (`krl-station`, `schedule`) — unverified; no credentialed call was made.
+- **Live KRL tracking sources:** community trackers claim ~30 s updates (trackmycommuter, krljakarta.com, botkita). Where do their positions come from? A dedicated audit would be its own subtopic (KRL realtime — likely a different upstream than schedules).
+- MRT Jakarta / LRT Jabodebek data sources (`jakartamrt.co.id` API was referenced by one community project) — candidates for umbrella subtopics.
+
+## Risks
+
+- **Comuline API:** frozen data; treat as reference only. Add freshness checks; do not build a "current timetable" product on it without another source.
+- **Upstream partner API:** authenticated; using the repo-published token is not acceptable use. Legitimate access = partner agreement.
+- Both are informal dependencies that can change or disappear without notice.
+
+## Ideas
+
+- Freshness monitor for the comuline API (alert when data updates — would signal the project waking up or a re-sync).
+- Multi-modal view: join [[transjakarta-live-data/index|TransJakarta live feed]] + KRL schedules to prototype Jabodetabek trip planning.
+- If current KRL timetables are needed long-term: evaluate official channels (KAI Commuter app/website) or pursue partner access properly.
--- a/comuline-krl-data/recipes.md
+++ b/comuline-krl-data/recipes.md
@ -0,0 +1,40 @@
+# Recipes — Comuline public API
+
+Base: `https://api.comuline.com` — no auth, documented. Docs: `https://api.comuline.com/docs`; OpenAPI spec: `https://api.comuline.com/openapi`.
+
+**Caveat:** data is a frozen snapshot (last sync ~2025-03-30). Use it for station/route reference and historical schedule structure — not for current timetables.
+
+## Stations
+
+```bash
+# all stations
+curl -s https://api.comuline.com/v1/station | jq '.data[] | {id, name, daop: .metadata.origin.daop}'
+
+# one station (MRI = Manggarai)
+curl -s https://api.comuline.com/v1/station/MRI | jq '.data'
+```
+
+## Departures at a station
+
+```bash
+curl -s https://api.comuline.com/v1/schedule/MRI | jq '.data[0:5]'
+# rows: { train_id, line, route, station_origin_id, station_destination_id, departs_at, arrives_at, metadata.origin.color }
+```
+
+## One train's full stop sequence
+
+```bash
+curl -s https://api.comuline.com/v1/route/1448B | jq '.data.routes[] | {station_name, departs_at}'
+```
+
+## Status
+
+```bash
+curl -s https://api.comuline.com/status
+```
+
+## If you need CURRENT timetables
+
+- The upstream partner API (`api-partner.krl.co.id`) is token-gated. Do **not** reuse the token published in a public repo — obtain legitimate partner access if you need it.
+- Community alternatives (not audited here): `tarif-krl` (fares/schedules), `Fannandya/api-krl` (network-data SaaS), plus the live trackers listed in [[comuline-krl-data/sources|Sources]].
+- Always add a freshness check when consuming third-party transit APIs — compare `updated_at`/timestamps against today.
--- a/comuline-krl-data/sources.md
+++ b/comuline-krl-data/sources.md
@ -0,0 +1,24 @@
+# Sources
+
+★ = directly verified during this audit (Oct 12, 2026). Unmarked items are community context, not re-verified.
+
+## Comuline (the audited project)
+
+- ★ `https://comuline.com` — web app ("Cari & Simpan Jadwal KRL 2024"). Vite SPA on Cloudflare; analytics via `analytics.zulio.me`.
+- ★ `https://api.comuline.com` — public API (no auth). Docs at `/docs` (Scalar), OpenAPI spec at `/openapi`. Endpoints: `/v1/station`, `/v1/station/{id}`, `/v1/schedule/{station_id}`, `/v1/route/{train_id}`, `/status`.
+- ★ Source repos: `github.com/comuline/web` (Vite/React; openapi-fetch + swr-openapi; i18n default `id`), `github.com/comuline/api` (Hono + Bun; Drizzle + Neon Postgres + Upstash Redis; sync scripts under `src/sync/`), plus `android` (Kotlin) and `ios` (Swift).
+- ★ Author: Abiel Zulio M — `github.com/abielzulio`; prior project `abielzulio/jadwal-krl`.
+- ★ Repo activity: `api` last push 2025-03-30 (116★), `web` 2025-04-03 (95★), `android` 2025-08-22, `ios` 2024-09-02. None archived; all dormant on the data side.
+
+## Upstream (official KAI Commuter)
+
+- ★ `https://api-partner.krl.co.id/krl-webs/v1` — official partner API of KRL (krl.co.id = KAI Commuter domain). Endpoints used by comuline's sync: `GET /krl-station`, `GET /schedule?stationid={id}&timefrom=00:00&timeto=23:00`.
+  - Auth: Bearer JWT. A long-lived token is hardcoded in the public repo (`comuline/api/src/sync/headers.ts`); its value is [REDACTED] here and was not used in this audit. Unauthenticated probe returned HTTP 404.
+  - Sync also sends browser-like headers and does a CORS `OPTIONS` preflight before each GET.
+- Context: KRL (Commuter Line) is operated by PT Kereta Commuter Indonesia (KAI Commuter), subsidiary of PT KAI. "Daop" = KAI operational region (daop 1 = Jakarta, daop 2 = Bandung, daop 6 = Yogyakarta).
+
+## Related ecosystem (community, for context)
+
+- `github.com/vyonizr/tarif-krl` — KRL fare calculator "based on the official APIs: `api-partner.krl.co.id` and `jakartamrt.co.id`" — independent confirmation that the partner API is the de-facto official source.
+- `github.com/Fannandya/api-krl` — KRL network-data SaaS (registration, API keys, daily quotas; curated reference data).
+- Live-ish KRL trackers (different approach, NOT audited): `trackmycommuter.com`, `krljakarta.com` ("updates every 30 seconds"), `botkita.com/krl-tracker`.
--- a/jakarta-public-transport-data/index.md
+++ b/jakarta-public-transport-data/index.md
@ -0,0 +1,46 @@
+---
+title: "Jakarta Public Transport Data"
+description: "Umbrella topic for Jakarta-area public transport data — how each operator's realtime/schedule systems actually work, what is publicly reachable, and what is usable for future projects. Subtopics: TransJakarta live bus feed; KRL/Comuline schedules."
+status: active
+category: technical
+tags: [jakarta-public-transport-data, transit-data, jakarta, jabodetabek, transjakarta, krl, commuter-line, open-data, reverse-engineering]
+draft: true
+created: 2026-10-12
+updated: 2026-10-12
+---
+
+# Jakarta Public Transport Data
+
+## Summary
+
+Umbrella topic for **Jakarta-area public transport data**: what data each operator exposes, how third-party apps actually get it, and what is usable for future projects. Each system lives in a subtopic below; the shared thread is reverse-engineering public transport data pipelines and documenting them for reuse.
+
+## Subtopic Map
+
+- [[transjakarta-live-data/index|TransJakarta Live Data]] — bus lens: anonymous MQTT fleet stream + guest REST API + official GTFS; verified via cross-capture (audited Oct 2026)
+- [[comuline-krl-data/index|Comuline / KRL Schedule Data]] — rail lens: KRL Commuter Line schedules replicated from KAI Commuter's official partner API; currently a frozen snapshot (audited Oct 2026)
+
+## Research Question
+
+Which Jakarta public transport data sources are publicly reachable today, how do they work end-to-end, and which are stable enough to build on?
+
+## Key Findings
+
+- **TransJakarta buses: live and open (as of Oct 2026).** Anonymous MQTT broker (`wss://mqtt.tj.co.id:8084/mqtt`) + guest-token REST API + official GTFS give the full realtime fleet (~3.3–3.7k buses). Usable today.
+- **KRL commuter line: schedules only, and stale.** Comuline replicates KAI Commuter's partner API, but its sync stopped ~Mar 2025; its public API still serves that snapshot. No live train positions there.
+- **Pattern across both:** operators rarely publish developer-facing APIs; practical access comes from app backends, partner endpoints, and open replication projects that document the pipelines. Access is undocumented and can change without notice.
+
+## Related Research
+
+- [[transjakarta-live-data/index|TransJakarta Live Data]] — first audit (origin of this umbrella)
+- [[comuline-krl-data/index|Comuline / KRL Schedule Data]] — second audit
+
+## Discussion
+
+Branched 2026-10-12 per user request after two audits: TransJakarta (via `transit.oyenza.com`) and KRL schedules (via `comuline.com`). Both were already documented as separate topics; this umbrella gathers them under one home and is the place to add future systems (MRT Jakarta, LRT Jabodebek, TransJabodetabek, etc.).
+
+## Next Steps
+
+- [ ] Add subtopics for other Jabodetabek systems as they get audited (MRT Jakarta, LRT Jakarta/Jabodebek, TransJabodetabek)
+- [ ] Re-verify access status of both existing feeds periodically (undocumented; can change)
+- [ ] Keep per-system "verified on" dates current in subtopics
--- a/transjakarta-live-data/index.md
+++ b/transjakarta-live-data/index.md
@ -3,7 +3,7 @@ title: "TransJakarta live data — verified stack: MQTT stream, app REST API, GT
 description: "How TransJakarta realtime bus data works and how to use it — anonymous MQTT broker, guest REST API, official GTFS. Verified by cross-capture, Oct 2026."
 status: concluded
 category: technical
-tags: [transjakarta, mqtt, realtime, gtfs, reverse-engineering, transit-data]
+tags: [transjakarta, mqtt, realtime, gtfs, reverse-engineering, transit-data, jakarta-public-transport-data]
 draft: true
 created: 2026-10-12
 updated: 2026-10-12
@ -41,3 +41,8 @@ updated: 2026-10-12
 - [[transjakarta-live-data/methodology|Methodology]] — how it was verified (repeatable)
 - [[transjakarta-live-data/recipes|Recipes]] — connection examples
 - [[transjakarta-live-data/next-steps|Next steps]] — risks + future ideas
+
+## Related Research
+
+- [[jakarta-public-transport-data/index|Jakarta Public Transport Data]] — umbrella topic
+- [[comuline-krl-data/index|Comuline / KRL Schedule Data]] — sibling audit (rail schedules)