research/gopay-merchant-qris-gateway/reverse-engineering.md

103 lines
11 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
title: Reverse engineering — methods & drift playbook
description: How the GoPay/GoBiz merchant APIs were reverse engineered — browser HAR captures and byte-for-byte replay, APK static analysis (apktool/androguard/strings), code archaeology, version-drift tracking, and the ethics/hygiene rules the researchers follow.
status: concluded
category: technical
tags: [reverse-engineering, har-capture, apk-analysis, gobiz, gopay, methodology]
draft: true
created: 2026-10-15
updated: 2026-10-15
---
# Reverse engineering — methods & drift playbook
> Part of [[gopay-merchant-qris-gateway/index|GoPay Merchant QRIS gateways]]
## 1. The three evidence channels
The API knowledge behind these gateways came from three complementary methods — plus one public spec that needed no reverse engineering at all:
| Channel | Object | Output | Fidelity |
|---|---|---|---|
| **A. Browser HAR captures** | `portal.gofoodmerchant.co.id` sessions (merchant's own login) | Exact request/response shapes, headers, token formats | **Highest** — live traffic, byte-for-byte |
| **B. APK static analysis** | GoPay Merchant 2.3.0 (APKM), GoFood Merchant 5.49.0 (XAPK) | ~250 endpoint paths, hosts, anti-tamper map, sister-app evidence | Static strings only — mostly **runtime-unverified** |
| **C. Code archaeology** | Earlier open-source repos (kavionn, warungerik, namtxs, alhifnywahid) | Hypotheses, working patterns, prior art | Variable — often unverified, to be capture-checked |
| **EMVCo QRIS spec** | Public standard | TLV structure, tag 54, CRC16 rules | N/A (public) |
All methods operate on the researchers' **own** merchant accounts and publicly distributed artifacts; no backend probing, no third-party data. [S9 §5][S11]
## 2. Channel A — HAR captures & byte-for-byte replay
Documented captures (all on the merchant's own account) [S6 §9, S7, S13]:
| Capture | Client | Size / entries | What it verified |
|---|---|---|---|
| `gobizverbrowser.har` | iPhone WebView (`mark.via.gp`) → portal | ~80 MB, 1063 entries, 77 API calls | OTP flow, merchants/search, users/me, analytics, journals (+aggregation variant), payouts — **plus 5 endpoints no repo had** |
| `GOTO.har` (19 Sep 2026) | Firefox desktop | ~51 MB, 589 entries (2-min session) | OTP login drift: `X-AppVersion` bump, bare phone format, split headers confirmed |
| `emialku.har.json` (19 Sep 2026) | Reqable | 169 entries (19 s) | Email+password flow verified end-to-end |
| Browser HAR (3 Oct 2026) | Browser → portal | (size not stated) | New build `v3.125.0`; auth calls moved to the portal host [S13] |
Method, as documented in the repo's reproduction guide and reports [S6][S7][S9]:
1. **Capture the portal session** with a normal browser/devtools export (Chrome/Firefox/WebView/Reqable) — i.e. record the merchant's own login and dashboard usage.
2. **Map every host/path**, isolate the merchant API calls (e.g. 30 calls to `api.gobiz.co.id` out of 589 entries), and pick the flows of interest.
3. **Diff captured requests against client code** header-by-header and field-by-field. Every mismatch is either a bug or drift; every match upgrades the client claim from [reported] to [verified].
4. **Cross-check the data endpoints** (`merchants/search`, `journals/search`, payouts, analytics) against the SDK's assumptions.
5. **Fix minimally, mirror the browser byte-for-byte** — the repo's stated principle ("tiru browser, byte-per-byte").
**Credential hygiene is explicit** — captures contain live access/refresh tokens, phone numbers, email, `otp_token` [S7 preamble]: HAR files are `.gitignore`d and kept outside the repo; every sensitive value is masked in reports ("format only, e.g. 11-digit bare national digits"); APKs are deleted after analysis and only hashes retained. [S9 §3.9, S11]
## 3. The drift playbook (what breaks, and how it was found)
The portal is a moving target; the documented fixes read like a clinician's notes [S6 §9, S7, S13]:
1. **`X-AppVersion` bumps** (the client build string sent on `/goid/*`): `platform-v3.107.0` (reference repos, Jun–Jul 2026) → `v3.119.0` (mid-Sep capture) → `v3.122.0` (19 Sep) → `v3.125.0` (3 Oct). Evidence that the server gates on it: an OTP flow that "doesn't work" was fixed by bumping nothing but this string + phone format. [S7 §3.1/§3.3/§4]
2. **Phone number normalisation:** the portal always sends **bare national digits** (`851…`); passing `0851…` or `+62851…` through produced server-side failures. Fixed with a normaliser that strips `+`, `62`, one leading `0`. [S7]
3. **`login_type` is vestigial:** reference repos send `login_type:"otp"` on the OTP request; the browser sends **no such field**; server ignores it either way. QMID removed it to match the browser exactly. [S6 §9.2]
4. **Auth host moved:** by Oct 2026 the browser sends `/goid/login/request` + `/goid/token` to `https://portal.gofoodmerchant.co.id` while data APIs stay on `https://api.gobiz.co.id`. Clients updated to route per-call. [S13][S10]
5. **Request-body drift:** the portal started adding `_source:[…]` to `merchants/search`; the older bare `{from,size}` shape was kept after being verified non-broken. [S7 §3.4]
6. **Split-header discipline:** device headers (`X-AppVersion`, `X-PhoneModel`, …) appear only on `/goid/*`; sending them everywhere or nowhere is a fingerprint mismatch. The SDK mirrors the split. [S7 §3.3]
[inference] The pattern across all six: GoBiz tolerates a *plausible* browser, not an arbitrary client. The RE work is less "break the gate" and more "stay indistinguishable from a current browser session" — an ongoing treadmill that explains why these repos need continuous maintenance and why version bumps break communities of users at once.
## 4. Channel B — APK static analysis
The QrisMerchantID project ran a full static-analysis pipeline on both merchant apps and published both the results **and the reproduction guide** [S8][S9]. Pipeline (from the guide) [verified — commands in S9]:
1. Download the container from APKMirror/APKPure (APKM for GoPay Merchant 2.3.0 — `com.gojek.gopaymerchant`; XAPK for GoFood Merchant 5.49.0).
2. Hash everything (`md5sum`/`sha1sum`/`sha256sum`), verify signature/certificates with **androguard** (`CN=Gojek` cert 2014–2039 + Google Play App Signing; v2-signed).
3. `apktool d` decode → smali; `strings` on the Flutter AOT blob (`libapp.so`, 22 MB) for the Dart-half of the app.
4. Grep for URL/host patterns in smali + libapp strings → **endpoint inventory**; scan for secrets (with mandatory masking), anti-tamper indicators (frida/xposed/root/ptrace/Play Integrity counts), crypto usage.
5. Write an honest report: findings graded (0 critical), everything unproven marked `BELUM TERVERIFIKASI` + TODO, dynamic analysis explicitly skipped (no device/emulator available), tools recorded for reproducibility.
Outputs relevant here [S8][S12]:
- ~150 GoPay-Merchant-app endpoints under groups: histories/reconciliation (`/api/v1/unified-histories`, `/api/v2/histories`), finance (`/api/v1/finance/balance/*`), transactions & QRIS (`/api/transactions/charge/qris-link`, `/api/v1/qris/shared`), payouts, auth (`/api/auth/token`, `/api/v1/login/complete`, `/api/v1/totp`), PIN endpoints, configs.
- Realtime **WebSocket** channel: `wss://gopay-merchant-raccoon.gojekapi.com/api/v1/events` (protobuf `odpf.raccoon.v1beta1`) — the app's push path, an eventual replacement for polling **if** someone traffic-verifies it.
- Hosts incl. `gopaymerchant.midtrans.com` (Midtrans gateway), `customer.gopayapi.com`, `app.gwk.gopayapi.com`, SSO `accounts.goto-products.com`.
- Anti-tamper is heavy (frida/xposed/root/emulator/Play Integrity checks in the app), so dynamic hooking of the *app* is hard; the *web portal* path (Channel A) avoided all of that, which is [inference] the strategic reason the tooling standardized on portal replay instead of app interception.
- **Sister-app evidence:** GoPay Merchant app and GoFood Merchant app share the same `CN=Gojek` signing certificate, 6 identical PIN SDK endpoint paths, and a shared module set — and GoPay Merchant's T&C links point at GoBiz. The maintainer of `gobiz-payment` had answered a user question (issue #3) that GoPay Merchant transactions are visible on the GoBiz web for the same account; the APK work independently confirmed "same account/system, different API gateways". [S8 §5.4][S4]
**Honesty caveat, verbatim in spirit:** the endpoint CSV marks all ~249 entries as *static strings, runtime use unverified* (file column values are all `BELUM TERVERIFIKASI`). Only Channel-A endpoints are capture-verified. [S12]
## 5. Channel C — code archaeology & cross-verification
The flagship research doc [S6] shows the working method: clone and read the prior repos (kavionn's `gobiz.js` read fully; warungerik's user-tested gateway; `merchantid` for ShopeePay; `namtxs/gopay-api` as the customer-side comparator), then **verify every inherited claim against captures** before shipping code. Examples of corrections this produced: `login_type` (dropped), `X-AppVersion` (bumped), phone format (normalized), analytics/journal shapes (confirmed 1:1). Unverified claims kept labelled `TODO`. Prior art is credited (README credits list) as a hard project rule. [S1][S6]
## 6. What the reverse engineering produced — and what it couldn't
**Produced** [verified]: a complete, capture-backed map of the merchant web-portal API subset (auth, merchants, analytics, journals, payouts), a static endpoint map of the mobile apps, and a working pattern (dynamic QRIS + watcher) replicated across at least 8 independent repos within ~4 months.
**Couldn't / didn't** [unverified, per the sources' own TODOs] [S15]:
- the app-side live flow (needs device + frida/mitmproxy; app has anti-tamper);
- the realtime WebSocket event protocol (identified, never hooked);
- request signing specifics — no static signing secret exists in the client and none was observed on the web path, so "how the server authenticates beyond the Bearer token" is simply *not answered*;
- impact of the app's embedded Google API key / staging hosts (static indicators, never tested).
## 7. Counterpoint: reverse engineering vs. readable results
The knowledge was gained by replaying the *merchant's own browser session*, but the flagship gateway repo (`gopay-api-gateaway`) then **closed the loop by obfuscating its own auth code** (`login.js`, `sessionManager.js` built with `javascript-obfuscator`; no plain strings leak). So the community can reuse the public API knowledge, but cannot audit what the most popular implementation does with the credentials it collects — a distribution choice, not an RE constraint. `kavionn/gobiz-payment` and `QrisMerchantID` went the other way: fully readable code + published methodology. [S3][S2][S1]
## 8. Assessment
[inference] This is a textbook example of **gray-box research on one's own entitlements**: using (a) your own account, (b) your own traffic, (c) publicly distributed apps and code — while staying away from probing, credential testing, and third-party data. The research outputs are unusually well-documented for this niche (reproduction guide, conflict register, responsible-disclosure doc, masking rules). The remaining ethical question isn't the method — it's the use: the same knowledge automates *your own* merchant account, and nothing in the pipeline prevents someone from pointing it at an account that isn't theirs (stolen session → payouts/balance visibility). Nothing in the audited repos encourages that; all three carry disclaimers, and the gateways self-host credentials rather than relaying them to a SaaS. [S1][S11]