Skip to content

fix: include paykit native symbols#1106

Open
ovitrif wants to merge 1 commit into
masterfrom
ovi/fix-native-symbol-build-ids
Open

fix: include paykit native symbols#1106
ovitrif wants to merge 1 commit into
masterfrom
ovi/fix-native-symbol-build-ids

Conversation

@ovitrif

@ovitrif ovitrif commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Fixes #1104

Description

This PR adds Paykit to Bitkit Android's existing native debug-symbol archive:

  1. Resolves Paykit's native-debug-symbols classifier alongside the existing bitkit-core, LDK, and VSS symbol artifacts.
  2. Requires libpaykit.so for both release ABIs before creating the Play upload archive.

pubky/paykit-rs#124 supplies the build-ID-enabled Paykit release. This PR will adopt that released coordinate before merge.

Preview

N/A — release artifact packaging only.

QA Notes

Manual Tests

N/A

Automated Checks

  • just compile
  • just test
  • just lint
  • just test file NativeReleaseConfigTest
  • sh -n scripts/create-native-debug-symbols.sh
  • shellcheck scripts/create-native-debug-symbols.sh
  • Authenticated ./gradlew :app:syncNativeDebugSymbolArtifacts --no-daemon --stacktrace
  • scripts/create-native-debug-symbols.sh generated native-debug-symbols-187.zip with all four required libraries for arm64-v8a and armeabi-v7a
  • NDK llvm-readelf -S confirmed full DWARF metadata in Paykit's symbol library for both release ABIs

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown

Greptile Summary

Adds release-time verification that native debug symbols contain GNU build IDs matching the libraries packaged in both release artifacts.

  • Adds Paykit to the native debug-symbol dependencies and required libraries.
  • Builds the release APK and AAB before symbol validation.
  • Validates DWARF metadata and build-ID parity across both release ABIs.
  • Adds executable fixtures and configuration tests for missing and mismatched build IDs.

Confidence Score: 5/5

The PR appears safe to merge, with no blocking failures eligible for this follow-up review.

No blocking failure remains.

Important Files Changed

Filename Overview
.github/workflows/release-internal.yml Builds both release artifact formats before running native-symbol validation.
app/build.gradle.kts Adds Paykit's native debug-symbol classifier to the resolved release artifacts.
scripts/create-native-debug-symbols.sh Requires four native libraries, validates GNU build IDs, and compares symbol copies against the final AAB and universal APK.
scripts/test-create-native-debug-symbols.sh Adds isolated shell fixtures covering absent GNU notes, mismatched IDs, and successful AAB/APK parity.
app/src/test/java/to/bitkit/build/NativeReleaseConfigTest.kt Expands release configuration coverage and executes the native-symbol fixture test.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Build release APK and AAB] --> B[Resolve four native-symbol archives]
    B --> C[Merge symbol libraries by ABI]
    C --> D[Verify DWARF metadata and GNU build IDs]
    D --> E[Extract packaged libraries from AAB and APK]
    E --> F[Compare packaged and symbol build IDs]
    F --> G[Create native debug-symbol archive]
Loading

Reviews (2): Last reviewed commit: "fix: verify native release build ids" | Re-trigger Greptile

@coreyphillips coreyphillips left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script itself holds up well. I traced it against dash, macOS sh, NDK 28 llvm-readelf and Info-ZIP, and these are all correct and need no changes:

  • build_id() awk parse. Real llvm-readelf -n puts the NT_GNU_BUILD_ID marker and the hash on separate lines, so $3 on the Build ID: line is the hash and the next never skips it. binutils readelf uses the identical two-line form, so the GNU fallback in find_readelf parses the same.
  • require_build_id's exit 1 inside a command substitution. POSIX gives an assignment the status of its last command substitution, and dash propagates it under set -e. A probe where every packaged lib lacks NT_GNU_BUILD_ID aborts with status 1 rather than passing vacuously. validate_symbol_tree also always runs first at both call sites and calls require_build_id in the current shell, so the symbol side can never be empty when parity runs.
  • llvm-strip preserves .note.gnu.build-id, so packaged-to-symbol parity is satisfiable in principle.
  • The fixture's trap re-installation and temp-dir cleanup, and the hardcoded AAB name against the FinalizeBundleTask rename block.
  • The LightningService change is forced, not scope creep: ElectrumSyncConfig gained additional_wallet_full_scan_batch_size and additional_wallet_full_scan_stop_gap in ldk-node rc.55 with no UDL defaults, so the generated Kotlin constructor requires both. The /** LDK's default */ KDoc is literally accurate against src/config.rs (5 and 20). Worth saying so in the PR body, since as written a reader cannot tell a forced binding update from an unrelated wallet-config change.

Blocking status is exactly as the body says. All four upstream PRs are still open and none of the four pinned versions are published (latest: ldk-node v0.7.0-rc.57, bitkit-core v0.4.3, vss-rust-client-ffi v0.5.20, paykit v0.1.0-rc39). That fully explains the red CI and is not a defect here.

Five things to resolve before this lands.

1. The paykit line needs a decision

0.1.1 is not rc33 plus six RCs. pubky/paykit-rs#124 targets release/0.1.1, and the CHANGELOG on that branch goes straight from [Unreleased] to [0.1.0-rc33], so rc34 through rc39 are not in it.

Meanwhile four open PRs here are on the newer line: codex/paykit-incoming-payment-requests-rc39 and codex/batch-paykit-contact-cleanup on rc39, codex/970-pubky-paykit-ui-parity and codex/paykit-watch-only-accounts on rc37. rc37 is breaking (session signup/signin/auth require the receiver Noise key explicitly) and rc38/rc39 are breaking (no implicit private-to-public fallback, callers choose the payment mode).

Whichever merges second loses something: keep 0.1.1 and the rc39 call sites no longer exist, keep 0.1.0-rc39 and create-native-debug-symbols.sh aborts at require_build_id with no NT_GNU_BUILD_ID for libpaykit.so. Can we get a build-ID-enabled release cut on top of rc39, or land this with paykit left at rc33 and add libpaykit.so to required_libs in a follow-up once the lines converge?

2. ADDITIONAL_WALLET_FULL_SCAN_STOP_GAP = 20u contradicts the derived-account branches

The four in-flight branches carrying the watch-only work all set 100/1000 for these. If one of them rebases onto master after this lands, the KDoc makes 20 look authoritative, and a Trezor account whose owner previously received beyond 20 consecutive unused addresses would come back with a missing balance.

Was 5/20 a deliberate divergence from the 100/1000 those branches already agreed on, or just preserving pre-rc.55 behavior? If the latter, worth either matching them or leaving a comment tying the value to that work so a later rebase does not silently keep 20.

3. The ldk-node jump needs wallet QA

rc.52 to rc.58 crosses six release candidates. rc.54's notes cover "corrected RBF input/change selection, replacement fee enforcement, foreign-input weight estimation, and reserve calculations across aggregated wallets", which touches boost and coin selection directly. rc.55/rc.56 add rolling receive-address lookahead for derived accounts.

The QA notes here are all symbol and build checks plus a launch on device. Could you add boost, send with anchor reserves, and restore-from-seed balance on regtest, or split the ldk-node bump into its own PR so it gets reviewed as a wallet change rather than as tooling?

4. QA note names an ABI the gate does not check

The notes end with "final build-ID parity passed for arm64-v8a and x86_64". validate_packaged_build_id_parity loops arm64-v8a armeabi-v7a, and the release abiFilters are armeabi-v7a plus arm64-v8a (x86_64 is debug only), so x86_64 is neither packaged nor inspected, and armeabi-v7a is not mentioned. Since CI cannot run here, the QA notes are the only evidence the gate was exercised end to end, and armeabi-v7a is the ABI most likely to diverge because it is a separate toolchain.

5. Changelog fragment

Native build IDs are release tooling rather than something a user observes, while the parts of this PR that are user facing (ldk-node rc.54's RBF and coin-selection corrections, bitkit-core 0.4.1 to 0.4.4) are not mentioned. Either drop the fragment or rewrite it to cover the user-visible change.

Minor, on naming: every other fragment in changelog.d/next/ is keyed to the PR number, including 1091.fixed.md which fixes issue #1034. 1104.fixed.md is the only one keyed to an issue.

Conflicts with #1107

git merge-tree gives conflicts in scripts/create-native-debug-symbols.sh, app/src/test/java/to/bitkit/build/NativeReleaseConfigTest.kt and gradle/libs.versions.toml. release-internal.yml and app/build.gradle.kts auto-merge cleanly.

The dangerous hunk: this PR adds libpaykit.so to required_libs, #1107 splits the same line into dependency_required_libs plus application_required_libs. Taking #1107's side drops libpaykit.so from the release symbol gate silently, and NativeReleaseConfigTest then fails whichever side is taken, where the least-effort green fix is to delete libpaykit.so from the script and cement the gap. Correct resolution:

dependency_symbols_dir=${NATIVE_SYMBOLS_DEPENDENCY_DIR:-app/build/intermediates/native-debug-symbol-artifacts}
dependency_required_libs="libbitkitcore.so libldk_node.so libpaykit.so libvss_rust_client_ffi.so"
application_required_libs="libimage_processing_util_jni.so libsurface_util_jni.so libandroidx.graphics.path.so"
required_libs="$dependency_required_libs $application_required_libs"

with the test assertion updated to match and both new @Test methods kept.

Also worth noting: #1107's check-16kb-compat.sh validates every .so in the final artifacts, including all four Rust libraries this PR bumps, and that gate runs only on release paths, never on PRs. I checked the current versions (ldk-node rc.52, paykit rc33, vss-client 0.5.20, bitkit-core) and all pass on both ABIs, but this PR replaces all four. Suggest running llvm-readelf -W -l and llvm-readelf -n over the jni/arm64-v8a and jni/armeabi-v7a .so files in the four new AARs once they publish, and pasting the results into the QA notes, so an unaligned or build-ID-less lib is caught before a release rather than during one.

@ovitrif
ovitrif force-pushed the ovi/fix-native-symbol-build-ids branch from 03758ad to fe83b1e Compare July 25, 2026 16:12
@ovitrif
ovitrif requested a review from coreyphillips July 25, 2026 16:13
@ovitrif

ovitrif commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — I reduced this PR to the Android release-symbol consumer change.

  • The rewritten head removes the four unpublished dependency-coordinate updates and the LightningService wallet-scan changes.
  • Paykit's symbol classifier and libpaykit.so remain required across both release ABIs.
  • The final script validates full DWARF metadata, GNU build-ID presence, and AAB/APK-to-symbol parity for all four Rust libraries before producing the upload archive.
  • Current master and the DataStore-only fix: update DataStore for 16 KB compatibility #1107 both merge conflict-free.
  • just compile, just test, just lint, the shell fixture, shellcheck, actionlint, and two fresh exact-head review passes are clean.

All four producer PRs remain required. Once their artifacts are published, Android will adopt the released coordinates through a dedicated dependency review and rerun the full release integration. That keeps the LDK 0.7.0-rc.52 to 0.7.0-rc.58 wallet migration visible and separately reviewable.

@ovitrif
ovitrif force-pushed the ovi/fix-native-symbol-build-ids branch from fe83b1e to c67ef8b Compare July 25, 2026 18:15
@ovitrif
ovitrif requested review from ben-kaufman and coreyphillips and removed request for ben-kaufman and coreyphillips July 25, 2026 18:17
@ovitrif

ovitrif commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator Author

I reopened this PR as the Android consumer in the separate Android ELF build-ID workstream tracked by #1105. GNU build IDs let Google Play match packaged native libraries to their uploaded debug symbols, enabling native symbolication as the outcome. This workstream remains independent of #1047, whose product correction is the DataStore-only #1107. Corey's simplification feedback remains active, so this PR is back in progress.

@ovitrif ovitrif closed this Jul 25, 2026
@ovitrif ovitrif reopened this Jul 25, 2026
@ovitrif
ovitrif force-pushed the ovi/fix-native-symbol-build-ids branch from c67ef8b to 3ee6b13 Compare July 26, 2026 16:56
@ovitrif ovitrif changed the title fix: verify native release build ids fix: include paykit native symbols Jul 26, 2026
@ovitrif

ovitrif commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — I reduced this PR to the Paykit consumer gap only.

  • The diff now changes three files: resolve Paykit's existing symbol classifier, require libpaykit.so, and assert that configuration.
  • The generalized build-ID framework, APK/AAB workflow expansion, dependency pins, wallet changes, changelog, and shell fixture are removed.
  • Paykit 0.1.0-rc33 already supplies full-DWARF symbols for both release ABIs, and the existing generator includes them in the final archive.
  • fix: add android elf build ids pubky/paykit-rs#124 remains the release dependency that adds GNU build IDs; this PR will adopt that released coordinate before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: include Paykit symbols for build-ID matching

2 participants