Skip to content

feat: authenticated IPFS uploads with client-side CID computation#289

Merged
odesenfans merged 2 commits into
mainfrom
od/authenticated-ipfs-uploads
Jun 11, 2026
Merged

feat: authenticated IPFS uploads with client-side CID computation#289
odesenfans merged 2 commits into
mainfrom
od/authenticated-ipfs-uploads

Conversation

@odesenfans

@odesenfans odesenfans commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Step 3 of the parity plan with the Rust SDK: authenticated IPFS uploads, with CIDs computed client-side via the new aleph-cid package (PyO3 bindings over the Rust reference implementation, see aleph-im/aleph-rs#263).

What changes

  • create_store(storage_engine=StorageEnum.ipfs) no longer does the legacy two-step (unauthenticated ipfs_push_file, then POST /messages). It now mirrors the storage engine path: compute the kubo-default CIDv0 locally, sign the STORE message, and send file + message in one multipart request to /api/v0/ipfs/add_file. Same double-broadcast safety net as the native path for older nodes.
  • New create_store_folder(folder_path, ...): packs a folder into a CARv1 file locally (UnixFS DAG with the exact root CIDv1 kubo would assign, HAMT sharding included) and posts it with the signed message to /api/v0/ipfs/add_car. CAR packing runs off the event loop via asyncio.to_thread (the Rust hasher releases the GIL).
  • _storage_push_file_with_message generalized to _push_file_with_message(url, ...) with optional file name and content type; the storage path uses it unchanged.
  • aleph-cid>=0.1 added as a required dependency (abi3 wheels for CPython 3.10+ on Linux x86_64/aarch64, macOS, Windows; sdist builds from source elsewhere). It is imported unconditionally: it has no system-library component, so if pip resolved the SDK, the import works. The legacy unauthenticated ipfs_push_file method remains available as public API.

Tests

tests/unit/test_authenticated_ipfs_upload.py (step 4 of the plan, golden parity):

  • compute_cid(b"") pinned to the kubo golden QmbFMke1...AwQH.
  • Authenticated file upload: asserts the multipart request carries the file and the signed message, and that item_content.item_hash is the locally computed CIDv0.
  • Folder upload: nested-directory fixture pinned to the kubo golden root bafybeidcclyz..., and the posted CAR is read back with read_carv1_root to confirm header root == message CID.
  • Rejection of non-directories for folder uploads.

All pass; the rest of the unit suite is unchanged vs main (the pre-existing test_vm_client.py failures with latest aiohttp + aioresponses 0.7.6 are unrelated). ruff, black, isort, pyproject-fmt, and mypy are clean on the changed files.

Notes for reviewers

  • The CIDs this produces are verified against the same kubo v0.30.0 golden vectors as the Rust SDK, so a STORE message for a given file or folder now has the identical item_hash regardless of which SDK created it.
  • /api/v0/ipfs/add_car requires a CCN that exposes it; create_store itself stays compatible with older nodes via the second broadcast.

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown

Failed to retrieve llama text: POST 502: Bad Gateway

foxpatch-aleph
foxpatch-aleph previously approved these changes Jun 11, 2026

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-structured PR implementing authenticated IPFS uploads with client-side CID computation via the new aleph-cid package. The code follows existing patterns, gracefully degrades when aleph-cid is unavailable, and includes thorough golden-vector tests. The double-broadcast safety net mirrors the native storage path. A minor note: create_store_folder is not declared in the abstract base class, but this is acceptable for a new feature.

src/aleph/sdk/client/authenticated_http.py (line 828): The asyncio.to_thread call here is correct since the Rust extension releases the GIL. However, note that the entire tempfile.TemporaryDirectory context manager wraps this — the car_path is created inside tmp_dir and read before exiting. This lifecycle is safe.

tests/unit/test_authenticated_ipfs_upload.py (line 34): The helper reaches into form._fields which is a private implementation detail of aiohttp.FormData. Pragmatic for testing, but fragile across aiohttp versions.

Bring create_store with the ipfs storage engine up to par with the
storage engine and the Rust SDK: the file CID is computed locally with
the aleph-cid package (kubo-default CIDv0), and the file travels with
the signed STORE message in a single multipart request to
/api/v0/ipfs/add_file. The legacy unauthenticated path remains as a
fallback when aleph-cid is not importable.

Also adds create_store_folder: packs a folder into a CARv1 file locally
(CIDv1 root, identical to kubo) and posts it with the signed message to
/api/v0/ipfs/add_car.

The shared upload helper _storage_push_file_with_message is generalized
to _push_file_with_message(url, ...) with optional file name and content
type. Tests pin the CIDs against kubo golden vectors from the aleph-cid
test suite.
@odesenfans odesenfans force-pushed the od/authenticated-ipfs-uploads branch from 8bff8a4 to e9ef68a Compare June 11, 2026 21:18
aleph-cid is a required dependency and has no system-library component,
so a guarded import could only trigger in environments that break the
declared dependencies, and silently downgrading to an unauthenticated
upload there would mask the problem. Fail loudly instead.

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean implementation of authenticated IPFS uploads with client-side CID computation via the new aleph-cid library. The code mirrors the existing native storage path pattern, the tests are thorough with golden CID vectors pinned against real kubo output, and the dependency addition is properly scoped. No correctness, security, or quality issues found.

@odesenfans odesenfans merged commit 5fb68ad into main Jun 11, 2026
37 checks passed
@odesenfans odesenfans deleted the od/authenticated-ipfs-uploads branch June 11, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants