feat: authenticated IPFS uploads with client-side CID computation#289
Conversation
|
Failed to retrieve llama text: POST 502: Bad Gateway |
foxpatch-aleph
left a comment
There was a problem hiding this comment.
Well-structured PR implementing authenticated IPFS uploads with client-side CID computation via the new aleph-cid package. The code follows existing patterns, gracefully degrades when aleph-cid is unavailable, and includes thorough golden-vector tests. The double-broadcast safety net mirrors the native storage path. A minor note: create_store_folder is not declared in the abstract base class, but this is acceptable for a new feature.
src/aleph/sdk/client/authenticated_http.py (line 828): The asyncio.to_thread call here is correct since the Rust extension releases the GIL. However, note that the entire tempfile.TemporaryDirectory context manager wraps this — the car_path is created inside tmp_dir and read before exiting. This lifecycle is safe.
tests/unit/test_authenticated_ipfs_upload.py (line 34): The helper reaches into form._fields which is a private implementation detail of aiohttp.FormData. Pragmatic for testing, but fragile across aiohttp versions.
Bring create_store with the ipfs storage engine up to par with the storage engine and the Rust SDK: the file CID is computed locally with the aleph-cid package (kubo-default CIDv0), and the file travels with the signed STORE message in a single multipart request to /api/v0/ipfs/add_file. The legacy unauthenticated path remains as a fallback when aleph-cid is not importable. Also adds create_store_folder: packs a folder into a CARv1 file locally (CIDv1 root, identical to kubo) and posts it with the signed message to /api/v0/ipfs/add_car. The shared upload helper _storage_push_file_with_message is generalized to _push_file_with_message(url, ...) with optional file name and content type. Tests pin the CIDs against kubo golden vectors from the aleph-cid test suite.
8bff8a4 to
e9ef68a
Compare
aleph-cid is a required dependency and has no system-library component, so a guarded import could only trigger in environments that break the declared dependencies, and silently downgrading to an unauthenticated upload there would mask the problem. Fail loudly instead.
foxpatch-aleph
left a comment
There was a problem hiding this comment.
Clean implementation of authenticated IPFS uploads with client-side CID computation via the new aleph-cid library. The code mirrors the existing native storage path pattern, the tests are thorough with golden CID vectors pinned against real kubo output, and the dependency addition is properly scoped. No correctness, security, or quality issues found.
Step 3 of the parity plan with the Rust SDK: authenticated IPFS uploads, with CIDs computed client-side via the new
aleph-cidpackage (PyO3 bindings over the Rust reference implementation, see aleph-im/aleph-rs#263).What changes
create_store(storage_engine=StorageEnum.ipfs)no longer does the legacy two-step (unauthenticatedipfs_push_file, thenPOST /messages). It now mirrors the storage engine path: compute the kubo-default CIDv0 locally, sign the STORE message, and send file + message in one multipart request to/api/v0/ipfs/add_file. Same double-broadcast safety net as the native path for older nodes.create_store_folder(folder_path, ...): packs a folder into a CARv1 file locally (UnixFS DAG with the exact root CIDv1 kubo would assign, HAMT sharding included) and posts it with the signed message to/api/v0/ipfs/add_car. CAR packing runs off the event loop viaasyncio.to_thread(the Rust hasher releases the GIL)._storage_push_file_with_messagegeneralized to_push_file_with_message(url, ...)with optional file name and content type; the storage path uses it unchanged.aleph-cid>=0.1added as a required dependency (abi3 wheels for CPython 3.10+ on Linux x86_64/aarch64, macOS, Windows; sdist builds from source elsewhere). It is imported unconditionally: it has no system-library component, so if pip resolved the SDK, the import works. The legacy unauthenticatedipfs_push_filemethod remains available as public API.Tests
tests/unit/test_authenticated_ipfs_upload.py(step 4 of the plan, golden parity):compute_cid(b"")pinned to the kubo goldenQmbFMke1...AwQH.item_content.item_hashis the locally computed CIDv0.bafybeidcclyz..., and the posted CAR is read back withread_carv1_rootto confirm header root == message CID.All pass; the rest of the unit suite is unchanged vs main (the pre-existing
test_vm_client.pyfailures with latest aiohttp + aioresponses 0.7.6 are unrelated). ruff, black, isort, pyproject-fmt, and mypy are clean on the changed files.Notes for reviewers
item_hashregardless of which SDK created it./api/v0/ipfs/add_carrequires a CCN that exposes it;create_storeitself stays compatible with older nodes via the second broadcast.🤖 Generated with Claude Code