Reconsider this approuch

[PGTBoos]

Wouldnt it be better to join and extend existing libraries, like pytorch , tensorflow, and some others ?.
Then you could do IF nvidea hardware do this api call... if not manual python call
That way a lot of existing code could more easily adapt to it.
And Python stays a uniform language acros HW.

[carterbox]

My understanding is you propose that NVIDIA should select a few deep learning frameworks and contribute to those instead of developing nvmath.

NVIDIA already does contribute to the frameworks you have mentioned, but our customers are not only machine learning framework users. We support other domains in scientific computing where a deep learning framework may not be appropriate. While we do support torch tensors as input/output tensors, we also aim to accept any tensor which implements NumPy's array interface protocol, Cupy's array interface protocol, and in the future, DLPack. Our strategy is to expose functionality to the python ecosystem that isn't currently available via existing tensor libraries. Contributing to a few existing tensor libraries instead of providing functions that accept any tensor library would be inappropriate because it would force users to adopt a specific tensor implementation to gain access to additional features. The goals of third-party tensor libraries may not align with our goal of exposing the full capabilities of NVIDIA math libraries to the python ecosystem (our device APIs and distributed execution APIs). A more detailed description of our goals is available in our documentation.

Specifically, it seems that you like how some of these frameworks have abstracted away hardware details; for example, torch tensor operations will run on both CPU and accelerators.

We are working on adding host execution APIs where available. Our FFT API if the first with the ability to choose between NVPL/MKL/CUDA backends. But since nvmath is mostly an interface library which wraps existing libraries, not every CUDA-accelerated function will have a corresponding CPU implementation. TBH, I don't know if we've thoroughly tested the experience of users without a CUDA device with our FFT APIs (the only API that currently has a CPU-execution option). If you have specific ideas on how our default execution space selection behavior could be improved, we would be interested in reading your thoughts.

I hope that provides sufficient explanation for our design decisions.

[PGTBoos]

Thank you for the detailed explanation - it's clear you've put considerable thought into the ecosystem implications. I appreciate that NVIDIA already contributes to existing frameworks and that nvmath serves broader scientific computing beyond just ML. Your point about tensor interface flexibility (NumPy, CuPy, DLPack) is great. That's actually a good middle ground that avoids forcing specific framework adoption while maintaining broad compatibility. I understand the challenge of exposing full NVIDIA math library capabilities when third-party frameworks may not align with those goals. My concern was more about Python ecosystem fragmentation, but your tensor interface approach does address much of that. Regarding the CPU fallback experience: It might be worth considering clearer documentation or runtime warnings when users without CUDA devices try to use GPU-only functions. This could help manage expectations and guide users toward appropriate alternatives. Thanks for taking the time to explain the design rationale It's helpful to understand the broader context beyond just the ML use cases I was initially thinking about.

…

________________________________ Van: Daniel Ching ***@***.***> Verzonden: maandag 22 september 2025 19:28 Aan: NVIDIA/nvmath-python ***@***.***> CC: PGTBoos ***@***.***>; Author ***@***.***> Onderwerp: Re: [NVIDIA/nvmath-python] Reconsider this approuch (Issue #44) [https://avatars.githubusercontent.com/u/9604511?s=20&v=4]carterbox left a comment (NVIDIA/nvmath-python#44)<#44 (comment)> My understanding is you propose that NVIDIA should select a few deep learning frameworks and contribute to those instead of developing nvmath. NVIDIA already does contribute to the frameworks you have mentioned, but our customers are not only machine learning framework users. We support other domains in scientific computing where a deep learning framework may not be appropriate. While we do support torch tensors as input/output tensors, we also aim to accept any tensor which implements NumPy's array interface protocol, Cupy's array interface protocol, and in the future, DLPack. Our strategy is to expose functionality to the python ecosystem that isn't currently available via existing tensor libraries. Contributing to a few existing tensor libraries instead of providing functions that accept any tensor library would be inappropriate because it would force users to adopt a specific tensor implementation to gain access to additional features. The goals of third-party tensor libraries may not align with our goal of exposing the full capabilities of NVIDIA math libraries to the python ecosystem (our device APIs and distributed execution APIs). A more detailed description of our goals is available in our documentation<https://docs.nvidia.com/cuda/nvmath-python/0.6.0/overview.html>. Specifically, it seems that you like how some of these frameworks have abstracted away hardware details; for example, torch tensor operations will run on both CPU and accelerators. We are working on adding host execution APIs where available. Our FFT API if the first with the ability to choose between NVPL/MKL/CUDA backends. But since nvmath is mostly an interface library which wraps existing libraries, not every CUDA-accelerated function will have a corresponding CPU implementation. TBH, I don't know if we've thoroughly tested the experience of users without a CUDA device with our FFT APIs (the only API that currently has a CPU-execution option). If you have specific ideas on how our default execution space selection behavior could be improved, we would be interested in reading your thoughts. I hope that provides sufficient explanation for our design decisions. — Reply to this email directly, view it on GitHub<#44 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFQQOGY7BTAGAXIMKHA3EWL3UAWUDAVCNFSM6AAAAACHESDLYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMRQGM2DEOBXHE>. You are receiving this because you authored the thread.Message ID: ***@***.***>