Skip to content

copyMemoryToImage function added#1079

Open
CrabExtra wants to merge 6 commits into
masterfrom
host_image_copy
Open

copyMemoryToImage function added#1079
CrabExtra wants to merge 6 commits into
masterfrom
host_image_copy

Conversation

@CrabExtra

Copy link
Copy Markdown
Contributor
  • Add hostImageCopy as MOVE_TO_LIMIT in device_features.json.
  • Query and enable VkPhysicalDeviceHostImageCopyFeaturesEXT through limits.hostImageCopy.
  • Add EUF_HOST_TRANSFER_BIT and map it to VK_IMAGE_USAGE_HOST_TRANSFER_BIT_EXT.
  • Wire VK_FORMAT_FEATURE_2_HOST_IMAGE_TRANSFER_BIT_EXT into hostImageTransfer format usage.
  • Add host image copy flags and SMemoryToImageCopy.
  • Add ILogicalDevice::copyMemoryToImage validation and CVulkanLogicalDevice backend.
  • Validate host-transfer image creation and EHICF_MEMCPY_BIT full-subresource rules.

Description

Testing

TODO list:

- Add hostImageCopy as MOVE_TO_LIMIT in device_features.json.
- Query and enable VkPhysicalDeviceHostImageCopyFeaturesEXT through limits.hostImageCopy.
- Add EUF_HOST_TRANSFER_BIT and map it to VK_IMAGE_USAGE_HOST_TRANSFER_BIT_EXT.
- Wire VK_FORMAT_FEATURE_2_HOST_IMAGE_TRANSFER_BIT_EXT into hostImageTransfer format usage.
- Add host image copy flags and SMemoryToImageCopy.
- Add ILogicalDevice::copyMemoryToImage validation and CVulkanLogicalDevice backend.
- Validate host-transfer image creation and EHICF_MEMCPY_BIT full-subresource rules.
Comment thread include/nbl/video/ILogicalDevice.h
Comment thread src/nbl/video/CVulkanPhysicalDevice.cpp Outdated
Comment on lines +1342 to +1344
//Not sure to remove this?
// retval.hostImageTransfer = anyFlag(features, VK_FORMAT_FEATURE_2_HOST_IMAGE_TRANSFER_BIT);
retval.hostImageTransfer = anyFlag(features, VK_FORMAT_FEATURE_2_HOST_IMAGE_TRANSFER_BIT_EXT);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to report because not every format suports

Comment thread src/nbl/video/utilities/CAssetConverter.cpp
CrabExtra added 5 commits June 9, 2026 01:49
…ImageCopyProperties`, including `identicalMemoryTypeRequirements`, `copySrcLayouts`, and `copyDstLayouts`.

- Updated `ILogicalDevice` to validate source and destination image layouts for host image copy operations in `copyMemoryToImage`, `copyImageToMemory`, `copyImageToImage`, and `transitionImageLayout`.
- Extended `device_limits.json` to include new fields for `hostImageCopySrcLayouts`, `hostImageCopyDstLayouts`, and `identicalMemoryTypeRequirements`.
Comment on lines +1755 to +1777
{
"comment": [
"VK_EXT_host_image_copy",
"HostImageCopyPropertiesEXT"
],
"entries": [
{
"type": "uint16_t",
"name": "hostImageCopySrcLayouts",
"value": 0
},
{
"type": "uint16_t",
"name": "hostImageCopyDstLayouts",
"value": 0
},
{
"type": "bool",
"name": "identicalMemoryTypeRequirements",
"value": false
}
]
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so this is wrong:
it's not a "limit" per se, or an extension/feature.
limits are either vulkan limits or vulkan features promoted to limits.
this fits neither. it's a physical device property.
Similar to SMemoryProperties you should have a SHostImageCopyProperties and function to query it, and member functions to fill.
it should be equivalent to https://docs.vulkan.org/refpages/latest/refpages/source/VkPhysicalDeviceHostImageCopyProperties.html

Look at how these work:

  • SMemoryProperties
  • SFormatImageUsages
  • SQueueFamilyProperties
  • or basically anything in SInitData

Comment on lines +117 to +118
/* VK_EXT_host_image_copy */
uint8_t optimalTilingLayoutUUID[VK_UUID_SIZE];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, should be in dedicate SHostImageCopyProperties

Comment on lines +600 to +603
layouts.copySrcLayoutCount = hostImageCopyProperties.copySrcLayoutCount;
layouts.pCopySrcLayouts = copySrcLayouts.data();
layouts.copyDstLayoutCount = hostImageCopyProperties.copyDstLayoutCount;
layouts.pCopyDstLayouts = copyDstLayouts.data();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug,
vectors go out of scope and you're holding onto stale pointer.
if you fix this previous issue first you'll realize why we used smart_refctd_dynamic_array for these types of things. it's basically a fixed sized dynamic allocation (no resize like vector + consistent pointers). it's also refcounted to ensure the memory doesn't get freed.

@Erfan-Ahmadi Erfan-Ahmadi Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I mistunderstood, It's ok.
but try to use smart_refctd_dynamic_array

properties.limits.minAccelerationStructureScratchOffsetAlignment = accelerationStructureProperties.minAccelerationStructureScratchOffsetAlignment;
}

if (isExtensionSupported(VK_EXT_HOST_IMAGE_COPY_EXTENSION_NAME))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this whole thing needs to fill initData's host image copy properties
(don't forget getters for this property)

Comment on lines +718 to +722
if (regions.size() > 0xffff'ffffull)
{
NBL_LOG_ERROR("`regions.size()` exceeds `uint32_t` range");
return false;
}

@Erfan-Ahmadi Erfan-Ahmadi Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh, I feel like this is the super protective stuff that AI generates, cool but we'd be having to do it everywhere if it was a real issue

Comment on lines +713 to +717
if (regions.empty())
{
NBL_LOG_ERROR("`regions` must not be empty");
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why return false?
user wants to copy 0 regions? done, you're good to go :D

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move that to the first check

Comment on lines +607 to +614
for (uint32_t i=0u; i<=static_cast<uint32_t>(layout_t::SHARED_PRESENT); i++)
{
const VkImageLayout vkLayout = getVkImageLayoutFromImageLayout(static_cast<layout_t>(i));
for (const VkImageLayout l : copySrcLayouts)
if (l==vkLayout) { properties.limits.hostImageCopySrcLayouts |= 1u<<i; break; }
for (const VkImageLayout l : copyDstLayouts)
if (l==vkLayout) { properties.limits.hostImageCopyDstLayouts |= 1u<<i; break; }
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I see what you're doing, instead of an array like vulkan's, you're creating a bitflag. that's actually a good idea for faster checks.

This double for loop seems weird af, you're iterating on all allowed src/dst layouts, per layout.

Why not do this instead:
loop over all copy[Src/Dst]Layouts, getImageLayout from Vk layout. check whether it's != UNDEFINED, and then put it in the correct place in the bitfield uint64_t seems good enough for the bitfield, we won't be dealing with 64+ layouts.

If there is no getImageLayout from Vk layout, implement it

@Erfan-Ahmadi Erfan-Ahmadi Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also good to assert that whatever you're doing with the bitshifts are less than the size of the storage. (e.g. i < 64)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw be careful about UNDEFINED because it is 0 in the enum and 1 << 0 is 1.
so we should never see the first bit set. because that would indicate host image copy is valid to/from undefined layout?

VkPhysicalDeviceProperties2 layouts2 = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2,&layouts };
vkGetPhysicalDeviceProperties2(vk_physicalDevice,&layouts2);
using layout_t = asset::IImage::LAYOUT;
for (uint32_t i=0u; i<=static_cast<uint32_t>(layout_t::SHARED_PRESENT); i++)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very risky, what if someone added a new layout to image layout, then SHARED_PRESENT wouldn't be the last item.
but anyways, this whole thing could be done with a single for loop

Comment on lines +686 to +705
bool validLayout = false;
switch (dstImageLayout)
{
case IGPUImage::LAYOUT::GENERAL:
case IGPUImage::LAYOUT::READ_ONLY_OPTIMAL:
case IGPUImage::LAYOUT::ATTACHMENT_OPTIMAL:
case IGPUImage::LAYOUT::TRANSFER_SRC_OPTIMAL:
case IGPUImage::LAYOUT::TRANSFER_DST_OPTIMAL:
case IGPUImage::LAYOUT::PRESENT_SRC:
case IGPUImage::LAYOUT::SHARED_PRESENT:
validLayout = true;
break;
default:
break;
}
if (!validLayout)
{
NBL_LOG_ERROR("Invalid `dstImageLayout`");
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vulkan has to make this validation because their flags are simple integers. we have explicit type for LAYOUT.
Just check if it's != UNDEFINED ? no need for switch case

Comment on lines +666 to +670
if ((flags.value & ~static_cast<decltype(flags.value)>(IGPUImage::EHICF_MEMCPY_BIT)) != 0u)
{
NBL_LOG_ERROR("Invalid host image copy flags");
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, no need for this, vulkan has to do these types of validations, cause their flags are just uint32_t's

typedef uint32_t VkFlags;
typedef VkFlags VkFormatFeatureFlags;

you don't have to make this validation. we have explicit enum class type for host image copy. and we shouldn't be resposnible for user doing anything weird with it. (we just translate this to vulkan flag and pass it down)
if vulkan gives validation error about this, we know it's us who did the translation incorrectly and passed incorrect flags

return false;
}
// device `pCopyDstLayouts` membership: https://registry.khronos.org/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-VkCopyMemoryToImageInfo-dstImageLayout-09060
if (!(getPhysicalDevice()->getLimits().hostImageCopyDstLayouts & (1u<<static_cast<uint32_t>(dstImageLayout))))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good check against bitflag, but it needs to change to physdev->gethostimagecopyproperties

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants