Skip to content

Support VDDK VMware VM migratiosn to CEPH storage#13270

Open
andrijapanicsb wants to merge 1 commit into
apache:mainfrom
andrijapanicsb:vddk-rbd-direct-import-main
Open

Support VDDK VMware VM migratiosn to CEPH storage#13270
andrijapanicsb wants to merge 1 commit into
apache:mainfrom
andrijapanicsb:vddk-rbd-direct-import-main

Conversation

@andrijapanicsb
Copy link
Copy Markdown
Contributor

@andrijapanicsb andrijapanicsb commented May 28, 2026

Description

This PR extends the existing VMware-to-KVM VDDK import flow with Ceph RBD support, while preserving the current VDDK/QCOW2 behaviour.

The existing VDDK/QCOW2 writer remains supported in both existing modes:

  • With forceconverttopool=true, CloudStack continues to let virt-v2v write QCOW2 output directly into the selected supported primary storage pool using -o local -os <pool-path> -of qcow2.
  • With forceconverttopool=false, CloudStack continues to use the staged flow: virt-v2v writes finalized QCOW2 disks to temporary conversion storage, and the import step then moves/copies those disks to the final destination pool.

This change adds a destination-aware writer model for VDDK imports. The current file/QCOW2 writer path remains, and a new Ceph RBD/raw writer path is added. This gives the VDDK import flow a cleaner foundation for adding more destination-specific writers later.

For Ceph RBD, two modes are supported:

  1. Direct RBD mode

    When usevddk=true, forceconverttopool=true, and convertinstancestoragepoolid points to an RBD primary storage pool, CloudStack can write the VMware disk data directly into raw RBD images.

    This path does not use the normal virt-v2v -o local -of qcow2 writer, because RBD is not a local filesystem target. Instead, the conversion host:

    • exposes the VMware source disks through VDDK/nbdkit
    • writes the data directly into raw RBD images using qemu-img convert
    • runs guest finalization in place on the newly-created RBD images using virt-v2v-in-place, or virt-v2v --in-place when supported

    The in-place finalization happens only on the destination RBD images. The VMware source disks are not modified.

  2. Staged RBD mode

    When usevddk=true and forceconverttopool=false, RBD can still be used as the final destination through the existing staged model.

    In this mode:

    • regular VDDK/virt-v2v creates finalized QCOW2 disks on temporary conversion storage
    • the import step then converts/copies those QCOW2 disks into raw RBD images

    This mode does not require in-place virt-v2v, because regular virt-v2v has already finalized the guest while creating the temporary QCOW2 disks.

Direct RBD mode requires newer conversion-host tooling. The practical recommended baseline is EL9-family KVM hosts, such as Oracle Linux 9, Rocky Linux 9, AlmaLinux 9, or RHEL 9, and Ubuntu 24.04-style hosts where virt-v2v-in-place or virt-v2v --in-place is available. EL8 and Ubuntu 22.04-style hosts should use staged RBD import unless compatible in-place tooling has been explicitly installed and detected by CloudStack.

The implementation adds KVM host capability detection for:

  • VDDK RBD direct import support
  • in-place virt-v2v support
  • qemu-img RBD support
  • qemu-based RBD copy support

Direct RBD mode is rejected early when the selected conversion host does not support the required in-place finalization path. Staged RBD remains available in that case.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

N/A

How Has This Been Tested?

Focused compile and unit test coverage was run for the changed API/core/server/KVM paths, including:

git diff --check
mvn -pl api,core -am -DskipTests "-Dcheckstyle.skip=true" compile
mvn -pl engine/orchestration -am -DskipTests "-Dcheckstyle.skip=true" compile
mvn -pl plugins/hypervisors/kvm -am -DskipTests "-Dcheckstyle.skip=true" compile
mvn -pl plugins/hypervisors/kvm -am "-Dtest=LibvirtCheckConvertInstanceCommandWrapperTest,LibvirtConvertInstanceCommandWrapperTest,LibvirtImportConvertedInstanceCommandWrapperTest" -DfailIfNoTests=false "-Dsurefire.failIfNoSpecifiedTests=false" test
mvn -pl server -am "-Dtest=UnmanagedVMsManagerImplTest#testValidateSelectedConversionStoragePoolForVddkSkipsTemporaryPoolForStagedImport+testSelectKVMHostForConversionInClusterDirectRbdAutoSelectsHostWithDirectSupport+testSelectKVMHostForConversionInClusterDirectRbdFailsWithoutDirectSupport+testCheckConversionStoragePoolRbdAllowedForVddkForceConvertToPool" -DfailIfNoTests=false "-Dsurefire.failIfNoSpecifiedTests=false" test

@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

Codecov Report

❌ Patch coverage is 26.43172% with 334 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.12%. Comparing base (c0ce5b4) to head (bd86d7c).

Files with missing lines Patch % Lines
.../wrapper/LibvirtConvertInstanceCommandWrapper.java 4.02% 187 Missing and 4 partials ⚠️
.../apache/cloudstack/vm/UnmanagedVMsManagerImpl.java 62.62% 31 Missing and 6 partials ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java 0.00% 25 Missing ⚠️
...com/cloud/agent/api/to/VmwareVddkSourceDiskTO.java 0.00% 21 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java 0.00% 18 Missing ⚠️
.../LibvirtImportConvertedInstanceCommandWrapper.java 72.54% 9 Missing and 5 partials ⚠️
.../java/com/cloud/agent/api/to/RemoteInstanceTO.java 0.00% 6 Missing ⚠️
.../org/apache/cloudstack/vm/UnmanagedInstanceTO.java 0.00% 6 Missing ⚠️
...va/com/cloud/agent/api/ConvertInstanceCommand.java 0.00% 6 Missing ⚠️
...loud/agent/api/ImportConvertedInstanceCommand.java 16.66% 5 Missing ⚠️
... and 2 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #13270      +/-   ##
============================================
+ Coverage     18.10%   18.12%   +0.01%     
- Complexity    16749    16797      +48     
============================================
  Files          6037     6038       +1     
  Lines        542796   543223     +427     
  Branches      66456    66506      +50     
============================================
+ Hits          98268    98450     +182     
- Misses       433492   433723     +231     
- Partials      11036    11050      +14     
Flag Coverage Δ
uitests 3.51% <ø> (ø)
unittests 19.28% <26.43%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@andrijapanicsb andrijapanicsb changed the title Support VDDK imports into RBD storage Support VDDK VMware VM migratiosn to CEPH storage May 28, 2026
@andrijapanicsb
Copy link
Copy Markdown
Contributor Author

@blueorangutan package kvm

@blueorangutan
Copy link
Copy Markdown

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18075

private String datacenterName;
private String clusterName;
private String hostName;
private String vmwareMoref;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private String vmwareMoref;
private String vmwareMoRef;

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snapshot internal ref of VMware as reported by vCenter

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public String getVmwareMoref() {
return vmwareMoref;
}

public void setVmwareMoref(String vmwareMoref) {
    this.vmwareMoref = vmwareMoref;

"r" is lowewrcase, don't capitalise it - or do it everywher :) ? Or....? You would know better


private String bootType;
private String bootMode;
private String vmwareMoref;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private String vmwareMoref;
private String vmwareMoRef;

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@andrijapanicsb
Copy link
Copy Markdown
Contributor Author

@blueorangutan test keepEnv

@blueorangutan
Copy link
Copy Markdown

@andrijapanicsb a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants