Repository navigation
QoS Min and Max IOPS Support and Volume Resize Feature - #14689
Open
suryag1201 wants to merge 10 commits into
Open
suryag1201 wants to merge 10 commits into
suryag1201 wants to merge 10 commits into
Conversation
Enable the managed-storage template cache flow for the ONTAP primary storage plugin so that VM root volumes are cloned from a template copy kept on the pool's FlexVolume instead of being copied from secondary storage for every deployment. - Advertise CAN_CREATE_VOLUME_FROM_VOLUME from the datastore driver. - Create the template cache on primary: an iSCSI LUN (identity tracked on template_spool_ref) or a qcow2 file for NFS, populated by the KVM host. - Clone new volumes from the cached template: LUN clone for iSCSI and ONTAP file clone (FileCloneRequest) for NFS, growing the clone when the disk offering is larger than the template. - Clean up the cached LUN/file when the template is removed from the pool; a missing file is treated as already deleted. - Move protocol-specific create/clone/resize logic into UnifiedSANStrategy and UnifiedNASStrategy, and add a StorageStrategy helper to poll an optional async job. - Add unit tests for the driver and the NAS/SAN strategies.
Stack the snapshot-clone paths onto the ONTAP template-cache branch so a volume or template can be created from a CloudStack volume snapshot that already lives on the same primary pool. - Clone a data volume from a snapshot on the same FlexVolume (iSCSI LUN clone or NFS file clone). A root snapshot still produces a data volume; the bootable path is create-template-from-snapshot. - Copy a snapshot into a temporary volume when creating a template, then remove that temporary volume record after the copy. - Grow the clone when the disk offering is larger than the snapshot, and delete the orphaned clone if resize fails. - Keep template-cache behavior for clone-from-template, including the existing pool types (Iscsi and NetworkFilesystem) on this branch. Co-authored-by: Cursor <[email protected]>
…Resize Changes for IOPS) (#102) QoS Support includes: 1- Capacity IOPS on the pool, 2- Create Disk/Compute offering with QoS 3- Resize changes for QoS - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] New feature (non-breaking change which adds functionality) - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) - [ ] Build/CI - [ ] Test (unit or integration test code) - [x] Major - [ ] Minor - [ ] BLOCKER - [ ] Critical - [x] Major - [ ] Minor - [ ] Trivial Performed the below testing for ISCSI and NFS both 1- Created the compute/disk offering with CustomIOPS and Fix IOPS both (min 2000, max 10000) 2- Created a Storage Pool with Total IOPS 5000 - It will just store the value in DB 3- Create a VM with Fix IOPS (ROOT + 1 DATA) - Created Lun/File, QoS policy got created and attached to file/lun 4- Create a VM with Custom IOPS (min 2000, max 5000) (ROOT + 1 DATA) - Created Lun/File, new QoS policy got created and attached to file/lun 5- Create a VM with Fix IOPS (ROOT + 1 DATA) again - Failed with error saying there is no more IOPS on a pool 6- Change the Total IOPS on pool - worked and new VM creation also went fine. 7- Resize the Volume, QoS existing field got populated and changed the min and max value - that also worked by creating new QoS if not found and attached to the file/Lun 8- Tested on FAS platform where Min QoS is set 1000 and it throw the error in logs, not on UI as UI is not using the thrown message from vendor 9- Tested on FAS platform where Min QoS is set 0 and VM created successfully. 10- Tested with No QoS - Success change? <!-- see how your change affects other areas of the code, etc. --> <!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document --> --------- Co-authored-by: Gupta, Surya <[email protected]>
…nfs (#100) Cloudstack volume resize supports grow for both NFS and Iscsi which on ontap leads to increase in size for a LUN in case of iscsi and file in case of NFS.Also consists of UT's for the changes and implementation for getusedbytes which is called in this flow to check if storagepool has enough space. This PR... <!--- Describe your changes in DETAIL - And how has behaviour functionally changed. --> <!-- For new features, provide link to FS, dev ML discussion etc. --> <!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. --> <!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged --> <!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" --> <!-- Fixes: # --> <!--- ******************************************************************************* --> <!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. --> <!--- PLEASE PUT AN 'X' in only **ONE** box --> <!--- ******************************************************************************* --> - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [X] New feature (non-breaking change which adds functionality) - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) - [ ] Build/CI - [ ] Test (unit or integration test code) - [ ] Major - [X] Minor - [ ] BLOCKER - [ ] Critical - [ ] Major - [ ] Minor - [ ] Trivial Before resizing volumes: <img width="1461" height="338" alt="Screenshot 2026-09-24 at 3 25 10 PM" src="https://github.com/user-attachments/assets/dfdcbf26-df4a-4d5d-b678-275eb804ab6a" /> Growing iscsi csvolume to 10GiB: <img width="1440" height="278" alt="Screenshot 2026-09-24 at 3 32 37 PM" src="https://github.com/user-attachments/assets/fb02809f-04e4-41ff-8415-d54545131a78" /> Lun on Ontap is grown to 10GiB: <img width="1539" height="638" alt="Screenshot 2026-09-24 at 3 30 02 PM" src="https://github.com/user-attachments/assets/69fbe426-abaa-439f-95dd-516233b9f2cf" /> Growing Nfs volume to 10GiB: <img width="1470" height="422" alt="Screenshot 2026-09-24 at 3 34 16 PM" src="https://github.com/user-attachments/assets/5b7bbff6-aae3-41c2-bf86-727eff97a917" /> virtual size of file on host side grown to 10GiB: <img width="1073" height="402" alt="Screenshot 2026-09-24 at 3 34 45 PM" src="https://github.com/user-attachments/assets/ccb40d12-5b5a-4702-b224-208ce902a192" /> maxsize by default is 1024 GiB in cloudstack for volume which is configurable if size greater than that throws an error: <img width="1448" height="431" alt="Screenshot 2026-09-24 at 3 37 08 PM" src="https://github.com/user-attachments/assets/51f2e5ed-d2b1-4c83-b718-a75a1a477fff" /> considering the overprovisiong factor, used space and current space, cloud stack calculates and propagates error in case value is over that: here get used bytes gets triggered: ISCSI CS-volume resize and get usedbytes here is triggered. <img width="1406" height="1039" alt="Screenshot 2026-09-28 at 5 47 46 PM" src="https://github.com/user-attachments/assets/a75f939b-f10a-4f78-9d08-7214eb23c5b8" /> NFS CS-volume resize and get usedbytes here is triggered. <img width="1409" height="944" alt="Screenshot 2026-09-28 at 5 49 24 PM" src="https://github.com/user-attachments/assets/21c3d9ab-6ea8-4612-a9ee-247b76bad976" /> <img width="1456" height="418" alt="Screenshot 2026-09-24 at 3 38 41 PM" src="https://github.com/user-attachments/assets/ad58307a-9f80-4a5c-b4fd-9d8d463c79a0" /> when Lun is deleted on ontap and cs volume is tried to resize , error is thrown: <img width="1463" height="430" alt="Screenshot 2026-09-24 at 3 40 17 PM" src="https://github.com/user-attachments/assets/be8255e3-d3c1-4ea5-a51f-c02f69dda727" /> when file is deleted on ONtap and resize is performed on volume: <img width="1427" height="410" alt="Screenshot 2026-09-24 at 3 41 40 PM" src="https://github.com/user-attachments/assets/49a87362-8f20-4e3e-a623-78ab4fbb9ee4" /> when shrink is performed on Iscsi volume with shrinkOk as true: <img width="1462" height="438" alt="Screenshot 2026-09-24 at 3 42 59 PM" src="https://github.com/user-attachments/assets/d870098e-40ed-4973-87e0-e035f25d1fc4" /> when shrink is performed on NFS volume : <img width="1464" height="396" alt="Screenshot 2026-09-24 at 3 43 56 PM" src="https://github.com/user-attachments/assets/0a3ad65f-5271-4738-b010-8ba5fe126c60" /> <!-- Please describe in detail how you tested your changes. --> <!-- Include details of your testing environment, and the tests you ran to --> change? <!-- see how your change affects other areas of the code, etc. --> <!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document --> --------- Co-authored-by: Gupta, Surya <[email protected]>
### Description Following changes are included: 1- Change Disk Offering 2- Accept Min as zero in create volume, resize volume and change disk offering for min iops field 3- Delete QoS 4- Create Volume from Snapshot with QoS <!--- Describe your changes in DETAIL - And how has behaviour functionally changed. --> <!-- For new features, provide link to FS, dev ML discussion etc. --> <!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. --> <!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged --> <!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" --> <!-- Fixes: # --> <!--- ******************************************************************************* --> <!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. --> <!--- PLEASE PUT AN 'X' in only **ONE** box --> <!--- ******************************************************************************* --> ### Types of changes - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] New feature (non-breaking change which adds functionality) - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) - [ ] Build/CI - [ ] Test (unit or integration test code) ### Feature/Enhancement Scale or Bug Severity #### Feature/Enhancement Scale - [x] Major - [ ] Minor #### Bug Severity - [ ] BLOCKER - [ ] Critical - [ ] Major - [ ] Minor - [ ] Trivial ### Screenshots (if appropriate): <img width="624" height="377" alt="image" src="https://github.com/user-attachments/assets/382fb6b2-736c-42ce-94b1-ffc8c455031e" /> <img width="616" height="518" alt="image" src="https://github.com/user-attachments/assets/16e41110-a299-4dd5-bafe-44a71987aab0" /> ### How Has This Been Tested? 1- Created a VM with root on AFF with custom IOPS - Success 2- Created a VM with root on FAS with custom IOPS with min 0 and max 1000 - Success 3- Created a VM with root on FAS with custom IOPS with min 100 and max 1000 - Throw error on UI, Expected Behaviour 4- Created a VM with root on AFF with Fix IOPS - Success 5- Created a VM with root on FAS with Fix IOPS with min 100 and max 1000 - Throw error on UI, Expected Behaviour 6- Created a volume with Custom IOPS on AFF with min 200 and max 400 and attache to VM- success 7- Created a volume with Custom IOPS on FAS with min 0 and max 400 and attache to VM - success 8- Created a Volume on FAS with Fix IOPS with min 100 and max 1000 - Throw error on UI, Expected Behaviour 9- Resize the Volume and change the QoS IOPS for AFF- Success 10- Resize the Volume and change the QoS IOPS for FAS with min 0 - Success 11- Resize the Volume and change the QoS IOPS for FAS with min 100 - Throw error on UI, Expected Behaviour 12- Change Disk Offering - Change from Custom to Fix on AFF - Success 13- Change Disk Offering - Change from Custom to Fix on FAS with min > 0 - Throw error on UI, Expected Behaviour 14- Change Disk Offering - Change from Fix to Custom on AFF - Success 15- Change Disk Offering - Change from Custom to Custom on FAS with min with 0 - Success 16- Delete VM - Delete QoS Policy - Success 17- Delete Volume - Delete QoS Policy - Success 18- Snapshot: Create a Snapshot of a volume which has QoS and created a volume from snapshot - success #### How did you try to break this feature and the system with this change? <!-- see how your change affects other areas of the code, etc. --> <!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document --> --------- Co-authored-by: Gupta, Surya <[email protected]>
…ol type and checkstyle/build fixes Co-authored-by: Cursor <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
QoS Min/Max IOPS Feature Support includes:
1- Capacity IOPS on NetApp Storage pool create/edit
2- Create Disk/Compute offering with fixed/custom IOPS
3- Volume Resize with Min/Max Custom IOPS
4- Change Disk Offering
5- Accept Min as zero in create volume, resize volume and change disk offering min iops field
6- Delete Volume / Delete VM with QoS
7- Create Volume from Snapshot with QoS
Volume Resize Feature:
Volume resize supports grow for both NFS and ISCSI
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
Storage Pool Create:
Storage Pool Edit:
Volume Resize:
How Has This Been Tested?
How did you try to break this feature and the system with this change?