feat(provisioning): extra format options (mkfs) added

[mhkarimi1383]

Pull Request template

Please, go through these steps before you submit a PR.

Why is this PR required? What issue does it fix?:

adding extra format (mkfs) options to newly created Volumes

also switched from deprecated utils/monut to mount-utils to be able to add this feature and moving from an old package to a maintained one

What this PR does?:

Refactor to new mount utils package and change format and mount function

Does this PR require any upgrade changes?:

No, But requires document updates for StorageClasses

If the changes in this PR are manually verified, list down the scenarios covered::

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

• Fixes Extra mkfs options #296
• PR Title follows the convention of <type>(<scope>): <subject>
• Has the change log section been updated?
• Commit has unit tests
• Commit has integration tests
• (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
• (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them:

[dsharma-dc]

How is the new formatOptions going to be set by user? via storageclass? There is no mkfsOptions unlike mountOptions as provided by storageclass.
Please write a test as well to show how this is used.

[tiagolobocastro]

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

[mhkarimi1383]

Hi

formatOptions is added to MountInfo just like mountOptions

So, It will work if you add it to storageclass

[mhkarimi1383]

Also there is no test for the mounter itself

[dsharma-dc]

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

[dsharma-dc]

Also there is no test for the mounter itself

You can run this manually and document here the steps to set formatOptions.

[mhkarimi1383]

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

GetVolAndMountInfo is parsing and building mountinfo from csi node publish request, but I don't see anything for format options in csi request. So I wonder how that's going to work. The storageclass parsing is being done in NewVolumeParams that's part of controller plugin.
cc: @abhilashshetty04

It's my first time that I'm contributing to this project so I just looked for where mountOptions is handled to handle formatOptions too

[mhkarimi1383]

Hello again

I'm using NixOS, and I can't run make, waiting on #351

[mhkarimi1383]

I think formatOptions need to be parsed from storage class at GetVolAndMountInfo ?

I checked out more and I saw that we have to put this on where we are mounting volume for the first time, So there is no need to add this into GetVolAndMountInfo, we have to handle that just like mountOptions

[mhkarimi1383]

Not able to test due to #353 and #352

[dsharma-dc]

@mhkarimi1383
Also, please refer to the change https://github.com/openebs/zfs-localpv/pull/542/files . Something similar is likely required here also for these formatOptions to be usable.
cc: @abhilashshetty04 @Abhinandan-Purkait

[mhkarimi1383]

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

[dsharma-dc]

@dsharma-dc

golint-ci returns error

pkg/driver/controller.go:338:6: declared and not used: vol (typecheck)
	var vol *lvmapi.LVMVolume
	   ^
pkg/lvm/mount.go:31:2: could not import k8s.io/mount-utils (-: cannot compile Go 1.22 code) (typecheck)
	"k8s.io/mount-utils"
	^
pkg/lvm/mount.go:73:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:97:14: undefined: mount (typecheck)
	mounter := &mount.SafeFormatAndMount{Interface: mount.New(""), Exec: utilexec.New()}
	           ^
pkg/lvm/mount.go:99:19: undefined: mount (typecheck)
	dev, ref, err := mount.GetDeviceNameFromMount(mounter, targetPath)
	                ^
make: *** [Makefile:263: golint] Error 1

I think to update to k8s.io/mount-utils we have to update go version

Would you like to do that as part of this PR? or instead try using a mount-utils that might work with go 1.19.

[mhkarimi1383]

I can do both, whatever is Ok :)

[dsharma-dc]

I can do both, whatever is Ok :)

Thanks, I'd say then try using a compatible mount-utils version(I hope no other issues pop up), and Go upgrade can be taken up as a new PR.

[mhkarimi1383]

@dsharma-dc

Ok, I will revert my changes to go.mod, etc. And will change the package again

[mhkarimi1383]

I have fixed package versions and switched to a correct version of mount utils

And I was able to build project successfully and tests passed

[mhkarimi1383]

Hello again,
Now I Understood that how volume creation works, after reading the code more deeply :)

and I have added new parameter to the correct locations (I think)
But When I create a storageclass and volume with new option The options does not appear in LVMVolume object

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: openebs-lvmpv
parameters:
  storage: "lvm"
  volgroup: "lvmvg"
  formatoptions: '-N "4000000000"'
provisioner: local.csi.openebs.io
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: csi-lvmpv
spec:
  storageClassName: openebs-lvmpv
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: fio
spec:
  restartPolicy: Never
  containers:
  - name: perfrunner
    image: openebs/tests-fio
    command: ["/bin/bash"]
    args: ["-c", "tail -f /dev/null"]
    volumeMounts:
       - mountPath: /datadir
         name: fio-vol
    tty: true
  volumes:
  - name: fio-vol
    persistentVolumeClaim:
      claimName: csi-lvmpv

Trying to figure out the problem

[tiagolobocastro]

formatoptions: '-N "4000000000"'

Trying to figure out the problem

Should be formatOptions

[mhkarimi1383]

formatoptions: '-N "4000000000"'
Trying to figure out the problem

Should be formatOptions

Parameters are converting to lowercase (in helpers.GetCaseInsensitiveMap), So it should be OK, I will test it out.

UPDATE: tested and not worked :/

[tiagolobocastro]

Could this be because the volume crd need a version bump?

[mhkarimi1383]

Could this be because the volume crd need a version bump?

I have checked that new field is applied in the CRD

[dsharma-dc]

@mhkarimi1383 Are you still on with testing your changes? Please let us know how we should proceed with this.

[mhkarimi1383]

@mhkarimi1383 Are you still on with testing your changes? Please let us know how we should proceed with this.

I'm in the phase of debugging this, getting no error and options are not getting reflected into the created LVM Volume object in K8s

Trying to debug that deeply to see what's the problem

[mhkarimi1383]

Problem Solved, In logs I found that spec.formatOptions is unknown, I found that make manifast and generate_manifasts.sh script is not generating CRDs that are in helm chart, Fixed by manually replacing CRDs

What I have to do to regenerate helm chart CRDs?

I'm so happy that I got this error :)

MountVolume.SetUp failed for volume "pvc-19e34f96-7d76-4ae6-a524-389739d750e4" : rpc error: code = Internal desc = failed to format and mount the volume error: format of disk "/dev/lvmvg/pvc-19e34f96-7d76-4ae6-a524-389739d750e4" failed: type:("ext4") target:("/var/lib/kubelet/pods/69700b5c-4607-4329-8622-3f5c0096e225/volumes/kubernetes.io~csi/pvc-19e34f96-7d76-4ae6-a524-389739d750e4/mount") options:("defaults") errcode:(exit status 1) output:(mke2fs 1.47.0 (5-Feb-2023)
mkfs.ext4: inode_size (256) * inodes_count (4000000000) too big for a
  filesystem with 1048576 blocks, specify higher inode_ratio (-i)
  or lower inode count (-N).

)

[mhkarimi1383]

Also a document update is required :)

[dsharma-dc]

Sounds good. Could you also add some unit tests for this. e.g. one positive(set format option and validate setting) and one negative case(set formatoption and expect failure like you did for inodecount above)

What I have to do to regenerate helm chart CRDs?

@abhilashshetty04

[mhkarimi1383]

Sounds good. Could you also add some unit tests for this. e.g. one positive(set format option and validate setting) and one negative case(set formatoption and expect failure like you did for inodecount above)

I should only test that if that everything successfully works, or I have to verify that options are propagated? Since checking this part is a bit complex

[mhkarimi1383]

I have to add some new tests to volumeCreationTest

[dsharma-dc]

If checking the set formatOption values turns out to be tricky you may try verifying functionally. For example - set a very low limit with -N and create more files that runs over this limit.

[mhkarimi1383]

If checking the set formatOption values turns out to be tricky you may try verifying functionally. For example - set a very low limit with -N and create more files that runs over this limit.

Will try to do that :)
I'm not so good at writing tests...

[mhkarimi1383]

Test is failing I looks like due to

W0319 20:37:42.950952       1 warnings.go:70] unknown field "spec.formatOptions"

See: https://github.com/openebs/lvm-localpv/actions/runs/13956138799/job/39067688864?pr=335#step:9:565

Since Helm CRDs are not updated yet

My Locally emulated test for provided options:

λ  truncate -s 5G /tmp/disk.img
λ  sudo losetup -f /tmp/disk.img --show
/dev/loop1
λ  tune2fs -l /dev/loop1 | grep "Inode count"
Inode count:              1000320

λ  sudo mkfs.ext4 /dev/loop1 -b 4096 -N 5000000
mke2fs 1.47.2 (1-Jan-2025)
/dev/loop1 contains a ext4 file system
	created on Wed Mar 19 23:39:49 2025
Proceed anyway? (y,N) y
Discarding device blocks: done                            
Creating filesystem with 1310720 4k blocks and 5001264 inodes
Filesystem UUID: c0e8f2a6-0bac-442e-966a-cab10de1a0f4
Superblock backups stored on blocks: 
	8608, 25824, 43040, 60256, 77472, 215200, 232416, 421792, 697248, 
	1076000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

[mhkarimi1383]

Also I think a cluster dump is needed after tests...

Sometimes tests pods logs are also needed