小白求解~o版openstack挂载逻辑卷报错
0
[待解决问题]
部署O版的openstack其他功能暂时正常，就是逻辑卷挂不上去。后端使用的是lvm+nfs。创建逻辑卷和虚机正常。创建虚机不能带逻辑卷。否则也会报错。
controller控制节点中：/var/log/cinder/scheduler.log暂时没有日志
cinder存储节点报错权限不够（下面是日志）：但是根据排查，权限都是cinder:cinder，不知道怎么解决，求大神帮助
2024-01-02 19:57:16.524 1621 INFO os_brick.remotefs.remotefs [req-6475b4de-62de-488c-be7f-781fca45c66b - - - - -] Already mounted: /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager [req-c2f0e6dc-f39f-4b9a-a0ec-f122a5dffbd6 5c8b08ba693d45ed9262910c4460da41 f3ee270ca75149108d84ef2924b64d79 - default default] Driver initialize connection failed (error: Unexpected error while running command.
Command: /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
Exit code: 1
Stdout: u''
Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Permission denied\n").: ProcessExecutionError: Unexpected error while running command.
Command: /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
Exit code: 1
Stdout: u''
Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Permission denied\n"
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager Traceback (most recent call last):
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1680, in initialize_connection
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager conn_info = self.driver.initialize_connection(volume, connector)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/nfs.py", line 130, in initialize_connection
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager info = self._qemu_img_info(path_to_vol, volume['name'])
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/nfs.py", line 537, in _qemu_img_info
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager run_as_root=True)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/remotefs.py", line 767, in _qemu_img_info_base
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager run_as_root=run_as_root)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/image/image_utils.py", line 90, in qemu_img_info
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager prlimit=QEMU_IMG_LIMITS)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 123, in execute
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 419, in execute
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager cmd=sanitized_cmd)
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager ProcessExecutionError: Unexpected error while running command.
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager Command: /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager Exit code: 1
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager Stdout: u''
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Permission denied\n"
2024-01-02 19:57:38.633 1621 ERROR cinder.volume.manager
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server [req-c2f0e6dc-f39f-4b9a-a0ec-f122a5dffbd6 5c8b08ba693d45ed9262910c4460da41 f3ee270ca75149108d84ef2924b64d79 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Driver initialize connection failed (error: Unexpected error while running command.
Command: /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
Exit code: 1
Stdout: u''
Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Permission denied\n").
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1688, in initialize_connection
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server raise exception.VolumeBackendAPIException(data=err_msg)
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Driver initialize connection failed (error: Unexpected error while running command.
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server Command: /usr/bin/python2 -m oslo_concurrency.prlimit --as=1073741824 --cpu=8 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server Exit code: 1
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server Stdout: u''
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Could not open '/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e': Permission denied\n").
2024-01-02 19:57:38.644 1621 ERROR oslo_messaging.rpc.server
2024-01-02 19:57:38.804 1621 INFO cinder.volume.manager [req-cedcfcf8-2f96-423d-9b7b-7c02b4be9549 5c8b08ba693d45ed9262910c4460da41 f3ee270ca75149108d84ef2924b64d79 - default default] Terminate volume connection completed successfully.
[root@cinder1 cinder]# ll /var/lib/ |grep cinder
drwxr-xr-x 6 cinder cinder 4096 Jan 2 16:17 cinder
[root@cinder1 cinder]# ll /var/lib/cinder/ |grep mnt
drwxrwxrwx 3 cinder cinder 46 Mar 17 2022 mnt
[root@cinder1 cinder]# ll /var/lib/cinder/mnt/
total 4
drwxrwxrwx 2 cinder cinder 4096 Jan 2 16:54 62ad70852aaab0962824cbf8d3f66ea9
[root@cinder1 cinder]#
所有回答(1)
0
从错误信息来看，主要问题是权限被拒绝。下面是可能的排查步骤：
检查目录权限：
确保 /var/lib/cinder/mnt/ 目录及其子目录的权限设置正确。你已经检查了 /var/lib/cinder/ 目录的权限，但仍然需要确认 /var/lib/cinder/mnt/ 目录及其子目录（例如 /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9）的权限。确保 cinder 用户有足够的权限读取和写入。
bash
Copy code
chmod -R 775 /var/lib/cinder/mnt/
chown -R cinder:cinder /var/lib/cinder/mnt/
检查 SELinux：
如果系统启用了 SELinux，它可能会导致权限问题。可以尝试禁用 SELinux 进行测试，或者按照 SELinux 的规则配置正确的权限。
bash
Copy code
setenforce 0 # 临时禁用 SELinux
检查挂载点权限：
确保挂载点 /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9 的权限也正确。这可能需要在挂载的时候设置正确的权限。
检查 cinder 用户权限：
确保 cinder 用户有足够的权限执行相关命令。可以尝试手动运行一下相关的命令，例如 qemu-img。
bash
Copy code
sudo -u cinder qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
检查 NFS 权限：
如果使用了 NFS，确保 NFS 服务器和客户端的配置正确，并且 cinder 用户在 NFS 共享上有足够的权限。
bash
Copy code
showmount -e # 查看 NFS 共享
确保检查以上步骤之后，问题没有解决。如果问题依然存在，可能需要更详细的调试信息，可以查看更多关于 cinder/volume 和 qemu-img 的日志以获取更多上下文信息。
[root@cinder1 selinux]# sudo -u cinder qemu-img info /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
image: /var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9/volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 0
[root@cinder1 62ad70852aaab0962824cbf8d3f66ea9]# pwd
/var/lib/cinder/mnt/62ad70852aaab0962824cbf8d3f66ea9
[root@cinder1 62ad70852aaab0962824cbf8d3f66ea9]# ll
total 0
-rw-rw---- 1 cinder cinder 2147483648 Jan 2 16:09 volume-2f7e0f16-21f0-4a45-a708-8bc5e705676e
[root@cinder1 62ad70852aaab0962824cbf8d3f66ea9]#
SELINUX关系的了‘
This file controls the state of SELinux on the system.
SELINUX= can take one of these three values:
enforcing - SELinux security policy is enforced.
permissive - SELinux prints warnings instead of enforcing.
disabled - No SELinux policy is loaded.
SELINUX=disabled
SELINUXTYPE= can take one of three two values:
targeted - Targeted processes are protected,
minimum - Modification of targeted policy. Only selected processes are protected.
mls - Multi Level Security protection.
SELINUXTYPE=disabled
[root@cinder1 62ad70852aaab0962824cbf8d3f66ea9]# showmount -e
Export list for cinder1:
/data/cinder_nfs1 *
[root@cinder1 62ad70852aaab0962824cbf8d3f66ea9]#
就很奇怪
