Scientific Linux Forum.org



  Reply to this topicStart new topicStart Poll

> KVM: 9p filesystem shared from KVM host to KVM guest via virtio 9pfs (no network involved), passthrough or mapped or squash access
helikaon
 Posted: Jul 25 2014, 01:55 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









Hi guys,
this was long coming for the RHEL based clones, finally doable with new release of SL 7 (CentOS 7, RHEL 7) , actually, if i say further SL 7 you can replace it with either of those.
What's that about?
The 9p FS kernel modules enable direct sharing of files between host and guest (similar to e.g. virtual box shared folder options).
Thing is, that no networking setup is involved, which adds a whole bunch of options where to use it (so no need for NFS or Samba setup to share the files).

Before you even start - your PC machine has to support virtualization on HW level..
in started linux:
CODE

[root@itstar70 ~]# lsmod | grep kvm
kvm_intel             137307  0
kvm                   424530  1 kvm_intel

You have to see both modules - if you see only 'kvm' module alone, something is wrong and your machine either doesn't support virtualization, or your virtualization option is turned off in BIOS of your PC (you should see kvm_amd on AMD cpu based machine)

Facts about 9pfs:
- it will not work with windows guests (not coded for windows yet)
- host machine: needs to be SL 7 - could be 6.5 too, but i wasn't able to compile right version of qemu there
- guest machine: SL 7 or SL 6.5 (no windows so far)
- it requires recompilation of vanilla kernel from kernel.org (stable) for SL 7 host and guest and dkms installation of 9p kernel module for SL 6.5 guest
- it requires recompilation of qemu 2.0 from EPEL source rpm (because EPEL guys forbidden to build some qemu packages from qemu package set in their rpm specfile - in order to 'fit' it with official RHEL 7.0 packages qemu build)
- it requires rebuilding of initramfs on host and guest, setting SELinux policies on host machine (if you want to keep selinux running there)
- finally it require the right choice of mount options for the 9p filesystem on guest, in respect with r/w rights and user rights for the mounted FS

So as you can see, you can (some of you) learn a few things along the way :]. And hopefully you'll be able to get to the mount point options stage and help me / others to fine tune them to best available scenario.

Project goal:
- guest can backup files to host, without any need of networking

Practical example:
- have SL 6 or 7 linux KVM guest on SL 7 host
- host will share FS (filesystem) to guest via 9p
- guest is insecure, facing directly to internet, having its own dedicated physical NIC (network card) from host and having public IP
- host is sitting in LAN having non-public IP
- we can not allow guest to 'see' into LAN (in case it's hacked), yet we want to do backups to host


Without further ado:

---Recompilation of vanilla kernel for SL 7 (RHEL 7 or CentOS 7) host machine---
q: why vanilla?
a: RH for some reason removed 9p support from kernel source it distributes

Prerequisities:

First let me make sidenote for the RHEL users about their repositories:
download both RH 7 DVDs and add them to fstab:
CODE

vi /etc/fstab
/opt/rhel-server-7.0-x86_64-dvd.iso     /repo   iso9660 loop    0 0
/opt/supp-server-7.0-rhel-7-x86_64-dvd.iso      /reposupl       iso9660 loop    0 0

[root@itstar70 ~]# mount -a
mount: /dev/loop0 is write-protected, mounting read-only
mount: /dev/loop1 is write-protected, mounting read-only

Create DVD repo file:
CODE

vi /etc/yum.repos.d/rhel-dvd.repo
[rhel-dvd]
name=RHEL 7.0 DVD1
baseurl=file:///repo
enabled=1
gpgcheck=0

[rhel-dvdsupl]
name=RHEL 7.0 DVD2
baseurl=file:///reposupl
enabled=1
gpgcheck=0


Installation of EPEL repo for RHEL 7.0 https://fedoraproject.org/wiki/EPEL
CODE

wget -c http://mirror.vutbr.cz/epel/beta/7/x86_64/epel-release-7-0.2.noarch.rpm
[root@itstar70 opt]# yum localinstall epel-release-7-0.2.noarch.rpm

don't forget to disable it (to keep your rpm database clean as much as possible):
CODE

vi /etc/yum.repos.d/epel.repo
enabled=0


Install needed dev packages:
CODE

yum install gcc ncurses ncurses-devel yum-utils rpm-build redhat-rpm-config rpmdevtools


Get kernel source (latest stable) from kernel.org (download and save it as normal user, not root)
e.g. (we will compile / recompile things as normal user (not root user!)
CODE

[lang@itstar70 Downloads]$ pwd
/home/lang/Downloads
wget -c https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.15.6.tar.xz


Set environment in user home (this will create ~/rpmbuild/* tree and ~/.rpmmacros file)
CODE

[lang@itstar70 ~]$ pwd
/home/lang
[lang@itstar70 ~]$ rpmdev-setuptree



Kernel compilation:
note: i did it on 8GB RAM, 2xCPU 2GHz Xeons (4cores per) machine -> 8 cores total and it took like 45minutes to crunch (i'm curios person, so i went through whole menuconfig and turned on lot's things that probably are not needed, but i like to test new things..)

CODE

[lang@itstar70 Downloads]$ pwd
/home/lang/Downloads

tar -xJvf linux-3.15.6.tar.xz
cd /home/lang/Downloads/linux-3.15.6/
make menuconfig
make rpm


tips to 'menuconfig' part
1. turn on all 9p modules - if you can't find it in the tree (just like i couldn't) just type "/" then you can search for the eg. '9p' and it will list the modules location in the tree
2. make sure selinux modules are completely same configured like with original kernel!
if you dont compile selinux modules right, you machine will freeze during bootup with new kernel at this part:
"OK reached target initrd default target"
3. in general->localversion state string that will identify your kernel (i put there -9pfs)
4. go through the tree and turn on other things you might consider helpful (eg. NTFS support, or varios wi-fi drivers if its for laptop etc ..)

you will end with something like (note the 9pfs identification string):
CODE

Wrote: /home/lang/rpmbuild/SRPMS/kernel-3.15.6_9pfs-1.src.rpm
Wrote: /home/lang/rpmbuild/RPMS/x86_64/kernel-3.15.6_9pfs-1.x86_64.rpm
Wrote: /home/lang/rpmbuild/RPMS/x86_64/kernel-headers-3.15.6_9pfs-1.x86_64.rpm
Wrote: /home/lang/rpmbuild/RPMS/x86_64/kernel-devel-3.15.6_9pfs-1.x86_64.rpm


as root:
CODE

[root@itstar70 x86_64]# pwd
/home/lang/rpmbuild/RPMS/x86_64

[root@itstar70 x86_64]# yum localinstall kernel-3.15.6_9pfs-1.x86_64.rpm kernel-headers-3.15.6_9pfs-1.x86_64.rpm



BEFORE you reboot to new kernel, pls read this TIP (for all distros):
hash out all ISO FS in your 'etc/fstab' that you put there at start, or your kernel will panic during boot - this we will fix after we booted new kernel:


REBOOT the host machine

When machine is up with new kernel:
If you use ISO FS in fstab (like the mounted DVDs), then you can get kernel panic during reboot. You need to rebuild initramfs and add modules there, because by default there is missing:
'loop.ko' and 'isofs.ko' module kernels (probably mistake), because in RHEL6 it's present
CODE

dracut --add-drivers "/lib/modules/3.15.6-9pfs/kernel/drivers/block/loop /lib/modules/3.15.6-9pfs/kernel/fs/isofs/isofs" /boot/initramfs-3.15.6-9pfs.img --force

Sideline dracut general tips:
- dont forget to add again (and again and again) all drivers you built in previously, because everytime you run dracut, it overwrites old initramfs file from scratch (so any previously added drivers are erased - unless you specify them again)
- the .ko extension is not used in dracut syntax .. actually dracut syntax is a bit mystical and man page is not 100% helpful in regards of correct syntax



Other checks after reboot:
1. search for exact module names:
CODE

cd /usr/lib/modules/3.15.6-9pfs
find . -name "*9p*.ko" -print
./kernel/net/9p/9pnet_rdma.ko
./kernel/net/9p/9pnet_virtio.ko

insert them
CODE

modprobe 9pnet_rdma
modprobe 9pnet_virtio


have a look to config for info:
CODE

[root@itstar70 3.15.6-9pfs]# cat /boot/config-3.15.6-9pfs | grep -i 9p
CONFIG_LOCALVERSION="-9pfs"
CONFIG_NET_9P=y
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_RDMA=m
CONFIG_NET_9P_DEBUG=y
CONFIG_9P_FS=y
CONFIG_9P_FSCACHE=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y

[root@itstar70 3.15.6-9pfs]# cat /boot/config-3.15.6-9pfs | grep -i selinux
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
# CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX is not set
CONFIG_DEFAULT_SECURITY_SELINUX=y
CONFIG_DEFAULT_SECURITY="selinux"


Rebuild INITRAMFS to add '9pnet_rdma' and '9pnet_virtio' modules (plus keep the 'loop' and 'isofs' modules we added before) so they are available from start
CODE

dracut --add-drivers "/lib/modules/3.15.6-9pfs/kernel/net/9p/9pnet_virtio  /lib/modules/3.15.6-9pfs/kernel/net/9p/9pnet_rdma /lib/modules/3.15.6-9pfs/kernel/drivers/block/loop /lib/modules/3.15.6-9pfs/kernel/fs/isofs/isofs" /boot/initramfs-3.15.6-9pfs.img --force

note: this is not probably needed to have 9p modules right in boot start, as the KVM machines start later in the machine boot (even if booting automatically), but i added it in there - you can test it without .. should be ok

Note:
how to check initramfs file to make sure the modules are really compiled in
CODE

cp /boot/initramfs-3.15.6-9pfs.img /tmp
mkdir /tmp/temp && cd /tmp/temp
/lib/dracut/skipcpio ../initramfs-3.15.6-9pfs.img | gunzip -c - | cpio -idm

ls
bin  dev  etc  init  lib  lib64  proc  root  run  sbin  shutdown  sys  sysroot  tmp  usr  var

find . -name "*9p*" -print
./usr/lib/modules/3.15.6-9pfs
./usr/lib/modules/3.15.6-9pfs/kernel/net/9p
./usr/lib/modules/3.15.6-9pfs/kernel/net/9p/9pnet_virtio.ko
./usr/lib/modules/3.15.6-9pfs/kernel/net/9p/9pnet_rdma.ko

ok, we can see the modules are there present, so all is OK.

This post has been edited by helikaon: Aug 4 2014, 07:21 AM

--------------------
PMEmail Poster
^
helikaon
 Posted: Jul 28 2014, 09:34 AM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









---QEMU WITH 9PFS SUPPORT INSTALLATION (on host machine - obviously you don't need hypervisor on the guests)---

So after having recompiled 7.0 kernel with proper 9p modules enabled, we have to also recompile qemu, because RH also removed 9p awareness from qemu binaries.
I'll save my breath, i'll just say, that this was the hardest part for me, because i tried to compile / recompile qemu and libvirt from various sources (Fedora 17,18,19,20 .. i even added 1 more CPU to my PC to compile faster :]] ), before i got it working.

Best solution turned out to be to recompile qemu 2.0 from EPEL repo (repo marked for usage for 7x ). I've been looking at those installation packages right from start, but one thing was baffling me - that there were not all of the qemu packages, you would expect there to be for the whole qemu install (and i was right).

Thing is, EPEL guys just edited the qemu.spec file in a way so that it does not produce all packages, in order to stay 'compatible' with RHEL 7.0 qemu packages. So if you just install qemu 2.0 from EPEL, you turn out with part packages of qemu 2.0 and part of qemu 1.5 from RHEL 7 and the 9PFS support missing.

Solution is simple - download src.rpm of qemu 2.0 from EPEL and recompile (all done -again- as normal user, no root):

1.
download qemu-2.0.0-1.el7.1.src.rpm from EPEL repo for RHEL 7 and save it
2.
as normal user:
CODE

rpm -ivh qemu-2.0.0-1.el7.1.src.rpm

3.
edit qemu.spec (in order to build all qemu packages)
CODE

edit qemu.spec
cd /home/lang/rpmbuild/SPECS/
cp qemu.spec qemu-2.0.epel.src.rpm.orig.spec (make backup of original)


vi qemu.spec

%if 0%{?rhel}
# RHEL-specific defaults:
%bcond_with    kvmonly          # enabled
%bcond_with    exclusive_x86_64 # enabled for my install -> i changed this 'without' -> 'with'
%bcond_with    rbd              # disabled
%bcond_without spice            # enabled
%bcond_without seccomp          # enabled
%bcond_without xfsprogs         # disabled
%bcond_with    separate_kvm     # disabled for EPEL but enabled for my install -> i changed this 'without' -> 'with'
%bcond_without gtk              # disabled
%else


4.
rebuild the rpm based on your edited spec file:
CODE

rpmbuild -ba qemu.spec /opt/epel-src/qemu-2.0.0-1.el7.1.src.rpm


It's gonna write files you your rpmbuild folder tree (that you already used for your kernel compilation) et. like this:
CODE

[lang@itstar70 x86_64]$ pwd
/home/lang/rpmbuild/RPMS/x86_64
[lang@itstar70 x86_64]$ ls -al
total 144964
drwxr-xr-x. 2 lang lang      4096 Jul 23 11:25 .
drwxrwxr-x. 3 lang lang        19 Jul 22 14:41 ..
-rw-rw-r--. 1 lang lang     52340 Jul 23 11:22 ksm-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     88076 Jul 23 11:23 libcacard-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     54116 Jul 23 11:23 libcacard-devel-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     54376 Jul 23 11:23 libcacard-tools-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     45164 Jul 23 11:22 qemu-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang    244880 Jul 23 11:22 qemu-common-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang 110944564 Jul 23 11:25 qemu-debuginfo-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang    146548 Jul 23 11:22 qemu-guest-agent-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang    538428 Jul 23 11:22 qemu-img-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     44216 Jul 23 11:22 qemu-kvm-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang     49092 Jul 23 11:23 qemu-kvm-tools-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1530872 Jul 23 11:22 qemu-system-alpha-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1967652 Jul 23 11:22 qemu-system-arm-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1108152 Jul 23 11:22 qemu-system-cris-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1094512 Jul 23 11:22 qemu-system-lm32-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1475988 Jul 23 11:22 qemu-system-m68k-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   2151912 Jul 23 11:22 qemu-system-microblaze-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   6722148 Jul 23 11:22 qemu-system-mips-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1075780 Jul 23 11:23 qemu-system-moxie-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1071052 Jul 23 11:22 qemu-system-or32-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1207672 Jul 23 11:22 qemu-system-s390x-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   2911244 Jul 23 11:23 qemu-system-sh4-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   1068500 Jul 23 11:23 qemu-system-unicore32-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   3542888 Jul 23 11:22 qemu-system-x86-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   2138924 Jul 23 11:23 qemu-system-xtensa-2.0.0-1.el7.1.x86_64.rpm
-rw-rw-r--. 1 lang lang   7058612 Jul 23 11:22 qemu-user-2.0.0-1.el7.1.x86_64.rpm


we can see, that finally we have built all the needed qemu packages!
for comparison, have a look at the EPEL repo where mainly the 'qemu-kvm' packages are not being built (because they dont want to override the priority of RH 'qemu-kvm' packages etc):
CODE

qemu-2.0.0-1.el7.1.x86_64.rpm   26-Apr-2014 19:46       45K
qemu-common-2.0.0-1.el7.1.x86_64.rpm    26-Apr-2014 19:53       236K
qemu-system-alpha-2.0.0-1.el7.1.x86_64.rpm      26-Apr-2014 19:51       1.5M
qemu-system-arm-2.0.0-1.el7.1.x86_64.rpm        26-Apr-2014 19:46       1.9M
qemu-system-cris-2.0.0-1.el7.1.x86_64.rpm       26-Apr-2014 19:44       1.1M
qemu-system-lm32-2.0.0-1.el7.1.x86_64.rpm       26-Apr-2014 19:48       1.0M
qemu-system-m68k-2.0.0-1.el7.1.x86_64.rpm       26-Apr-2014 19:52       1.4M
qemu-system-microblaze-2.0.0-1.el7.1.x86_64.rpm 26-Apr-2014 19:42       2.0M
qemu-system-mips-2.0.0-1.el7.1.x86_64.rpm       26-Apr-2014 19:39       6.4M
qemu-system-moxie-2.0.0-1.el7.1.x86_64.rpm      26-Apr-2014 19:56       1.0M
qemu-system-or32-2.0.0-1.el7.1.x86_64.rpm       26-Apr-2014 19:47       1.0M
qemu-system-s390x-2.0.0-1.el7.1.x86_64.rpm      26-Apr-2014 20:00       1.1M
qemu-system-sh4-2.0.0-1.el7.1.x86_64.rpm        26-Apr-2014 19:43       2.8M
qemu-system-unicore32-2.0.0-1.el7.1.x86_64.rpm  26-Apr-2014 19:39       1.0M
qemu-system-x86-2.0.0-1.el7.1.x86_64.rpm        26-Apr-2014 19:39       3.4M
qemu-system-xtensa-2.0.0-1.el7.1.x86_64.rpm     26-Apr-2014 19:50       2.0M
qemu-user-2.0.0-1.el7.1.x86_64.rpm      26-Apr-2014 19:35       6.7M


5.
installation of built qemu-2.0 packages + installation of libvirt
CODE

/home/lang/rpmbuild/RPMS/x86_64
yum list all | grep -i qemu

yum remove qemu-guest-agent.x86_64 qemu-img.x86_64 qemu-kvm.x86_64 qemu-kvm-common.x86_64 qemu-kvm-tools.x86_64
yum remove libcacard-1.5.3-60.el7.x86_64
yum localinstall qemu* ksm* libcacard-2.0.0-1.el7.1.x86_64.rpm
yum localinstall libcacard-tools-2.0.0-1.el7.1.x86_64.rpm libcacard-devel-2.0.0-1.el7.1.x86_64.rpm
yum install spice-gtk3 spice-glib vinagre
yum install libvirt libvirt-daemon-kvm libvirt-java.noarch libvirt-docs.x86_64 libvirt-cim.x86_64
yum install virt-manager


simply uninstall any conflicting package and keep cleaning untill you can make it all installed

OK
that's all, now if you run 'virt-manager' and open any existing VM (virtual machine) there - go:
- 'show virtual hw details' icon
- left bottom corner 'add hardware' icon
- pick 'filesystem' from hw list

Now:
Driver: default
Mode: mapped
Source path: Filesystem mountpoint you want to share from host
Target path: identification string for the FS - eh. 'my-9p-filesystem'

Note:
1 source path you cant share just folder, or the guest see whole disk you need really share some FS - i created EXT4 FS for this purpose (would be nice to try out XFS and other FS to see how rights etc behave on these)
1. target path is not a path on guest, just ident string, so really the name doesnt matter
2. modes
- mapped
- passthrough
- squash

mapped
this is only mode i got reasonably working after mounting it up to guest in regards propagation of user rights and 'rw' usage

passthrough
i didnt get this working at for 'rw' i cant write through this one

squash
i got it working 'rw' but user rights are 'sqashed' to qemu (uid 107) user at host and at guest side of mounted FS




---SETTING OF SHARED FS ON HOST SIDE---

1.
create the fs for sharing - eg. i created 15GB ext4 FS mounted to '/9p01' mountpoint :
CODE

df -mT
/dev/mapper/vg_sys-lv_9p01 ext4         14991      41     14198   1% /9p01


2.
change owner and rights and set selinux context for the folder:
CODE

yum list all | grep selinux
yum install policycoreutils-gui.x86_64 policycoreutils-sandbox.x86_64 policycoreutils-python.x86_64 selinux-policy-minimum.noarch selinux-policy-mls.noarch setools-console-3.3.7-46.el7.x86_64


default rights
CODE

ls -alZ /
drwxr-xr-x. root root system_u:object_r:etc_runtime_t:s0 9p01


I decided to go with "public_content_rw_t"
CODE

seinfo -t | grep public
  publicfile_exec_t
  public_content_rw_t
  postfix_public_t
  publicfile_content_t
  public_content_t
  sssd_public_t
  publicfile_t

man semanage-fcontext

semanage fcontext -a -t public_content_rw_t "/9p01(/.*)?"
restorecon -R -v /9p01

ls -alZ /
drwxr-xr-x. root root system_u:object_r:public_content_rw_t:s0 9p01


3. change owner:
CODE

chown qemu:qemu /9p01

ls -alZ /
drwxr-xr-x. qemu qemu system_u:object_r:public_content_rw_t:s0 9p01



Debatable:
1. which SELinux context to use?
2. which owner to use and rights for the FS mountpoint
3. use of ACL for FS mountpoint ?
4. combination of above with setting right FS mode (passthrough, mapped, squash)?

Combination of above factors, plus variety of mount options on the guest side makes this really tough to choose the right combination. I would gladly hear any info from anyone who tried this.

This post has been edited by helikaon: Aug 4 2014, 07:21 AM

--------------------
PMEmail Poster
^
helikaon
 Posted: Jul 30 2014, 11:14 AM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









---SETTING KVM GUEST AND MOUNTING 9p SHARED FILESYSTEM---

SL (RHEL OR CENTOS) 7 guest

This is actually very easy.
- take your custom build kernel that you installed on your host (that has those 9p modules compiled in) and install it on guest
- rebuild initramfs so that the modules are available right from start (just like on host)
- as i recall i didn't have to set any special rights on guest side, neither selinux context
- mount the filesystem eg. like:
CODE

vi /etc/fstab

9p01fs   /backup   9p   trans=virtio,version=9p2000.L,rw,posixacl,access=user   0 0

more on mount options e.g.:
wiki qemu - 9p setup

Note: as said earlier (during guest HW setup on host), while you add the 'filesystem' to KVM guest config, you setup the identification string in "Target path". In my example that string is the '9p01fs'.


SL (RHEL OR CENTOS) 6 guest

This is not as easy as 7. Why? Because RH doesnt care about 9p support, so even though the 9p module is present in the 6x kernel (not compiled in though) they dont apply any patches to it.

This proven as big problem.
I wanted to stay with 'stock' kernel, so i recompiled the newest 2.6.32-431.20.5.el6.x86_64 kernel (with 9p module) and repackaged it.
After install of kernel and setting up the 9p mountpoint i got this error:
CODE

mount 9p can't read superblock

And i wasn't able to mount it on guest. This is known error and is already patched long ago, but unfortunately not in rhel kernel.

Solution could be either:
- compile vanilla kernel with 9p supp (like we did on SL 7 host)
- build own kernel module fro
- build or download patch, edit specfile of kernel.src.rpm and repackage
- i found quick solution on
https://github.com/adrahon/vagrant-kvm/issues/218
(guys build dkms module for specific version of kernel: 2.6.32-431.el6.x86_64 )
CODE

yum -y install kernel-devel- 2.6.32-431.el6.x86_64

yum -y install unzip
yum --enablerepo=epel install dkms

wget -O /tmp/centos-9p-2.6.38.5.zip https://github.com/antst/centos-9p/archive/master.zip
unzip -d /tmp /tmp/centos-9p-2.6.38.5.zip

mv /tmp/centos-9p-master /usr/src/centos-9p-2.6.38.5

dkms add -m centos-9p -v 2.6.38.5
dkms build -m centos-9p -v 2.6.38.5
dkms install -m centos-9p -v 2.6.38.5

dkms status -->>
centos-9p, 2.6.38.5, 2.6.32-431.el6.x86_64, x86_64: installed


Thats all. Not completely viable, because of specific kernel restriction, but workable for the moment for me.

This post has been edited by helikaon: Jul 30 2014, 12:51 PM

--------------------
PMEmail Poster
^
helikaon
 Posted: Jul 30 2014, 11:31 AM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11










---SETTING SELINUX POLICY FOR THE 9PFS SHARED FILESYSTEM ON HOST---

Ok,
we have now installed host, setup SL 7 (RHEL 7 or CentOS 7) guest and we have our 9p FS mounted.

Problem can be SELinux on host side. If you cant write anything on the FS (while you're on guest), getting 'permission denied' message, check:
CODE

/var/log/audit/audit.log

if you see messages there like:
CODE

type=UNKNOWN[1327] msg=audit(1406217950.823:1576): proctitle=2F7573722F62696E2F71656D752D73797374656D2D7838365F3634002D6D616368696E6500616363656C3D6B766D002D6E616D650072683730002D53002D6D616368696E650070632D6934343066782D322E302C616363656C3D6B766D2C7573623D6F6666002D6370750050656E72796E2C2B6463612C2B7064636D2C2B7874
type=AVC msg=audit(1406217950.825:1577): avc:  denied  { write } for  pid=9310 comm="pool" name="k" dev="dm-4" ino=12 scontext=system_u:system_r:svirt_t:s0:c384,c596 tcontext=system_u:object_r:public_content_rw_t:s0 tclass=file
type=SYSCALL msg=audit(1406217950.825:1577): arch=c000003e syscall=280 success=no exit=-13 a0=ffffffffffffff9c a1=7fb198001250 a2=7fb2058293e0 a3=100 items=0 ppid=1 pid=9310 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="pool" exe="/usr/bin/qemu-system-x86_64" subj=system_u:system_r:svirt_t:s0:c384,c596 key=(null)

then SELinux on host deny access
also:
CODE

audit2allow -a

returns something like:
CODE

#============= svirt_t ==============
allow svirt_t public_content_rw_t:dir write


Solution: build your own SELinux policy:
1. turn SELinux on host to permissive mode:
CODE

setenforce 0

2. switch to guest and perform all operation on the shared FS like creating dir, changing rights on file, deleting file, deleting folder, creating link etc etc
this is done in order so that audit log gathers all this informations on the SELinux denials
3. swithc back to host and
CODE

audit2allow -a

now returns (you can see much more info now there):
CODE

#============= svirt_t ==============
allow svirt_t public_content_rw_t:dir { write rmdir setattr remove_name create add_name };
allow svirt_t public_content_rw_t:file { write rename create unlink setattr };

4. policy build
CODE

grep svirt_t /var/log/audit/audit.log | audit2allow -M svirt-9p-access-2-public_content_rw_t-01

creates policy file (in my case)
CODE

svirt-9p-access-2-public_content_rw_t-01.pp

5. install policy module (file) and put selinux back to enforcing:
CODE

semodule -i svirt-9p-access-2-public_content_rw_t-01.pp
setenforce 1


Done, now the shared FS is writeable from kvm guest too.

--------------------
PMEmail Poster
^
spottyrover
 Posted: Aug 31 2014, 12:29 PM
Quote Post


SLF Junior
**

Group: Members
Posts: 40
Member No.: 98
Joined: 25-April 11









Thanks for all of your effort.
I have not tried it yet because I am still deciding which way to go with the host SL6.5, SL7 (when it comes out ) or Ubuntu 14.04 Arch linux has just been thrown into the mix.

My question is would I have to do this every time I update or is there a chance that SL will add a kernel which has this compiled into it?

Dave
PM
^
helikaon
 Posted: Aug 31 2014, 05:14 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









QUOTE (spottyrover @ Aug 31 2014, 12:29 PM)
Thanks for all of your effort.
I have not tried it yet because I am still deciding which way to go with the host SL6.5, SL7 (when it comes out ) or Ubuntu 14.04 Arch linux has just been thrown into the mix.

My question is would I have to do this every time I update or is there a chance that SL will add a kernel which has this compiled into it?

Dave

Hi,
you'd have to exclude kernel and qemu-kvm from being updated on your host and exclude kernel from being updated on guest.

To keep up, you'd have to recompile again. But thing is why you need new kernel ... you simply dont - unless some serious security bug was discovered.

SL almost surely won't add the support for it, because RH is not supporting it.

As for other distros, i'm not sure with 9p support - didnt check it, but it can't be worse than RH where there is none :]

cheers,

--------------------
PMEmail Poster
^
gecata
 Posted: Sep 1 2014, 08:03 PM
Quote Post


SLF Newbie


Group: Members
Posts: 1
Member No.: 3206
Joined: 1-September 14









Many thanks for this excellent tutorial! http://th166.photobucket.com/albums/u117/rdshear/Smiley%20Faces/th_smiley-face-thumbs-up.gif

Just keep in mind You can use kernel-ml from elrepo repository. Kernel-ml have enabled all needed 9p modules. Now i'm gonna try it with xfs file system on host and will be back to share results! biggrin.gif

Best regards!

--------------------
No way to no way!
PM
^
helikaon
 Posted: Sep 3 2014, 09:46 AM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









QUOTE (gecata @ Sep 1 2014, 08:03 PM)
Many thanks for this excellent tutorial!  http://th166.photobucket.com/albums/u117/rdshear/Smiley%20Faces/th_smiley-face-thumbs-up.gif

Just keep in mind You can use kernel-ml from elrepo repository. Kernel-ml have enabled all needed 9p modules. Now i'm gonna try it with xfs file system on host and will be back to share results!  biggrin.gif

Best regards!


Hi :]
good, the more ppl test it, the better. I didn't test it on XFS - yet (have some more urgent things atm).
As for 9p modules - they might be enables in SL/CentOS/RHEL 6.5 from ELRepo, but thing is, unless it is also patched, it will not work for 6.5

I did recompile 6.5 kernel with all needed modules (just turned them 'on' because they are there), but because RH does not apply patches to it's unenabled moddules in kernel, you'd have to not just enable them, but also apply all patches to get it working.

As i wrote earlier, i was able to setup 9p mountpoint on the RHEL 6.5 guest running RHEL7.0 host, but i was NOT able to mount it on the guest (on 9p mail list this was discussed like 1yr ago or something and fixed - so apparently not fixed in those modules present in RH kernel).
Maybe you'll have better luck, i'm really anxious to see if you get better result :]
cheers,

This post has been edited by helikaon: Sep 3 2014, 09:48 AM

--------------------
PMEmail Poster
^
spottyrover
 Posted: Sep 23 2014, 04:31 AM
Quote Post


SLF Junior
**

Group: Members
Posts: 40
Member No.: 98
Joined: 25-April 11









I am trying the ml kernel from epel
uname -r 3.16.3-1.el7.elrepo.x86_64

I have 2 questions
1. Do I need to add any other modules?
2. Any Idea why I get this error?
when trying to add filesystem support to a VM using virtual machine manager I get an error
"Not supported for this hypervisor/libvirt combination"

This is what I have done
CODE
Check hardware

lscpu
Virtualization:        AMD-V
L1d cache:             16K
L1i cache:             64K
L2 cache:              2048K
L3 cache:              8192K
NUMA node0 CPU(s):     0-5
[david@david-host ~]$

[david@david-host ~]$ grep -E "(vmx|svm|0xc0f)" --color=always /proc/cpuinfo
Worked



lsmod | grep kvm
kvm_amd                59131  0
kvm                   413027  1 kvm_amd

Hardware looks ok


lsmod | grep virtio
empty


I added them to /etc/modules-load.d/virtio.conf

# Load virtio-net.ko at boot
virtio-net
#load    block device (virtio-blk)
virtio-blk
#Load    controller device (virtio-scsi)
virtio-scsi
#load    balloon device (virtio-balloon)
virtio-balloon
virtio-pci
virtio

lsmod | grep virtio
virtio_pci             17746  0
virtio_balloon         13540  0
virtio_scsi            18381  0
virtio_blk             18129  0
virtio_net             28256  0
virtio_ring            21225  5 virtio_blk,virtio_net,virtio_pci,virtio_balloon,virtio_scsi
virtio                 14474  5 virtio_blk,virtio_net,virtio_pci,virtio_balloon,virtio_scsi


When I done this in manjaro (file pass through worked) I get an extra line

scsi_mod              142915  4 libata,sd_mod,sr_mod,virtio_scsi


cat /boot/config-3.16.3-1.el7.elrepo.x86_64 | grep -i 9p
CONFIG_NET_9P=m
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_RDMA=m
# CONFIG_NET_9P_DEBUG is not set
# CONFIG_9P_FS is not set

helikaon gets
cat /boot/config-3.15.6-9pfs | grep -i 9p
CONFIG_LOCALVERSION="-9pfs"
CONFIG_NET_9P=y
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_RDMA=m
CONFIG_NET_9P_DEBUG=y
CONFIG_9P_FS=y
CONFIG_9P_FSCACHE=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y


Is there other modules that need to be added?

CODE

Then I installed qemu and virt-manger


groupadd libvirt
gpasswd -a david libvirt


sudo systemctl start libvirtd

sudo systemctl enable libvirtd

uncomment the following
#unix_sock_group = "libvirt"
#unix_sock_ro_perms = "0777"
#unix_sock_rw_perms = "0770"
#auth_unix_ro = "none"
#auth_unix_rw = "none"

in /etc/libvirt/libvirtd.conf  


Installing and starting a VM seems ok.

Any Idea why I get this error when trying to add filesystem support to a VM using virtual machine manager ?
"Not supported for this hypervisor/libvirt combination"

thanks

Dave

PM
^
helikaon
 Posted: Sep 23 2014, 12:10 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









Hi Dave,
i had exactly same error in the virt-manager while i tried to install new KVM machine and i did NOT have turned on the virtualization support in the BIOS.

In your case, however, the reason is different, as i see that you have both kvm modules loaded ... the 'kvm_amd' and 'kvm'.
What exactly you doing? the 'adding' FS support to existing VM' .. can you elaborate a bit?

cheers,

--------------------
PMEmail Poster
^
spottyrover
 Posted: Sep 23 2014, 12:55 PM
Quote Post


SLF Junior
**

Group: Members
Posts: 40
Member No.: 98
Joined: 25-April 11









I have tried to add the shared folder before installing and after installing.
By going to VM information in virtual manager and then clicking on the "add hardware" button.
In this window the file system button is grey. When you click on it you get the error described above.

lsmod virtio in the guest shows the virtio modules are loaded, the same as above.

Looking back through the how to. I assume that I have to recompile qemu also.

Thanks

Dave

p.s. just going through this one step at a time (hoping using a different kernel (ml) would make it easier)

PM
^
helikaon
 Posted: Sep 23 2014, 01:42 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









QUOTE (spottyrover @ Sep 23 2014, 12:55 PM)
I have tried to add the shared folder before installing and after installing.
By going to VM information in virtual manager and then clicking on the "add hardware" button.
In this window the file system button is grey. When you click on it you get the error described above.

lsmod virtio in the guest shows  the virtio modules are loaded, the same as above.

Looking back through the how to. I assume that I have to recompile qemu also.

Thanks

Dave

p.s. just going through this one step at a time (hoping using a different kernel (ml) would make it easier)

Hi,
yes, you need to recompile quemu, because they removed the support from it too at Red Hat.
Regarding kernel - you can see very quickly if you have modules turned 'on' in kernel - if you just 'cat' the:
'/boot/config*kernel-version* like:
cat /boot/config-2.6.32-220.el6.x86_64 | grep -i 9p

cheers,

--------------------
PMEmail Poster
^
spottyrover
 Posted: Sep 24 2014, 05:28 AM
Quote Post


SLF Junior
**

Group: Members
Posts: 40
Member No.: 98
Joined: 25-April 11









I will start a new post as I do not want to pollute your how to
I have done a fresh install and have different problem now.

Thanks for your help

Dave

PM
^
SignalStrength
 Posted: Feb 19 2015, 09:30 PM
Quote Post


SLF Newbie


Group: Members
Posts: 4
Member No.: 3369
Joined: 19-February 15









Hello helikaon,

Thanks for your excellent post on how to add 9p filesystem support to CentOS 7/RHEL 7. I am following your tutorial to add 9p support on Centos 7 system. I have rebuilt linux-3.15.6 kernel with 9p support and was able to boot machine with new kernel. I am working on compiling qemu with 9pfs support. I have downloaded qemu-2.0.0-1.el7.3.src.rpm from http://dl.fedoraproject.org/pub/epel/7/SRPMS/repoview/qemu.html (I was not able to find qemu-2.0.0-1.el7.1.src.rpm from epel). While trying to install source rpm (rpm -ivh qemu-2.0.0-1.el7.1.src.rpm) as a normal user, it was showing this warning "Warning: user mockbuild does not exist: using root" . I went ahead and installed mock package to remove the warning.

Now I am facing the same issue I believe you were facing mentioned in this post ( http://scientificlinuxforum.org/index.php?showtopic=2852 ) . Could you please explain how you solved the error "qemu-2.0.0-1.el7.3.src.rpm does not appear to be a specfile" to get a successful rpm build?

Thanks
PMEmail Poster
^
SignalStrength
 Posted: Feb 21 2015, 05:39 AM
Quote Post


SLF Newbie


Group: Members
Posts: 4
Member No.: 3369
Joined: 19-February 15









Though my rpmbuild is erroring out, I believe it is building all the qemu rpm packages ( It is showing error during the cleanup part after it writes all the rpm)

My understanding based on what I read from the tutorial on this thread, there are two parts qemu (process emulator) and kernel. Both need to support 9p for filesharing to work . For my custom CentOS 7 kernel, I am positive it has 9p support now.

For qemu I have done everything exactly as described (except for the src.rpm issue ( epel src rpm is different from what helikaon used), which seems to have not affected my install). Still I am not able to pick 'filesystem' from hw list when I run virt-manager(not supported for this hypervisor/libvirt combination). Can anyone please help me with this issue?
PMEmail Poster
^
helikaon
 Posted: Feb 23 2015, 03:12 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









Hi there,
sorry for late reply, been down with flu past week.

I think you need to go back to my second post in this thread and rebuild the "qemu*.src.rpm" the ritght way, make sure, your spec file has those changes done...

I think you have same src.rpm package as i had - e.g. here:

epel src rpm repo

and downloaded the "qemu-2.0.0-1.el7.3.src.rpm"

The problems mentioned with quemu rebuilding came from the different distros incompatibilities - i tried src.rpm packages from Fedora 18,19,20 ... etc and it didnt work.


cheers :]

--------------------
PMEmail Poster
^
SignalStrength
 Posted: Mar 1 2015, 09:13 PM
Quote Post


SLF Newbie


Group: Members
Posts: 4
Member No.: 3369
Joined: 19-February 15










Thanks Helikaon. I finally was able to add filesystem hardware to virtual machine in virt-manager when I select qemu as hypervisor ( When I select kvm as hypervisor in virt-manager, the option to add filesystem still shows "Not supported for this hypervisor/libvirt combination")
PMEmail Poster
^
helikaon
 Posted: Mar 5 2015, 02:45 PM
Quote Post


SLF Administrator
*******

Group: Admins
Posts: 836
Member No.: 4
Joined: 8-April 11









QUOTE (SignalStrength @ Mar 1 2015, 09:13 PM)
Thanks Helikaon. I finally was able to add filesystem hardware to virtual machine in virt-manager when I select qemu as hypervisor ( When I select kvm as hypervisor in virt-manager, the option to add filesystem still shows "Not supported for this hypervisor/libvirt combination")


Great! :]
Let me (us) know if you get some experience with it. I'm interested eg. with file rights propagation, which filesystem you used etc ..

Thanks,
cheers ]

--------------------
PMEmail Poster
^
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

Topic Options Reply to this topicStart new topicStart Poll