kerberos/nfs and SL6.1

I've got a working kerberos setup with SL6.0, however if I use the same configuration under SL6.1 I have problems with nfs+kerberos.

This shows the working ticket checkout (this is the SL6.1 machine):

[s123456@msclnx1 ~]0% kinit zzxtty
Password for zzxtty@IOP.KCL.AC.UK:
[s123456@msclnx1 ~]0% klist
Ticket cache: FILE:/tmp/krb5cc_2000
Default principal: zzxtty@IOP.KCL.AC.UK

Valid starting Expires Service principal
09/13/11 09:26:51 09/14/11 09:26:44 krbtgt/IOP.KCL.AC.UK@IOP.KCL.AC.UK
renew until 12/12/11 08:26:44
[s123456@msclnx1 ~]0%

Under SL6.0 this works:
[s123456@msclnx1 ~]0% ls /home/zzxtty
ls: cannot open directory /home/zzxtty: No such file or directory
[s123456@msclnx1 ~]2% ls /home/zzxtty

I've started rpc.gssd with debug flags:

[root@msclnx1 ~]# rpc.gssd -f -vvvv
beginning poll
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt178
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt177
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
dir_notify_handler: sig 37 si 0x7fffe9f84bb0 data 0x7fffe9f84a80
handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt17a)
handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt17a)
process_krb5_upcall: service is '<null>'
Full hostname for 'hawker.iop.kcl.ac.uk' is 'hawker.iop.kcl.ac.uk'
Full hostname for 'msclnx1.iop.kcl.ac.uk' is 'msclnx1.iop.kcl.ac.uk'
No key table entry found for MSCLNX1.IOP.KCL.AC.UK$@IOP.KCL.AC.UK while getting keytab entry for 'MSCLNX1.IOP.KCL.AC.UK$@IOP.KCL.AC.UK'
Success getting keytab entry for 'root/msclnx1.iop.kcl.ac.uk@IOP.KCL.AC.UK'
Successfully obtained machine credentials for principal 'root/msclnx1.iop.kcl.ac.uk@IOP.KCL.AC.UK' stored in ccache 'FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK'
INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK' are good until 1315990165
using FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK as credentials cache for machine creds
using environment variable to select krb5 ccache FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK
creating context using fsuid 0 (save_uid 0)
ERROR: GSS-API: error in gss_set_allowable_enctypes(): GSS_S_NO_CRED (No credentials were supplied, or the credentials were unavailable or inaccessible) - Unknown error
WARNING: Failed while limiting krb5 encryption types for user with uid 0
WARNING: Failed to create machine krb5 context with credentials cache FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK for server hawker.iop.kcl.ac.uk
WARNING: Machine cache is prematurely expired or corrupted trying to recreate cache for server hawker.iop.kcl.ac.uk
Full hostname for 'hawker.iop.kcl.ac.uk' is 'hawker.iop.kcl.ac.uk'
Full hostname for 'msclnx1.iop.kcl.ac.uk' is 'msclnx1.iop.kcl.ac.uk'
No key table entry found for MSCLNX1.IOP.KCL.AC.UK$@IOP.KCL.AC.UK while getting keytab entry for 'MSCLNX1.IOP.KCL.AC.UK$@IOP.KCL.AC.UK'
Success getting keytab entry for 'root/msclnx1.iop.kcl.ac.uk@IOP.KCL.AC.UK'
INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK' are good until 1315990165
INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK' are good until 1315990165
using FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK as credentials cache for machine creds
using environment variable to select krb5 ccache FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK
creating context using fsuid 0 (save_uid 0)
ERROR: GSS-API: error in gss_set_allowable_enctypes(): GSS_S_NO_CRED (No credentials were supplied, or the credentials were unavailable or inaccessible) - Unknown error
WARNING: Failed while limiting krb5 encryption types for user with uid 0
WARNING: Failed to create machine krb5 context with credentials cache FILE:/tmp/krb5cc_machine_IOP.KCL.AC.UK for server hawker.iop.kcl.ac.uk
WARNING: Failed to create machine krb5 context with any credentials cache for server hawker.iop.kcl.ac.uk
doing error downcall
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380
dir_notify_handler: sig 37 si 0x7fffe9f804b0 data 0x7fffe9f80380

Unfortunately this doesn't mean a great deal to me, nor google. The disk server (hawker) is a solaris 10 server, normal nfsv4 mounts work perfectly well. NFS+krb works on SL6.0:

[s123456@cnslnx1 ~]0% ls /home/zzxtty
afile afile2 bert bob check_time.pl Desktop Documents fred time2.log time.log
[s123456@cnslnx1 ~]0% mount | grep zzxtty
hawker:/zfs_r08/zzxtty on /home/zzxtty type nfs4 (rw,sec=krb5i,sloppy,addr=xx.xx.xx.xx,clientaddr=xx.xx.xx.xx)
[123456@cnslnx1 ~]0%

Any ideas?

C.

zzxtty: any success?

Well, yes and no. I've just built a 6.1 vm and tested it, it displays all the symptoms above. I removed the nfs-utils and nfs-utils-lib packages and replaced them with the ones available from the 6.0 repository, it now works. I guess this points at a problem in one of these two packages.

Thanks for the info. I'll do the same until we get a bug-fix for those packages. Hopefully this workaround won't cause too much problems with other update ...

I haven't tested this heavily, I just got a single mount to work. I have no idea if it will corrupt data, or do other bad things. I was only trying to narrow down the problem. I'm not about to roll it out to systems with real data.

I suspect this needs to be brought to the attention of either the upstream vendor or the nfs-utils project. I suspect nfs-utils will ask for someone to test the latest version (1.2.4), I'll see if I can build and test this but I'm rather busy here.

This is probably the first time i wish for a support contract with upstream vendor. Until now, i could always manage without ...

C'est la vie

Tell me about it!

You can rule out nfs-utils-lib, I upgraded that and everything still worked. I've downloaded and built nfs-utils-1.2.4 from here:

http://sourceforge.net/projects/nfs/files/nfs-utils/

After a "rpm -e --nodeps nfsutils" I installed the above build and can still successfully do a:

mount -tnfs4 -o sec=krb5i hawker:/zfs_r08/zzxtty /a

mount | grep zzx
hawker:/zfs_r08/zzxtty on /a type nfs4 (rw,sec=krb5i,addr=......

There isn't much else I can do, I'll assume upstream packager problem.

Hi guys,
i see you were busy :-) Thanks for information. I'm having problem with nfs v.4 mounts now too. We moved some servers to NFS v.4.
We mount the shared folder from server to client, no problem, everything works. But some files are copied and saved with no rights, e.g. like:

----------. 1 fp4rim fp4rim 52 Sep 21 06:17 aaa

Problem is we didnt experience this on test, but in production yes, strange thing is this happens on couple of files from like tens of thousands. But script fails then on taring/zipping .... bleh.
Tcpdump is yummy, in one minute gets like 50MB.

NFS is hell.
Cant figure this out, RH support nothing usefull so far too.

well ... *sigh* ./rant off

helikaon, you are able to mount a remote file system. Can we assume you are using auth=sys instead one of the krb5 flavours?