On importing hive tables from H2o, sometimes it fails with the bellow error message on some Kerberized Environments.
H2OServerError: HTTP 500 Server Error:
Server error java.lang.RuntimeException:
Error: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://xxx.bankofamerica.com:10000/;principal=hive/xxx.bankofamerica.com@xxx.BANKOFAMERICA.COM;ssl=true;sslTrustStore=/pathto/cacerts;mapreduce.job.queuename=queue.name: GSS initiate failed
Request: None
Stacktrace: java.lang.RuntimeException: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://xxx.bankofamerica.com:10000/;principal=hive/xxx.bankofamerica.com@xxx.BANKOFAMERICA.COM;ssl=true;sslTrustStore=/pathto/cacerts;mapreduce.job.queuename=queue.name: GSS initiate failed
water.api.Handler.handle(Handler.java:68)
water.api.RequestServer.serve(RequestServer.java:472)
water.api.RequestServer.doGeneric(RequestServer.java:303)
water.api.RequestServer.doPost(RequestServer.java:227)
javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
ai.h2o.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
ai.h2o.org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
ai.h2o.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
The issue occurs as krb5 configs are not read in h2o driver Program , To pass krb5 configs to h2o driver , we need to pass krb5 relation configs with option -JJ to hadoop command as below.
-JJ -Djava.security.krb5.conf=krb5.conf -JJ -Djavax.security.auth.useSubjectCredsOnly=false -JJ -Dsun.security.krb5.debug=true
Example Command:-
hadoop jar h2odriver.jar -libjars hive-jdbc-2.1.1-cdh6.2.1-standalone.jar -Dmapreduce.job.queuename=queue.name -JJ -Djava.security.krb5.conf=krb5.conf -JJ -Djavax.security.auth.useSubjectCredsOnly=false -JJ -Dsun.security.krb5.debug=true -nodes 1 -mapperXmx 4G -hiveJdbcUrlPattern "jdbc:hive2://hive_host.com:port/;ssl=true;sslTrustStore=/pathto/cacerts;trustStorePassword=changeit;principal=hive/hive_host@xxx.xxxx.COM;" -hiveHost hive_host:port -hivePrincipal hive_principal -jobname H2O -form_auth -hash_login -login_conf realm.properties -hash_login -user_name user -principal user@xxxx.COM -keytab user.keytab -refreshHiveTokens -disown -notify notify.log -jks h2o.jks -jks_pass changeit