trino与ranger集成
田佐
集成
trino配置
-
将编译后的ranger-trino插件包拷贝到Trino的coordinator节点，然后解压。
-
修改
install.properties文件：# ranger admin url POLICY_MGR_URL=http://nn23.fff.com:6080/ REPOSITORY_NAME=trinodev # 审计日志根据需要 # trino 安装目录 COMPONENT_INSTALL_DIR_NAME=/apps/trino-server-378 XAAUDIT.SUMMARY.ENABLE=false -
运行
enable-trino-plugin.sh -
查看trino安装目录的etc目录下面多了一些与ranger有关的配置
-rw-r--r-- 1 root root 168 May 11 14:33 access-control.properties -rw-r--r-- 1 root root 7529 May 11 14:33 core-site.xml -rw-r--r-- 1 root root 509 May 11 11:15 log4j.properties -rw-r--r-- 1 root root 166 Apr 29 15:49 node.properties -rw-r--r-- 1 root root 2236 May 11 14:02 ranger-policymgr-ssl.xml -rw-r--r-- 1 root root 67 May 10 10:58 ranger-security.xml -rw-r--r-- 1 root root 13912 May 10 10:57 ranger-trino-audit.xml -rw-r--r-- 1 root root 2237 May 11 14:18 ranger-policymgr-ssl.xml -rw-r--r-- 1 root root 3160 May 10 10:55 ranger-trino-security.xml -
修改
ranger-policymgr-ssl.xml的文件名：mv ranger-policymgr-ssl.xml ranger-trino-policymgr-ssl.xml
-
由于ranger集成了kerberos，所以需要修改
access-control.properties增加安全配置access-control.name=ranger ranger.hadoop_config=core-site.xml ranger.keytab=/etc/security/keytabs/trino.service.keytab ranger.principal=trino/trino34.fff.com@FFF.COM ranger.use_ugi=true
-
启动trino
ranger配置
Service Name : 一定要与trino插件安装时install.properteis文件中的REPOSITORY_NAME一致 Username: 生成默认策略的超级用户 policy.download.auth.users: 如果开启了kerberos认证，则要与access-control.properties的principal一致
这个服务连接不通也没有关系，只是在新建策略的时候不能自动补全。
问题
ranger admin ui页面没有集成trino插件
-
在ranger的/agents-common/src/main/resources/service-defs路径下找到ranger-servicedef-trino.json文件。
-
输入
curl -u admin:password -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d @ranger-servicedef-trino.json "http://nn23.fff.com:6080/service/public/v2/api/servicedef"将trino定义的策略，上传到ranger admin的服务器中，登录就可以查看到多出了trino选项。
trino启动后无法看到ranger插件日志
在trino安装目录的etc目录下创建log4j.properties文件：
# Configure logging for testing: optionally with log file log4j.rootLogger=debug, stdout log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - %m%n
trino无法从ranger下载策略
ranger admin日志报错：
Request failed. loginId=null, logMessage=Unauthenticated access not allowed
从日志上看，是匿名访问未认证通过。
通过阅读源码发现：
类org.apache.ranger.authorization.trino.authorizer.TrinoRangerPlugin
@Override
public Iterable<SystemAccessControlFactory> getSystemAccessControlFactories()
{
ArrayList<SystemAccessControlFactory> list = new ArrayList<>();
SystemAccessControlFactory factory = new RangerSystemAccessControlFactory();
//add factory
list.add(factory);
return list;
}Trino源码io.trino.security.AccessControlManager
private SystemAccessControl createSystemAccessControl(File configFile)
{
log.info("-- Loading system access control %s --", configFile);
//这个配置文件就是etc下面的access-control.properties
configFile = configFile.getAbsoluteFile();
//读取配置文件
Map<String, String> properties;
try {
properties = new HashMap<>(loadPropertiesFrom(configFile.getPath()));
}
catch (IOException e) {
throw new UncheckedIOException("Failed to read configuration file: " + configFile, e);
}
//配置文件中的access-control.name配置项
String name = properties.remove(NAME_PROPERTY);
checkState(!isNullOrEmpty(name), "Access control configuration does not contain '%s' property: %s", NAME_PROPERTY, configFile);
//上面ranger的TrinoRangerPlugin类中set的工厂类RangerSystemAccessControlFactory
SystemAccessControlFactory factory = systemAccessControlFactories.get(name);
checkState(factory != null, "Access control '%s' is not registered: %s", name, configFile);
SystemAccessControl systemAccessControl;
try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(factory.getClass().getClassLoader())) {
//调用工厂的create方法创建访问控制类-next
systemAccessControl = factory.create(ImmutableMap.copyOf(properties));
}
log.info("-- Loaded system access control %s --", name);
return systemAccessControl;
}Ranger的org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControlFactory工厂类:
主要是使用传入的map参数构建RangerConfig类，再创建RangerSystemAccessControl类。
@Override
public SystemAccessControl create(Map<String, String> config) {
System.out.println("SystemAccessControl create(Map<String, String> config)");
requireNonNull(config, "config is null");
LOG.info("ranger.keytab : "+config.get("ranger.keytab"));
LOG.info("ranger.principal : "+config.get("ranger.principal"));
System.out.println("ranger.keytab : "+config.get("ranger.keytab"));
System.out.println("ranger.principal : "+config.get("ranger.principal"));
try {
Bootstrap app = new Bootstrap(
binder -> {
configBinder(binder).bindConfig(RangerConfig.class);
binder.bind(RangerSystemAccessControl.class).in(Scopes.SINGLETON);
}
);
Injector injector = app
.strictConfig()
.doNotInitializeLogging()
.setRequiredConfigurationProperties(config)
.initialize();
//创建RangerSystemAccessControl类
return injector.getInstance(RangerSystemAccessControl.class);
} catch (Exception e) {
throwIfUnchecked(e);
throw new RuntimeException(e);
}
}在ranger的ranger-trino-plugin-shim包中的RangerSystemAccessControl类中会使用反射初始化ranger-trino包中的RangerSystemAccessControl类
private static final String RANGER_TRINO_AUTHORIZER_IMPL_CLASSNAME = "org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl";
final private RangerPluginClassLoader rangerPluginClassLoader;
final private SystemAccessControl systemAccessControlImpl;
@Inject
public RangerSystemAccessControl(RangerConfig config) {
try {
rangerPluginClassLoader = RangerPluginClassLoader.getInstance(RANGER_PLUGIN_TYPE, this.getClass());
······
systemAccessControlImpl = cls.getDeclaredConstructor(Map.class).newInstance(configMap);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
deactivatePluginClassLoader();
}
}在ranger-trino包中的RangerSystemAccessControl类中会使用trino的access-control.properties文件中配置的keytab和principal进行认证。并加载根据配置加载hadoop配置文件。还会初始化RangerPlugin类，并初始化。
public RangerSystemAccessControl(Map<String, String> config) {
super();
Configuration hadoopConf = new Configuration();
if (config.get(RANGER_CONFIG_HADOOP_CONFIG) != null) {
URL url = hadoopConf.getResource(config.get(RANGER_CONFIG_HADOOP_CONFIG));
if (url == null) {
LOG.warn("Hadoop config " + config.get(RANGER_CONFIG_HADOOP_CONFIG) + " not found");
} else {
LOG.info("hadoopConf.addResource"+url);
hadoopConf.addResource(url);
}
} else {
URL url = hadoopConf.getResource(RANGER_TRINO_DEFAULT_HADOOP_CONF);
if (LOG.isDebugEnabled()) {
LOG.debug("Trying to load Hadoop config from " + url + " (can be null)");
}
if (url != null) {
hadoopConf.addResource(url);
}
}
UserGroupInformation.setConfiguration(hadoopConf);
if (config.get(RANGER_CONFIG_KEYTAB) != null && config.get(RANGER_CONFIG_PRINCIPAL) != null) {
String keytab = config.get(RANGER_CONFIG_KEYTAB);
String principal = config.get(RANGER_CONFIG_PRINCIPAL);
LOG.info("Performing kerberos login with principal " + principal + " and keytab " + keytab);
try {
UserGroupInformation.loginUserFromKeytab(principal, keytab);
} catch (IOException ioe) {
LOG.error("Kerberos login failed", ioe);
throw new RuntimeException(ioe);
}
}
if (config.getOrDefault(RANGER_CONFIG_USE_UGI, "false").equalsIgnoreCase("true")) {
useUgi = true;
}
rangerPlugin = new RangerBasePlugin(RANGER_TRINO_SERVICETYPE, RANGER_TRINO_APPID);
//初始化RangerPlugin
rangerPlugin.init();
rangerPlugin.setResultProcessor(new RangerDefaultAuditHandler());
}RangerPlugin的init方法中，会启动定时刷新策略的线程PolicyRefresher
public void init() {
cleanup();
AuditProviderFactory providerFactory = AuditProviderFactory.getInstance();
if (!providerFactory.isInitDone()) {
if (pluginConfig.getProperties() != null) {
providerFactory.init(pluginConfig.getProperties(), getAppId());
} else {
LOG.error("Audit subsystem is not initialized correctly. Please check audit configuration. ");
LOG.error("No authorization audits will be generated. ");
}
}
if (!pluginConfig.getPolicyEngineOptions().disablePolicyRefresher) {
refresher = new PolicyRefresher(this);
LOG.info("Created PolicyRefresher Thread(" + refresher.getName() + ")");
refresher.setDaemon(true);
//启动刷新策略
refresher.startRefresher();
}
for (RangerChainedPlugin chainedPlugin : chainedPlugins) {
chainedPlugin.init();
}
}在PolicyRefresher类中会调用自身的loadPolicy()方法加载策略，最终调用 org.apache.ranger.admin.client.RangerAdminRESTClient#getServicePoliciesIfUpdated方法获取策略
@Override
public ServicePolicies getServicePoliciesIfUpdated(final long lastKnownVersion, final long lastActivationTimeInMillis) throws Exception {
if (LOG.isDebugEnabled()) {
LOG.debug("==> RangerAdminRESTClient.getServicePoliciesIfUpdated(" + lastKnownVersion + ", " + lastActivationTimeInMillis + ")");
}
final ServicePolicies ret;
if (isRangerCookieEnabled && policyDownloadSessionId != null && isValidPolicyDownloadSessionCookie) {
ret = getServicePoliciesIfUpdatedWithCookie(lastKnownVersion, lastActivationTimeInMillis);
} else {
ret = getServicePoliciesIfUpdatedWithCred(lastKnownVersion, lastActivationTimeInMillis);
}
if (LOG.isDebugEnabled()) {
LOG.debug("<== RangerAdminRESTClient.getServicePoliciesIfUpdated(" + lastKnownVersion + ", " + lastActivationTimeInMillis + "): " + ret);
}
return ret;
}private ServicePolicies getServicePoliciesIfUpdatedWithCred(final long lastKnownVersion, final long lastActivationTimeInMillis) throws Exception {
if (LOG.isDebugEnabled()) {
LOG.debug("==> RangerAdminRESTClient.getServicePoliciesIfUpdatedWithCred(" + lastKnownVersion + ", " + lastActivationTimeInMillis + ")");
}
final ServicePolicies ret;
//获取当前全程的UGI用户，如果hadoop的配置文件未加载，或者安全策略是SIMPLE，这个用户就是启动trino的用户
final UserGroupInformation user = MiscUtil.getUGILoginUser();
//是否安全模式，如果hadoop的配置文件未加载，或者安全策略是SIMPLE就代表非安全模式
final boolean isSecureMode = user != null && UserGroupInformation.isSecurityEnabled();
//获取策略
final ClientResponse response = getRangerAdminPolicyDownloadResponse(lastKnownVersion, lastActivationTimeInMillis, user, isSecureMode);
......
}在上述的异常中，由于ranger-trino插件中是非安全模式，所以插件以匿名方式访问Ranger Admin以获取策略，最终导致失败。所以需要修改trino的access-control.properties文件增加：
ranger.hadoop_config=core-site.xml ranger.keytab=/etc/security/keytabs/trino.service.keytab ranger.principal=trino/trino34.gdmp.com@GDMP.COM
并且把core-site.xml文件拷贝到trino安装目录的etc目录才会被加载。
插件已经可以从ranger admin加载策略，但是本地策略未更新
在trino日志中发现:
2022-05-11T13:37:18.223+0800 INFO main stdout 2022-05-11 13:37:18,222 ERROR [org.apache.ranger.plugin.service.RangerBasePlugin] - setPolicies: policy engine initialization failed! Leaving current policy engine as-is. Exception : java.lang.NullPointerException at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:991) at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:229) at org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:180) at org.apache.ranger.plugin.policyengine.PolicyEngine.<init>(PolicyEngine.java:212) at org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.<init>(RangerPolicyEngineImpl.java:104) at org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:325) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:270) at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:145) at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:223) at org.apache.ranger.authorization.trino.authorizer.RangerSystemAccessControl.<init>(RangerSystemAccessControl.java:120)
查看源码,找到对应的行：
LOG.debug("audit policy evaluation order: " + this.auditPolicyEvaluators.size() + " policies");
order = 0;
for (RangerPolicyEvaluator policyEvaluator : this.auditPolicyEvaluators) {
RangerPolicy policy = policyEvaluator.getPolicy();
LOG.debug("rowFilter policy evaluation order: #" + (++order) + " - policy id=" + policy.getId() + "; name=" + policy.getName() + "; evalOrder=" + policyEvaluator.getEvalOrder());
}this.auditPolicyEvaluators这个final变量为null，我们没有开启任何审计，这个地方修改一下源码重新编译：
if(auditPolicyEvaluators!=null) {
LOG.debug("audit policy evaluation order: " + this.auditPolicyEvaluators.size() + " policies");
order = 0;
for (RangerPolicyEvaluator policyEvaluator : this.auditPolicyEvaluators) {
RangerPolicy policy = policyEvaluator.getPolicy();
LOG.debug("rowFilter policy evaluation order: #" + (++order) + " - policy id=" + policy.getId() + "; name=" + policy.getName() + "; evalOrder=" + policyEvaluator.getEvalOrder());
}
}
类似资料：