配置UDF以从不使用外部查询的分布式缓存中获取值

Question

我们已经用Java编写了一个配置单元UDF来从添加到分布式缓存中的文件中获取值，该文件在选择查询中非常有效，如下所示：

查询1。

select country_key, MyFunction(country_key,"/data/MyData.txt") as capital from tablename;

 create table new_table 
    as 
    select country_key, MyFunction(country_key,"/data/MyData.txt") as capital from tablename;

select t.capital from 
(
select country_key, MyFunction(country_key,"/data/MyData.txt") as capital from tablename
) t;

public class CountryMap extends UDF{

    Map<Integer, String> countryMap =  null;

    public String evaluate(Integer keyCol, String mapFile) {


        if (countryMap == null){
            //read comma delimited data from mapFile and build a hashmap
                countryMap.put(key, value);
            }

        if (countryMap.containsKey(keyCol)) {
                return countryMap.get(keyCol);
            }
        return "NA";
    }
}

ADD JAR /data/CountryMap-with-dependencies.jar;
ADD FILE /data/MyData.txt;
CREATE TEMPORARY FUNCTION MyFunction as 'CountryMap';

Answer 1

你使用什么版本的Hive？在0.14.0之前，您必须设置hive.cache.expr.evaluation=false；来绕过bug。