登录
当前位置： 首页 > 知识库问答 >
问题：

AWS Cloudsearch上传文档

翟卓君
2023-03-14

我正在尝试将json对象上传到AWS Cloudsearch。以下是我创建索引字段的全部代码：

  require 'aws-sdk'
  require 'pp'

  class CloudSearch
    @@client = Aws::CloudSearch::Client.new({region: 'us-east-1',credentials: Aws::Credentials.new('KEY', 'SECRET')})
    def self.create_index_fields
      Aws.config.update({
        region: 'us-east-1'
      })

      client = @@client
      fields = {'id' => 'metadata',
                'name' => 'search',
                'rating' => 'sort',
                'review_count' => 'sort',
                'price' => 'sort',
                'category_name' => 'search/filter',
                'rank_in_category' => 'sort',
                'brand_name' => 'search/filter',
                'award_winner' => 'filter'
              }
      fields.each do |key, value|
        options = {}
        indexFields = value.split("/")
        options[:domain_name] = "dev-purchxapp-com"
        options[:index_field] = {}
        options[:index_field][:index_field_name] = key
        type = :text_options
        if key == "price"
          options[:index_field][:index_field_type] = "double"
          type = :double_options
        elsif key == "review_count" || key == "rating"
          options[:index_field][:index_field_type] = "int"
          type = :int_options
        else
          options[:index_field][:index_field_type] = "literal"
          type = :literal_options
        end      
        options[:index_field][type] = {}
        if indexFields.include?('search')
          options[:index_field][type][:search_enabled] = true
        end
        if indexFields.include?('sort') || indexFields.include?('filter')
          options[:index_field][type][:facet_enabled] = true
        end
        options[:index_field][type][:return_enabled] = true

        res = client.define_index_field(options)
        pp(res)
      end
    end

    def self.list_index_fields
      client = @@client
      res=client.describe_index_fields({
        domain_name: "dev-purchxapp-com"
      })
      res
    end

    def self.delete_index_field(field)
      client = @@client
      res = client.delete_index_field({
        domain_name: "dev-purchxapp-com", # required
        index_field_name: field, # required
      })
    end

    def self.delete_all_index_fields
      client = @@client
          fields = {'id' => 'metadata',
                'name' => 'search',
                'rating' => 'sort',
                'review_count' => 'sort',
                'price' => 'sort',
                'category_name' => 'search/filter',
                'rank_in_category' => 'sort',
                'brand_name' => 'search/filter',
                'award_winner' => 'filter'
              }
      fields.each do |key, value|
        res = client.delete_index_field({
          domain_name: "dev-purchxapp-com", # required
          index_field_name: key, # required
        })
      end
    end

    def self.create_cloudsearch_domain
      client = @@client
      res = client.create_domain({domain_names: ["dev-purchxapp-com"]})
    end

    def self.list_cloudsearch_domains
      res = @@client.describe_domains({domain_names: ["dev-purchxapp-com"]})
    end

    def self.index_documents
      res = @@client.index_documents()
    end
  end

当从rails控制台运行create_index_字段时，它显示字段已创建，但当我去上传文档时，它表示找不到具有我定义的不同字段名的字段。

定义散列以转换为json对象：

    def cloudsearch_product_json
      fields = {}
      fields[:award_winner] = !self.consumr_approved_at.nil? ? 1 : 0
      fields[:brand_name] = !self.brand.nil? ? self.brand.name.to_s : ""
      fields[:category_name] = !self.category.nil? ? self.category.name.to_s : ""
      fields[:description] = ""
      fields[:id] = self.id
      fields[:name] = self.name
      fields[:price] = self.price.to_s
      fields[:rank_in_category] = self.rank_in_category.to_s
      fields[:rating] = self.rating
      fields[:review_count] = self.review_count

      data = {}
      data[:type] = "add"
      data[:id] = "product-#{self.id }"
      data[:fields] = fields
      data
    end

和文档上传rake任务：

    require 'aws-sdk'
    require 'pp'

    namespace :cloud_search do
      task :index_all_products => :environment do
        Aws.config.update({
          region: 'us-east-1', access_key_id: 'KEY', secret_access_key: 'SECRET'
        })
        client = Aws::CloudSearchDomain::Client.new(endpoint:AppConfig.cloud_search_host)
        product = "[#{Product.first.cloudsearch_product_json.to_json}]"
        resp = client.upload_documents({documents: product, content_type: "application/json",})
      end
    end

最后是我的错误：

    Aws::CloudSearchDomain::Errors::DocumentServiceException: { ["Field "award_winner" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "brand_name" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "category_name" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "description" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "id" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "name" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "price" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "rank_in_category" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "rating" does not exist in domain configuration (near operation with index 1; document_id product-1)","Field "review_count" does not exist in domain configuration (near operation with index 1; document_id product-1)"] }

我是不是忘了初始化什么东西或者发生了什么？任何帮助都将不胜感激。希望我已经给了你足够多的时间来帮助我解决这个问题。我已经读了几周AWS SDK文档，试图弄明白这一点，但没有运气。http://docs.aws.amazon.com/sdkforruby/api/Aws/CloudSearch/Client.html http://docs.aws.amazon.com/cloudsearch/latest/developerguide/configuring-index-fields.html

多谢你的帮忙

以下是我在AWS CLI中列出索引字段的结果：

C:\Users\ndalton>aws cloudsearch describe-index-fields --domain-name dev-purchxa
pp-com
{
    "IndexFields": [
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:54:00.568Z",
                "UpdateVersion": 200,
                "UpdateDate": "2015-08-18T15:08:47.629Z"
            },
            "Options": {
                "LiteralOptions": {
                    "FacetEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "award_winner"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:59.962Z",
                "UpdateVersion": 199,
                "UpdateDate": "2015-08-18T15:08:46.704Z"
            },
            "Options": {
                "LiteralOptions": {
                    "FacetEnabled": true,
                    "SearchEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "brand_name"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:58.440Z",
                "UpdateVersion": 197,
                "UpdateDate": "2015-08-18T15:08:45.556Z"
            },
            "Options": {
                "LiteralOptions": {
                    "FacetEnabled": true,
                    "SearchEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "category_name"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T21:11:20.083Z",
                "UpdateVersion": 73,
                "UpdateDate": "2015-08-12T21:58:32.590Z"
            },
            "Options": {
                "LiteralOptions": {
                    "SearchEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "description"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:54.808Z",
                "UpdateVersion": 192,
                "UpdateDate": "2015-08-18T15:08:41.349Z"
            },
            "Options": {
                "LiteralOptions": {
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "id"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:55.684Z",
                "UpdateVersion": 193,
                "UpdateDate": "2015-08-18T15:08:42.233Z"
            },
            "Options": {
                "LiteralOptions": {
                    "SearchEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "name"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:57.682Z",
                "UpdateVersion": 196,
                "UpdateDate": "2015-08-18T15:08:44.738Z"
            },
            "Options": {
                "DoubleOptions": {
                    "FacetEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "double",
                "IndexFieldName": "price"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:59.214Z",
                "UpdateVersion": 198,
                "UpdateDate": "2015-08-18T15:08:46.145Z"
            },
            "Options": {
                "LiteralOptions": {
                    "FacetEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "literal",
                "IndexFieldName": "rank_in_category"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:56.494Z",
                "UpdateVersion": 194,
                "UpdateDate": "2015-08-18T15:08:43.476Z"
            },
            "Options": {
                "IntOptions": {
                    "FacetEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "int",
                "IndexFieldName": "rating"
            }
        },
        {
            "Status": {
                "PendingDeletion": false,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:53:57.158Z",
                "UpdateVersion": 195,
                "UpdateDate": "2015-08-18T15:08:44.008Z"
            },
            "Options": {
                "IntOptions": {
                    "FacetEnabled": true,
                    "ReturnEnabled": true
                },
                "IndexFieldType": "int",
                "IndexFieldName": "review_count"
            }
        },
        {
            "Status": {
                "PendingDeletion": true,
                "State": "RequiresIndexDocuments",
                "CreationDate": "2015-08-11T20:22:56.541Z",
                "UpdateVersion": 201,
                "UpdateDate": "2015-08-18T15:11:34.982Z"
            },
            "Options": {
                "TextOptions": {
                    "AnalysisScheme": "_en_default_"
                },
                "IndexFieldType": "text",
                "IndexFieldName": "testfield"
            }
        }
    ]
}

我的硬编码JSON文件看起来像这样：

     [{"type":"add","id":"product-1","fields":{"award_winner":0,"brand_name":"","category_name":"","description":"","id":1,"name":"CNSMRBLKHL","price":"","rank_in_category":"","rating":"4.67","review_count":3}}]

错误保持不变：

C:\Users\ndalton>aws cloudsearchdomain --endpoint-url http://doc-dev-purchxapp-c
om-rjxkouy2hppztmth47dn2oowua.us-east-1.cloudsearch.amazonaws.com upload-documen
ts --content-type application/json --documents \\OGDC1\og-users\ndalton\Document
s\test.json

A client error (DocumentServiceException) occurred when calling the UploadDocume
nts operation: { ["Field "award_winner" does not exist in domain configuration (
near operation with index 1; document_id product-1)","Field "brand_name" does no
t exist in domain configuration (near operation with index 1; document_id produc
t-1)","Field "category_name" does not exist in domain configuration (near operat
ion with index 1; document_id product-1)","Field "description" does not exist in
 domain configuration (near operation with index 1; document_id product-1)","Fie
ld "id" does not exist in domain configuration (near operation with index 1; doc
ument_id product-1)","Field "name" does not exist in domain configuration (near
operation with index 1; document_id product-1)","Field "price" does not exist in
 domain configuration (near operation with index 1; document_id product-1)","Fie
ld "rank_in_category" does not exist in domain configuration (near operation wit
h index 1; document_id product-1)","Field "rating" does not exist in domain conf
iguration (near operation with index 1; document_id product-1)","Field "review_c
ount" does not exist in domain configuration (near operation with index 1; docum
ent_id product-1)"] }

共有1个答案

洪和风
2023-03-14

你需要重新索引你的域，以使新添加的字段处于活动状态。它们当前显示为“状态”：“RequiredIndexDocuments”，但应处于“状态”：“活动”。

运行aws cloudsearch索引文档--domain name dev purchxapp com应该可以解决这个问题。有关为域编制索引的更多信息，请参阅：http://docs.aws.amazon.com/cloudsearch/latest/developerguide/indexing.html

 类似资料：
  • 文件上传

    在Yii里上传文件通常使用 yii\web\UploadedFile 类， 它把每个上传的文件封装成 UploadedFile 对象。 结合 yii\widgets\ActiveForm 和 models，你可以轻松实现安全的上传文件机制。 创建模型 和普通的文本输入框类似，当要上传一个文件时，你需要创建一个模型类并且用其中的某个属性来接收上传的文件实例。 你还需要声明一条验证规则以验证上传的文件

  • 文件上传

    大多数的 Web 应用都不可避免的，会涉及到文件上传。文件上传，不过是一种适配 HTTP 输入流的方式。 为此，Nutz.Mvc 内置了一个专门处理文件上传的适配器 org.nutz.mvc.upload.UploadAdaptor 这个适配器专门解析形式为 <form target="hideWin" enctype="multipart/form-data" method="post">

  • 上传文件

    哦，上传文件可是个经典的好问题了。文件上传的基本概念实际上非常简单， 他基本是这样工作的: 一个 <form> 标签被标记有 enctype=multipart/form-data ，并且 在里面包含一个 <input type=file> 标签。 服务端应用通过请求对象上的 files 字典访问文件。 使用文件的 save() 方法将文件永久地 保存在文件系统上的某处。 一点点介绍 让我们建立一

  • ①. 文件上传

    Django提供了一些类实现管理数据分页，这些类位于django/core/paginator.py中 Paginator对象 Paginator(列表,int)：返回分页对象，参数为列表数据，每面数据的条数 属性 count：对象总数 num_pages：页面总数 page_range：页码列表，从1开始，例如[1, 2, 3, 4] 方法 page(num)：下标以1开始，如果提供的页码不存在

  • 文件上传

    SDK 详细代码可参考sdk-java模块代码，位于单元测试文件中 /** * 上传文件，读取本地文件 * * @throws IOException */ @Test public void testUpload() throws IOException { FileUploadRequest request = new F

  • 文件上传

    请求接口时带上文件 客户端调用 DemoFileUploadRequest request = new DemoFileUploadRequest(); DemoFileUploadModel model = new DemoFileUploadModel(); model.setRemark("上传文件参数"); request.setBizModel(model); List<Upload

  • 文件上传

    ThinkCMF封装了文件上传，开发者只要在模板中使用已经封装好的 js 方法就可以了，相关方法在 admin.js和frontend.js,相关函数如下： 上传对话框 /** * 打开文件上传对话框 * @param dialog_title 对话框标题 * @param callback 回调方法，参数有（当前dialog对象，选择的文件数组，你设置的extra_params） * @

  • 上传文件

    接口说明 顺序上传文件的所有分片 API地址 POST /api/upload/1.0.0/upload 是否需要登录 是 请求字段说明 参数 类型 请求类型 是否必须 说明 dataguid string form 是 数据标识 file MultipartFile form 是 上传的文件 响应字段说明 参数 类型 说明 md5 String 文件md5校验码 响应成功示例 { "code

相关问答

PHP文件上传移动上传文件不工作在android webview上上传文件Laravel 4上传文件Android WebView文件上传selenium webdriver上传文件

相关文章

jQuery文件上传始终失败，文件上传中止Spring boot实现文件上传实例（多文件上传)Jsp+Servlet实现文件上传下载 文件上传（一）FTPClient-Java，上传文件Java applet上传文件

相关阅读

7.3.1 Android 文件上传Spring MVC文件上传Struts2文件上传例子RESTful JAX-RS文件上传示例Struts2上传多个文件例子

相关工具

Stream 上传插件photo批量上传ftpjQuery异步上传插件flash 头像上传裁剪预览流传

相关文档

MathJax 中文文档PhoneGap 中文文档MooTools 中文文档Snap.svg 英文文档prototype.js 中文文档
快捷导航： 新手教程 算法原理 架构设计 Java进阶 数据库进阶 大厂专栏 面试经验 编程笔记 编程问答 所有专题 文档资料 工具软件 电子书籍 小牛导航
在线工具： 房贷计算器 个税计算器 Linux命令查询 Json格式化 正则表达式 颜色转换 AES加解密 SHA1加密 MD5加密 毒鸡汤 字数统计 随机密码生成 进制转换 Base64编解码 励志句子
Copyright © 2019-2024 小牛知识库@xnip.cn. All Rights Reserved.