无法运行mongo-connector

Question

我已经在mongodb服务器中安装了mongo-connector。

我通过发出命令来执行

mongo-connector -m [remote mongo server IP]:[remote mongo server port] -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager.py

我还尝试了此操作，因为mongo在具有默认端口的同一服务器上运行。

mongo-connector -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager.py

我遇到错误

Traceback (most recent call last):
  File "/usr/local/bin/mongo-connector", line 9, in <module>
    load_entry_point('mongo-connector==2.3.dev0', 'console_scripts', 'mongo-connector')()

  File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/util.py", line 85, in wrapped
    func(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 1037, in main
    conf.parse_args()

  File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/config.py", line 118, in parse_args
    option, dict((k, values.get(k)) for k in option.cli_names))

  File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 820, in apply_doc_managers
    module = import_dm_by_name(dm['docManager'])

  File "/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/connector.py", line 810, in import_dm_by_name
    "Could not import %s." % full_name)

**mongo_connector.errors.InvalidConfiguration: Could not import mongo_connector.doc_managers.elastic_doc_manager.py.**

注意：我正在使用python2.7和mongo-connector 2.3

elasticsearch服务器是2.2

有什么建议 ？

[edit] 应用Val的建议后：

2016-02-29 19:56:59,519 [CRITICAL] mongo_connector.oplog_manager:549 - Exception during collection dump

Traceback (most recent call last):

File “/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/oplog_manager.py”, line 501, in do_dump

upsert_all(dm)

File “/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/oplog_manager.py”, line 485, in upsert_all dm.bulk_upsert(docs_to_dump(namespace), mapped_ns, long_ts)

File “/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/util.py”, line 32, in wrapped

return f(args, *kwargs)

File “/usr/local/lib/python2.7/dist-packages/mongo_connector-2.3.dev0-py2.7.egg/mongo_connector/doc_managers/elastic_doc_manager.py”, line 190, in bulk_upsert

for ok, resp in responses:

File “/usr/local/lib/python2.7/dist-packages/elasticsearch-1.9.0-py2.7.egg/elasticsearch/helpers/init.py”, line 160, in streaming_bulk

for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):

File “/usr/local/lib/python2.7/dist-packages/elasticsearch-1.9.0-py2.7.egg/elasticsearch/helpers/init.py”, line 132, in _process_bulk_chunk

raise BulkIndexError(‘%i document(s) failed to index.’ % len(errors), errors)

BulkIndexError: (u‘2 document(s) failed to index.’,..document_class=dict, tz_aware=False, connect=True, replicaset=u’mss’), u’local’), u’oplog.rs’)

2016-02-29 19:56:59,835 [ERROR] mongo_connector.connector:302 - MongoConnector: OplogThread unexpectedly stopped! Shutting down

嗨，瓦尔，

我连接了另一个mongodb实例，该实例只有一个数据库，并且具有一个包含30,000+条记录的集合，并且能够成功执行它。先前的mongodb集合有多​​个数据库（大约7个），内部有多个集合（每个数据库大约5到15个），并且所有集合中都有大量文档（范围从500到50,000）。

Mongo连接器是否由于mongo数据库中存在大量数据而发生故障？

我还有其他疑问

一个。是否可以仅对位于不同数据库中的mongodb中的特定集合进行索引编制？我只想索引特定的集合（而不是整个数据库）。我怎样才能做到这一点？

b。在elasticsearch中，我可以看到一个集合的重复索引。第一个是数据库名称（如预期的那样），另一个是数据库名称mongodb_meta，它们都具有相同的数据，如果我要更改集合，则两个集合中都将进行更新。

C。是否可以配置输出索引名称或任何其他参数？

Answer 1

我认为唯一的问题是您.py在文档管理器上具有扩展名（在mongo-connector 2.0之前需要它），您只需要删除它即可：

mongo-connector -m [remote mongo server IP]:[remote mongo server port] -t [elastic search server IP]:[elastic search server Port]  -d elastic_doc_manager