mongo-connector is a real-time sync service as a package of python, which is a generic connection system that you can use to integrate MongoDB with another system with simple CRUD operational semantics. It creates a pipeline from a MongoDB cluster to one or more target systems, such as Solr, Elasticsearch, or another MongoDB cluster.
mongo-connector needs mongo to run in replica-set mode, sync data in mongo to the target then tail the mongo oplog, keeping up with operations in MongoDB in real-time. It needs a package named “elastic2_doc_manager” to write data to ES.
mongo-connector copies your documents from MongoDB to your target system. Afterwards, it constantly performs updates on the target system to keep MongoDB and the target in sync. The connector supports both Sharded Clusters and standalone Replica Sets, hiding the internal complexities such as rollbacks and chunk migrations.
Reference link is : mongo-connector
Elasticsearch provides ability to enhance the basic functionality by plugins, which are easy to use and develop. They can be used for analysis, discovery, monitoring, data synchronization and many others. Rivers is a group of plugins used for data synchronization between database and elasticsearch.
There is a mongoDB river plugin for data synchronization, named “elasticsearch-river-mongodb”.
When document is inserted to MongoDB, database is created (if it doesn’t exist), along with schema for that particular record. Then, our data is stored. When more data comes in, the schema is updated. After inserting document in MongoDB configured as replica set, it is also stored in oplog collection.The mentioned collection is operations log configured as capped collection, which keeps a rolling record of all operations that modify the data stored in databases.
River plugin monitors this collection and forwards new operations to elasticsearch according to its configuration. That means that all insert, update and delete operations are forwarded to elasticsearch automatically. Missing index with default configuration was created automatically while indexing data in ES.
Reference link is: elasticsearch-river-mongodb