How to copy selective documents from one index to a new index in Elasticsearch?

In this post we have shown how to copy an index and all its contents to a new index in Elasticsearch.

In this post we are going to show how to copy selective documents from one index to a new index in Elasticsearch.

reindex with query

As shown in the other post, we still use a reindex by specifying the source and destination but this time we also specify a query in source along with the term which indicates that the documents with state = ‘ny’ will be filtered from the source.

So only documents with state ny will be copied to the new index account_v3 with this reindex operation.

 curl -X POST localhost:9200/_reindex  -H 'Content-Type: application/json' -d'
 {
   "source": {
     "index": "account",
     "query": {
       "term": {
         "state": "ny"
       }
     }
   },
   "dest": {
     "index": "account_v3"
   }
 }'
 {
 "took": 591,
 "timed_out": false,
 "total": 20,
 "updated": 0,
 "created": 20,
 "deleted": 0,
 "batches": 1,
 "version_conflicts": 0,
 "noops": 0,
 "retries": {
 "bulk": 0,
 "search": 0
 },
 "throttled_millis": 0,
 "requests_per_second": -1.0,
 "throttled_until_millis": 0,
 "failures": []
 } 

Let’s list the indices in elasticsearch. There you go we see 3 indices the new index account_v3 has only 20 documents because we only have 20 documents with state = ny in account.

 [osboxes@wk1 ~]$  curl http://localhost:9200/_cat/indices?v
 health status index      uuid                   pri rep docs.count docs.deleted store.size pri.store.size
 yellow open   account_v2 Kws5DlAdQuCv8845NZPUhQ   1   1        993            0    393.5kb        393.5kb
 yellow open   account_v3 vqWW9i3KSD-wjqFH5kFWwg   1   1         20            0     17.4kb         17.4kb
 yellow open   account    avtO6o3jTmWtgAyQwTTM6Q   1   1        993            7    400.9kb        400.9kb