1.SUMMARY Node : Any running instance of ElasticSearch Engine is known as NODE. Nodes can be multiple. Cluster : Cluster is a collection of nodes. Index : An ElasticSearch index is a logical namespace that holds multiple shards. Shard : Shard where we store data. Documents : Document is a collection of fields which in turn, are key-value pairs. Document is a JSON object.
2.CHECK CLUSTER STATUS Request: curl -k -X GET "https://localhost:9200/_cat/health?pretty" -H 'Content-Type: application/json' -u username:password Response: 1725881802 11:36:42 elasticsearch yellow 1 1 26 26 0 0 21 0 - 55.3% GREEN : All primary and replica shards are active. YELLOW : All primary shards are active, but some replica shards are not active. RED : Some primary shards are not active
3.LOG FILE tail -f /var/log/elasticsearch/elasticsearch.log
4. ELASTIC CONFIGURATION CHANGES - PORT [root@localhost ~]# grep http.port /etc/elasticsearch/elasticsearch.yml http.port: 9200 - SSL SETTINGS [root@localhost~]# grep xpack.security /etc/elasticsearch/elasticsearch.yml -A 4 xpack.security.enabled: true xpack.security.enrollment.enabled: true xpack.security.http.ssl: enabled: true keystore.path: certs/http.p12 - DATA PATH [root@localhost ~]# grep path.data /etc/elasticsearch/elasticsearch.yml path.data: /var/lib/elasticsearch
7. SIMPLE QUERIES - Health Check https://localhost:9200/_cluster/health Request: curl -k -X GET "https://localhost:9200/_cat/health?pretty" -H 'Content-Type: application/json' -u username:password Response: 1725881802 11:36:42 elasticsearch yellow 1 1 26 26 0 0 21 0 - 55.3% - Check pending tasks of cluster https://localhost:9200/_cluster/pending_tasks?pretty Request: curl -k -X GET "https://localhost:9200/_cluster/pending_tasks?pretty" -u username:password Response: { "tasks" : [ ] } - Check shards https://localhost:9200/_cat/shards?v Request: curl -k -X GET "https://localhost:9200/_cat/shards?pretty" -H 'Content-Type: application/json' -u username:password Response: mail-test_com-202409 0 p STARTED 1 12.8kb 12.8kb 192.168.40.72 localhost.test.com mail-test_com-202409 0 r UNASSIGNED expire-test_com-248 0 p STARTED 0 249b 249b 192.168.40.72 localhost.test.com ................... - List of indices https://localhost:9200/_cat/indices?v Request: curl -k -X GET "https://localhost:9200/_cat/indices?pretty" -H 'Content-Type: application/json' -u username:password Response: yellow open expire-test_com-202408-000001 _ARWflCXSV2nw-rUtyo2vA 1 1 0 0 249b 249b 249b yellow open test 7c322HqXQ9CDn41TDzk8qw 1 1 2 0 9.5kb 9.5kb 9.5kb yellow open audit-test_com-202408-000001 Ls1F41YaRDOydDToGth9uQ 1 1 0 0 249b 249b 249b .......................... - Check data in shard https://localhost:9200/audit-test_com-202406/_search?pretty Request: curl -k -X POST "https://localhost:9200/audit-test_com-202406*/_search" -H 'Content-Type: application/json' -u username:password Response: { "took" : 1, "timed_out" : false, "_shards" : { "total" : 0, "successful" : 0, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max_score" : 0.0, "hits" : [ .........] } } - Check data in shard with filter Request: curl -k -X POST "https://localhost:9200/audit-elk_com-202406*/_search" -H 'Content-Type: application/json' -u username:password -d ‘{ "query": { "bool": { "must": [ { "range": { "date": { "gte": "2024-04-01T00:00:00", "lte": "2024-04-30T19:00:00" } } } ] } }, "size": "10", "from": 1, "sort": [ { "date": { "order": "asc" } } ] }’ Response: { "took" : 1, "timed_out" : false, "_shards" : { "total" : 0, "successful" : 0, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max_score" : 0.0, "hits" : [............... ] } } - Check Status of Index https://localhost:9200/mail-test_com/stats?v Request: curl -k -X GET "https://localhost:9200/mail-test_com/_stats?pretty" -u username:password Response: { "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_all" : { "primaries" : { "docs" : { "count" : 595, "deleted" : 0 }, "shard_stats" : { "total_count" : 1 }, ........... - Count number of data in shard https://localhost:9200/mail-nstest_com/stats?v Request: curl -k -X POST "https://localhost:9200/audit-elk_com-202406*/_count?pretty" -H 'Content-Type: application/json' -u username:password Response: { "count" : 595, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } - Check Thread Pool https://localhost:9200/_cat/thread_pool?v Request: curl -X GET "https://localhost:9200/_cat/thread_pool?v" -u username:password Response: node_name name active queue rejected rocky93.netcorecloud.com analyze 0 0 0 rocky93.netcorecloud.com auto_complete 0 0 0 rocky93.netcorecloud.com azure_event_loop 0 0 0 rocky93.netcorecloud.com ccr 0 0 0 rocky93.netcorecloud.com cluster_coordination 0 0 0 rocky93.netcorecloud.com downsample_indexing 0 0 0 rocky93.netcorecloud.com esql 0 0 0 rocky93.netcorecloud.com esql_worker 0 0 0 rocky93.netcorecloud.com fetch_shard_started 0 0 0 rocky93.netcorecloud.com fetch_shard_store 0 0 0 rocky93.netcorecloud.com flush 0 0 0 ..................
8.INFORMATION ABOUT SCRIPT
1. To Search Indices for Specific Domain
[root@NDL4099 progs]# go run /opt/powerrelay/gologparser/src/progs/searchIndices.go
Usage of /tmp/go-build3038188552/b001/exe/searchIndices:
-d string
Your Domain Name (shorthand)
-date string
Specify Year Month. Eg. 202405
-domain string
Your Domain Name
-h Show help (shorthand)
-help
Show help
-t string
Specify Year Month (shorthand)
eg:
go run /opt/powerrelay/gologparser/src/progs/searchIndices.go -d test.com -t 202405 2. To Search Data from Specific Index.
[root@NDL4099 progs]# go run /opt/powerrelay/gologparser/src/progs/searchData.go
Usage of /tmp/go-build3900704349/b001/exe/searchData:
-d string
Your Domain Name (shorthand)
-domain string
Your Domain Name
-enddate string
Specify End Date Eg. 2024-05-14T14:38:41
-from string
Specify from
-h Show help (shorthand)
-help
Show help
-pageNumber int
Specify the page number (default 1)
-pageSize int
Specify the limit (default 10)
-rtype string
Specify Report Type Eg. mail|audit
-sortcolumn string
Specify sort column (default "time")
-sortorder string
Specify sort order (default "desc")
-startdate string
Specify Start Date Eg. 2024-05-14T14:38:41
-subj string
Specify subject
-to string
Specify to
eg: go run /opt/powerrelay/gologparser/src/progs/searchData.go -d test.com -startdate 2024-05-14T14:38:41 -enddate 2024-05-14T14:38:41 -pageNumber 1 -pageSize 10 -rtype mail -sortcolumn desc -subj test -to test@domain.com -from test@domain.com9. ROLLOVER(ROTATION) POLICY[root@powerrelay ~]# curl -k -X GET "https://localhost:9200/_ilm/policy/powerrelay_logs_policy?pretty" -u username:password
"powerrelay_logs_policy" : {
"version" : 1,
"modified_date" : "2024-06-18T06:12:49.729Z",
"policy" : {
"phases" : {
"warm" : {
"min_age" : "30d",
"actions" : {
"set_priority" : {
"priority" : 50
},
"forcemerge" : {
"max_num_segments" : 1
}
}
},
"cold" : {
"min_age" : "90d",
"actions" : {
"set_priority" : {
"priority" : 20
}
}
},
"hot" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_primary_shard_size" : "1gb"
}
}
}
}Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article