tonglin0325的个人主页

ElasticSearch学习笔记——插件开发

参考

1
2
3
4
5
6
https://dzone.com/articles/elasticsearch5-how-to-build-a-plugin-and-add-a-lis
https://github.com/chrisshayan/es-changes-feed-plugin
https://blog.csdn.net/qq_16164711/article/details/87872383
https://blog.gaiaproject.club/es-develop-plugin/
https://blog.51cto.com/13755625/2117995

Es的插件主要有如下几种类型,参考

1
2
https://github.com/elastic/elasticsearch/tree/master/server/src/main/java/org/elasticsearch/plugins

 

API Extension Plugins API拓展插件:API extension plugins add new functionality to Elasticsearch by adding new APIs or features, usually to do with search or mapping.

Analysis Plugins 解析器插件:Analysis plugins extend Elasticsearch by adding new analyzers, tokenizers, token filters, or character filters to Elasticsearch.

Alerting Plugins 告警插件:Alerting plugins allow Elasticsearch to monitor indices and to trigger alerts when thresholds are breached.

Discovery Plugins 发现插件:Discovery plugins extend Elasticsearch by adding new discovery mechanisms that can be used instead of Zen Discovery.

Ingest Plugins 摄取插件:The ingest plugins extend Elasticsearch by providing additional ingest node capabilities.

Management Plugins 管理插件:Management plugins offer UIs for managing and interacting with Elasticsearch.

Mapper Plugins Mapper插件:Mapper plugins allow new field datatypes to be added to Elasticsearch.

Security Plugins 安全插件:Security plugins add a security layer to Elasticsearch.

Snapshot/Restore Repository Plugins 快照插件:Repository plugins extend the Snapshot/Restore functionality in Elasticsearch by adding repositories backed by the cloud or by distributed file systems: 

Store Plugins 存储插件:Store plugins offer alternatives to default Lucene stores.

 

ActionPlugin:An additional extension point for {@link Plugin}s that extends Elasticsearch’s scripting functionality

1
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ActionPlugin.java

CircuitBreakerPlugin 断路器插件:An extension point for {@link Plugin} implementations to add custom circuit breakers

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/CircuitBreakerPlugin.java

ClusterPlugin 集群插件:An extension point for {@link Plugin} implementations to customer behavior of cluster management.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ClusterPlugin.java

EnginePlugin 引擎插件:A plugin that provides alternative engine implementations.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/EnginePlugin.java

ExtensiblePlugin 拓展插件:This class provides a callback for extensible plugins to be informed of other plugins which extend them.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ExtensiblePlugin.java

IndexStorePlugin 索引存储插件:A plugin that provides alternative directory implementations.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/IndexStorePlugin.java

NetworkPlugin 网络插件:Plugin for extending network and transport related classes

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/NetworkPlugin.java

PersistentTaskPlugin 持续任务插件:Plugin for registering persistent tasks executors.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/PersistentTaskPlugin.java

ReloadablePlugin Reload插件:An extension point for {@link Plugin}s that can be reloaded. There is no clear definition about what reloading a plugin actually means. When a plugin is reloaded it might rebuild any internal members. Plugins usually implement this interface in order to reread the values of {@code SecureSetting}s and then rebuild any dependent internal members.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ReloadablePlugin.java

RestCompatibilityPlugin Rest兼容性插件:An extension point for Compatible API plugin implementation.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/RestCompatibilityPlugin.java

ScriptPlugin 脚本插件:An additional extension point for {@link Plugin}s that extends Elasticsearch’s scripting functionality.

1
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ScriptPlugin.java

SearchPlugin 搜索插件:Plugin for extending search time behavior.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java

SystemIndexPlugin 系统索引插件:Plugin for defining system indices. Extends {@link ActionPlugin} because system indices must be accessed via APIs added by the plugin that owns the system index, rather than standard APIs.

1
2
https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/SystemIndexPlugin.java

 

编写es插件的时候需要提供2类文件

1
2
3
4
lintong@master:~/software/apache/elasticsearch-6.2.4/plugins/analysis-ik$ ls
commons-codec-1.9.jar elasticsearch-analysis-ik-6.2.4.jar httpcore-4.4.4.jar
commons-logging-1.2.jar httpclient-4.5.2.jar plugin-descriptor.properties

一个是plugin-descriptor.properties文件,里面是插件的一些信息,比如

1
2
3
4
5
6
7
description=自定义插件
version=1.0-SNAPSHOT
name=my plugin
classname=com.xxxx.xx.MetadataListenerPlugin
java.version=1.8
elasticsearch.version=6.2.4

一个是插件的jar包文件

在es启动的过程中,首先会启动节点node,之后PluginsService会加载这个node目录下的插件,可以看到my plugin这个插件已经成功加载

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[2021-04-29T10:14:15,061][INFO ][o.e.n.Node               ] [] initializing ...
[2021-04-29T10:14:15,223][INFO ][o.e.e.NodeEnvironment ] [gl4ygFd] using [1] data paths, mounts [[/media/xxx/14201D6AD04D90DA (/dev/sdb5)]], net usable_space [421.2gb], net total_space [466gb], types [fuseblk]
[2021-04-29T10:14:15,223][INFO ][o.e.e.NodeEnvironment ] [gl4ygFd] heap size [990.7mb], compressed ordinary object pointers [true]
[2021-04-29T10:14:15,587][INFO ][o.e.n.Node ] node name [gl4ygFd] derived from node ID [gl4ygFdaRCKyTcLB6SoYJg]; set [node.name] to override
[2021-04-29T10:14:15,588][INFO ][o.e.n.Node ] version[6.2.4], pid[1162], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/4.4.0-165-generic/amd64], JVM[Private Build/OpenJDK 64-Bit Server VM/1.8.0_222/25.222-b10]
[2021-04-29T10:14:15,590][INFO ][o.e.n.Node ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.P9YagNAJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/lintong/software/apache/elasticsearch-6.2.4, -Des.path.conf=/home/lintong/software/apache/elasticsearch-6.2.4/config]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [aggs-matrix-stats]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [analysis-common]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [ingest-common]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [lang-expression]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [lang-mustache]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [lang-painless]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [mapper-extras]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [parent-join]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [percolator]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [rank-eval]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [reindex]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [repository-url]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [transport-netty4]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded module [tribe]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded plugin [analysis-ik]
[2021-04-29T10:14:16,150][INFO ][o.e.p.PluginsService ] [gl4ygFd] loaded plugin [my plugin]