Ubuntu16.04安装zkui

2020-01-20

tonglin0325

1.git clone

1 2	git clone git@github.com:DeemOpen/zkui.git

2.打包

1 2	mvn clean install

3.复制config.cfg到target目录，并修改zk地址

4.配置 supervisor zkui.conf

[program:zkui]
directory=/home/lintong/software/zkui/target
command = java -jar zkui-2.0-SNAPSHOT-jar-with-dependencies.jar
user = lintong
autostart = false
autoresart = true
stderr_logfile = /var/log/supervisor/zkui.stderr.log
stdout_logfile = /var/log/supervisor/zkui.stdout.log

5.启动zookeeper

6.启动zkui

全文 >>

antlr解析hive语句

2020-01-19

tonglin0325

hive是使用antlr来解析的

parser要做的事情，是从无结构的字符串里面，解码产生有结构的数据结构（a parser is a function accepting strings as input and returning some structure as output），参考 Parser_combinator wiki

parser分成两种，一种是parser combinator，一种是parser generator，区别可以参考王垠的文章——对 Parser 的误解

全文 >>

面试题目——leetcode总结

2019-10-30

tonglin0325

有东西被加密了, 请输入密码查看.

全文 >>

Elasticsearch学习笔记——索引模板

2019-10-29

tonglin0325

在索引模板里面,date类型的字段的format支持多种类型,在es中全部会转换成long类型进行存储,参考

1 2	https://zhuanlan.zhihu.com/p/34240906

一个索引模板范例

{
  "order": 0,
  "index_patterns": [
    "ssssssssssssssss_test*"
  ],
  "settings": {
    "index": {
      "codec": "best_compression",
      "routing": {
        "allocation": {
          "require": {
            "box_type": "master"
          },
          "total_shards_per_node": "3"
        }
      },
      "refresh_interval": "60s",
      "unassigned": {
        "node_left": {
          "delayed_timeout": "900m"
        }
      },
      "number_of_shards": "3",
      "number_of_replicas": "1"
    }
  },
  "mappings": {
    "_default_": {
      "_source": {
        "enabled": true
      },
      "dynamic_templates": [
        {
          "string_template": {
            "mapping": {
              "type": "keyword"
            },
            "match_mapping_type": "string",
            "match": "*"
          }
        }
      ],
      "properties": {
        "a1": {
          "search_analyzer": "ik_max_word",
          "analyzer": "ik_max_word",
          "type": "text"
        },
        "a2": {
          "type": "float"
        },
        "a3": {
          "type": "integer"
        },
        "untitled": {
          "type": "float"
        },
        "a4": {
          "type": "long"
        },
        "a5": {
          "type": "double"
        },
        "a6": {
          "type": "ip"
        },
        "a8": {
          "type": "keyword"
        }
      },
      "_all": {
        "enabled": false
      }
    }
  },
  "aliases": {}
}

全文 >>

Elasticsearch6.2.1安装elasticsearch-sq插件

2019-10-27

tonglin0325

参考

1 2	https://github.com/NLPchina/elasticsearch-sql

1.下载插件

1 2	wget https://github.com/NLPchina/elasticsearch-sql/releases/download/6.2.1.0/elasticsearch-sql-6.2.1.0.zip

2.安装

1 2	./bin/elasticsearch-plugin instal file:///home/lintong/下载/elasticsearch-sql-6.2.1.0.zip

如果遇到 Exception in thread “main” java.lang.IllegalArgumentException: Unknown properties in plugin descriptor: [jvm, site]

解压zip文件,然后修改 plugin-descriptor.properties 文件

去掉下面两行后重新压缩成zip,然后再安装

1
2
3

site=
jvm=

参考

1 2	https://github.com/NLPchina/elasticsearch-sql/issues/610

3.下载前端组件

1 2	wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/es-sql-site-standalone.zip

解压

cd site-server
npm --registry https://registry.npm.taobao.org install express --save
node node-server.js

4.执行sql

1
2
3

curl -X GET "localhost:9200/_sql" -H 'Content-Type: application/json' -d'select * from es limit 10'
{"took":7,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":20000,"max_score":1.0,"hits":[{"_index":"es","_type":"people","_id":"0","_score":1.0,"_source":{"playerId":122,"logId":0,"dateTime":"2012-01-02 00:21:00","action":"私聊:нужен парень 0965188229"}},{"_index":"es","_type":"people","_id":"14","_score":1.0,"_source":{"playerId":343,"logId":14,"dateTime":"2012-01-02 00:35:00","action":"私聊:с новым годом!скучно девченкам.0988939122"}},{"_index":"es","_type":"people","_id":"19","_score":1.0,"_source":{"playerId":219,"logId":19,"dateTime":"2012-01-02 00:42:00","action":"私聊:ищу взрослую женщину звоните 0661464552"}},{"_index":"es","_type":"people","_id":"22","_score":1.0,"_source":{"playerId":822,"logId":22,"dateTime":"2012-01-02 00:45:00","action":"私聊:Жду смс от солидного мужчины за 47  0952342589"}},{"_index":"es","_type":"people","_id":"24","_score":1.0,"_source":{"playerId":111,"logId":24,"dateTime":"2012-01-02 00:47:00","action":"私聊:П27 ищу горячую снегурочку 0505618815"}},{"_index":"es","_type":"people","_id":"25","_score":1.0,"_source":{"playerId":204,"logId":25,"dateTime":"2012-01-02 00:48:00","action":"私聊:снегурочька хочет деда мороза.тома.0661796082"}},{"_index":"es","_type":"people","_id":"26","_score":1.0,"_source":{"playerId":789,"logId":26,"dateTime":"2012-01-02 00:49:00","action":"私聊:молодой парень 31год из сел.местн. Очень хочет познакомиться с м"}},{"_index":"es","_type":"people","_id":"29","_score":1.0,"_source":{"playerId":328,"logId":29,"dateTime":"2012-01-02 00:54:00","action":"私聊:ВСЕХ ПАРНЕЙ С НОВЫМ ГОДОМ ЛЮБВИ СЧАСТЬЯ.С УВ.СВЕТА 0666339477.ЗАДЕРЖ"}},{"_index":"es","_type":"people","_id":"40","_score":1.0,"_source":{"playerId":35,"logId":40,"dateTime":"2012-01-02 01:06:00","action":"私聊:ищю парня до21"}},{"_index":"es","_type":"people","_id":"41","_score":1.0,"_source":{"playerId":750,"logId":41,"dateTime":"2012-01-02 01:06:00","action":"私聊:ВСЕХ ДЕВУШЕК С НОВЫМ ГОДОМ Я ВАС ВСЕХ ЛЮБЛЮ 0507143632 СЕРЁГА"}}]}}

5.web界面

配置在site_configuration.json

全文 >>

MySQL学习笔记——乐观锁和悲观锁

2019-10-23

tonglin0325

场景：两个用户同时读取了数据库中的一条记录，此时用户A对其中一个字段的值进行了修改操作并进行了提交，后来用户B也对这个字段进行了修改，用户B的提交将会覆盖用户A提交的值

乐观锁和悲观锁

悲观锁：

每次去取数据，很悲观，都觉得会被别人修改，所以在拿数据的时候都会上锁。

简言之，共享资源每次都只给一个线程使用，其他线程阻塞，等第一个线程用完后再把资源转让给其他线程。

select … for update，synchronized和ReentranLock等都是悲观锁思想的体现。

全文 >>

Nexus上传npm包

2019-10-18

tonglin0325

1.创建npm仓库

私服仓库npm-hosted

全文 >>

Nginx使用笔记

2019-10-16

tonglin0325

1.请求转发

比如说我要将127.0.0.1/topics上的所有请求转发到xxx:xxx/上

修改 sudo vim /etc/nginx/nginx.conf

server {
       listen       80;
        server_name  127.0.0.1;
        location /topics {
            #root   html;
            #index  index.html index.htm;
        proxy_pass http://xxx:xxx;
        }
    }

2.代理前端和后端服务

3.Nginx使用本地缓存

4.代理

1.正向代理和反向代理

正向代理：代理服务器位于客户端和目标服务器之间，代表客户端向目标服务器发送请求。客户端需要配置代理服务器。

用途: 访问控制、内容过滤、匿名浏览、缓存。我们用来kexue上网的工具就属于正向代理。

反向代理：代理服务器位于客户端和目标服务器之间，代表服务器处理客户端请求。客户端不知道反向代理的存在。

用途: 负载均衡、安全防护、缓存、SSL加速。比如使用Nginx，HAProxy，Apache HTTP Server等作为反向代理。

2.透明代理和非透明代理

透明代理和非透明代理都是正向代理，位于客户端和目标服务器之间，代理客户端向目标服务器发送请求。

透明代理：网络管理员在网络边界部署透明代理来过滤不良内容。用户浏览网站时，代理自动拦截和检查内容，而用户并不知道代理的存在。

非透明代理：用户配置浏览器使用非透明代理，以便匿名访问互联网。用户在浏览器中手动设置代理服务器地址和端口。

5.X-Forwarded-For，X-Real-IP和Remote Address

X-Forwarded-For通常用于标识通过 HTTP 代理或负载均衡器的原始客户端 IP 地址。X-Forwarded-For 是一个 HTTP 扩展头部。HTTP/1.1（RFC 2616）协议并没有对它的定义，它最开始是由 Squid 这个缓存代理软件引入，用来表示 HTTP 请求端真实 IP。如今它已经成为事实上的标准，被各大 HTTP 代理、负载均衡等转发服务广泛使用，并被写入 RFC 7239（Forwarded HTTP Extension）标准之中。示例值：

全文 >>

使用filebeat发送nginx日志到kafka

2019-10-14

tonglin0325

1.配置filebeat_nginx.yml

filebeat.modules:
- module: nginx
  access:
    enabled: true
    var.paths: ["/var/log/nginx/access.log*"]
  error:
    enabled: true
    var.paths: ["/var/log/nginx/error.log*"]

#----------------------------------Kafka output--------------------------------#
output.kafka:
  version: "1.0.1"
  enabled: true
  hosts: ['xxx:9092', 'xxx:9092', 'xxx:9092']
  topic: 'temp'
  required_acks: 1  #default
  compression: gzip #default
  max_message_bytes: 1000000 #default
  codec.format:
    string: '%{[message]}'

2.启动filebeat

1 2	./filebeat -e -c filebeat_nginx.yml

3.访问nginx

1 2	tail -f /var/log/nginx/access.log

日志文件输出

{"ts":"2019-10-14 10:53:22","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:23","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:23","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:30","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:31","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}

kafka输出

{"ts":"2019-10-14 10:53:23","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:23","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:22","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:30","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}
{"ts":"2019-10-14 10:53:31","host":"127.0.0.1","clientip":"127.0.0.1","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"localhost","url":"/index.html","domain":"localhost","xff":"-","referer":"-","status":"304"}

全文 >>

由crt和key文件生成keystore文件

2019-10-14

tonglin0325

该图转自知乎海棠依旧

全文 >>

tonglin0325的个人主页

Ubuntu16.04安装zkui

antlr解析hive语句

面试题目——leetcode总结

Elasticsearch学习笔记——索引模板

Elasticsearch6.2.1安装elasticsearch-sq插件

MySQL学习笔记——乐观锁和悲观锁

乐观锁和悲观锁

悲观锁：

Nexus上传npm包

Nginx使用笔记

1.请求转发

2.代理前端和后端服务

3.Nginx使用本地缓存

4.代理

1.正向代理和反向代理

2.透明代理和非透明代理

5.X-Forwarded-For，X-Real-IP和Remote Address

使用filebeat发送nginx日志到kafka

由crt和key文件生成keystore文件

tonglin0325

标签

标签云

文章归档

最近文章