Zk学习笔记——应用场景

2020-07-12

tonglin0325

1.Master选举

在分布式系统中，需要选举一台机器作为master或者leader。

这时候，可以选择一个跟节点，比如/master，然后多台机器同时像这个节点创建一个子节点/master/lock，利用zookeeper的特性，最终只有一台机器能否创建成功，成功的那台机器就是Master；

其他机器注册watch到这个子节点，然后当master宕机的时候，其他机器就会重新开始选举。

全文 >>

Zk学习笔记——ZkClient

2020-07-11

tonglin0325

ZkClient是开源的zk客户端，对Zookeeper原生的java api进行了封装，实现了诸如session超时重连，watcher反复注册等功能。

依赖的话有

<dependency>
    <groupId>com.101tec</groupId>
    <artifactId>zkclient</artifactId>
    <version>0.10</version>
</dependency>

和

<dependency>
    <groupId>com.github.sgroschupf</groupId>
    <artifactId>zkclient</artifactId>
    <version>0.1</version>
</dependency>

前者使用较为广泛，使用其的项目包括kafka，dubbo等

全文 >>

Zk学习笔记——修改节点

2020-07-11

tonglin0325

参考：从Paxos到Zookeeper分布式一致性原理和实践

使用的zk依赖是cdh5.16.2的3.4.5

<!-- zookeeper -->
<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.5-cdh5.16.2</version>
</dependency>

代码，创建一个临时节点，并赋值，此时初始的节点version=0

使用version=-1对这个节点进行修改，然后查看这个stat，version=1，

使用version=1对这个节点进行修改，成功，然后查看这个stat，version=2，

再次使用version=1对这个节点进行修改，失败，抛出BadVersionException

package com.bigdata.zookeeper;


import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

import java.util.List;
import java.util.concurrent.CountDownLatch;

public class ZkExample implements Watcher {

    public static CountDownLatch connectedSemaphore = new CountDownLatch(1);
    private static ZooKeeper zk;

    public static void main(String[] args) throws Exception {
        zk = new ZooKeeper("master:2181", 5000, new ZkExample());
        System.out.println(zk.getState());
        try {
            connectedSemaphore.await();
            // 创建一个节点
            String path = "/app6";
            zk.create(path, "123".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
            zk.getData(path, true, null);

            // version=-1，修改节点
            Stat stat = zk.setData(path, "555".getBytes(), -1);
            System.out.println(stat.getVersion());

            // version=1，修改节点
            Stat stat2 = zk.setData(path, "55555".getBytes(), stat.getVersion());
            System.out.println(stat2.getVersion());

            // version=1，再次修改节点
            Stat stat3 = zk.setData(path, "555555555".getBytes(), stat.getVersion());

            Thread.sleep(10000); // 10秒延时

        } catch (InterruptedException e) {
            System.out.println("Zk session established" + e);
        }
    }

    @Override
    public void process(WatchedEvent watchedEvent) {
        System.out.println(watchedEvent);
        if (Event.KeeperState.SyncConnected == watchedEvent.getState()) {
            if (Event.EventType.None == watchedEvent.getType() &amp;&amp; null == watchedEvent.getPath()) {
                connectedSemaphore.countDown();
            } else if (watchedEvent.getType() == Event.EventType.NodeChildrenChanged) {
                try {
                    System.out.println(zk.getChildren(watchedEvent.getPath(), true));
                } catch (Exception e) {

                }
            }

        }
    }
}

输出

全文 >>

Zk学习笔记——检测节点是否存在

2020-07-11

tonglin0325

参考：从Paxos到Zookeeper分布式一致性原理和实践

使用的zk依赖是cdh5.16.2的3.4.5

<!-- zookeeper -->
<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.5-cdh5.16.2</version>
</dependency>

全文 >>

Pac代理文件

2020-07-09

tonglin0325

用于将本地特定规则的请求转发到某个ip port代理

mac系统需要编辑~/.$hadow$ocksX-NG/gfwli$t.js中的FindProxyForURL函数

var proxy = "PROXY 127.0.0.1:xxxx; DIRECT;";

var rules = [
  "||xxx.xxx.com",
];

/*
* This file is part of Adblock Plus <http://adblockplus.org/>,
* Copyright (C) 2006-2014 Eyeo GmbH
*
* Adblock Plus is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 3 as
* published by the Free Software Foundation.
*
* Adblock Plus is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Adblock Plus.  If not, see <http://www.gnu.org/licenses/>.
*/

function createDict()
{
    var result = {};
    result.__proto__ = null;
    return result;
}

function getOwnPropertyDescriptor(obj, key)
{
    if (obj.hasOwnProperty(key))
    {
        return obj[key];
    }
    return null;
}

function extend(subclass, superclass, definition)
{
    if (Object.__proto__)
    {
        definition.__proto__ = superclass.prototype;
        subclass.prototype = definition;
    }
    else
    {
        var tmpclass = function(){}, ret;
        tmpclass.prototype = superclass.prototype;
        subclass.prototype = new tmpclass();
        subclass.prototype.constructor = superclass;
        for (var i in definition)
        {
            if (definition.hasOwnProperty(i))
            {
                subclass.prototype[i] = definition[i];
            }
        }
    }
}

function Filter(text)
{
    this.text = text;
    this.subscriptions = [];
}
Filter.prototype = {
    text: null,
    subscriptions: null,
    toString: function()
    {
        return this.text;
    }
};
Filter.knownFilters = createDict();
Filter.elemhideRegExp = /^([^\/\*\|\@"!]*?)#(\@)?(?:([\w\-]+|\*)((?:\([\w\-]+(?:[$^*]?=[^\(\)"]*)?\))*)|#([^{}]+))$/;
Filter.regexpRegExp = /^(@@)?\/.*\/(?:\$~?[\w\-]+(?:=[^,\s]+)?(?:,~?[\w\-]+(?:=[^,\s]+)?)*)?$/;
Filter.optionsRegExp = /\$(~?[\w\-]+(?:=[^,\s]+)?(?:,~?[\w\-]+(?:=[^,\s]+)?)*)$/;
Filter.fromText = function(text)
{
    if (text in Filter.knownFilters)
    {
        return Filter.knownFilters[text];
    }
    var ret;
    if (text[0] == "!")
    {
        ret = new CommentFilter(text);
    }
    else
    {
        ret = RegExpFilter.fromText(text);
    }
    Filter.knownFilters[ret.text] = ret;
    return ret;
};

function InvalidFilter(text, reason)
{
    Filter.call(this, text);
    this.reason = reason;
}
extend(InvalidFilter, Filter, {
    reason: null
});

function CommentFilter(text)
{
    Filter.call(this, text);
}
extend(CommentFilter, Filter, {
});

function ActiveFilter(text, domains)
{
    Filter.call(this, text);
    this.domainSource = domains;
}
extend(ActiveFilter, Filter, {
    domainSource: null,
    domainSeparator: null,
    ignoreTrailingDot: true,
    domainSourceIsUpperCase: false,
    getDomains: function()
    {
        var prop = getOwnPropertyDescriptor(this, "domains");
        if (prop)
        {
            return prop;
        }
        var domains = null;
        if (this.domainSource)
        {
            var source = this.domainSource;
            if (!this.domainSourceIsUpperCase)
            {
                source = source.toUpperCase();
            }
            var list = source.split(this.domainSeparator);
            if (list.length == 1 &amp;&amp; list[0][0] != "~")
            {
                domains = createDict();
                domains[""] = false;
                if (this.ignoreTrailingDot)
                {
                    list[0] = list[0].replace(/\.+$/, "");
                }
                domains[list[0]] = true;
            }
            else
            {
                var hasIncludes = false;
                for (var i = 0; i < list.length; i++)
                {
                    var domain = list[i];
                    if (this.ignoreTrailingDot)
                    {
                        domain = domain.replace(/\.+$/, "");
                    }
                    if (domain == "")
                    {
                        continue;
                    }
                    var include;
                    if (domain[0] == "~")
                    {
                        include = false;
                        domain = domain.substr(1);
                    }
                    else
                    {
                        include = true;
                        hasIncludes = true;
                    }
                    if (!domains)
                    {
                        domains = createDict();
                    }
                    domains[domain] = include;
                }
                domains[""] = !hasIncludes;
            }
            this.domainSource = null;
        }
        return this.domains;
    },
    sitekeys: null,
    isActiveOnDomain: function(docDomain, sitekey)
    {
        if (this.getSitekeys() &amp;&amp; (!sitekey || this.getSitekeys().indexOf(sitekey.toUpperCase()) < 0))
        {
            return false;
        }
        if (!this.getDomains())
        {
            return true;
        }
        if (!docDomain)
        {
            return this.getDomains()[""];
        }
        if (this.ignoreTrailingDot)
        {
            docDomain = docDomain.replace(/\.+$/, "");
        }
        docDomain = docDomain.toUpperCase();
        while (true)
        {
            if (docDomain in this.getDomains())
            {
                return this.domains[docDomain];
            }
            var nextDot = docDomain.indexOf(".");
            if (nextDot < 0)
            {
                break;
            }
            docDomain = docDomain.substr(nextDot + 1);
        }
        return this.domains[""];
    },
    isActiveOnlyOnDomain: function(docDomain)
    {
        if (!docDomain || !this.getDomains() || this.getDomains()[""])
        {
            return false;
        }
        if (this.ignoreTrailingDot)
        {
            docDomain = docDomain.replace(/\.+$/, "");
        }
        docDomain = docDomain.toUpperCase();
        for (var domain in this.getDomains())
        {
            if (this.domains[domain] &amp;&amp; domain != docDomain &amp;&amp; (domain.length <= docDomain.length || domain.indexOf("." + docDomain) != domain.length - docDomain.length - 1))
            {
                return false;
            }
        }
        return true;
    }
});

function RegExpFilter(text, regexpSource, contentType, matchCase, domains, thirdParty, sitekeys)
{
    ActiveFilter.call(this, text, domains, sitekeys);
    if (contentType != null)
    {
        this.contentType = contentType;
    }
    if (matchCase)
    {
        this.matchCase = matchCase;
    }
    if (thirdParty != null)
    {
        this.thirdParty = thirdParty;
    }
    if (sitekeys != null)
    {
        this.sitekeySource = sitekeys;
    }
    if (regexpSource.length >= 2 &amp;&amp; regexpSource[0] == "/" &amp;&amp; regexpSource[regexpSource.length - 1] == "/")
    {
        var regexp = new RegExp(regexpSource.substr(1, regexpSource.length - 2), this.matchCase ? "" : "i");
        this.regexp = regexp;
    }
    else
    {
        this.regexpSource = regexpSource;
    }
}
extend(RegExpFilter, ActiveFilter, {
    domainSourceIsUpperCase: true,
    length: 1,
    domainSeparator: "|",
    regexpSource: null,
    getRegexp: function()
    {
        var prop = getOwnPropertyDescriptor(this, "regexp");
        if (prop)
        {
            return prop;
        }
        var source = this.regexpSource.replace(/\*+/g, "*").replace(/\^\|$/, "^").replace(/\W/g, "\\$&amp;").replace(/\\\*/g, ".*").replace(/\\\^/g, "(?:[\\x00-\\x24\\x26-\\x2C\\x2F\\x3A-\\x40\\x5B-\\x5E\\x60\\x7B-\\x7F]|$)").replace(/^\\\|\\\|/, "^[\\w\\-]+:\\/+(?!\\/)(?:[^\\/]+\\.)?").replace(/^\\\|/, "^").replace(/\\\|$/, "$").replace(/^(\.\*)/, "").replace(/(\.\*)$/, "");
        var regexp = new RegExp(source, this.matchCase ? "" : "i");
        this.regexp = regexp;
        return regexp;
    },
    contentType: 2147483647,
    matchCase: false,
    thirdParty: null,
    sitekeySource: null,
    getSitekeys: function()
    {
        var prop = getOwnPropertyDescriptor(this, "sitekeys");
        if (prop)
        {
            return prop;
        }
        var sitekeys = null;
        if (this.sitekeySource)
        {
            sitekeys = this.sitekeySource.split("|");
            this.sitekeySource = null;
        }
        this.sitekeys = sitekeys;
        return this.sitekeys;
    },
    matches: function(location, contentType, docDomain, thirdParty, sitekey)
    {
        if (this.getRegexp().test(location) &amp;&amp; this.isActiveOnDomain(docDomain, sitekey))
        {
            return true;
        }
        return false;
    }
});
RegExpFilter.prototype["0"] = "#this";
RegExpFilter.fromText = function(text)
{
    var blocking = true;
    var origText = text;
    if (text.indexOf("@@") == 0)
    {
        blocking = false;
        text = text.substr(2);
    }
    var contentType = null;
    var matchCase = null;
    var domains = null;
    var sitekeys = null;
    var thirdParty = null;
    var collapse = null;
    var options;
    var match = text.indexOf("$") >= 0 ? Filter.optionsRegExp.exec(text) : null;
    if (match)
    {
        options = match[1].toUpperCase().split(",");
        text = match.input.substr(0, match.index);
        for (var _loopIndex6 = 0; _loopIndex6 < options.length; ++_loopIndex6)
        {
            var option = options[_loopIndex6];
            var value = null;
            var separatorIndex = option.indexOf("=");
            if (separatorIndex >= 0)
            {
                value = option.substr(separatorIndex + 1);
                option = option.substr(0, separatorIndex);
            }
            option = option.replace(/-/, "_");
            if (option in RegExpFilter.typeMap)
            {
                if (contentType == null)
                {
                    contentType = 0;
                }
                contentType |= RegExpFilter.typeMap[option];
            }
            else if (option[0] == "~" &amp;&amp; option.substr(1) in RegExpFilter.typeMap)
            {
                if (contentType == null)
                {
                    contentType = RegExpFilter.prototype.contentType;
                }
                contentType &amp;= ~RegExpFilter.typeMap[option.substr(1)];
            }
            else if (option == "MATCH_CASE")
            {
                matchCase = true;
            }
            else if (option == "~MATCH_CASE")
            {
                matchCase = false;
            }
            else if (option == "DOMAIN" &amp;&amp; typeof value != "undefined")
            {
                domains = value;
            }
            else if (option == "THIRD_PARTY")
            {
                thirdParty = true;
            }
            else if (option == "~THIRD_PARTY")
            {
                thirdParty = false;
            }
            else if (option == "COLLAPSE")
            {
                collapse = true;
            }
            else if (option == "~COLLAPSE")
            {
                collapse = false;
            }
            else if (option == "SITEKEY" &amp;&amp; typeof value != "undefined")
            {
                sitekeys = value;
            }
            else
            {
                return new InvalidFilter(origText, "Unknown option " + option.toLowerCase());
            }
        }
    }
    if (!blocking &amp;&amp; (contentType == null || contentType &amp; RegExpFilter.typeMap.DOCUMENT) &amp;&amp; (!options || options.indexOf("DOCUMENT") < 0) &amp;&amp; !/^\|?[\w\-]+:/.test(text))
    {
        if (contentType == null)
        {
            contentType = RegExpFilter.prototype.contentType;
        }
        contentType &amp;= ~RegExpFilter.typeMap.DOCUMENT;
    }
    try
    {
        if (blocking)
        {
            return new BlockingFilter(origText, text, contentType, matchCase, domains, thirdParty, sitekeys, collapse);
        }
        else
        {
            return new WhitelistFilter(origText, text, contentType, matchCase, domains, thirdParty, sitekeys);
        }
    }
    catch (e)
    {
        return new InvalidFilter(origText, e);
    }
};
RegExpFilter.typeMap = {
    OTHER: 1,
    SCRIPT: 2,
    IMAGE: 4,
    STYLESHEET: 8,
    OBJECT: 16,
    SUBDOCUMENT: 32,
    DOCUMENT: 64,
    XBL: 1,
    PING: 1,
    XMLHTTPREQUEST: 2048,
    OBJECT_SUBREQUEST: 4096,
    DTD: 1,
    MEDIA: 16384,
    FONT: 32768,
    BACKGROUND: 4,
    POPUP: 268435456,
    ELEMHIDE: 1073741824
};
RegExpFilter.prototype.contentType &amp;= ~ (RegExpFilter.typeMap.ELEMHIDE | RegExpFilter.typeMap.POPUP);

function BlockingFilter(text, regexpSource, contentType, matchCase, domains, thirdParty, sitekeys, collapse)
{
    RegExpFilter.call(this, text, regexpSource, contentType, matchCase, domains, thirdParty, sitekeys);
    this.collapse = collapse;
}
extend(BlockingFilter, RegExpFilter, {
    collapse: null
});

function WhitelistFilter(text, regexpSource, contentType, matchCase, domains, thirdParty, sitekeys)
{
    RegExpFilter.call(this, text, regexpSource, contentType, matchCase, domains, thirdParty, sitekeys);
}
extend(WhitelistFilter, RegExpFilter, {
});

function Matcher()
{
    this.clear();
}
Matcher.prototype = {
    filterByKeyword: null,
    keywordByFilter: null,
    clear: function()
    {
        this.filterByKeyword = createDict();
        this.keywordByFilter = createDict();
    },
    add: function(filter)
    {
        if (filter.text in this.keywordByFilter)
        {
            return;
        }
        var keyword = this.findKeyword(filter);
        var oldEntry = this.filterByKeyword[keyword];
        if (typeof oldEntry == "undefined")
        {
            this.filterByKeyword[keyword] = filter;
        }
        else if (oldEntry.length == 1)
        {
            this.filterByKeyword[keyword] = [oldEntry, filter];
        }
        else
        {
            oldEntry.push(filter);
        }
        this.keywordByFilter[filter.text] = keyword;
    },
    remove: function(filter)
    {
        if (!(filter.text in this.keywordByFilter))
        {
            return;
        }
        var keyword = this.keywordByFilter[filter.text];
        var list = this.filterByKeyword[keyword];
        if (list.length <= 1)
        {
            delete this.filterByKeyword[keyword];
        }
        else
        {
            var index = list.indexOf(filter);
            if (index >= 0)
            {
                list.splice(index, 1);
                if (list.length == 1)
                {
                    this.filterByKeyword[keyword] = list[0];
                }
            }
        }
        delete this.keywordByFilter[filter.text];
    },
    findKeyword: function(filter)
    {
        var result = "";
        var text = filter.text;
        if (Filter.regexpRegExp.test(text))
        {
            return result;
        }
        var match = Filter.optionsRegExp.exec(text);
        if (match)
        {
            text = match.input.substr(0, match.index);
        }
        if (text.substr(0, 2) == "@@")
        {
            text = text.substr(2);
        }
        var candidates = text.toLowerCase().match(/[^a-z0-9%*][a-z0-9%]{3,}(?=[^a-z0-9%*])/g);
        if (!candidates)
        {
            return result;
        }
        var hash = this.filterByKeyword;
        var resultCount = 16777215;
        var resultLength = 0;
        for (var i = 0, l = candidates.length; i < l; i++)
        {
            var candidate = candidates[i].substr(1);
            var count = candidate in hash ? hash[candidate].length : 0;
            if (count < resultCount || count == resultCount &amp;&amp; candidate.length > resultLength)
            {
                result = candidate;
                resultCount = count;
                resultLength = candidate.length;
            }
        }
        return result;
    },
    hasFilter: function(filter)
    {
        return filter.text in this.keywordByFilter;
    },
    getKeywordForFilter: function(filter)
    {
        if (filter.text in this.keywordByFilter)
        {
            return this.keywordByFilter[filter.text];
        }
        else
        {
            return null;
        }
    },
    _checkEntryMatch: function(keyword, location, contentType, docDomain, thirdParty, sitekey)
    {
        var list = this.filterByKeyword[keyword];
        for (var i = 0; i < list.length; i++)
        {
            var filter = list[i];
            if (filter == "#this")
            {
                filter = list;
            }
            if (filter.matches(location, contentType, docDomain, thirdParty, sitekey))
            {
                return filter;
            }
        }
        return null;
    },
    matchesAny: function(location, contentType, docDomain, thirdParty, sitekey)
    {
        var candidates = location.toLowerCase().match(/[a-z0-9%]{3,}/g);
        if (candidates === null)
        {
            candidates = [];
        }
        candidates.push("");
        for (var i = 0, l = candidates.length; i < l; i++)
        {
            var substr = candidates[i];
            if (substr in this.filterByKeyword)
            {
                var result = this._checkEntryMatch(substr, location, contentType, docDomain, thirdParty, sitekey);
                if (result)
                {
                    return result;
                }
            }
        }
        return null;
    }
};

function CombinedMatcher()
{
    this.blacklist = new Matcher();
    this.whitelist = new Matcher();
    this.resultCache = createDict();
}
CombinedMatcher.maxCacheEntries = 1000;
CombinedMatcher.prototype = {
    blacklist: null,
    whitelist: null,
    resultCache: null,
    cacheEntries: 0,
    clear: function()
    {
        this.blacklist.clear();
        this.whitelist.clear();
        this.resultCache = createDict();
        this.cacheEntries = 0;
    },
    add: function(filter)
    {
        if (filter instanceof WhitelistFilter)
        {
            this.whitelist.add(filter);
        }
        else
        {
            this.blacklist.add(filter);
        }
        if (this.cacheEntries > 0)
        {
            this.resultCache = createDict();
            this.cacheEntries = 0;
        }
    },
    remove: function(filter)
    {
        if (filter instanceof WhitelistFilter)
        {
            this.whitelist.remove(filter);
        }
        else
        {
            this.blacklist.remove(filter);
        }
        if (this.cacheEntries > 0)
        {
            this.resultCache = createDict();
            this.cacheEntries = 0;
        }
    },
    findKeyword: function(filter)
    {
        if (filter instanceof WhitelistFilter)
        {
            return this.whitelist.findKeyword(filter);
        }
        else
        {
            return this.blacklist.findKeyword(filter);
        }
    },
    hasFilter: function(filter)
    {
        if (filter instanceof WhitelistFilter)
        {
            return this.whitelist.hasFilter(filter);
        }
        else
        {
            return this.blacklist.hasFilter(filter);
        }
    },
    getKeywordForFilter: function(filter)
    {
        if (filter instanceof WhitelistFilter)
        {
            return this.whitelist.getKeywordForFilter(filter);
        }
        else
        {
            return this.blacklist.getKeywordForFilter(filter);
        }
    },
    isSlowFilter: function(filter)
    {
        var matcher = filter instanceof WhitelistFilter ? this.whitelist : this.blacklist;
        if (matcher.hasFilter(filter))
        {
            return !matcher.getKeywordForFilter(filter);
        }
        else
        {
            return !matcher.findKeyword(filter);
        }
    },
    matchesAnyInternal: function(location, contentType, docDomain, thirdParty, sitekey)
    {
        var candidates = location.toLowerCase().match(/[a-z0-9%]{3,}/g);
        if (candidates === null)
        {
            candidates = [];
        }
        candidates.push("");
        var blacklistHit = null;
        for (var i = 0, l = candidates.length; i < l; i++)
        {
            var substr = candidates[i];
            if (substr in this.whitelist.filterByKeyword)
            {
                var result = this.whitelist._checkEntryMatch(substr, location, contentType, docDomain, thirdParty, sitekey);
                if (result)
                {
                    return result;
                }
            }
            if (substr in this.blacklist.filterByKeyword &amp;&amp; blacklistHit === null)
            {
                blacklistHit = this.blacklist._checkEntryMatch(substr, location, contentType, docDomain, thirdParty, sitekey);
            }
        }
        return blacklistHit;
    },
    matchesAny: function(location, docDomain)
    {
        var key = location + " " + docDomain + " ";
        if (key in this.resultCache)
        {
            return this.resultCache[key];
        }
        var result = this.matchesAnyInternal(location, 0, docDomain, null, null);
        if (this.cacheEntries >= CombinedMatcher.maxCacheEntries)
        {
            this.resultCache = createDict();
            this.cacheEntries = 0;
        }
        this.resultCache[key] = result;
        this.cacheEntries++;
        return result;
    }
};
var defaultMatcher = new CombinedMatcher();

var direct = 'DIRECT;';

for (var i = 0; i < rules.length; i++) {
    defaultMatcher.add(Filter.fromText(rules[i]));
}

function FindProxyForURL(url, host) {
    if (defaultMatcher.matchesAny(url, host) instanceof BlockingFilter) {
        return proxy;
    }
    return direct;
}

全文 >>

Zk学习笔记——创建节点

2020-07-05

tonglin0325

参考：从Paxos到Zookeeper分布式一致性原理和实践

使用的zk依赖是cdh5.16.2的3.4.5

<!-- zookeeper -->
<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.5-cdh5.16.2</version>
</dependency>

Zk在创建节点的有同步创建和异步创建，但是Zk是不支持递归创建节点，即父节点必须存储。

在创建同名的节点的时候，会抛出NodeExistsException异常。

Zookeeper节点内容只支持字节数组（byte[]）

同步创建节点代码，其中创建了一个永久节点，key是/app1，value是123

package com.bigdata.zookeeper;


import org.apache.zookeeper.*;
import java.util.concurrent.CountDownLatch;

public class ZkExample implements Watcher {

    public static CountDownLatch connectedSemaphore = new CountDownLatch(1);

    public static void main(String[] args) throws Exception {
        ZooKeeper zk = new ZooKeeper("master:2181", 5000, new ZkExample());
        System.out.println(zk.getState());
        try {
            connectedSemaphore.await();
            String path = zk.create("/app1", "123".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            System.out.println("success create znode: " + path);
        } catch (InterruptedException e) {
            System.out.println("Zk session established" + e);
        }
    }

    @Override
    public void process(WatchedEvent watchedEvent) {
        System.out.println(watchedEvent);
        if (Event.KeeperState.SyncConnected == watchedEvent.getState()) {
            connectedSemaphore.countDown();
        }
    }
}

输出，使用Zkui查看zk中的数据，能发现多出了/app1的节点，节点的值是123

全文 >>

Zk学习笔记——读取节点

2020-07-05

tonglin0325

参考：从Paxos到Zookeeper分布式一致性原理和实践

使用的zk依赖是cdh5.16.2的3.4.5

<!-- zookeeper -->
<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.5-cdh5.16.2</version>
</dependency>

代码，在初始化zk的时候，会触发一个watchEvent，将CountDownLatch-1=0，从而开始读取/app1的value，此时为12345，

然后将其修改成123，会再次触发一个watchEvent，此时watchEvent的type是NodeDataChanged，之后10秒延时过后，输出/app1的value，此时为123

读取节点的getData放中的true表示是否注册一个watch，注意这里的watch是一次性的，触发通过之后就失效，需要反复注册watch

package com.bigdata.zookeeper;


import org.apache.zookeeper.AsyncCallback;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;

import java.util.concurrent.CountDownLatch;

public class ZkExample implements Watcher {

    public static CountDownLatch connectedSemaphore = new CountDownLatch(1);
    private static Stat stat = new Stat();

    public static void main(String[] args) throws Exception {
        ZooKeeper zk = new ZooKeeper("master:2181", 5000, new ZkExample());
        System.out.println(zk.getState());
        try {
            connectedSemaphore.await();
            System.out.println(new String(zk.getData("/app1", true, stat)));
            Thread.sleep(10000); // 10秒延时
            System.out.println(new String(zk.getData("/app1", true, stat)));
        } catch (InterruptedException e) {
            System.out.println("Zk session established" + e);
        }
    }

    @Override
    public void process(WatchedEvent watchedEvent) {
        System.out.println(watchedEvent);
        if (Event.KeeperState.SyncConnected == watchedEvent.getState()) {
            connectedSemaphore.countDown();
        }
    }
}

输出

异步代码

package com.bigdata.zookeeper;


import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

import java.util.List;
import java.util.concurrent.CountDownLatch;

public class ZkExample implements Watcher {

    public static CountDownLatch connectedSemaphore = new CountDownLatch(1);
    private static Stat stat = new Stat();
    private static ZooKeeper zk = null;

    public static void main(String[] args) throws Exception {
        zk = new ZooKeeper("master:2181", 5000, new ZkExample());
        System.out.println(zk.getState());
        try {
            connectedSemaphore.await();
            // 注册一个watch，将能异步获取到为空的子节点
            zk.getChildren("/app1", true, new IChildren2Callback(), null);
            // 创建一个子节点
            zk.create("/app1/app12", "".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

            Thread.sleep(10000); // 10秒延时

        } catch (InterruptedException e) {
            System.out.println("Zk session established" + e);
        }
    }

    @Override
    public void process(WatchedEvent watchedEvent) {
        System.out.println(watchedEvent);
        if (Event.KeeperState.SyncConnected == watchedEvent.getState()) {
            if (Event.EventType.None == watchedEvent.getType() &amp;&amp; null == watchedEvent.getPath()) {
                connectedSemaphore.countDown();
            } else if (watchedEvent.getType() == Event.EventType.NodeChildrenChanged) {
                try {
                    System.out.println(zk.getChildren(watchedEvent.getPath(), true));
                } catch (Exception e) {

                }
            }

        }
    }
}

class IChildren2Callback implements AsyncCallback.Children2Callback {

    @Override
    public void processResult(int i, String s, Object o, List<String> list, Stat stat) {
        System.out.println("Get children znode : [response code: " + i + ", path: " + s +
                ", children list" + list + ", stat : " + stat);
    }
}

输出

全文 >>

Zk学习笔记——连接Zookeeper

2020-07-05

tonglin0325

参考：从Paxos到Zookeeper分布式一致性原理和实践

使用的zk依赖是cdh5.16.2的3.4.5

<!-- zookeeper -->
<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.5-cdh5.16.2</version>
</dependency>

代码，其中CountDownLatch计数器参考：Java多线程——其他工具类CyclicBarrier、CountDownLatch和Exchange

初始化的计数器为1，当zk client连接到zk集群后，zk集群返回的状态为connected，计数器-1，触发InterruptedException

package com.bigdata.zookeeper;

import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

import java.util.concurrent.CountDownLatch;

public class ZkExample implements Watcher {

    public static CountDownLatch connectedSemaphore = new CountDownLatch(1);

    public static void main(String[] args) throws Exception {
        ZooKeeper zk = new ZooKeeper("master:2181", 5000, new ZkExample());
        System.out.println(zk.getState());
        try {
            connectedSemaphore.await();
        } catch (InterruptedException e) {
            System.out.println("Zk session established");
        }
    }

    @Override
    public void process(WatchedEvent watchedEvent) {
        System.out.println(watchedEvent);
        if (Event.KeeperState.SyncConnected == watchedEvent.getState()) {
            connectedSemaphore.countDown();
        }
    }
}

输出

全文 >>

小米手机卸载小爱同学

2020-06-14

tonglin0325

1.将系统进行root

手机的miui版本需要是开发版，并且进行了root，这样才能获取到卸载系统自带软件的权限

尤其是小爱同学这个语音助手，消耗了过多的系统资源，使得手机变得卡顿，所以建议卸载掉

像通过安全中心进行卸载的方式是不能完全卸载掉的，会随着系统后台再次进行安装，所以只能获取root权限后，使用root权限进行删除

如果需要获取root权限，就需要刷开发版的rom，下载ROM请去

1 2	https://xiaomirom.com/

2.在手机上安装termux app

这个app用于在手机上执行终端命令

1 2	https://github.com/termux/termux-app/releases

3.手机安装termux-adb

这个用于在termux上连上手机adb模式，开启调试

参考：https://github.com/MasterDevX/Termux-ADB

1 2	apt update && apt install wget && wget https://github.com/MasterDevX/Termux-ADB/raw/master/RemoveTools.sh && bash RemoveTools.sh

4.使用pm uninstall命令将小爱同学卸载掉

然后如图所示在termux终端中连上adb，输入命令进行卸载操作，就可以卸载小爱同学了

如果还需要卸载其他的系统应用的话，参考下面文章

1 2	https://powersee.github.io/2019/09/MIUI-adb/

卸载某些系统应用会导致系统异常，下面列出了我验证过的可以卸载的应用

pm uninstall com.xiaomi.mimobile.noti
pm uninstall com.xiaomi.jr.security
pm uninstall -k --user 0 com.miui.voiceassist
pm uninstall -k --user 0 com.miui.yellowpage
pm uninstall -k --user 0 com.miui.bugreport
pm uninstall -k --user 0 com.miui.hybrid
pm uninstall -k --user 0 com.xiaomi.ab
pm uninstall -k --user 0 com.xiaomi.vipaccount
pm uninstall -k --user 0 com.miui.milivetalk
pm uninstall -k --user 0 com.xiaomi.payment
pm uninstall -k --user 0 com.miui.hybrid.accessory
pm uninstall -k --user 0 com.sohu.inputmethod.sogou.xiaomi
pm uninstall -k --user 0 com.xiaomi.gamecenter
pm uninstall -k --user 0 com.miui.analytics
pm uninstall -k --user 0 com.mipay.wallet
pm uninstall -k --user 0 com.xiaomi.joyose
pm uninstall -k --user 0 com.xiaomi.miplay
pm uninstall -k --user 0 com.miui.video
pm uninstall -k --user 0 com.miui.klo.bugreport
pm uninstall -k --user 0 com.baidu.input_mi
pm uninstall -k --user 0 com.xiaomi.pass
pm uninstall -k --user 0 com.miui.virtualsim
pm uninstall -k --user 0 com.miui.player
pm uninstall -k --user 0 com.miui.compass
pm uninstall -k --user 0 com.miui.systemAdSolution
pm uninstall -k --user 0 com.android.browser
pm uninstall -k --user 0 com.miui.contentextension
pm uninstall -k --user 0 com.xiaomi.gamecenter.sdk.service

全文 >>

Flink学习笔记——Flink on YARN

2020-05-31

tonglin0325

Flink集群部署的方式有以下几种，在本文中主要介绍Flink on yarn：

Yarn
Mesos
Docker/Kubernetes
Standalone

全文 >>

tonglin0325的个人主页

Zk学习笔记——应用场景

Zk学习笔记——ZkClient

Zk学习笔记——修改节点

Zk学习笔记——检测节点是否存在

Pac代理文件

Zk学习笔记——创建节点

Zk学习笔记——读取节点

Zk学习笔记——连接Zookeeper

小米手机卸载小爱同学

1.将系统进行root

2.在手机上安装termux app

3.手机安装termux-adb

4.使用pm uninstall命令将小爱同学卸载掉

Flink学习笔记——Flink on YARN

tonglin0325

标签

标签云

文章归档

最近文章