tonglin0325的个人主页

logback，log4j和log4j2的区别

tonglin0325

1.logback的使用，参考：logback的使用和logback.xml详解

pom文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  <logback.version>1.1.7</logback.version>
  <slf4j.version>1.7.21</slf4j.version>
</properties>

<dependencies>
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>${slf4j.version}</version>
    <scope>compile</scope>
  </dependency>
  <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-core</artifactId>
    <version>${logback.version}</version>
  </dependency>
  <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>${logback.version}</version>
    </dependency>
</dependencies>

代码

1
2
3
4
5
import org.slf4j.Logger
import org.slf4j.LoggerFactory

private static final Logger log = LoggerFactory.getLogger(LoggingDemo.class);

　　　　

2.log4j的使用

pom文件

1
2
3
4
5
6
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>

代码

1
2
3
4
import org.apache.log4j.Logger;

private final Logger LOGGER = Logger.getLogger(LoggingDemo.class.getName());

全文 >>

安装ubuntu和安装ubuntu后要安装的软件列表

tonglin0325

重装ubuntu并做些笔记以及ubuntu下使用的一些开发环境和软件

我现在的ubuntu桌面

ubuntu_desktop

<1>安装ubuntu

使用的电脑是宏碁的4750G，机械硬盘500G+SSG120G，进行win7+ubuntu 14.04 32位双系统的安装。如果要在双硬盘下安装双系统的话，建议先把win7安装在/dev/hda的那块硬盘下，不然会出现不能进入系统的问题。 前期准备：一个u盘和两台电脑，方便查资料和制作启动盘。 这里采用的引导方式为通过win7引导选择进入ubuntu，所以先安装win7的32位版本。

全文 >>

数据治理基本概念

tonglin0325

1.数据治理解决的问题#

1.数据易用性（取数复杂度&速度，需要数据建模，不能都从原始表来查，需要数据仓库设计）

2.数据质量（日志定义口径，指标定义，数据波动报警，和钱相关的一般使用阻塞式）

3.研发成本（研发复杂度&周期，历史负担，数据地图）

4.数据的安全性（加密&脱敏&审计）

Java基本语法笔记

tonglin0325

1.基本格式

1
2
3
4
5
6
7
8
9
10
11
12
public class HelloWprdApp {

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        int num = 10;
        num = 30;
        System.out.println("num="+num);
        System.out.println("num*num="+num*num);
    }

}

String args[]：传统的写法，c、c++都用该种写法；

String[] args ：Java的写法，但Java也能用上面的方法定义。

实际上String[] args 是定义一字符串数组变量。

在java中，String[] args和String args[] 都可以定义数组。二者没有什么区别。为规范编码，推荐使用String[] args。另外args可以随意改，和其它变量一样，它只不过是一变量名，只要符合变量名规则即可。

全文 >>

Flink学习笔记——报错合集

tonglin0325

遇到的flink任务各种报错合集

1.Exception in thread “main” java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;

1
2
3
4
5
6
7
8
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
	at org.apache.flink.runtime.entrypoint.parser.CommandLineOptions.<clinit>(CommandLineOptions.java:27)
	at org.apache.flink.runtime.entrypoint.DynamicParametersConfigurationParserFactory.options(DynamicParametersConfigurationParserFactory.java:43)
	at org.apache.flink.runtime.entrypoint.DynamicParametersConfigurationParserFactory.getOptions(DynamicParametersConfigurationParserFactory.java:50)
	at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:42)
	at org.apache.flink.runtime.entrypoint.ClusterEntrypointUtils.parseParametersOrExit(ClusterEntrypointUtils.java:70)
	at org.apache.flink.yarn.entrypoint.YarnApplicationClusterEntryPoint.main(YarnApplicationClusterEntryPoint.java:87)

这是由于commons-cli包版本过低导致的，从1.2.升级到1.5.0可以解决这个问题

2.Caused by: java.lang.ClassCastException: org.codehaus.janino.CompilerFactory cannot be cast to org.codehaus.commons.compiler.ICompilerFactory

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Unable to instantiate java compiler
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:836)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:247)
	at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1078)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1156)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
	at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1156)
Caused by: java.lang.IllegalStateException: Unable to instantiate java compiler
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:428)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.load3(JaninoRelMetadataProvider.java:374)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:109)
	at org.apache.flink.calcite.shaded.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:3951)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
	at org.apache.flink.calcite.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.create(JaninoRelMetadataProvider.java:469)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:481)
	at org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:95)
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:784)
	at org.apache.calcite.rel.rules.ReduceExpressionsRule$FilterReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:151)
	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:243)
	at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
	at org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:64)
	at org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:78)
	at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.$anonfun$optimize$1(FlinkChainedProgram.scala:59)
	at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:156)
	at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:156)
	at scala.collection.Iterator.foreach(Iterator.scala:937)
	at scala.collection.Iterator.foreach$(Iterator.scala:937)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
	at scala.collection.IterableLike.foreach(IterableLike.scala:70)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:156)
	at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:154)
	at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
	at org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:55)
	at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:175)
	at org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:82)
	at org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:75)
	at org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:306)
	at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:186)
	at org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1656)
	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:782)
	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:861)
	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:701)
	at com.xxx.data.FlinkSQLDemo.main(FlinkSQLDemo.java:415)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
	... 11 more
Caused by: java.lang.ClassCastException: org.codehaus.janino.CompilerFactory cannot be cast to org.codehaus.commons.compiler.ICompilerFactory
	at org.codehaus.commons.compiler.CompilerFactoryFactory.getCompilerFactory(CompilerFactoryFactory.java:129)
	at org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:79)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:426)
	... 63 more

在尝试了网上搜到的各种解决方案没有效果之后，将flink任务提交的命令从

全文 >>

JavaScript排序算法——冒泡排序

tonglin0325
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312" />
<title>&aring;&dagger;&rsquo;&aelig;&sup3;&iexcl;</title>
<!--<link rel="stylesheet" type="text/css" href="../style/fdt.css" />-->
<script type="text/javascript" src="../js/jquery-1.6.2.min.js"></script>
<script type="text/javascript" src="../js/jquery.easydrag.handler.beta2.js"></script>
<script type="text/javascript">

	$(document).ready(
		function() { 
			
			var array_1 = [9,8,7,6,5,4,3,2,1];
			alert(array_1);
			/*bubbleSort*/
		    alert(bubbleSort(array_1));

		}
	); 



</script>

<style type="text/css">
	
	* { padding:0; margin:0; }

	body {
		padding: 100px;
		font-size: 15px; 
	}

	


</style>


<script type="text/javascript">
	function bubbleSort(array){
		var i = 0;len = array.length;
		var j, d;
		for (; i < len; i++) {
		    for (j = 0; j < len; j++) {
		       	if (array[i] < array[j]) {
		            d = array[j];
		            array[j] = array[i];
		            array[i] = d;
		        }
		    }
		}
		return array;
	}


</script>


</head>




<body>
	&Atilde;&deg;&Aring;&Yacute;&Aring;&Aring;&ETH;&ograve;
</body>
</html>

全文 >>

JavaScript排序算法——堆排序

tonglin0325
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>堆排序</title>
<!--<link rel="stylesheet" type="text/css" href="../style/fdt.css" />-->
<script type="text/javascript" src="../js/jquery-1.6.2.min.js"></script>
<script type="text/javascript" src="../js/jquery.easydrag.handler.beta2.js"></script>
<script type="text/javascript">

    $(document).ready(
        function() { 
            
            // var array_1 = [9,8,7,6,5,4,3,2,1];
            var array_1 = [4,5,3,1,2,0];
            alert(array_1);
            /*heapSort*/
            alert(heapSort(array_1));

        }
    ); 



</script>

<style type="text/css">
    
    * { padding:0; margin:0; }

    body {
        padding: 100px;
        font-size: 15px; 
    }

    


</style>


<script type="text/javascript">
    function heapSort(elements){
        


        //调整函数
        function headAdjust(elements, pos, len){
          //将当前节点值进行保存
          var swap = elements[pos];

          //定位到当前节点的左边的子节点
          var child = pos * 2 + 1;

          //递归，直至没有子节点为止
          while(child < len){
            //如果当前节点有右边的子节点，并且右子节点较大的场合，采用右子节点和当前节点进行比较
            if(child + 1 < len &amp;&amp; elements[child] < elements[child + 1]){
              child += 1;
            }

            //比较当前节点和最大的子节点，小于则进行值交换，交换后将当前节点定位于子节点上
            if(elements[pos] < elements[child]){
              elements[pos] = elements[child];
              pos = child;
              child = pos * 2 + 1;
            }
            else{
              break;
            }

            elements[pos] = swap;
          }
        }

        //构建堆
        function buildHeap(elements){
          //从最后一个拥有子节点的节点开始，将该节点连同其子节点进行比较，
          //将最大的数交换与该节点,交换后，再依次向前节点进行相同交换处理，
          //直至构建出大顶堆（升序为大顶，降序为小顶）
          for(var i=elements.length/2; i>=0; i--){
            headAdjust(elements, i, elements.length);
          }
        }

        function sort(elements){
          //构建堆
          buildHeap(elements);

          //从数列的尾部开始进行调整
          for(var i=elements.length-1; i>0; i--){
            //堆顶永远是最大元素，故，将堆顶和尾部元素交换，将
            //最大元素保存于尾部，并且不参与后面的调整
            alert(elements);
            var swap = elements[i];
            elements[i] = elements[0];
            elements[0] = swap;
            alert(elements);
            //进行调整，将最大的元素调整至堆顶
            headAdjust(elements, 0, i);
            alert(elements);
          }
        }

        sort(elements);
        return elements;
        

    }


</script>


</head>




<body>
    堆排序
</body>
</html>

全文 >>

tonglin0325

标签

标签云

AWS Airbnb Android CDH Doris ELK Flink Git Grafana HAProxy HBase Hadoop Hive Hudi InfluxDB JDBC JMeter JVM Java JavaWeb LaTex Linkedin Linux ML ML Infra MPP Maven MySQL Nexus OpenTSDB Paper Play Prometheus Python React Redis RocksDB Scala Solr Spark SpringBoot SpringMVC Thrift YARN antlr arthas avro cassandra clickhouse confluent docker filebeat flume golang google gradle hexo impala jenkins k8s kafka kerberos kudu ldap mac mongo mybatis nginx nlp open-falcon openwrt parquet presto ranger scyllaDB twitter zookeeper 刷题 前端 图存储及计算 多线程 广告系统 开发工具 数据结构 杂谈 正则表达式 算法 系统设计 计算机网络 设计模式

文章归档

最近文章