开发环境配置备注

===========brew==============

mac安装神器brew

安装: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"   

用法:

brew update  更新brew；

brew install {应用名，如git} 安装软件

brew cask install {应用名，如git} 也是下载安装，与上面的区别，请查看https://www.zhihu.com/question/22624898

更多用法请 brew help

按照指定版本软件：

brew search xxx;

例如：

brew search thrift

==> Formulae

thrift                                                                                                 thrift@0.9

选择 0.9版本的进行安装

brew install thrift@0.9

安装 brew： ruby -e"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

查看brew的当前源(依赖的应该是国外的资源，可能被qiang):

cd /usr/local

cd Homebrew

git remote -v

origin  https://github.com/Homebrew/brew (fetch)

origin  https://github.com/Homebrew/brew (push)

更改brew镜像源(超级快)

cd /usr/local

git remote set-url origin git://mirrors.tuna.tsinghua.edu.cn/homebrew.git

清华镜像源: git://mirrors.tuna.tsinghua.edu.cn/homebrew.git

中科大镜像源: http://mirrors.ustc.edu.cn/homebrew.git

更新brew镜像源之后再执行brew update, 这时提示你删除下面目录:

/usr/local/share/doc/homebrew

那就删除此目录: rm -rf /usr/local/share/doc/homebrew

=============pip===============

下载pip：

sudo easy_install pip 

安装：

sudo pip install pyotp

brew install expect

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

python get-pip.py

sudo python get-pip.py

sudo pip install pyserial

sudo pip install psutil

sudo pip install Flask

sudo pip install flask_login

=================thrift================

安装thrift

https://blog.csdn.net/u010827436/article/details/44303897

--安装boost

./bootstrap.sh —prefix=PATH_TO_BOOST

sudo ./b2 threading=multi address-model=64 variant=release stage install

--安装libevent

./configure —prefix=/usr/local  

make  

sudo make install  

--安装 libevent 缺少openssl

git clone https://github.com/openssl/openssl  

cp -R ./openssl/1.0.2o_1/include/openssl /Users/didi/utils/libevent-2.1.8-stable

cd /usr/bin

sudo mv bison bison.2.3

sudo cp /usr/local/Cellar/bison/3.0.4/bin/bison bison

https://www.cnblogs.com/smartloli/p/4220545.html

http://www.zhimengzhe.com/mac/134615.html

1.下载thrift0.9.3的安装包，进行解压。 

2. 执行configure

./configure --prefix=/usr/local/ --with-boost=/usr/local --with-libevent=/usr/local --without-ruby --without-perl --without-php --without-nodejs

make

sudo make install

thrift --version

THRIFT_HOME=/usr/local/thrift-0.9.1

PATH=$JAVA_HOME/bin:$PATH:$THRIFT_HOME/bin

export JAVA_HOME CLASSPATH PATH THRIFT_HOME

============搭建springmvc+mybaitis 框架============

SSH Spring+SpringMVC+Hibernate SSM Spring+SpringMVC+Mybatis

https://www.cnblogs.com/jlustone/articles/7398454.html

https://www.cnblogs.com/inspurhaitian/p/4647485.html

https://blog.csdn.net/sinat_33982461/article/details/75202499

https://blog.csdn.net/bhuds/article/details/52674850

https://www.cnblogs.com/quickcodes/p/da-jianSpringMVCMyBatis-kai-fa-kuang-jia-yi.html

https://www.cnblogs.com/xiaoL/p/7753130.html

https://blog.csdn.net/qq_37989365/article/details/78361798

==============es==================

安装ES & Kibana:

$ brew update

$ brew install elasticsearch

$ brew install kibana

Elasticsearch下载X-Pack

　　在Es的根目录（每个节点），运行 bin/elasticsearch-plugin进行安装。

    bin/elasticsearch-plugin install x-pack

Kibana下载X-Pack

　　在Kibana根目录运行 bin/kibana-plugin 进行安装。

　　先进入kibana的安装目录，然后键入以下命令：

bin/kibana-plugin install x-pack

文档：

https://www.cnblogs.com/zhangs1986/p/7325504.html

https://www.jianshu.com/p/18074e1e719f

es教程：

https://www.cnblogs.com/ajianbeyourself/p/5529575.html

es快速上手：

https://www.cnblogs.com/aaanthony/p/7380662.html

es优化

https://www.cnblogs.com/skyblue/p/5533527.html

https://www.cnblogs.com/liyafei/p/8530522.html

创建新的索引

put：

http://localhost:9200/test_idx_1

查看所有的index

get：

http://localhost:9200/_cat/indices?v

==============flink==================

version:flink-1.5.2

sources:

https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/batch/index.html

flink下载：

https://flink.apache.org/downloads.html

安装flink：

tar zxf flink-1.5.2-bin-hadoop27-scala_2.11.tgz

cd flink-1.5.2

启动flink集群

./bin/start-cluster.sh 

本地flink-ui

http://localhost:8081/#/overview

bin 启动 停止脚本

log 存放日志

conf 配置集群 master slaver zoo. zk信息

http://ifeve.com/apache-flink-introduction/

https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/batch/index.html

https://www.jianshu.com/p/ea80d15e9b5e

================scala================

idea中安装scala插件，然后新建项目，SDK选择create 下载scala

https://blog.csdn.net/zhousishuo/article/details/71598718

===============kafka==================

version: 1.1.0

sources:

http://kafka.apache.org/downloads

brew update;

brew install kafka;

启动zk、启动kafka

zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties >>/dev/null &

zookeeper-server-stop /usr/local/etc/kafka/zookeeper.properties & kafka-server-stop /usr/local/etc/kafka/server.properties

启动Kafka，其中">>/dev/null"（不做记录）表示将日志信息输出到"黑洞"，

其中"2>&1"表示将错误信息和前面的日志信息一样，也输出到"黑洞"，

末尾的"&"表示以后台方式启动kafka：

bin/kafka-server-start.sh config/server.properties >>/dev/null 2>&1 &

进入kafka相关目录：/usr/local/Cellar/kafka/1.1.0/bin

创建topic

kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看创建的topic

kafka-topics --list --zookeeper localhost:2181  

查看Topic的详细信息

kafka-topics --describe --topic test --zookeeper localhost:2181

生产数据

kafka-console-producer --broker-list localhost:9092 --topic test

消费数据

kafka-console-consumer --zookeeper localhost:2181 --topic test --from-beginning

kafka-console-consumer --bootstrap-server 127.0.0.1:9092 --topic test --from-beginning

kafka-console-consumer --bootstrap-server localhost:9092 --topic test --from-beginning

kafka-console-consumer --bootstrap-server localhost:9092 --topic test1 --from-beginning

备注：--from-beginning （latest） 将从第一个消息还是最近接收

删除topic

kafka-topics --delete --zookeeper localhost:2181 --topic test

===如果遇到没有master的问题，修改server.properties

设置：listeners=PLAINTEXT://localhost:9092

==============gradle==================

GRADLE_HOME=/usr/local/Cellar/gradle/4.9

export GRADLE_HOME

export PATH=$PATH:$GRADLE_HOME/bin

source /etc/bashrc

==============flink-demo===================

https://www.cnblogs.com/jiashengmei/p/9084057.html

https://www.cnblogs.com/huxi2b/p/7219792.html

flink 分三种 windows模式 滚动窗口 滑动窗口 session窗口

nohup hadoop fs -cat /user/xxxx.sh >>./test_log.txt 2>&1 &

crontab 

-e：编辑该用户的计时器设置；

-l：列出该用户的计时器设置；

-r：删除该用户的计时器设置；

-u<用户名称>：指定要设定计时器的用户名称。

/bin/flink run -d  -c com.daxin.batch.App flinkwordcount.jar

./bin/flink run -d ./examples/batch/WordCount.jar

./bin/flink run ./examples/batch/WordCount.jar