近期评论

图数据库及图计算

概述

arangodb一种内存数据,一种基于document的nosql数据库。

比起neo4j性能和设计都要好很多。用户界面也好很多。支持单点模式和集群模式。

相比传统的数据库,和新型的nosql数据,图数据库有一个非常特别的地方,就是图计算,这是它的优势。

劣势在于如何把现实生活的数据抽象成图数据,也就是建模过程。这个反而更难些。

这里做一个开端,普及一些基础的图计算知识。

部署

下载源码

官方3.0,3.2这两个版本的源码没法编译成功。3.1.10版本存在bug,选择3.1.10.1版本。

git clone \

-b 3.1.10.1 \

https://github.com/arangodb/arangodb.git  \

arangodb-3.1.10.1

编译安装

编译arangodb的源码需要用到4.9以上的gcc,gcc-c++和python-argparse

yum -y install python-argparse

BASE_DIR=/usr/local/src/arangodb-3.1.10.1/build

rm -rf $BASE_DIR

mkdir -p $BASE_DIR

cd $BASE_DIR

export LD_LIBRARY_PATH=/usr/local/gcc-4.9.3/lib64/

rm -rf /usr/local/arangodb-3.1.10.1

mkdir /usr/local/arangodb-3.1.10.1

ln -s /usr/local/arangodb-3.1.10.1 /usr/local/arangodb

/usr/local/cmake-3.5.2/bin/cmake ../ \

-DCMAKE_INSTALL_PREFIX=/usr/local/arangodb-3.1.10.1 \

-DPCH=1 \

-DDEBUG=0 \

-DCMAKE_C_COMPILER=/usr/local/gcc-4.9.3/bin/gcc \

-DCMAKE_CXX_COMPILER=/usr/local/gcc-4.9.3/bin/g++

time make -j14

make install

基础环境

arangodb集群默认要求使用arangdb用户启动和关闭

groupadd arangodb

useradd arangodb -g arangodb

chown -R arangodb.arangodb /usr/local/arangodb*

mkdir -p /usr/local/arangodb/{data,logs,pids,tmp}

mkdir -p /usr/local/arangodb/tmp/{agency,primary,coordinator}

mkdir -p /usr/local/arangodb/data/{agency,primary,coordinator}

chown -R arangodb.arangodb /usr/local/arangodb*

集群模式

初始化和重启

关闭脚本

对应关闭,检查和启动的脚本如下:

cat stop_coordinator.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh stop_cordinator.sh

NODE=coordinator

ps -ef |grep arangodb |grep $NODE |grep -v grep | grep -v stop | awk ‘{print $2}’ |xargs -t -i kill {}

rm -rf /usr/local/arangodb/pids/$NODE.pid

# 检查进程是否存在

ps -ef |grep arangodb |grep $NODE |grep -v grep | grep -v stop

echo “stop $NODE finished!”

———————————————————————

cat stop_primary.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh stop_primary.sh

NODE=primary

ps -ef |grep arangodb |grep $NODE |grep -v grep |grep -v stop | awk ‘{print $2}’ |xargs -t -i kill {}

rm -rf /usr/local/arangodb/pids/$NODE.pid

# 检查进程是否存在

ps -ef |grep arangodb |grep $NODE |grep -v grep |grep -v stop

echo “stop $NODE finished!”

———————————————————————

cat stop_agency.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh stop_agency.sh

NODE=agency

ps -ef |grep arangodb |grep $NODE |grep -v grep |grep -v stop| awk ‘{print $2}’ |xargs -t -i kill {}

rm -rf /usr/local/arangodb/pids/$NODE.pid

# 检查进程是否存在

ps -ef |grep arangodb |grep $NODE |grep -v grep |grep -v stop

echo “stop $NODE finished!”

检查脚本

ps -ef |grep arango |grep -v su |grep -v bash |grep -v grep |grep -v ps

启动脚本

cat start_agency.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh start_agency.sh

NODE=agency

export LD_LIBRARY_PATH=/usr/local/arangodb/lib64/

/usr/local/arangodb/sbin/arangod –configuration /usr/local/arangodb/etc/arangodb3/roles/$NODE.conf

echo “start $NODE finished!”

———————————————————————

cat start_primary.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh start_primary.sh

NODE=primary

export LD_LIBRARY_PATH=/usr/local/arangodb/lib64/

/usr/local/arangodb/sbin/arangod –configuration /usr/local/arangodb/etc/arangodb3/roles/$NODE.conf

ps -ef |grep $NODE |grep -v grep |grep -v start

echo “start $NODE finished!”

———————————————————————

cat start_coordinator.sh

#!/bin/sh

# author  : tanghaiyang

# date    : Fri Jun 23 07:17:22 EDT 2017

# usage   : sh start_cordinator.sh

NODE=coordinator

export LD_LIBRARY_PATH=/usr/local/arangodb/lib64/

/usr/local/arangodb/sbin/arangod –configuration /usr/local/arangodb/etc/arangodb3/roles/$NODE.conf

echo “start $NODE finished!”

关闭arngodb集群

先关闭coordinator节点,再关闭primary节点,最后关闭agency节点

stopcoordinator

stopprimary

stopagency

# 检查配置文件

cat  /usr/local/arangodb/etc/arangodb3/roles/primary.conf |egrep ‘endpoint|my-add’ |grep -v ^# |grep -v agency

初始化清理数据文件

首次启动时,或者反复安装时,或者数据文件破坏,或者已经备份,需要恢复集群时,可以考虑清理数据文件,执行如下命令:

rm -rf /usr/local/arangodb/logs/*

rm -rf /usr/local/arangodb/data

rm -rf /usr/local/arangodb/pids/*

rm -rf /usr/local/arangodb/var/lib/arangodb3-apps/*

rm -rf /tmp/arangod_*

mkdir -p /usr/local/arangodb/data/{agency,coordinator,primary}

chown -R arangodb.arangodb /usr/local/arangodb*

启动arangodb

启动arangodb时,先启动agency,然后启动primary,最后启动coordinator节点

agency类似zookeeper,primary向agency注册集群信息。

startagency

startprimary

startcoordinator

配置文件

agency节点

3.0.12版本的agency配置文件中有个配置项id,

但是3.1.10以后版本没有了,不能在3.1.10版本里面存在这个配置项。

示例配置如下(配置项必须包含[agency]模块,否则primary和coordinator节点无法启动,2017.8.5踩到这个坑,直到第二天仔细检查配置才发现,复制粘贴时漏掉了):

vim /usr/local/arangodb/etc/arangodb3/roles/agency.conf

##############################

supervisor = true

pid-file = /usr/local/arangodb/pids/agency.pid

[database]

directory = /usr/local/arangodb/data/agency

[server]

endpoint = tcp://dm0:5020

authentication = false

statistics = true

uid = arangodb

jwt-secret = 123456

[scheduler]

[javascript]

startup-directory = /usr/local/arangodb/share/arangodb3/js

app-path = /usr/local/arangodb/var/lib/arangodb3-apps

[foxx]

[log]

level = info

file = /usr/local/arangodb/logs/agency.log

use-local-time = true

[temp]

path = /usr/local/arangodb/tmp/agency

[agency]

activate = true

size = 1

supervision = true

primary节点

primary的配置文件除了模块的下面三项不一样以外,其他配置项配每台都一样:

[server]

endpoint = tcp://dm0:18558

……

[cluster]

my-address = tcp://dm0:18558

my-local-info = primary_dm0

注意:

不能添加my-id,如果增加了my-id,coordinator节点无法启动,debug日志,提示endpoint无效

另外agency的配置每台都一样

agency-endpoint = tcp://dm0:5020

示例配置如下:

/usr/local/arangodb/etc/arangodb3/roles/primary.conf

##############################

supervisor = true

pid-file = /usr/local/arangodb/pids/primary.pid

[database]

directory = /usr/local/arangodb/data/primary

[server]

endpoint = tcp://dm0:18558

authentication = false

statistics = true

uid = arangodb

jwt-secret = 123456

[scheduler]

[javascript]

startup-directory = /usr/local/arangodb/share/arangodb3/js

app-path = /usr/local/arangodb/var/lib/arangodb3-apps

[foxx]

[log]

level = info

file = /usr/local/arangodb/logs/primary.log

[temp]

path = /usr/local/arangodb/tmp/primary

[cluster]

my-role = PRIMARY

my-address = tcp://dm0:18558

my-local-info = primary_dm0

agency-endpoint = tcp://dm0:5020

coordinator节点

coordinator节点也可以有多个节点。

示例配置如下:

/usr/local/arangodb/etc/arangodb3/roles/coordinator.conf

##############################

supervisor = true

pid-file = /usr/local/arangodb/pids/coordinator.pid

[database]

directory = /usr/local/arangodb/data/coordinator

[server]

endpoint = tcp://dm0:18530

authentication = false

statistics = true

uid = arangodb

jwt-secret = 123456

[scheduler]

[javascript]

startup-directory = /usr/local/arangodb/share/arangodb3/js

app-path = /usr/local/arangodb/var/lib/arangodb3-apps

v8-contexts = 10

[foxx]

[log]

level = info

file = /usr/local/arangodb/logs/coorniantor.log

[cluster]

[temp]

path = /usr/local/arangodb/tmp/coorniantor

[cluster]

my-role = COORDINATOR

my-address = tcp://dm0:18530

my-local-info = coord_171

agency-endpoint = tcp://dm0:5020

单点模式

单点模式的web界面左侧没有NODE一栏,对应服务器只有一个arangod进程。

启动脚本也很简单,如下:

vim /usr/local/arangodb/bin/start_single.sh

#!/bin/sh

export LD_LIBRARY_PATH=/usr/local/arangodb/lib64/

/usr/local/arangodb/sbin/arangod \

–database.directory /usr/local/arangodb/data/primary \

–javascript.startup-directory /usr/local/arangodb/share/arangodb3/js \

–javascript.app-path /usr/local/arangodb/var/lib/arangodb3-apps \

–server.endpoint tcp://dm9:18530 \

–server.authentication false

注意:

集群模式也可以只启用一个primary节点作为数据节点存放数据,但毕竟还是集群模式,而创建分片或者创建库和表时,有时候会受到限制,比如下面的命令在集群单个primary节点下,无法创建过大replication,因为存在分片。

curl -X POST –data-binary @- –dump – http://${ARANGO_HOST}:${ARANGO_PORT}/_db/${ARANGO_DB}/_api/collection <<EOF

{

“type” : “${FILE_EXT}”,

“name” : “${FILE_NAME}”,

“numberOfShards”: 12,

“replicationFactor”: 1,

“waitForSync”: true

}

EOF

149 Responses to “图数据库及图计算”

  • 增达网说道:

    如果上天再给我一次机会,我会对你的博客说,下次还来看你!

  • 笑八达说道:

    世间博客不少,风景这边独好!

  • 三五游戏说道:

    这里来不够,看不够,玩不够!

  • 快乐赚说道:

    这个博客不简单,看了还想接着看。

  • 沃八达说道:

    没事就来转一转,每天多吃两碗饭!

  • 能让人来访的乐此不疲,这里就是有那么大的魅力!

  • Daron Cheatom说道:

    I like what you guys are up also. Such clever work and reporting! Carry on the excellent works guys I?¦ve incorporated you guys to my blogroll. I think it’ll improve the value of my site :)

  • 沃八达说道:

    人生短暂,开心每一天!

  • 沃八达说道:

    每逢佳节倍思亲,看你博客很用心!

  • Thanks for sharing such a nice thinking, post is pleasant, thats why i have read it fully|

  • Today, I went to the beachfront with my kids. I found a sea shell and gave it to my 4 year old daughter and said “You can hear the ocean if you put this to your ear.” She put the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear. She never wants to go back! LoL I know this is totally off topic but I had to tell someone!|

  • Egli Diana Pinto说道:

    Hello there, just became alert to your blog through Google, and found that it’s really informative. I am gonna watch out for brussels. I’ll be grateful if you continue this in future. A lot of people will be benefited from your writing. Cheers!|

  • I think the admin of this web page is truly working hard in favor of his web site, since here every information is quality based material.|

  • I have been browsing online more than three hours as of late, but I never found any interesting article like yours. It’s beautiful value enough for me. Personally, if all web owners and bloggers made good content material as you did, the web can be a lot more helpful than ever before.|

  • My brother suggested I might like this web site. He was totally right. This post truly made my day. You cann’t imagine simply how much time I had spent for this info! Thanks!|

  • Hi there! I know this is kind of off topic but I was wondering which blog platform are you using for this site? I’m getting tired of WordPress because I’ve had problems with hackers and I’m looking at alternatives for another platform. I would be awesome if you could point me in the direction of a good platform.|

  • whoah this weblog is fantastic i really like reading your articles. Keep up the good work! You know, a lot of individuals are hunting round for this information, you can help them greatly. |

  • I just could not leave your web site before suggesting that I actually loved the usual info an individual supply on your visitors? Is going to be back incessantly in order to check out new posts|

  • Everything is very open with a clear clarification of the issues. It was truly informative. Your website is very helpful. Thank you for sharing!|

  • Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.|

  • continuously i used to read smaller articles or reviews that as well clear their motive, and that is also happening with this piece of writing which I am reading at this place.|

  • MAGA说道:

    Hello there, You’ve done an incredible job. I will certainly digg it and personally suggest to my friends. I am confident they’ll be benefited from this web site.|

  • When some one searches for his vital thing, thus he/she desires to be available that in detail, thus that thing is maintained over here.|

  • Oh my goodness! Impressive article dude! Thank you, However I am having troubles with your RSS. I don’t know the reason why I can’t join it. Is there anybody having the same RSS problems? Anybody who knows the solution can you kindly respond? Thanx!!|

  • It’s very trouble-free to find out any topic on web as compared to textbooks, as I found this piece of writing at this website.|

  • Hey very nice website!! Guy .. Excellent .. Wonderful .. I’ll bookmark your blog and take the feeds additionally? I am happy to seek out a lot of helpful info here within the publish, we need work out more strategies in this regard, thanks for sharing. . . . . .|

  • I visited many sites except the audio quality for audio songs existing at this web site is really superb.|

  • Thank you a bunch for sharing this with all people you actually recognise what you are speaking approximately! Bookmarked. Kindly also talk over with my site =). We could have a link alternate agreement among us|

  • Hey there! I could have sworn I’ve been to this site before but after reading through some of the post I realized it’s new to me. Anyhow, I’m definitely happy I found it and I’ll be bookmarking and checking back frequently!|

  • I am regular visitor, how are you everybody? This paragraph posted at this web site is genuinely nice.|

  • Itamar Serpa说道:

    Hi there, just became alert to your blog through Google, and found that it’s truly informative. I am going to watch out for brussels. I will appreciate if you continue this in future. A lot of people will be benefited from your writing. Cheers!|

  • Umanizzare说道:

    Greetings I am so glad I found your site, I really found you by accident, while I was browsing on Google for something else, Nonetheless I am here now and would just like to say thank you for a fantastic post and a all round entertaining blog (I also love the theme/design), I don’t have time to go through it all at the minute but I have book-marked it and also included your RSS feeds, so when I have time I will be back to read more, Please do keep up the excellent b.|

  • I’m impressed, I must say. Seldom do I come across a blog that’s both equally educative and interesting, and let me tell you, you have hit the nail on the head. The issue is an issue that too few people are speaking intelligently about. I am very happy I came across this during my hunt for something relating to this.|

  • Pretty! This was a really wonderful post. Thanks for providing this information.|

  • What i don’t understood is in reality how you are not really much more smartly-favored than you may be now. You are very intelligent. You realize therefore significantly on the subject of this matter, produced me for my part believe it from so many various angles. Its like women and men don’t seem to be involved except it’s one thing to do with Girl gaga! Your personal stuffs nice. Always deal with it up!|

  • It’s going to be ending of mine day, except before ending I am reading this enormous paragraph to improve my knowledge.|

  • Oh my goodness! Amazing article dude! Thank you so much, However I am going through issues with your RSS. I don’t understand why I am unable to join it. Is there anybody getting identical RSS issues? Anybody who knows the solution will you kindly respond? Thanks!!|

  • Excellent beat ! I wish to apprentice while you amend your website, how could i subscribe for a blog website? The account aided me a acceptable deal. I had been a little bit acquainted of this your broadcast provided bright clear concept|

  • Egli Diana Pinto说道:

    Good response in return of this question with firm arguments and telling everything about that.|

  • I am no longer positive the place you are getting your information, however good topic. I must spend a while finding out much more or working out more. Thanks for great information I was searching for this info for my mission.|

  • I absolutely love your blog and find almost all of your post’s to be just what I’m looking for. Does one offer guest writers to write content available for you? I wouldn’t mind creating a post or elaborating on a number of the subjects you write with regards to here. Again, awesome blog!|

  • I am actually grateful to the holder of this website who has shared this great piece of writing at at this time.|

  • Pabnacle说道:

    I am am excited too with this question. Tell to me, please – where I can find more information on this question?
    [url=http://rexuiz.top/]Rexuiz FPS Game[/url]

  • Lelio Junior说道:

    I believe everything posted made a ton of sense. But, think on this, what if you were to write a awesome headline? I ain’t suggesting your information isn’t solid., however suppose you added a post title to possibly get people’s attention? I mean %BLOG_TITLE% is a little boring. You should look at Yahoo’s front page and watch how they write post headlines to grab viewers to open the links. You might add a related video or a related picture or two to get people interested about what you’ve written. In my opinion, it could make your blog a little bit more interesting.|

  • ip cameras说道:

    I am really loving the theme/design of your web site. Do you ever run into any internet browser compatibility problems?
    A few of my blog audience have complained about my site not operating correctly in Explorer but looks great in Firefox.
    Do you have any ideas to help fix this issue?

  • Have you ever thought about adding a little
    bit more than just your articles? I mean, what you say is
    important and everything. But just imagine if you added some great visuals or videos to give your
    posts more, “pop”! Your content is excellent but with pics and video clips, this site could definitely be
    one of the very best in its niche. Terrific blog!

  • timex enjoy说道:

    It’s not my first time to pay a visit this website, i am browsing this web site dailly
    and take pleasant facts from here every day.

  • Very good post! We will be linking to this great content on our site.
    Keep up the great writing.

  • Gerald说道:

    Excellent blog here! Also your website loads up very fast! What
    web host are you using? Can I get your affiliate link to your host?

    I wish my website loaded up as fast as yours lol

  • I’m very happy to uncover this page. I wanted to thank you for ones time just for
    this wonderful read!! I definitely loved every bit of it and i also have you book-marked to look at new stuff in your web site.

  • Hello colleagues, how is everything, and what you want to say on the topic of this paragraph,
    in my view its genuinely awesome in support of me.

  • Are you gonna be concerned the ones you love?

  • asbestos inside说道:

    Does your site have a contact page? I’m having trouble locating it but, I’d like to shoot you an email.
    I’ve got some suggestions for your blog you
    might be interested in hearing. Either way, great website and I look forward to seeing it develop over time.

  • asbestos study说道:

    Hi there, after reading this awesome post i am as well cheerful to share
    my know-how here with colleagues.

Leave a Reply