How to implement connection and query between Nebula Graph and SpringBoot environment
- PHPzforward
- 2023-05-15 22:01:041285browse
说明
Nebula Graph 是一款开源的、分布式的、易扩展的原生图数据库，能够承载包含数千亿个点和数万亿条边的超大规模数据集，并且提供毫秒级查询。
当前Nebula Graph的最新版本是3.2.1, 根据官方的文档进行配置
支持分布式. 相对于Neo4j, TigerGraph这些图数据库, Nebula 是面向分布式设计的, 因此对集群的支持比较完备, 在规模上上限要高很多. 在实际项目中存储了180亿的点边, 这个数量对于Neo4j和TigerGraph是比较困难的.
支持图空间. 各个图空间的ID是互不干扰的, 但是在同一个图空间里ID的类型和长度必须一致. 注意这个一致约束的是所有的点和边. Nebula 可以使用int64作为ID, 也可以用字符串, 但是字符串需要指定一个长度, 例如64个字节. 相对于只能用长整数的Neo4j, ID设计上更自由灵活.
-
点对应的类型叫TAG, 边对应的类型叫EDGE
TAG和EDGE都会对应一组的属性(map, 或者说dict)
一个点可以对多个TAG, 每个TAG一组属性, 多组属性. 项目中建议一开始不要用多TAG, 在整个图结构稳定后, 再做合并
一个边只对应一个EDGE, 一组属性
Nebula 用的是自定义的查询语法 GQL, 和 cypher 语法基本一样
除了点边的ID和关联关系外, 只有带索引的属性可以查询. 这点和其它图数据库不一样, 其它数据库即使没有索引, 慢是慢点但是不报错, Nebula直接给你返回错误.
对于返回数量较大的查询, Nebula会强制查询必须带limit
Nebula 单节点稳定性是有问题的, 在3.2.1版本中观察到偶尔会出现服务自行退出, 如果在生产环境使用, 需要有后台监控进行心跳检测和自动启动
GQL 常用查询
下面列出一些常用的查询
-- 列出图空间 SHOW SPACES; -- 列出tag(点类型)和edge(边类型), 需要先 USE 一个图空间 SHOW TAGS; SHOW EDGES;
列出某一类型的点和边
MATCH ()-[e:follow]-() RETURN e MATCH (v:player) RETURN v
带条件的查询, 在结果数量较多时必须带limit, 否则Nebula会报错
match (v:ADDRESS)-[e]-() where id(v)==\"ADD:82388116\" return v,e limit 100
基础配置和使用
在上面的链接中, 提供了最小的配置和测试代码
pom.xml 增加包依赖
对于Nebula Graph 3.2.1, 需要使用3.0.0的版本. client的每个版本只能对应特定的一两个服务端版本
<dependency> <groupId>com.vesoft</groupId> <artifactId>client</artifactId> <version>3.0.0</version> </dependency>
Java调用
Java调用主要是三部分, 创建连接池, 创建会话, 执行查询
创建 NebulaPool 连接池
连接到地址127.0.0.1, 端口9669, 连接池大小100. 注意地址和端口是一个列表, Nebula是支持集群的. 连接时不需要用户和密码
NebulaPool pool = new NebulaPool();
try {
NebulaPoolConfig nebulaPoolConfig = new NebulaPoolConfig();
nebulaPoolConfig.setMaxConnSize(100);
List<HostAddress> addresses = Arrays.asList(new HostAddress("127.0.0.1", 9669));
Boolean initResult = pool.init(addresses, nebulaPoolConfig);
if (!initResult) {
log.error("pool init failed.");
return;
}
} catch ()
//...创建 Session 会话
创建会话时需要用户名和密码
Session session = pool.getSession("root", "nebula", false);执行查询
创建一个SPACE, 然后使用这个SPACE, 创建一个TAG person, 创建一个EDGE like
String createSchema = "CREATE SPACE IF NOT EXISTS test(vid_type=fixed_string(20)); "
+ "USE test;"
+ "CREATE TAG IF NOT EXISTS person(name string, age int);"
+ "CREATE EDGE IF NOT EXISTS like(likeness double)";
ResultSet resp = session.execute(createSchema);
if (!resp.isSucceeded()) {
log.error(String.format("Execute: `%s', failed: %s",
createSchema, resp.getErrorMessage()));
System.exit(1);
}添加一个点记录
String insertVertexes = "INSERT VERTEX person(name, age) VALUES "
+ "'Bob':('Bob', 10), "
+ "'Lily':('Lily', 9), "
+ "'Tom':('Tom', 10), "
+ "'Jerry':('Jerry', 13), "
+ "'John':('John', 11);";
ResultSet resp = session.execute(insertVertexes);
if (!resp.isSucceeded()) {
log.error(String.format("Execute: `%s', failed: %s",
insertVertexes, resp.getErrorMessage()));
System.exit(1);
}查询
String query = "GO FROM \"Bob\" OVER like "
+ "YIELD $^.person.name, $^.person.age, like.likeness";
ResultSet resp = session.execute(query);
if (!resp.isSucceeded()) {
log.error(String.format("Execute: `%s', failed: %s",
query, resp.getErrorMessage()));
System.exit(1);
}
printResult(resp);在 SpringBoot 项目中使用 Nebula Graph
pom.xml 增加包依赖
<dependency> <groupId>com.vesoft</groupId> <artifactId>client</artifactId> <version>3.0.0</version> </dependency>
Session工厂: NebulaSessionFactory.java
配合@Bean(destroyMethod = "close"), 创建一个工厂类, 接收pool并实现close()方法
public class NebulaSessionFactory {
private final NebulaPool pool;
private final String username;
private final String password;
public NebulaSessionFactory(NebulaPool pool, String username, String password) {
this.pool = pool;
this.username = username;
this.password = password;
}
public Session getSession() {
try {
return pool.getSession(username, password, false);
} catch (NotValidConnectionException|IOErrorException|AuthFailedException|ClientServerIncompatibleException e) {
throw new RuntimeException("Nebula session exception", e);
}
}
public void close() {
pool.close();
}
}为什么不直接将 NebulaPool 配置为Bean? 因为 Session 每次创建时需要带用户名密码, 将密码作为config注入到每个Service中肯定是大家都不愿意看到的.
配置修改: application.yml
这里的值如果不打算使用profile配置, 可以直接写入
hosts是逗号分隔的地址端口列表, 例如
10.22.33.33:9669,10.22.33.34:9669
myapp:
nebula:
hosts: @nebula.hosts@
username: @nebula.username@
password: @nebula.password@
max-conn: @nebula.max-conn@Spring启动配置: NebulaGraphConfig.java
应用启动时读取配置, 创建 NebulaPool, 并实例化 NebulaSessionFactory, destroyMethod = "close", 这个表示在项目shutdown时会调用Bean的close方法释放资源.
@Configuration
public class NebulaGraphConfig {
@Value("${myapp.nebula.hosts}")
private String hosts;
@Value("${myapp.nebula.max-conn}")
private int maxConn;
@Value("${myapp.nebula.username}")
private String username;
@Value("${myapp.nebula.password}")
private String password;
@Bean(destroyMethod = "close")
public NebulaSessionFactory nebulaSessionFactory() {
List<HostAddress> hostAddresses = new ArrayList<>();
String[] hostList = hosts.split(",[ ]*");
for (String host : hostList) {
String[] hostParts = host.split(":");
if (hostParts.length != 2 || !hostParts[1].matches("\\d+")) {
throw new RuntimeException("Invalid host name set for Nebula: " + host);
}
hostAddresses.add(new HostAddress(hostParts[0], Integer.parseInt(hostParts[1])));
}
NebulaPoolConfig poolConfig = new NebulaPoolConfig();
poolConfig.setMaxConnSize(maxConn);
NebulaPool pool = new NebulaPool();
try {
pool.init(hostAddresses, poolConfig);
} catch (UnknownHostException e) {
throw new RuntimeException("Unknown Nebula hosts");
}
return new NebulaSessionFactory(pool, username, password);
}
}Service调用
在 Service 中进行调用
@Service
@Slf4j
public class GraphServiceImpl implements GraphService {
@Autowired
private NebulaSessionFactory sessionFactory;
@Override
public <T> NebulaResult<T> query(String graphSpace, String gql) {
Session session = null;
try {
log.info("GQL: {}", gql);
session = sessionFactory.getSession();
NebulaResult<Void> res = query(session, "USE " + graphSpace);
if (!res.isSuccess() || res.getResults() == null || res.getResults().size() == 0) {
log.error("Failed to use space:{}", graphSpace);
return null;
}
if (!graphSpace.equals(res.getResults().get(0).getSpaceName())) {
log.error("Failed to use space:{}, result:{}", graphSpace, res.getResults().get(0).getSpaceName());
return null;
}
return query(session, gql);
} catch (IOErrorException e) {
log.error(e.getMessage(), e);
return null;
} finally {
if (session != null) {
session.release();
}
}
}
private <T> NebulaResult<T> query(Session session, String gql) throws IOErrorException {
String json = session.executeJson(gql);
return JacksonUtil.extractByType(json, new TypeReference<>() {});
}
}辅助类 NebulaResult.java 等
外层结构
这里定义了 json 格式响应的外层结构
@Data
public class NebulaResult<T> implements Serializable {
private List<Error> errors;
private List<Result<T>> results;
@JsonIgnore
public boolean isSuccess() {
return (errors != null && errors.size() == 1 && errors.get(0).getCode() == 0);
}
@Data
public static class Error implements Serializable {
private int code;
}
@Data
@JsonIgnoreProperties(ignoreUnknown = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public static class Result<T> implements Serializable {
private String spaceName;
private List<Element<T>> data;
private List<String> columns;
private Error errors;
private long latencyInUs;
}
@Data
public static class Element<T> implements Serializable {
private List<Meta<T>> meta;
private List<Serializable> row;
}
@Data
public static class Meta<T> implements Serializable {
private String type;
private T id;
}
}内层因为区分Edge和Vertex, 结构不一样. 如果是混合返回的结果, 可以用 Serializable
String gql = "match (v:ADDR)-[e]-() where id(v)==\"ADD:123123\" return v,e limit 100";
NebulaResult<Serializable> res = graphService.query("insurance", gql);
log.info(JacksonUtil.compress(res));
Assertions.assertThat(res).isNotNull();对于边, 需要使用结构化的ID
@Data
@JsonIgnoreProperties(ignoreUnknown = true)
@JsonInclude(JsonInclude.Include.NON_NULL)
public class EdgeId implements Serializable {
private int ranking;
private int type;
private String dst;
private String src;
private String name;
}用这个结构进行查询
NebulaResult3bd3c46109d7c459f7949a02ed75b5bf res3 = graphService.query("t_test1", "MATCH ()-[e:follow]-() RETURN e");
对于点, ID就是String
NebulaResultf7e83be87db5cd2d9a8a0b8117b38cd4 res2 = graphService.query("t_test1", "MATCH (v:player) RETURN v");
The above is the detailed content of How to implement connection and query between Nebula Graph and SpringBoot environment. For more information, please follow other related articles on the PHP Chinese website!
Related articles
See more- Does MVC Have a Place in Android Development?
- How do I append data to a file in Java using FileWriter without overwriting existing content?
- Should You Always Use `final` for Method Parameters and Local Variables in Java?
- How to Play .WAV Files in Java with Ease?
- How to Convert Between java.time.LocalDateTime and java.util.Date?