Neo4j/Spring-Data 中的延迟/急切加载/获取

发布于 2024-12-27 03:26:48 字数 7538 浏览 6 评论 0原文

我有一个简单的设置并遇到了一个令人费解的(至少对我来说)问题:

我有三个相互关联的pojo:

@NodeEntity
public class Unit {
    @GraphId Long nodeId;
    @Indexed int type;
    String description;
}


@NodeEntity
public class User {
    @GraphId Long nodeId;
    @RelatedTo(type="user", direction = Direction.INCOMING)
    @Fetch private Iterable<Worker> worker;
    @Fetch Unit currentUnit;

    String name;

}

@NodeEntity
public class Worker {
    @GraphId Long nodeId;
    @Fetch User user;
    @Fetch Unit unit;
    String description;
}

所以你有一个带有“currentunit”的用户工作单元,它在用户中标记允许直接跳转到“当前单位”。每个用户可以有多个工人,但一名工人只能分配给一个单元(一个单元可以有多个工人)。

我想知道如何控制“User.worker”上的 @Fetch 注释。我实际上希望仅在需要时才加载它,因为大多数时候我只与“Worker”一起工作。

我浏览了 http:// static.springsource.org/spring-data/data-neo4j/docs/2.0.0.RELEASE/reference/html/ 我不太清楚:

  • worker是可迭代的,因为它应该是只读的(传入关系)- 在文档中明确说明了这一点,但在示例中大多数时候使用“设置”。为什么?或者没关系...
  • 我如何让工作人员仅在访问时加载? (延迟加载)
  • 为什么我需要用@Fetch注释即使是简单的关系(worker.unit)。难道就没有更好的办法吗?我有另一个具有许多这样简单关系的实体 - 我真的想避免仅仅因为我想要一个对象的属性就必须加载整个图。
  • 我是否缺少弹簧配置,以便它可以延迟加载?
  • 有没有办法通过额外的调用加载任何关系(未标记为@Fetch)?

从我的角度来看,只要我需要一个 Worker,这个构造就会加载整个数据库,即使我大多数时候并不关心 User。

我发现的唯一解决方法是使用存储库并在需要时手动加载实体。

-----更新 -------

我已经使用 neo4j 有一段时间了,找到了上述问题的解决方案,不需要一直调用 fetch (因此不会加载整个图表)。唯一的缺点:它是一个运行时方面:

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.mapping.model.MappingException;
import org.springframework.data.neo4j.annotation.NodeEntity;
import org.springframework.data.neo4j.support.Neo4jTemplate;

import my.modelUtils.BaseObject;

@Aspect
public class Neo4jFetchAspect {

    // thew neo4j template - make sure to fill it 
    @Autowired private Neo4jTemplate template;

    @Around("modelGetter()")
    public Object autoFetch(ProceedingJoinPoint pjp) throws Throwable {
        Object o = pjp.proceed();
        if(o != null) {
            if(o.getClass().isAnnotationPresent(NodeEntity.class)) {
                if(o instanceof BaseObject<?>) {
                    BaseObject<?> bo = (BaseObject<?>)o;
                    if(bo.getId() != null && !bo.isFetched()) {
                        return template.fetch(o);
                    }
                    return o;
                }
                try {
                    return template.fetch(o);
                } catch(MappingException me) {
                    me.printStackTrace();
                }
            }
        }
        return o;
    }

    @Pointcut("execution(public my.model.package.*.get*())")
    public void modelGetter() {}

}

您只需调整应应用该方面的类路径:my.model.package..get())")

我将该方面应用于所有这需要一些先决条件:

  • 您必须在模型类中使用 getter(该方面不适用于公共属性 - 无论如何您都不应该使用)
  • 所有模型类都在同一个包中(因此您需要调整代码a小) - 我想你可以调整过滤器
  • 方面,因为需要运行时组件(当你使用 tomcat 时有点棘手) - 但它有效:)
  • 所有模型类必须实现 BaseObject 接口,它提供:

    公共接口BaseObject { 公共布尔 isFetched(); }

可以防止重复获取。我只是检查强制的子类或属性(即名称或除nodeId 之外的其他内容)以查看它是否确实被获取。 Neo4j 将创建一个对象,但仅填充 nodeId,而其他所有内容保持不变(因此其他所有内容均为 NULL)。

即,

@NodeEntity
public class User implements BaseObject{
    @GraphId
    private Long nodeId;

        String username = null;

    @Override
    public boolean isFetched() {
        return username != null;
    }
}

如果有人找到一种方法来做到这一点,而不需要那种奇怪的解决方法,请添加您的解决方案:)因为这个有效,但我会喜欢一个没有方面的解决方案。

不需要自定义字段检查的基础对象设计

一种优化是创建一个基类而不是实际使用布尔字段(加载布尔值)并对其进行检查的接口(因此您不需要担心手动检查)

public abstract class BaseObject {
    private Boolean loaded;
    public boolean isFetched() {
        return loaded != null;
    }
    /**
     * getLoaded will always return true (is read when saving the object)
     */
    public Boolean getLoaded() {
        return true;
    }

    /**
     * setLoaded is called when loading from neo4j
     */
    public void setLoaded(Boolean val) {
        this.loaded = val;
    }
}

这是有效的,因为在保存对象时返回“true”以进行加载。当方面查看对象时,它使用 isFetched() ,当尚未检索到对象时,它将返回 null。一旦检索到对象,就会调用 setLoaded 并将已加载的变量设置为 true。

如何防止杰克逊触发延迟加载?

(作为评论中问题的答案 - 请注意,我还没有尝试过,因为我没有这个问题)。

对于 Jackson,我建议使用自定义序列化器(请参阅 ie http://www.baeldung.com/杰克逊自定义序列化)。这允许您在获取值之前检查实体。您只需检查它是否已获取,然后继续整个序列化或仅使用 id:

public class ItemSerializer extends JsonSerializer<BaseObject> {
    @Override
    public void serialize(BaseObject value, JsonGenerator jgen, SerializerProvider provider)
      throws IOException, JsonProcessingException {
        // serialize the whole object
        if(value.isFetched()) {
            super.serialize(value, jgen, provider);
            return;
        }
        // only serialize the id
        jgen.writeStartObject();
        jgen.writeNumberField("id", value.nodeId);
        jgen.writeEndObject();
    }
}

Spring 配置

这是我使用的示例 Spring 配置 - 您需要根据您的情况调整包项目:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j-2.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

    <context:annotation-config/>
    <context:spring-configured/>

    <neo4j:repositories base-package="my.dao"/> <!-- repositories = dao -->

    <context:component-scan base-package="my.controller">
        <context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/> <!--  that would be our services -->
    </context:component-scan>
    <tx:annotation-driven mode="aspectj" transaction-manager="neo4jTransactionManager"/>    
    <bean class="corinis.util.aspects.Neo4jFetchAspect" factory-method="aspectOf"/> 
</beans>

AOP 配置

这是 /META-INF/aop.xml 以便其工作:

<!DOCTYPE aspectj PUBLIC
        "-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">
    <aspectj>
        <weaver>
            <!-- only weave classes in our application-specific packages -->
            <include within="my.model.*" />
        </weaver>
        <aspects>
            <!-- weave in just this aspect -->
            <aspect name="my.util.aspects.Neo4jFetchAspect" />
        </aspects>
    </aspectj>

I have a simple setup and encountered a puzzling (at least for me) problem:

I have three pojos which are related to each other:

@NodeEntity
public class Unit {
    @GraphId Long nodeId;
    @Indexed int type;
    String description;
}


@NodeEntity
public class User {
    @GraphId Long nodeId;
    @RelatedTo(type="user", direction = Direction.INCOMING)
    @Fetch private Iterable<Worker> worker;
    @Fetch Unit currentUnit;

    String name;

}

@NodeEntity
public class Worker {
    @GraphId Long nodeId;
    @Fetch User user;
    @Fetch Unit unit;
    String description;
}

So you have User-Worker-Unit with a "currentunit" which marks in user that allows to jump directly to the "current unit". Each User can have multiple workers, but one worker is only assigned to one unit (one unit can have multiple workers).

What I was wondering is how to control the @Fetch annotation on "User.worker". I actually want this to be laoded only when needed, because most of the time I only work with "Worker".

I went through http://static.springsource.org/spring-data/data-neo4j/docs/2.0.0.RELEASE/reference/html/ and it isn't really clear to me:

  • worker is iterable because it should be read only (incoming relation) - in the documentation this is stated clarly, but in the examples ''Set'' is used most of the time. Why? or doesn't it matter...
  • How do I get worker to only load on access? (lazy loading)
  • Why do I need to annotate even the simple relations (worker.unit) with @Fetch. Isn't there a better way? I have another entity with MANY such simple relations - I really want to avoid having to load the entire graph just because i want to the properties of one object.
  • Am I missing a spring configuration so it works with lazy loading?
  • Is there any way to load any relationships (which are not marked as @Fetch) via an extra call?

From how I see it, this construct loads the whole database as soon as I want a Worker, even if I don't care about the User most of the time.

The only workaround I found is to use repository and manually load the entities when needed.

------- Update -------

I have been working with neo4j quite some time now and found a solution for the above problem that does not require calling fetch all the time (and thus does not load the whole graph). Only downside: it is a runtime aspect:

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.mapping.model.MappingException;
import org.springframework.data.neo4j.annotation.NodeEntity;
import org.springframework.data.neo4j.support.Neo4jTemplate;

import my.modelUtils.BaseObject;

@Aspect
public class Neo4jFetchAspect {

    // thew neo4j template - make sure to fill it 
    @Autowired private Neo4jTemplate template;

    @Around("modelGetter()")
    public Object autoFetch(ProceedingJoinPoint pjp) throws Throwable {
        Object o = pjp.proceed();
        if(o != null) {
            if(o.getClass().isAnnotationPresent(NodeEntity.class)) {
                if(o instanceof BaseObject<?>) {
                    BaseObject<?> bo = (BaseObject<?>)o;
                    if(bo.getId() != null && !bo.isFetched()) {
                        return template.fetch(o);
                    }
                    return o;
                }
                try {
                    return template.fetch(o);
                } catch(MappingException me) {
                    me.printStackTrace();
                }
            }
        }
        return o;
    }

    @Pointcut("execution(public my.model.package.*.get*())")
    public void modelGetter() {}

}

You just have to adapt the classpath on which the aspect should be applied: my.model.package..get())")

I apply the aspect to ALL get methods on my model classes. This requires a few prerequesites:

  • You MUST use getters in your model classes (the aspect does not work on public attributes - which you shouldn't use anyways)
  • all model classes are in the same package (so you need to adapt the code a little) - I guess you could adapt the filter
  • aspectj as a runtime component is required (a little tricky when you use tomcat) - but it works :)
  • ALL model classes must implement the BaseObject interface which provides:

    public interface BaseObject {
    public boolean isFetched();
    }

This prevents double-fetching. I just check for a subclass or attribute that is mandatory (i.e. the name or something else except nodeId) to see if it is actually fetched. Neo4j will create an object but only fill the nodeId and leave everything else untouched (so everything else is NULL).

i.e.

@NodeEntity
public class User implements BaseObject{
    @GraphId
    private Long nodeId;

        String username = null;

    @Override
    public boolean isFetched() {
        return username != null;
    }
}

If someone finds a way to do this without that weird workaround please add your solution :) because this one works, but I would love one without aspectj.

Base object design that doenst require a custom field check

One optimization would be to create a base-class instead of an interface that actually uses a Boolean field (Boolean loaded) and checks on that (so you dont need to worry about manual checking)

public abstract class BaseObject {
    private Boolean loaded;
    public boolean isFetched() {
        return loaded != null;
    }
    /**
     * getLoaded will always return true (is read when saving the object)
     */
    public Boolean getLoaded() {
        return true;
    }

    /**
     * setLoaded is called when loading from neo4j
     */
    public void setLoaded(Boolean val) {
        this.loaded = val;
    }
}

This works because when saving the object "true" is returned for loaded. When the aspect looks at the object it uses isFetched() which - when the object is not yet retrieved will return null. Once the object is retrieved setLoaded is called and the loaded variable set to true.

How to prevent jackson from triggering the lazy loading?

(As an answer to the question in the comment - note that I didnt try it out yet since I did not have this issue).

With jackson I suggest to use a custom serializer (see i.e. http://www.baeldung.com/jackson-custom-serialization ). This allows you to check the entity before getting the values. You simply do a check if it is already fetched and either go on with the whole serialization or just use the id:

public class ItemSerializer extends JsonSerializer<BaseObject> {
    @Override
    public void serialize(BaseObject value, JsonGenerator jgen, SerializerProvider provider)
      throws IOException, JsonProcessingException {
        // serialize the whole object
        if(value.isFetched()) {
            super.serialize(value, jgen, provider);
            return;
        }
        // only serialize the id
        jgen.writeStartObject();
        jgen.writeNumberField("id", value.nodeId);
        jgen.writeEndObject();
    }
}

Spring Configuration

This is a sample Spring configuration I use - you need to adjust the packages to your project:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j-2.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

    <context:annotation-config/>
    <context:spring-configured/>

    <neo4j:repositories base-package="my.dao"/> <!-- repositories = dao -->

    <context:component-scan base-package="my.controller">
        <context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/> <!--  that would be our services -->
    </context:component-scan>
    <tx:annotation-driven mode="aspectj" transaction-manager="neo4jTransactionManager"/>    
    <bean class="corinis.util.aspects.Neo4jFetchAspect" factory-method="aspectOf"/> 
</beans>

AOP config

this is the /META-INF/aop.xml for this to work:

<!DOCTYPE aspectj PUBLIC
        "-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">
    <aspectj>
        <weaver>
            <!-- only weave classes in our application-specific packages -->
            <include within="my.model.*" />
        </weaver>
        <aspects>
            <!-- weave in just this aspect -->
            <aspect name="my.util.aspects.Neo4jFetchAspect" />
        </aspects>
    </aspectj>

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

遇见了你 2025-01-03 03:26:48

我自己找到了所有问题的答案:

@Iterable:是的,iterable 可用于只读

@load on access:默认情况下不会加载任何内容。并且自动延迟加载不可用(至少据我所知)

其余的:
当我需要建立关系时,我必须使用 @Fetch 或使用 neo4jtemplate.fetch 方法:

@NodeEntity
public class User {
    @GraphId Long nodeId;
    @RelatedTo(type="user", direction = Direction.INCOMING)
    private Iterable<Worker> worker;
    @Fetch Unit currentUnit;

    String name;

}

class GetService {
  @Autowired private Neo4jTemplate template;

  public void doSomethingFunction() {
    User u = ....;
    // worker is not avaiable here

    template.fetch(u.worker);
    // do something with the worker
  }  
}

Found the answer to all the questions myself:

@Iterable: yes, iterable can be used for readonly

@load on access: per default nothing is loaded. and automatic lazy loading is not available (at least as far as I can gather)

For the rest:
When I need a relationship I either have to use @Fetch or use the neo4jtemplate.fetch method:

@NodeEntity
public class User {
    @GraphId Long nodeId;
    @RelatedTo(type="user", direction = Direction.INCOMING)
    private Iterable<Worker> worker;
    @Fetch Unit currentUnit;

    String name;

}

class GetService {
  @Autowired private Neo4jTemplate template;

  public void doSomethingFunction() {
    User u = ....;
    // worker is not avaiable here

    template.fetch(u.worker);
    // do something with the worker
  }  
}
素食主义者 2025-01-03 03:26:48

不透明,但仍然延迟获取< /a>.

template.fetch(person.getDirectReports());

@Fetch 会按照您的答案中所述进行急切的获取。

Not transparent, but still lazy fetching.

template.fetch(person.getDirectReports());

And @Fetch does the eager fetching as was already stated in your answer.

笑梦风尘 2025-01-03 03:26:48

我喜欢使用方面方法来解决当前 spring-data 处理延迟加载方式的限制。

@niko - 我已将您的代码示例放入一个基本的 Maven 项目中,并尝试让该解决方案发挥作用,但收效甚微:

https://github.com/samuel-kerrien/neo4j-aspect-auto-fetching

由于某些原因,方面正在初始化,但建议似乎没有得到执行。要重现该问题,只需运行以下 JUnit 测试:

playground.neo4j.domain.UserTest

I like the aspect approach to work around the limitation of the current spring-data way to handle lazy loading.

@niko - I have put your code sample in a basic maven project and tried to get that solution to work with little success:

https://github.com/samuel-kerrien/neo4j-aspect-auto-fetching

For some reasons the Aspect is initialising but the advice doesn't seem to get executed. To reproduce the issue, just run the following JUnit test:

playground.neo4j.domain.UserTest
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文