Java中有SoftHashMap吗?

发布于 2024-07-08 13:36:31 字数 258 浏览 6 评论 0原文

我知道 java.util 中有一个 WeakHashMap,但由于它对所有内容都使用 WeakReference,因此仅由此 引用Map,引用的对象将在下一个GC周期中丢失。 因此,如果您想缓存随机数据,那么它几乎没有用处,因为在其余时间很可能会再次请求这些数据而不会进行硬链接。 最好的解决方案是一张地图,它使用 SoftReference 来代替,但我在 Java RT 包中没有找到。

I know there is a WeakHashMap in java.util, but since it uses WeakReferences for everything, which is only referenced by this Map, referenced objects will get lost on the next GC cycle. So it's nearly useless if you want to cache random data, which is very likely to be requested again without being Hard-linked the rest of the time. The best solution would be a map, which uses SoftReferences instead, but I didn't find one in the Java RT Package.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(7

徒留西风 2024-07-15 13:36:31

编辑(2012 年 8 月):

事实证明,目前最好的解决方案可能是 Guava 13.0 的 Cache 类,解释于 Guava 的 Wiki - 这就是我要使用的。
它甚至支持构建 SoftHashMap(请参阅 CacheBuilder.newBuilder().softKeys()),但这可能不是您想要的,正如 Java 专家 Jeremy Manson 所解释的那样(如下)你会找到链接)。


不是我知道的(2008 年 11 月),但你会发现一些SoftHashMap 在网络上的实现。

就像这个: SoftHashMap这个


编辑(2009 年 11 月)
正如 Matthias 在评论中提到的,Google Guava MapMaker 确实使用了软引用:

ConcurrentMap 构建器,提供以下功能的任意组合:

  • 软键或弱键,
  • 软值或弱值、
  • 定时到期以及
  • 值的按需计算。

正如此帖子中提到的,另一个 JSR166y 候选者:

jsr166y.ConcurrentReferenceHashMap

它为 Google 实现提供了另一种并发参考映射(依赖后台线程来逐出条目)


编辑(2012 年 8 月)

Google 实现仅在请求条目定时过期时才使用后台线程。 特别是,它只是使用 java.util.Timer,这不像拥有单独的后台线程那样具有侵入性。

Jeremy Manson 建议,对于任何缓存,使用此功能来避免 SoftReference 的危险:
http://jeremymanson.blogspot.de/2009/ 07/how-hotspot-decides-to-clear_07.html

还有一个来自 Apache Commons,即 org.apache.commons.collections.map.ReferenceMap; 它不支持定时删除,但它支持选择是否应按同一性或相等性比较键。 此外,此实现不是并发的 - 它可以同步,但在多个线程访问的情况下效果较差。

Edit (Aug. 2012):

It turns out that currently the best solution are probably Guava 13.0's Cache classes, explained on Guava's Wiki - that's what I'm going to use.
It even supports building a SoftHashMap (see CacheBuilder.newBuilder().softKeys()), but it is probably not what you want, as Java expert Jeremy Manson explains (below you'll find the link).


Not that I know of (Nov. 2008), but you kind find some implementation of SoftHashMap on the net.

Like this one: SoftHashMap or this one.


Edit (Nov. 2009)
As Matthias mentions in the comments, the Google Guava MapMaker does use SoftReferences:

A ConcurrentMap builder, providing any combination of these features:

  • soft or weak keys,
  • soft or weak values,
  • timed expiration, and
  • on-demand computation of values.

As mentioned in this thread, another JSR166y candidate:

jsr166y.ConcurrentReferenceHashMap

It provides an alternative concurrent reference map to the Google implementation (which relies on a background thread to evict entries)


Edit (August 2012)

The Google implementation uses a background thread only when timed expiration of entries is requested. In particular, it simply uses java.util.Timer, which is not so intrusive as having a separate background thread.

Jeremy Manson recommends, for any cache, using this feature to avoid the dangers of SoftReference:
http://jeremymanson.blogspot.de/2009/07/how-hotspot-decides-to-clear_07.html

There's another implementation from Apache Commons, namely org.apache.commons.collections.map.ReferenceMap; it does not support timed removal, but it does support choosing whether keys should be compared by identity or by equality. Moreover, this implementation is not concurrent - it can be made synchronized, but that works less well under accesses from multiple threads.

川水往事 2024-07-15 13:36:31

我熟悉两个提供 SoftHashMap 实现的库:

  1. Apache Commons:org.apache。 commons.collections.map.ReferenceMap

  2. Google 集合:com.google.common.collect.ReferenceMap

I am familiar with two libraries that offer a SoftHashMap implementation:

  1. Apache Commons: org.apache.commons.collections.map.ReferenceMap

  2. Google Collections: com.google.common.collect.ReferenceMap

淡忘如思 2024-07-15 13:36:31

98期java专家通讯中有一个示例实现

There is an example implementation in 98 issue of java specialists newsletter

何以心动 2024-07-15 13:36:31

Apache Shiro 附带了一个专为缓存设计的 SoftHashMap。 它基于上面 jb 发布的文章,并在 Apache v2 下获得许可。 您可以在此处找到文档和源代码此处

Apache Shiro comes with a SoftHashMap designed for caching. Its based on the article posted by jb above and licensed under Apache v2. You can find the documentation here and the source code here.

假面具 2024-07-15 13:36:31

您是否考虑过使用 LRUMap 而不是软 HashMap? 您可以更好地控制存储的内容(或至少存储多少)。

Have you considered using an LRUMap instead of a soft HashMap? You get more control over what gets stored (or at least, how much).

淡淡的优雅 2024-07-15 13:36:31

如果您想实现缓存,软引用肯定比弱引用更好,但它会将整个缓存删除策略交给垃圾收集器。 这可能不是你想要的。

如果缓存删除策略很重要,您将需要自己执行此操作,很可能使用常规引用。 但是,您必须决定何时弹出项目以及弹出哪些项目。 如果您只想在堆空间耗尽时丢失一些东西,您可以通过以下方式查询可用的堆空间:

Runtime.getRuntime().getFreeMemory();

然后,一旦可用内存下降到一定数量以下,您就可以开始删除项目。 或者您可以只实现缓存的最大大小,并使用它来决定何时删除内容。

这是我设计的LRU缓存插入、删除和查找时间为 O(1),具有可配置的最大元素数量。 如果你想要一个缓存,恕我直言,这将是比 SoftHashMap 更好的解决方案。

软引用是创建可增长缓存的好方法。 因此,理想的解决方案是使用 SoftHashMap 以及常规的固定大小缓存。 让所有到缓存的插入都进入固定缓存和软哈希映射,然后引用某些东西,看看它是否在软哈希映射中(并更新缓存中的引用时间)。 这样,您所有最重要的项目(根据您选择的策略 LRU、MFU 等)将永远不会被删除,因为它们在缓存中被硬引用,但您也将保留更多的东西(没有策略控制)因为有足够的内存。

If you want to implement a cache softreferences are definetly a better idea than weak references, but it puts your entire cache removal policy in the hands of the garbage collector. which is probably not what you want.

If cache removal policy is important your are going to need to do it on your own most likely using regular references. However you are going to have to decide when to eject items and which to eject. If you only want to lose things when you are running out of heap space you can query available heap space via:

Runtime.getRuntime().getFreeMemory();

Then once free memory drops below a certain amount you can start either dropping items. Or you could just implement a max size for the cache and use that to decide when to drop things.

here's an LRU cache i designed with O(1) insertion, deletion and lookup time, that has a configurable max number of elements. If you want a cache this is going to be a better solution imho than a SoftHashMap.

The softreferences are a great way to create a growable cache. So the ideal solution would be to use a SoftHashMap along with a regular fixed size cache. have all inserts into the cache go into both the fixed cache and the soft hash map then to reference something just see if its in the soft hashmap (and update the reference time in the cache). this way all your most important items (according to your chosen policy LRU, MFU,...) will never be removed because they are hard referenced in the cache but you will also hold on to more things (with no policy control) as long as there is sufficient memory.

紫轩蝶泪 2024-07-15 13:36:31

例如,您可以使用此实现线程安全的软引用 HashMap:

package cz.b2b.jcl.util;

import java.util.*;
import java.lang.ref.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.AbstractMap.SimpleImmutableEntry;

public class ConcurrentSoftHashMap<K, V> extends AbstractMap {

    /**
     The internal HashMap that will hold the SoftReference.
     */
    private final Map<Object, SoftReference> hash = new ConcurrentHashMap<>();
    /**
     The number of "hard" references to hold internally.
     */
    private final int HARD_SIZE;
    /**
     The FIFO list of hard references, order of last access.
     */
    private final ConcurrentLinkedSetQueue hardCache = new ConcurrentLinkedSetQueue();
    /**
     Reference queue for cleared SoftReference objects.
     */
    private final ReferenceQueue queue = new ReferenceQueue();

    public ConcurrentSoftHashMap(int hardSize) {
        HARD_SIZE = hardSize;
    }

    @Override
    public Object get(Object key) {
        Object result = null;
        // We get the SoftReference represented by that key
        SoftReference soft_ref = hash.get(key);
        if (soft_ref != null) {
            // From the SoftReference we get the value, which can be
            // null if it was not in the map, or it was removed in
            // the processQueue() method defined below
            result = soft_ref.get();
            if (result == null) {
                // If the value has been garbage collected, remove the
                // entry from the HashMap.
                hash.remove(key);
            } else {
                // We now add this object to the beginning of the hard
                // reference queue.  
                hardCache.enqueue(result);
                if (hardCache.size() > HARD_SIZE) {
                    // Remove the last entry if list longer than HARD_SIZE
                    hardCache.dequeue();
                }
            }
        }
        return result;
    }

    /**
     Here we put the key, value pair into the HashMap using
     a SoftValue object.
     @param key
     @param value
     @return
     */
    @Override
    public Object put(Object key, Object value) {
        processQueue(); // throw out garbage collected values first
        return hash.put(key, new SoftValue(value, key, queue));
    }

    @Override
    public Object remove(Object key) {
        processQueue(); // throw out garbage collected values first
        return hash.remove(key);
    }

    @Override
    public void clear() {
        hardCache.clear();
        processQueue(); // throw out garbage collected values
        hash.clear();
    }

    @Override
    public int size() {
        processQueue(); // throw out garbage collected values first
        return hash.size();
    }

    @Override
    public boolean containsKey(Object key) {
        processQueue(); // throw out garbage collected values first
        return hash.containsKey(key);
    }

    @Override
    public Set entrySet() {
        Set<Map.Entry> entry = new HashSet<>();
        Map.Entry simpleImmutableEntry = null;
        Object result = null;
        processQueue(); // throw out garbage collected values first
        for (Map.Entry<Object, SoftReference> item : hash.entrySet()) {
            if (item == null) {
                continue;
            }
            Object key = item.getKey();
            SoftReference soft_ref = item.getValue();
            if (soft_ref != null) {
                result = soft_ref.get();
                if (result == null) {
                    hash.remove(key);
                } else {
                    hardCache.enqueue(result);
                    if (hardCache.size() > HARD_SIZE) {
                        hardCache.dequeue();
                    }
                    simpleImmutableEntry = new SimpleImmutableEntry(key, result);
                    entry.add(simpleImmutableEntry);

                }
            }

        }

        return entry;
    }

    private class ConcurrentLinkedSetQueue<E> extends ConcurrentLinkedQueue<E> {

        public void enqueue(E o) {
            if (!contains(o)) {
                add(o);
            }
        }

        public E dequeue() {
            return poll();
        }

    }

    /**
     We define our own subclass of SoftReference which contains
     not only the value but also the key to make it easier to find
     the entry in the HashMap after it's been garbage collected.
     */
    private static class SoftValue extends SoftReference {

        private final Object key; // always make data member final

        /**
         Did you know that an outer class can access private data
         members and methods of an inner class? I didn't know that!
         I thought it was only the inner class who could access the
         outer class's private information. An outer class can also
         access private members of an inner class inside its inner
         class.
         */
        private SoftValue(Object k, Object key, ReferenceQueue q) {
            super(k, q);
            this.key = key;
        }
    }

    /**
     Here we go through the ReferenceQueue and remove garbage
     collected SoftValue objects from the HashMap by looking them
     up using the SoftValue.key data member.
     */
    private void processQueue() {
        SoftValue sv;
        while ((sv = (SoftValue) queue.poll()) != null) {
            hash.remove(sv.key); // we can access private data!
        }
    }

}

You can use for example this implementation thread-safe Soft reference HashMap:

package cz.b2b.jcl.util;

import java.util.*;
import java.lang.ref.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.AbstractMap.SimpleImmutableEntry;

public class ConcurrentSoftHashMap<K, V> extends AbstractMap {

    /**
     The internal HashMap that will hold the SoftReference.
     */
    private final Map<Object, SoftReference> hash = new ConcurrentHashMap<>();
    /**
     The number of "hard" references to hold internally.
     */
    private final int HARD_SIZE;
    /**
     The FIFO list of hard references, order of last access.
     */
    private final ConcurrentLinkedSetQueue hardCache = new ConcurrentLinkedSetQueue();
    /**
     Reference queue for cleared SoftReference objects.
     */
    private final ReferenceQueue queue = new ReferenceQueue();

    public ConcurrentSoftHashMap(int hardSize) {
        HARD_SIZE = hardSize;
    }

    @Override
    public Object get(Object key) {
        Object result = null;
        // We get the SoftReference represented by that key
        SoftReference soft_ref = hash.get(key);
        if (soft_ref != null) {
            // From the SoftReference we get the value, which can be
            // null if it was not in the map, or it was removed in
            // the processQueue() method defined below
            result = soft_ref.get();
            if (result == null) {
                // If the value has been garbage collected, remove the
                // entry from the HashMap.
                hash.remove(key);
            } else {
                // We now add this object to the beginning of the hard
                // reference queue.  
                hardCache.enqueue(result);
                if (hardCache.size() > HARD_SIZE) {
                    // Remove the last entry if list longer than HARD_SIZE
                    hardCache.dequeue();
                }
            }
        }
        return result;
    }

    /**
     Here we put the key, value pair into the HashMap using
     a SoftValue object.
     @param key
     @param value
     @return
     */
    @Override
    public Object put(Object key, Object value) {
        processQueue(); // throw out garbage collected values first
        return hash.put(key, new SoftValue(value, key, queue));
    }

    @Override
    public Object remove(Object key) {
        processQueue(); // throw out garbage collected values first
        return hash.remove(key);
    }

    @Override
    public void clear() {
        hardCache.clear();
        processQueue(); // throw out garbage collected values
        hash.clear();
    }

    @Override
    public int size() {
        processQueue(); // throw out garbage collected values first
        return hash.size();
    }

    @Override
    public boolean containsKey(Object key) {
        processQueue(); // throw out garbage collected values first
        return hash.containsKey(key);
    }

    @Override
    public Set entrySet() {
        Set<Map.Entry> entry = new HashSet<>();
        Map.Entry simpleImmutableEntry = null;
        Object result = null;
        processQueue(); // throw out garbage collected values first
        for (Map.Entry<Object, SoftReference> item : hash.entrySet()) {
            if (item == null) {
                continue;
            }
            Object key = item.getKey();
            SoftReference soft_ref = item.getValue();
            if (soft_ref != null) {
                result = soft_ref.get();
                if (result == null) {
                    hash.remove(key);
                } else {
                    hardCache.enqueue(result);
                    if (hardCache.size() > HARD_SIZE) {
                        hardCache.dequeue();
                    }
                    simpleImmutableEntry = new SimpleImmutableEntry(key, result);
                    entry.add(simpleImmutableEntry);

                }
            }

        }

        return entry;
    }

    private class ConcurrentLinkedSetQueue<E> extends ConcurrentLinkedQueue<E> {

        public void enqueue(E o) {
            if (!contains(o)) {
                add(o);
            }
        }

        public E dequeue() {
            return poll();
        }

    }

    /**
     We define our own subclass of SoftReference which contains
     not only the value but also the key to make it easier to find
     the entry in the HashMap after it's been garbage collected.
     */
    private static class SoftValue extends SoftReference {

        private final Object key; // always make data member final

        /**
         Did you know that an outer class can access private data
         members and methods of an inner class? I didn't know that!
         I thought it was only the inner class who could access the
         outer class's private information. An outer class can also
         access private members of an inner class inside its inner
         class.
         */
        private SoftValue(Object k, Object key, ReferenceQueue q) {
            super(k, q);
            this.key = key;
        }
    }

    /**
     Here we go through the ReferenceQueue and remove garbage
     collected SoftValue objects from the HashMap by looking them
     up using the SoftValue.key data member.
     */
    private void processQueue() {
        SoftValue sv;
        while ((sv = (SoftValue) queue.poll()) != null) {
            hash.remove(sv.key); // we can access private data!
        }
    }

}
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文