为什么要多处理不通过其他进程阻止对象使用的锁?

发布于 2025-02-09 13:09:29 字数 939 浏览 1 评论 0原文

以下代码是一家商店,该商店有5个项目和3个客户要求一项。

import multiprocessing as mp
 
class Shop:
    def __init__(self, stock=5):
        self.stock = stock

    def get_item(self, l, x):
        l.acquire()
        if self.stock >= x:
            self.stock -= x
            print(f"{self.stock} = remaining")
        l.release()

if __name__ == "__main__":
    l = mp.Lock()
    obj = Shop()

    p1 = mp.Process(target=obj.get_item, args=(l, 1))
    p2 = mp.Process(target=obj.get_item, args=(l, 1))
    p3 = mp.Process(target=obj.get_item, args=(l, 1))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock)

但是,我得到的输出如下

4 = remaining
4 = remaining
4 = remaining
Final:  5

,因为我使用锁定我期望它会

4 = remaining
3 = remaining
2 = remaining
Final:  2

质疑:如何仅使用锁来实现上述输出(并且没有过程通信,即没有管道/队列)?

The following code is of a shop that has 5 items and three customers each demanding one item.

import multiprocessing as mp
 
class Shop:
    def __init__(self, stock=5):
        self.stock = stock

    def get_item(self, l, x):
        l.acquire()
        if self.stock >= x:
            self.stock -= x
            print(f"{self.stock} = remaining")
        l.release()

if __name__ == "__main__":
    l = mp.Lock()
    obj = Shop()

    p1 = mp.Process(target=obj.get_item, args=(l, 1))
    p2 = mp.Process(target=obj.get_item, args=(l, 1))
    p3 = mp.Process(target=obj.get_item, args=(l, 1))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock)

The output that I got is as follows

4 = remaining
4 = remaining
4 = remaining
Final:  5

However, since I'm using Lock I was expecting it to be

4 = remaining
3 = remaining
2 = remaining
Final:  2

Question: How to achieve the above output just with Locks(and no process communication i.e without Pipe/Queue)?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

烧了回忆取暖 2025-02-16 13:09:29

此代码无法正常工作的原因是,由于多处理没有与子流程共享其状态。这意味着您启动的每个过程,p1p2p3,获取class对象的副本shop shop < /代码>。它不是同一对象。 有两种方法可以解决此问题,与进程共享实例属性,或共享整个对象本身。如果商店对象拥有需要在流程之间共享的其他数据,则第二种方法可能会更好。

方法1

仅共享库存实例变量的值,您可以使用

shared_int = multiprocessing.Value('i', 5)
print(f'Value is {shared_int.value}')  # 5 

适应您的用例,代码将成为:

import multiprocessing


class Shop:
    def __init__(self, stock=5):
        self.stock = multiprocessing.Value('i', stock)


    def get_item(self, l, x):
        l.acquire()
        if self.stock.value >= x:
            self.stock.value -= x
            print(f"{self.stock.value} = remaining")
        l.release()


if __name__ == "__main__":
    l = multiprocessing.Lock()
    obj = Shop()

    p1 = multiprocessing.Process(target=obj.get_item, args=(l, 1))
    p2 = multiprocessing.Process(target=obj.get_item, args=(l, 1))
    p3 = multiprocessing.Process(target=obj.get_item, args=(l, 1))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock.value)

output> output>

4 = remaining
3 = remaining
2 = remaining
Final:  2

方法2

共享整个复杂对象是一个更具参与的过程。我最近有回答 详细介绍了有关共享复杂对象的类似问题(例如,在这种情况下是类商店的对象),还介绍了下面提供的代码背后的推理。我建议您对其进行阅读,因为它可以更详细地说明底部提供的代码背后的逻辑。此用例的唯一主要区别是您要使用 supperrocess ,多处理,而不是多处理。该库与内置的多处理相同,除了它提供了我们需要的更好的腌制支持。

基本上,您将需要使用 nofollow noreferrer“> supperrocessing.managers.managers ,以及适合访问该州的代理。 objproxy在下面的代码中提供的代理是共享名称空间和实例方法的代理(除了受保护/私有属性外)。一旦拥有这些,您只需要使用经理和代理创建类shop的对象即可。这是使用新添加的创建shop> shop的方法完成的。这是一个类构造函数,shop> shop的所有对象应仅使用此方法创建,而不是直接调用构造函数。完整代码:

import multiprocess
from multiprocess import Manager, Process
from multiprocess.managers import NamespaceProxy, BaseManager
import types


class ObjProxy(NamespaceProxy):
    """Returns a proxy instance for any user defined data-type. The proxy instance will have the namespace and
    functions of the data-type (except private/protected callables/attributes). Furthermore, the proxy will be
    pickable and can its state can be shared among different processes. """

    def __getattr__(self, name):
        result = super().__getattr__(name)
        if isinstance(result, types.MethodType):
            def wrapper(*args, **kwargs):
                return self._callmethod(name, args, kwargs)
            return wrapper
        return result


class Shop:

    def __init__(self, stock=5):
        self.stock = stock

    @classmethod
    def create(cls, *args, **kwargs):

        # Register class
        class_str = cls.__name__
        BaseManager.register(class_str, cls, ObjProxy, exposed=tuple(dir(cls)))

        # Start a manager process
        manager = BaseManager()
        manager.start()

        # Create and return this proxy instance. Using this proxy allows sharing of state between processes.
        inst = eval("manager.{}(*args, **kwargs)".format(class_str))
        return inst

    def get_item(self, l, x):
        with l:
            if self.stock >= x:
                self.stock -= x
                print(f"{self.stock} = remaining")

    def k(self, l, n):
        pass


if __name__ == "__main__":
    manager = Manager()
    l = manager.Lock()
    obj = Shop.create()
    p1 = Process(target=obj.get_item, args=(l, 1, ))
    p2 = Process(target=obj.get_item, args=(l, 1, ))
    p3 = Process(target=obj.get_item, args=(l, 1, ))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock) 

输出

4 = remaining
3 = remaining
2 = remaining
Final:  2

注意:这两行的说明:

manager = Manager()
l = manager.Lock()

我们不需要在您的锁之前为锁创建经理(随后是代理)的原因示例概述在这里。它之所以不使用上述代码而不创建代理的原因是因为我们不再在主过程中创建进程,并且在当前过程中不存在锁定(因为为我们的复杂对象创建管理器为了共享其状态,产生了自己的服务器流程)

The reason this code is not working as you expect it to is because multiprocessing does not share its state with child processes. This means that each of the process you start, p1, p2 and p3, get a copy of the object of class Shop. It is NOT the same object. There are two ways you can fix this, share the instance attribute stock with the processes, or share the whole object itself. The second way is probably better for your larger use case if the shop object holds other data that needs to be shared between the processes to.

Method 1:

To share the value of only the stock instance variable, you can use multiprocessing.Value. The way to create shared integers using this and also access their value is here:

shared_int = multiprocessing.Value('i', 5)
print(f'Value is {shared_int.value}')  # 5 

Adapting to your use case, the code will then become:

import multiprocessing


class Shop:
    def __init__(self, stock=5):
        self.stock = multiprocessing.Value('i', stock)


    def get_item(self, l, x):
        l.acquire()
        if self.stock.value >= x:
            self.stock.value -= x
            print(f"{self.stock.value} = remaining")
        l.release()


if __name__ == "__main__":
    l = multiprocessing.Lock()
    obj = Shop()

    p1 = multiprocessing.Process(target=obj.get_item, args=(l, 1))
    p2 = multiprocessing.Process(target=obj.get_item, args=(l, 1))
    p3 = multiprocessing.Process(target=obj.get_item, args=(l, 1))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock.value)

Output

4 = remaining
3 = remaining
2 = remaining
Final:  2

Method 2

Sharing the whole complex object is a more involved process. I had recently answered a similar question in detail about sharing complex objects (like the object of class Shop in this case), which also covered the reasoning behind the code provided below. I recommend that you give it a read since it explains the logic behind the code provided at the bottom in greater detail. The only major difference for this use-case is that you will want to use multiprocess, a fork of multiprocessing, instead of multiprocessing. This library works identically to the built-in multiprocessing except for the fact that it offers better pickling support which we will need.

Basically, you will want to use multiprocessing.Managers to share the state, and a suitable proxy to access the state. The ObjProxy provided in below code is one such proxy which shares the namespace as well as instance methods (apart from protected/private attributes). Once you have these, you just need to create the objects of class Shop using the manager and the proxy. This is done using the newly added create method of class Shop. This is a class constructor and all objects of Shop should be created using this method only rather than directly calling the constructor. Full code:

import multiprocess
from multiprocess import Manager, Process
from multiprocess.managers import NamespaceProxy, BaseManager
import types


class ObjProxy(NamespaceProxy):
    """Returns a proxy instance for any user defined data-type. The proxy instance will have the namespace and
    functions of the data-type (except private/protected callables/attributes). Furthermore, the proxy will be
    pickable and can its state can be shared among different processes. """

    def __getattr__(self, name):
        result = super().__getattr__(name)
        if isinstance(result, types.MethodType):
            def wrapper(*args, **kwargs):
                return self._callmethod(name, args, kwargs)
            return wrapper
        return result


class Shop:

    def __init__(self, stock=5):
        self.stock = stock

    @classmethod
    def create(cls, *args, **kwargs):

        # Register class
        class_str = cls.__name__
        BaseManager.register(class_str, cls, ObjProxy, exposed=tuple(dir(cls)))

        # Start a manager process
        manager = BaseManager()
        manager.start()

        # Create and return this proxy instance. Using this proxy allows sharing of state between processes.
        inst = eval("manager.{}(*args, **kwargs)".format(class_str))
        return inst

    def get_item(self, l, x):
        with l:
            if self.stock >= x:
                self.stock -= x
                print(f"{self.stock} = remaining")

    def k(self, l, n):
        pass


if __name__ == "__main__":
    manager = Manager()
    l = manager.Lock()
    obj = Shop.create()
    p1 = Process(target=obj.get_item, args=(l, 1, ))
    p2 = Process(target=obj.get_item, args=(l, 1, ))
    p3 = Process(target=obj.get_item, args=(l, 1, ))

    p1.start()
    p2.start()
    p3.start()

    p1.join()
    p2.join()
    p3.join()

    print("Final: ", obj.stock) 

Output

4 = remaining
3 = remaining
2 = remaining
Final:  2

Note : Explanation for these 2 lines:

manager = Manager()
l = manager.Lock()

The reason why we didn't need to create a manager (and subsequently a proxy) for the lock before in your example is outlined here. The reason why it does not work with the above code without creating a proxy is because we are no longer creating the processes in the main process, and the lock does not exist in the current processes memory space (since creating a manager for our complex object to share its state spawned its own server process)

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文