Python 中的线程、多进程、Twisted

发布于 2024-10-30 18:03:47 字数 1293 浏览 1 评论 0原文

我可以获得将此代码从线程转换为多进程的帮助吗? 那么任何人都可以帮助使用inf twins 转换此代码。

使用twisted上传数据库会有好处吗
Python 内与外部工具。

import  os, pyodbc, sys, threading, Queue


class WorkerThread(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while 1:
            try: # take a job from the queue
                type  = self.queue.get_nowait()

            except Queue.Empty:
                raise SystemExit

            try:
               cxn = pyodbc.connect('DSN=MySQL;PWD=MLML;Option=3') 
               csr = cxn.cursor()    
               # Inserts,update, CRUD

            except:
               # count = count +1
                print 'DB Error', type

if __name__ == '__main__':
    connections =  25

    sml = ('A', 'B', 'C','D',)
    # build a queue with tuples
    queue = Queue.Queue()

    for row in sml:
        if not row or row[0] == "#":
            continue
        queue.put(row) 

    threads = []
    for dummy in range(connections):
        t = WorkerThread(queue)
        t.start()
        threads.append(t)

    # wait for all threads to finish
    for thread in threads:
        thread.join()

    sys.stdout.flush()

#csr.close()
#cxn.close()
print 'Finish'  

Can I get help converting this code from Threading to Mutliprocess.
Then can anyone help convert this code usinf twisted.

Would there be a gain from using twisted to upload db
within Python vs External tools.

import  os, pyodbc, sys, threading, Queue


class WorkerThread(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while 1:
            try: # take a job from the queue
                type  = self.queue.get_nowait()

            except Queue.Empty:
                raise SystemExit

            try:
               cxn = pyodbc.connect('DSN=MySQL;PWD=MLML;Option=3') 
               csr = cxn.cursor()    
               # Inserts,update, CRUD

            except:
               # count = count +1
                print 'DB Error', type

if __name__ == '__main__':
    connections =  25

    sml = ('A', 'B', 'C','D',)
    # build a queue with tuples
    queue = Queue.Queue()

    for row in sml:
        if not row or row[0] == "#":
            continue
        queue.put(row) 

    threads = []
    for dummy in range(connections):
        t = WorkerThread(queue)
        t.start()
        threads.append(t)

    # wait for all threads to finish
    for thread in threads:
        thread.join()

    sys.stdout.flush()

#csr.close()
#cxn.close()
print 'Finish'  

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

你如我软肋 2024-11-06 18:03:47

更新:JP提到了txpostgrestxmysql 模块,我不知道。这些允许 Twisted 异步访问两个数据库(不使用线程池)。

请注意,如果您决定使用 Twisted 的企业 adbapi,它将最终使用线程池来处理数据库连接,大致相当于您现有的示例。请参阅有关使用 Twisted 企业数据库模块的文档

您应该能够直接转换基于线程/队列的代码以使用多处理模块,方法是将线程替换为 Process 实例,并使用多处理 Queue 实现。请参阅多处理文档

Updated: JP mentioned the txpostgres and txmysql modules, which I wasn't aware of. These allow Twisted to access both databases asynchronously (without using a thread pool).

Note that if you decide to use Twisted's enterprise adbapi, it will end up using a thread pool to handle database connections, roughly the equivalent of your existing example. See the docs on using Twisted's enterprise database module.

You should be able to directly translate your thread/queue-based code to use the multiprocessing module, by replacing your threads with Process instances, and uing the multiprocessing Queue implementation. See the multiprocessing docs.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文