Python/Json - 检查多个文件中的特定对象

发布于 2025-01-09 17:32:50 字数 436 浏览 0 评论 0原文

我有大量 json 文件 (4000)，我需要检查每个文件中的特定对象。我的代码如下：

import os
import json

files = sorted(os.listdir("my files path"))
for f in files:
    if f.endswith(".json"): 
        myFile = open("my path\\" + f)
        myJson = json.load(bayesFile)
        if myJson["something"]["something"]["what im looking for"] == "ACTION"
            #do stuff
        myFile.close()

正如你可以想象的，这花费了大量的执行时间，我想知道是否有更快的方法......？

原文

I have a huge amount of json files (4000) and I need to check every single one of them for a specific object. My code is like the following:

import os
import json

files = sorted(os.listdir("my files path"))
for f in files:
    if f.endswith(".json"): 
        myFile = open("my path\\" + f)
        myJson = json.load(bayesFile)
        if myJson["something"]["something"]["what im looking for"] == "ACTION"
            #do stuff
        myFile.close()

As you can imagine this is taking a lot of execution time and I was wondering if there is a quicker way...?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

风筝在阴天搁浅。 2025-01-16 17:32:50

这是一种可能对您有所帮助的多线程方法：

from glob import glob
import json
from concurrent.futures import ThreadPoolExecutor
import os

BASEDIR = 'myDirectory' # the directory containing the json files

def process(filename):
    with open(filename) as infile:
        data = json.load(infile)
        if data.get('foo', '') == 'ACTION':
            pass # do stuff

def main():
    with ThreadPoolExecutor() as executor:
        executor.map(process, glob(os.path.join(BASEDIR, '*.json')))

if __name__ == '__main__':
    main()

Here's a multithreaded approach that may help you:

from glob import glob
import json
from concurrent.futures import ThreadPoolExecutor
import os

BASEDIR = 'myDirectory' # the directory containing the json files

def process(filename):
    with open(filename) as infile:
        data = json.load(infile)
        if data.get('foo', '') == 'ACTION':
            pass # do stuff

def main():
    with ThreadPoolExecutor() as executor:
        executor.map(process, glob(os.path.join(BASEDIR, '*.json')))

if __name__ == '__main__':
    main()

回复收藏 0 原文

~没有更多了~