如何有效地迭代具有数百万行的 pandas 数据集并将函数传递给每一行？

Question

我有一个 pandas 数据框，其中包含 700 万个飞行数据实例。航班数据带有我用来获取当时天气的位置和时间。目前，对于 1000 个实例，我的代码花费了 83 秒。考虑到我要经历 700 万，这太长了。我不知道该怎么做。我读到itertuples会更快一点，或者可能使用Dask或多线程。

import warnings
import pandas as pd
from datetime import datetime
from meteostat import Point, Daily
import airportsdata
import time
import threading
import os

warnings.simplefilter(action='ignore', category=FutureWarning)

airports = airportsdata.load('IATA')
all1 = pd.DataFrame()
all2 = pd.DataFrame()

first_mil = pd.read_csv('first_mil.csv')

def weather_from_IATA(airport, time):
    """
    uses airport data from "airportsdata" to find latitude and longitude 
    and pass lat and lon into meteostat, which will return the weather at
    that airport at that time.
    """
    lon, lat = airports[airport]['lat'], airports[airport]['lon']

    # Set time period
    start, end = time, time

    # Create Point for airport
    location = Point(lon, lat)

    # Get daily data for location
    data = Daily(location, start, end)
    weather_time = data.fetch()

    # =================================================================
    return weather_time

for air in range(1000):
    Iata = (first_mil.iloc[air]['ORIGIN'])
    flight_time = (first_mil.iloc[air]['FL_DATE'])
    weather_data = weather_from_IATA(Iata, flight_time)
    all1 = all1.append(weather_data)

end1 = time.time()
print("iteration for append took: ", (end1 - start), "sec")