Python 中匹配地理空间数据集的快速方法

Question

我有一组 2000 个地理空间点（经度/纬度），我需要将其与其他几个地理空间数据集（我正在使用 Geopandas GeoDataFrames）进行匹配。我使用 sklearn BallTree 函数来查找每个点一定半径内的邻居（在下面的函数中，point 是 2000 个点之一，right_gdf 是我需要从中获取邻居的数据集）。

我目前正在使用 for 循环来遍历所有 2000 个点并找到每个点的邻居。但是，根据 right_gdf 的大小，这可能需要很长时间。我确信有一种方法可以加快这个过程，可能是通过并行计算，但我正在努力寻找它。我尝试使用 Dask 延迟来并行化循环（请参阅下面的代码），但不知何故，这比简单的 for 循环花费的时间更长。

# Function that finds a point's neighbors within a certain radius
def neighbours_radius(point, right_gdf, R=1):

  # Create tree from the right gdf (use haversine for lat/lon coordinates)
  tree = BallTree(right_gdf, leaf_size=40, metric='haversine')

  # Find indices of all neighbors within R
  indices = tree.query_radius(point, r=r)[0]

return indices


# Function that loops through the 2000 points
def knn_gpd(right_gdf, R=75):

  # Load the gdf with the 2000 points
  base = gpd.read_file(...)

  # Empty list to fill in the indices of the neighbors
  neighbors = []

  # Loop through the points and find the neighbors within R.
  for i in range(len(base)):
      point = base.iloc[i:i+1,:]
      ind = neighbours_radius(point, right_gdf, R=R)

      # append index lists
      neighbors.append(ind)

return neighbors


# Function that loops through the 2000 points with Dask delayed
def knn_gpd_dask(right_gdf, R=75):

  # Load the gdf with the 2000 points
  base = gpd.read_file(...)

  # Empty list to fill in the indices of the neighbors
  neighbors = []

  # Loop through the points and find the neighbors within R.
  for i in range(len(base):
    point = base.iloc[i:i+1,:]
    ind = delayed(neighbours_radius)(point, right_gdf, R=R)

    # append index list
    neighbors.append(ind)
  
  result = compute(neighbors)

return result

谁能帮助我加快这个过程？

Answer 1

如果您分析您的代码，我怀疑您会发现创建 BallTree 占用了大部分时间，因为您创建了它 2000 次。您应该尝试仅创建一次，例如：

# Function that loops through the 2000 points
def knn_gpd(right_gdf, R=75):

  # Load the gdf with the 2000 points
  base = gpd.read_file(...)

  # Create tree from the right gdf (use haversine for lat/lon coordinates)
  tree = BallTree(right_gdf, leaf_size=40, metric='haversine')

  # Empty list to fill in the indices of the neighbors
  neighbors = []

  # Loop through the points and find the neighbors within R.
  for i in range(len(base)):
      point = base.iloc[i:i+1,:]

      # Find indices of all neighbors within R
      indices = tree.query_radius(point, r=r)[0]

      # append index lists
      neighbors.append(indices )

return neighbors