将长数据帧写入 SQL
我有一个包含 338 行的数据框(由 8 列组成)。其中一列具有 wkt 格式(即它包含 MultiPolygon 的表示)。我正在尝试使用以下代码将我的数据帧写入 SQL:
engine = sqlalchemy.create_engine(
'mysql+pymysql://root:maria@localhost:3306/topology?unix_socket=/data/mysql/mysql.sock', pool_recycle=3600)
#After researching, I found that if I added pool_recycle=3600, it would prevent a SQL timeout
#Since the Polygon is more than some letters, it is necessary to specify the length:
datatype={
"coordinates": LONGTEXT
}
#Write to Database:
dataframe.to_sql(
name='Federal_Info',
con=engine,
index=False,
dtype=datatype,
if_exists='replace')
#The next two lines of code convert the wkt format to a geom type
engine.execute('''ALTER TABLE `topology`.`Federal_Info`
ADD COLUMN `geom` Polygon;''')
engine.execute('''UPDATE `topology`.`Federal_Info`
SET geom = ST_GeomFromText(coordinates) ;''')
但是,我不断收到以下错误:
Traceback (most recent call last):
File "home/.local/lib/python3.6/site-packages/pymysql/connections.py", line 713, in _write_bytes
self._sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe
...
File "/home/.local/lib/python3.6/site-packages/pymysql/connections.py", line 718, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
pymysql.err.OperationalError: (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")
有办法解决这个问题吗? wkt 专栏是相当大的文本。
我尝试过的:
我尝试将 pool_recycle 参数添加到我的 sql 创建引擎函数中,但这似乎没有成功。
I have a dataframe (consisting of 8 columns) with 338 rows. One of these columns has a wkt format (i.e. it contains a representation of a MultiPolygon). I'm trying to write my dataframe to SQL with the following code:
engine = sqlalchemy.create_engine(
'mysql+pymysql://root:maria@localhost:3306/topology?unix_socket=/data/mysql/mysql.sock', pool_recycle=3600)
#After researching, I found that if I added pool_recycle=3600, it would prevent a SQL timeout
#Since the Polygon is more than some letters, it is necessary to specify the length:
datatype={
"coordinates": LONGTEXT
}
#Write to Database:
dataframe.to_sql(
name='Federal_Info',
con=engine,
index=False,
dtype=datatype,
if_exists='replace')
#The next two lines of code convert the wkt format to a geom type
engine.execute('''ALTER TABLE `topology`.`Federal_Info`
ADD COLUMN `geom` Polygon;''')
engine.execute('''UPDATE `topology`.`Federal_Info`
SET geom = ST_GeomFromText(coordinates) ;''')
However, I keep getting the following error:
Traceback (most recent call last):
File "home/.local/lib/python3.6/site-packages/pymysql/connections.py", line 713, in _write_bytes
self._sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe
...
File "/home/.local/lib/python3.6/site-packages/pymysql/connections.py", line 718, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
pymysql.err.OperationalError: (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")
Is there anyway to solve this issue? The wkt column is quite a large piece of text.
What I tried:
I tried to add a pool_recycle parameter to my sql create engine function but that didn't seem to do the trick.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论