JDBC模板批量更新带有雪花数据库非常慢
我有一个连接到雪花数据库的Spring Boot应用程序，并上传记录（大约50列不同数据类型）。我正在使用
jdbctemplate.batchupdate（insertsql，value，type）
来执行批量插入。目前，它正在消耗 100 秒数的 50,000记录。我想提高批处理性能。但找不到最佳解决方案。
我提到并尝试了的解决方案T根本有帮助。任何建议都将不胜感激
I have a spring boot application that connects to the Snowflake Database and uploads records (approx 50 columns of different data types). I am using
JdbcTemplate.batchUpdate(insertSql, values, types)
to do the bulk insert. Currently, it is consuming around 100 seconds for 50,000 records. I want to improve the batch performance. but not able to find an optimal solution.
I referred to and tried the solution mentioned in this post, but it didn't help at all. Any suggestions will be highly appreciated
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我从批量插入转向 snowflake 复制命令 使用 JDBC。速度快如闪电。使用复制命令，只需 2-3 秒即可从 XS（超小）大小的 Dataware house 的 CSV 文件加载 50000 条记录。
此外，如果出现错误，消息非常清晰，可以在information_schema.load_history中查看。可以加载不同的文件格式，并且有多种选项可以自定义加载过程。
就我而言，我首先将 CSV 文件加载到内部暂存区域（需要不到 1 秒），运行复制命令（需要 1-2 秒），验证 information_schema.load_history 表中的加载状态（需要几毫秒），然后相应地进行
这篇文章也对 使用 JDBC 运行复制命令
I moved away from batch insert to snowflake copy command using JDBC. It is lightning fast. With the copy command, it is barely taking 2-3 seconds to load 50000 records from a CSV file with XS (extra small) size Dataware house.
Moreover, in case of error, messages are very clear and can be viewed in information_schema.load_history. Different file formats can be loaded and there are a variety of options to customize load process.
In my case, I am first loading the CSV file to the internal staging area (takes less than 1 sec), Run Copy command (takes 1-2 seconds), verifying load status in information_schema.load_history table (takes a few milliseconds) and proceed accordingly
This article was also helpful for running copy command with JDBC